Chuan Chuan Law

DevOps | Software Automation | Continuous Integration

Category: Docker

Ansible – How to fix Failed to import docker-py – No module named websocket. Try `pip install docker-py`

Set the path of Python site-packages in your playbook:

environment:
PYTHONPATH: “{{ lookup(‘env’,’PYTHONPATH’) }}:/Users/claw/Library/Python/2.7/lib/python/site-packages:/Users/claw/Library/Python/2.7/lib/python/site-packages”

Docker – How To Set Locale In Ubuntu

Add the following in your Dockerfile to set Locale in Ubuntu

RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

Docker – How To Source A File In Dockerfile

If you want to source /etc/profile.d/apache-maven.sh, do this in your Dockerfile

RUN echo ‘. /etc/profile.d/apache-maven.sh’ >> ~/.bashrc

Jenkinsfile – Build & Publish Docker In Docker

The Jenkinsfile below shows how build and publish a  Docker image to Docker registry on a Dockerized Jenkins node:

//Running on Docker node

node(DOCKER_IMAGE){

deleteDir()

checkout([$class: ‘GitSCM’, branches: [[name: “${git_branch}”]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs:     [[credentialsId: ‘abc’, url: GIT_URL]]])

stage(‘docker build, tag & push’){

//credentials for Docker registry

withCredentials([usernamePassword(credentialsId: ‘dockerpush’, passwordVariable: ‘pass’, usernameVariable: ‘user’)]) {

dir(“${dockerfile_path}”)
{

//Build the Docker image
def dockerImage=docker.build(“${docker_source_image_tag}”)

//Tag the image

sh ‘docker tag “${docker_source_image_tag}” “${docker_target_image_tag}”‘

docker.withRegistry(‘https://docker-test.com’, ‘dockerpush’) {

//Log into the Docker registry

sh “docker login -u ${user} -p ${pass} https://docker-test.com”

//Push the image

dockerImage.push(‘latest’)
}

}

}
}

}

Jenkins – Making Data Persistent In Docker Slave

As in Jenkins Docker Slave, every time a job runs, a new Docker container is started and terminated upon job completion. This will mean it will download all the dependencies every time. To avoid this, we will use the Volume function in Docker.

As we are using the Docker Plugin, there is a field to do this – Volumes.

Docker Jenkins plugin

 

In the example above, we map path /var/lib/jenkins/tools on the Slave machine to path /home/jenkins/tools in Docker container, and /home/jenkins/.m2 on Slave machine to /home/jenkins/.m2 on Docker container. We can specify the mode of Read/Write, Read-Only, etc, by default is Read/Write.

Therefore, all Jenkins tools and Maven dependencies will be stored on the host or Slave Machine every time Docker container runs and will not need to be downloaded upon next Docker container starts. So build time will be saved.

 

How To Set Up Jenkins 2.0 Master & Slaves On Docker

Jenkins 2.0 Master In Docker

We can just use the Jenkins Docker official image, or you can just install Jenkins 2.0 normally.

 Jenkins Slave In Docker

Setup

  • Install Docker
  • Open up TCP port by adding the following in /etc/default/docker
    DOCKER_OPTS="-H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock"
  • On Jenkins Master, install Docker Plugin on Master->Manage Jenkins->Manage Plugins
  • On Master->Manage Jenkins->Cloud configure the communication with Jenkins slave node
  • Docker URL is the IP of the Slave machine in TCP
  • Click on Test Connection, if successful will show the Docker version and Docker API versionDocker Plugin

 

  • Have Docker Jenkins Slave images in the slave box
  • On Master->Manage Jenkins->Manage Plugins->Cloud->Add Docker Template
  • The SSH credentials to access the Docker Jenkins Slave container is the Jenkins user setup in Dockerfile and Jenkins MasterScreen Shot 2017-03-27 at 4.29.05 PM
Docker Slave Image

There are some Docker images that you can use like Docker Slave image , or you can write your own Dockerfile.

Docker image contains a minimum of:

  • Ubuntu
  • Java
  • Jenkins user and password
  • Git
  • OpenSSH for Jenkins Master to SSH to Slave machine
  • Maven, Ruby, etc depends on what your project needs
  • Version managers like NPM or RVM cannot be installed in Docker due to we cannot “source” files like .bashrc

It is a sshd service running on port 22, therefore in the Dockerfile, you will need:

EXPOSE 22

CMD [“/usr/sbin/sshd”, “-D”]

Running
  • We restrict Jenkins jobs to run based on the label we give in Docker template
  • When job runs, Jenkins Master will spin up the Docker image on the Slave machine
  • When job completes, the Docker image will be terminated
  • Therefore, we can have multiple Docker images for different types of jobs, Ruby, Maven, NodeJS on a single Slave machine

Docker – Using Tiller To Overwrite Environment Specific Variables

To use Tiller, in Dockerfile, we specify:

ADD deploy/tiller/*.yaml /etc/tiller/
ADD deploy/tiller/environments/dev /etc/tiller/environments/
ADD deploy/tiller/templates/* /etc/tiller/templates/

Tiller folder comprises of the following:

  • environments

This is where the yaml for environment specific variables are written:

#These are the variables to overwrite service.conf.erb template

service.conf.erb:
target: /etc/appleapi/application.ini
config:
environment_name: “prod”

#These are the variables to overwrite template.conf.erb template

template.conf.erb:
target: /etc/appleapi/application.conf

#database
appleapi_database_url: jdbc:postgresql://appledb:5432/appleapi
deliveryapi_database_user: apple
deliveryapi_database_password: apple

  • templates

There are the files in the server to be overwritten:

# PostgreSQL configuration (default datasource)
db.default.driver=org.postgresql.Driver
db.default.url=”<%=appleapi_database_url%>”
db.default.username=”<%=appleapi_database_user%>”
db.default.password=”<%=appleapi_database_password%>”

  • common.yaml

data_sources: [ “defaults” , “file” , “environment” ]
template_sources: [ file ]
deep_merge: true

Load the defaults, file and then environment data source to overwrite the templates

  • default.yaml

global:
application_name: “appleapi”
application_port: 9001

Default variables that are shared regardless of environment

Run it in Dockerfile:

RUN tiller -b /etc/tiller -n -v -e production

Docker – How To Run A Debian Package In Docker

Below are the major steps on how to run a Debian package in Docker

Dockerfile
  • Download your Debian file

# Download .deb fileRUN wget $debian_url \
&& sudo dpkg -i appleapi.deb \
&& apt-get install -f

  • Using Ruby Tiller to overwrite environment specific configs (optional)

#Tiller
RUN sudo apt-get update && sudo apt-get -y install ruby
RUN gem install tiller

ADD deploy/tiller/*.yaml /etc/tiller/
ADD deploy/tiller/environments/dev /etc/tiller/environments/
ADD deploy/tiller/templates/* /etc/tiller/templates/

  • Run Tiller and tell it to use development environment configuration

RUN tiller -b /etc/tiller -n -v -e development

  • Expose the port of your app

EXPOSE 9001

  • Start your app via Supervisord (optional as you could just use CMD)

CMD [“/usr/bin/supervisord”]

  • Build the Docker file

docker build  -t appleapi .

  • Run Docker in localhost

docker run -p 9001:9001 appleapi

supervisord.conf

[supervisord]
nodaemon=true

[program:appleapi]
startsecs = 0
autorestart = false
command=/bin/bash -c “appleapi/bin/appleapi”

© 2019 Chuan Chuan Law

Theme by Anders NorenUp ↑