Chuan Chuan Law

DevOps | Software Automation | Continuous Integration

Page 2 of 12

Docker – How To Source A File In Dockerfile

If you want to source /etc/profile.d/apache-maven.sh, do this in your Dockerfile

RUN echo ‘. /etc/profile.d/apache-maven.sh’ >> ~/.bashrc

Consul – Integrating Nagios Checks

We can integrate script checks into Consul.  To do so with Nagios:

  • Install nagios-plugins-basic. That will contain some basic checks such as disk util, and cpu load

apt-get install nagios-plugins-basic

  • There are also other custom Nagios checks on the internet for eg: checks for Open Connections

Put this script in:

/usr/lib/nagios/plugins

  • Create JSON config (connections.json) and put it under /etc/consul.d/client

{
“check”: {
“name”: “Open Connections”,
“interval”: “60s”,
“args”: [“/usr/lib/nagios/plugins/check_connections”, “-c”, “{{connection_limit_critical}}”, “-w”, “{{connection_limit_warn}}” ],
“status”: “passing”
}

}

  • Restart Consul service
  • You should see this on Consul UI

How To Publish Artifacts To Artifactory Using Curl

We can publish an artifact onto Artifactory using unconventional way such as Curl.

This is how we can do that:

  • Generate a pom file using a bash script, so the artifact will have versioning, etc.
#!/usr/bin/env bash
cat  << EOF
<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>1.0.0</modelVersion>
  <groupId>com.org.test</groupId>
  <artifactId>apple</artifactId>
  <version>apple</version>
  <packaging>tar</packaging>
</project>
EOF
  • Post onto Artifactory using Curl

curl -i -X PUT -u username:password –data-binary @”apple.tar” “https://artifactory/snapshot/com/org/test/apple/apple.tar”

Starting Dropwizard Application Using Jar Packaging & SysV

By default Debian package will generate an Upstart and SysV script.

This blog will show how we can ship an application in Jar file and start it using SysV

  • Put your jar file somewhere. E.g: /opt
  • Copy the Debian package generated config in /etc into the new app config directory. You might need to reformat the YAML file a bit.

For example:

Some vars might look like this:

timeout: ${sys.TIMEOUT!”5000ms” }

We want to change it to look like:

timeout: 5000ms

  • Uninstall Debian packages

apt-get purge <package>

  • Delete the generated Upstart (/etc/init/<package>.conf) and SysV script (/etc/init.d/<package>)
  • Create a SysV script which looks like this:
#!/bin/bash
#
# dropwizard     This shell script takes care of starting and stopping Dropwizard applications
#
# chkconfig: - 80 20
#
### BEGIN INIT INFO
# Provides: dropwizard
# Required-Start: $network $syslog
# Required-Stop: $network $syslog
# Default-Start:
# Default-Stop:
# Short-Description: start and stop dropwizard
### END INIT INFO

#You just need to replace the {{ var }} below for your application
APPLICATION_NAME="{{ app_name }}"
APPLICATION_USER="{{ app_user }}"
APPLICATION_HOME="{{ app_home }}"
APPLICATION_JAR="{{ app_name }}.jar"
APPLICATION_CONFIG="{{ app_name }}.yml"
APPLICATION_CONFIG_DIR="{{ app_config_dir}}"
APPLICATION_CMD="java {{ additional_java_args }} -jar ${APPLICATION_HOME}/${APPLICATION_JAR} server ${APPLICATION_CONFIG_DIR}/${APPLICATION_CONFIG}"
APPLICATION_SHUTDOWN_WAIT=120

dropwizard_pid() {
    echo `ps aux | grep "${APPLICATION_CMD}" | grep -v grep | awk '{ print $2 }'`
}

start() {
    pid=$(dropwizard_pid)
    if [ -n "$pid" ]
    then
        echo "${APPLICATION_NAME} is already running (pid: $pid)"
    else
        # Start dropwizard
        echo "Starting ${APPLICATION_NAME}"
        su ${APPLICATION_USER} -c "cd ${APPLICATION_HOME}; ${APPLICATION_CMD} > /dev/null &"
    fi
    return 0
}

stop() {
    pid=$(dropwizard_pid)
    if [ -n "$pid" ]
    then
        echo "Stopping ${APPLICATION_NAME}"
        kill $pid

    let kwait=$APPLICATION_SHUTDOWN_WAIT
    count=0
    count_by=5
    until [ `ps -p $pid | grep -c $pid` = '0' ] || [ $count -gt $kwait ]
    do
        echo "Waiting for processes to exit. Timeout before we kill the pid: ${count}/${kwait}"
        sleep $count_by
        let count=$count+$count_by;
    done

    if [ $count -gt $kwait ]; then
        echo "Killing processes which didn't stop after ${APPLICATION_SHUTDOWN_WAIT} seconds"
        kill -9 $pid
    fi
    else
        echo "${APPLICATION_NAME} is not running"
    fi

    return 0
}

status(){
    pid=$(dropwizard_pid)
    if [ -n "$pid" ]; then
        echo "${APPLICATION_NAME} is running with pid: $pid"
    else
        echo "${APPLICATION_NAME} is not running"
    exit 1
    fi
}


case $1 in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        start
        ;;
    status)
        status
        ;;
    *)
    echo "Usage: $0 {start|stop|restart|status}"
    exit 1
    ;;

esac

exit 0
  • Finally, we can start the application by:

service <package> start

Jenkinsfile – Credentials Binding Plugin – sshUserPrivateKey

Example below shows how to use the sshUserPrivateKey of a server to do Git clone in Jenkinsfile:

node(‘jenkins_node’){

stage(‘checkout code’){

//Private key for server is stored in Jenkins with id ‘private_key_for_server’ and can be accessible via variable ‘private_key’

withCredentials([sshUserPrivateKey(credentialsId: ‘private_key_for _server’, keyFileVariable: ‘private_key’, passphraseVariable: ”, usernameVariable: ”)]){

// start ssh-agent
sh ‘ssh-agent /bin/bash’

// add private key to ssh-agent, check if private key is successfully added and git clone using the private key

sh ‘eval $(ssh-agent) && ssh-add ${private_key} && ssh-add -l && git clone git@git.test.com:test.git’

}

}

}

 

How To Trigger Downstream Job And Pass In Parameters

This is a Jenkinsfile that:

  • Takes in parameter “test_branch”
  • Runs Job-1 and pass in parameter “test_branch” to it
  • Upon successful run of Job-1, it will trigger Job-2 and pass in the “test_branch” parameter
  • Note that Job-1 and Job-2 are 2 separate Jenkins jobs, and the Jenkinsfile below belongs to the Jenkins job that triggers both Job-1 and Job-2

node(DOCKER_IMAGE){

deleteDir()

checkout([$class: ‘GitSCM’, branches: [[name: “${test_branch}”]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs:              [[credentialsId: ‘abc’, url: GIT_URL]]])

stage(‘job 1’){

build job: ‘Job-1’, parameters: [[$class: ‘StringParameterValue’, name: ‘test_branch’, value: “${test_branch}”]]
}
stage(‘job 2’){

build job: ‘Job-2’, parameters: [[$class: ‘StringParameterValue’, name: ‘test_branch’, value: “${test_branch}”]]
}

Jenkins 2 – How To Use Credentials Binding Plugin

This post will be demonstrated using Secret File.

  • Select the Credentials “Kind” – “Secret File”

Screen Shot 2017-12-27 at 10.39.56 AM

 

  • Enter the password in a plain text file
  • Enter a ID, Description and upload the plain text file
  • Using Credentials Binding Plugin, enter the following in the Jenkinsfile where credentialsId refers to ID field above

withCredentials([file(credentialsId: ‘ID’, variable: ‘FILE’)]) {
ansiblePlaybook(
playbook: ‘playbook.yml’,
sudo: true

)

}
}

 

 

Jenkinsfile – Build & Publish Docker In Docker

The Jenkinsfile below shows how build and publish a  Docker image to Docker registry on a Dockerized Jenkins node:

//Running on Docker node

node(DOCKER_IMAGE){

deleteDir()

checkout([$class: ‘GitSCM’, branches: [[name: “${git_branch}”]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs:     [[credentialsId: ‘abc’, url: GIT_URL]]])

stage(‘docker build, tag & push’){

//credentials for Docker registry

withCredentials([usernamePassword(credentialsId: ‘dockerpush’, passwordVariable: ‘pass’, usernameVariable: ‘user’)]) {

dir(“${dockerfile_path}”)
{

//Build the Docker image
def dockerImage=docker.build(“${docker_source_image_tag}”)

//Tag the image

sh ‘docker tag “${docker_source_image_tag}” “${docker_target_image_tag}”‘

docker.withRegistry(‘https://docker-test.com’, ‘dockerpush’) {

//Log into the Docker registry

sh “docker login -u ${user} -p ${pass} https://docker-test.com”

//Push the image

dockerImage.push(‘latest’)
}

}

}
}

}

How To Write Jenkinsfile

Jenkinsfile is another great feature from Jenkins2.

Below is an example of a Jenkinsfile:

properties(

[             

   //Parameters of a Jenkins build  
parameters(
[
text(defaultValue: ”, description: ‘URL’, name: ‘ARTIFACT’),
choice(choices: ‘qa’, description: ‘Deploy_Env’, name: ‘DEPLOY_ENV’),
string(defaultValue: ‘master’ , description: ‘ Branch’,name:’BRANCH’)
]
)
]
)

//Which node the job should run on

node(‘master’){

//Delete directory before job starts

deleteDir()

//Git checkout certain branch using defined Git credentials

checkout([$class: ‘GitSCM’, branches: [[name: “${branch}”]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: ‘abc’, url: GIT_URL]]])

//Name of which stage of task that is running
stage(‘deploy’){

//Credentials with secret file configured in Jenkins

withCredentials([file(credentialsId: ‘PASS’, variable: ‘FILE’)]) {

//Execute shell script

sh ‘ansible-galaxy install -r requirements.yml –force’

//Ansible command

ansiblePlaybook(
playbook: ‘deploy.yml’,
inventory: ‘inventory/qa.inventory’,
extraVars:[
artifact_url: “${ARTIFACT}”,
],
extras: ‘–diff –vault-password-file ${FILE} –tags ${ACTION}’,
colorized: true

)

}
}

}

Enter Jenkinsfile into Jenkins2 as below:

Screen Shot 2017-10-24 at 11.14.39 AM

References on Jenkinsfile

Screen Shot 2017-10-20 at 1.28.07 PM

How To Install Tomcat8 On Ubuntu 14 Trusty By Extracting Package

Introduction

Tomcat8 package is yet currently available for Ubuntu 14.04. However, we can set up Tomcat8.0.32 on Ubuntu 14.04 by using the package for Tomcat 16.

A Tomcat8 package consists of the following packages:

  • authbind
  • libcommons-collections3-java
  • libcommons-dhcp-java
  • libcommons-pool-java
  • libecj-java
  • libtomcat8-java
  • tomcat8
  • tomcat8-common

Steps:

  • Install Tomcat8.0.32 on a Ubuntu 16 box
  • Inspect the contents of the package

dpkg -c ./path/to/tomcat8_8.0.32-1ubuntu1.3_all.deb

The path showed will be a reference of file structure of a Tomcat 8 installation which we will need to mimic in the Ubuntu 14.04 box

  • Download Tomcat8.0.32 package for Ubuntu 16
  • Extract files from the Debian package:

ar -x tomcat8_8.0.32-1ubuntu1.3_all.deb

  • Extract files from data.tar.gz using tar

tar -xf data.tar.gz

  • The above will produce usr, etc and var folders
  • Move the folders above into /usr, /etc and /var respectively
  • Do the same for libtomcat8-java and tomcat8-common packages
  • Apt-get install the rest of the required packages:

– authbind

– libcommons-collections3-java

– libcommons-dbcp-java

– libcommons-pool-java

– libecj-java

Note: if you are getting dependency issue during the install of libecj-java, fix with “apt-get install -f

  • Create tomcat8 user

groupadd tomcat8

useradd -s /bin/false -g tomcat8 -d /usr/share/tomcat8 tomcat8

  • /usr/share/tomcat8/lib contains symlinks to /usr/share/java/tomcat8 which are extracts from libtomcat8-java
  • Fix file permissions and ownership in /usr, /etc, /var. Some will require ownership of root:tomcat8
« Older posts Newer posts »

© 2019 Chuan Chuan Law

Theme by Anders NorenUp ↑