CI/CD pipeline using Dynamic Distributed Cluster

5 min readJul 17, 2020

Task -

Create container image that’s has Linux and other basic configuration required to run Slave for Jenkins. ( example here we require kubectl to be configured )

2. When we launch the job it should automatically starts job on slave based on the label provided for dynamic approach.

3. Create a job chain of job1 & job2 using build pipeline plugin in Jenkins

4. Job1 : Pull the Github repo automatically when some developers push repo to Github and perform the following operations as:

a. Create the new image dynamically for the application and copy the application code into that corresponding docker image

b. Push that image to the docker hub (Public repository)

( Github code contain the application code and Dockerfile to create a new image )

5. Job2 ( Should be run on the dynamic slave of Jenkins configured with Kubernetes kubectl command): Launch the application on the top of Kubernetes cluster performing following operations:

1. If launching first time then create a deployment of the pod using the image created in the previous job. Else if deployment already exists then do rollout of the existing pod making zero downtime for the user.

2. If Application created first time, then Expose the application. Else don’t expose it.

Let’s get started :

Step 1. Creating the container image that’s has Linux and other basic configuration required to run Slave for Jenkins. ( example here we require kubectl to be configured ).

First I have created one workspace named as devtask in / directory. In this workspace I have created one Dockerfile for build the image that has linux and other basic configuration required to run Slave for Jenkins. ( example here we require kubectl to be configured ).In this workspace I have added one hidden folder “.kube” in which all the certificates ,key and config file that is required by kubectl to connect to kubernetes is added.

This is the Dockerfile that I’ve created. I have copied the “.kube” directory that contains all the requirements for kubectl to connect with kubernetes.

Now simply build a Docker image using Dockerfile using following command :

$ docker build -t jenkins4:v1 .

For our requirement we need to tag this image in order to upload it on docker hub. We can tag a image using following command.

$ docker tag jenkins4:v1 ameet333/jenkins4:v1

To push this image on Dokcer hub use following command

$ docker push ameet333/jenkins4:v1

After the image has been uploaded to docker hub successfully we need to do some settings so that jenkins can be able to create dynamic slave for running the second job over it.

For this go to the docker host . In docker host first run “ systemctl status docker”. After this go to file — /usr/lib/systemd/system/docker.service and edit this file by command.

After adding this line to this file we have to run two commands :

$ systemctl daemon-reload
$ systemctl restart docker

These commands are use to apply the changes we have done in the above file.

Step 2. After launching the job jenkins should automatically start the job on the dynamic slave based on the labels , so for that we need to configure the jenkins and we have to also install 2 plugins “Docker and Yet Another Docker”.

Go to jenkins → Manage Nodes and clouds — -> Configure clouds

In the Docker Host Uri — give the ip of dockerhost

For better understanding of creating a Distributed Jenkins master/slave arch. refer my previous blog : https://www.linkedin.com/posts/amit-tiwari-97347b171_vimaldaga-righteducation-iiecabrrise-activity-6673555584689815552-TSro

Step 3. Creating the first job for the task-

Through this job jenkins will pull the Github repo automatically when some developers push repo to Github.

In this Github repository developer uploaded a application code and a Dockerfile.

This job will simply create a new docker image dynamically for the application and copy the application code into that corresponding docker image push that image to the docker hub (Public repository)

Step 4. creating the second job that will run on dynamic slave. In this job we are giving the labels that we have assign at the time of configure the cloud so that by using this labels Jenkins will know that they have to run this job on dynamic slave and start creating slave dynamically for running this job.

Job 2 should run on the dynamic slave of Jenkins configured with K8s kubectl command.

Launch the application on the top of K8s cluster performing following operations:

If launching for the first time then create a deployment of the pods using the image created in the previous job. Else if deployment already exists then do rollout of the existing pod making zero downtime for the user.
If application created first time, then expose the application. Else don’t expose it

As you can see the deployment has been created successfully on top of kubernetes cluster . All the pods are running fine and service has also created for exposing this deployment to outside world. The pods are running with the image that we have created in the first job.

The site is up running successfully on top of K8s.