CI/CD using Jenkins: AI End-to-End Series (Part — 8)

By Hiren Rupchandani, Abhinav Jangir, and Ashish Lepcha

In our previous article, we looked at how to deploy a docker container in Kubernetes using Google Kubernetes Engine.

Let’s see how we can automate the process of updating our code through development and integration cycles.

What is CI/CD?

• CI/CD or CICD generally refers to the combined practices of continuous integration and either continuous delivery or continuous deployment.
• It bridges the gaps between development and operation activities and teams by enforcing automation in the building, testing, and deployment of applications.

What is Jenkins?

• Jenkins is one of the most famous Open Source CI/CD tools.
• The leading open-source automation server
• It provides hundreds of plugins to support building, deploying, and automating any project

Advantages of Jenkins:

• Platform independent as it is developed in java.
• It is an Open-Source tool
• Has a huge community with more than 1000 plug-ins.

Jenkins provides us with various interfaces and tools in order to automate the entire process.

A typical CI/CD process with Jenkins

1. We will create a git repository or use an existing one
2. Dev team have the responsibility to commit the code to Dev-Branch
3. Jenkins will fetch the code from Github/GitLab/Repository and will map with the job enabled for a specific task.
4. We will make sure that CI and CD are done for the job/task.
5. Jenkins will pull the code and will enter the commit phase of the task.
6. Jenkins then will compile the code and this is called the build phase of the task.
7. It is deployed by Jenkins after the code is merged to the Master branch by the DevOps team and the job is started for a specific application.
8. We enter the deployment phase cycle once the code is ready to be deployed.
9. The code, after being deployed from Jenkins then gets deployed to the server using the docker container.
10. After the code is working fine in the staging server with unit testing, the same code then is deployed on the production server.

• With all the above steps, Jenkins is responsible for the delivery phase with automating the deployment.
• But before starting with Jenkins, we need to set up a repository. We will use GitLab for the same.

Setting Up Git and GitLab

Photo by Pankaj Patel on Unsplash

• Download and Install Git on Windows using this link.
• While the download and installation finish, go to GitLab and create an account.
• On creating the account, click on Create a project.

Creating a project

• Then click on Blank project. You can also copy the existing repository from GitHub using the import repository option.

Create a blank project or import one if you have a repository

• Make the repository public:

Make the repository public

Create a Jenkinsfile

• Jenkinsfile will be used by Jenkins to set up the pipeline and run the container accordingly.
• You can create Jenkinsfile in the server or can directly create one file in GitLab.

JenkinsFile

Pushing files (Local system -> GitLab repository)

• Go to the file directories in the system.

Setting your git credentials

• Push the files to the GitLab repository:

Commands to Push your files into your repository

• Reload your GitLab repository and you will see all the files present.

Installing Jenkins

• It is available on a variety of systems, including Windows, Linux, Unix, and macOS.
• It is a free source that can handle any kind of build or continuous integration.
• You can integrate Jenkins with a number of testing and deployment technologies.
• We will use the Jenkins container to run the Jenkins service for our purpose.
• Pull and run the Jenkins container using the following command:

docker run -it -d --rm -p 8080:8080 \
-v /var/run/docker.sock:/var/run/docker.sock \
--name jenkins \
jenkins/jenkins:lts

• Output:

Output

• We can see our Jenkins container running.
• Open the Container bash to install docker using:

docker exec -it -u root <Container ID>  bash

• Write the command given below in the bash and hit Enter to install Docker:

apt-get update && \
  apt-get -y install apt-transport-https \
      ca-certificates \
      curl \
      gnupg2 \
      software-properties-common && \
  curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg > /tmp/dkey; apt-key add /tmp/dkey && \
  add-apt-repository \
    "deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \
    $(lsb_release -cs) \
    stable" && \
  apt-get update && \
  apt-get -y install docker-ce

• After the installation is complete, you can check the version of Docker.

Checking Docker Privileges and version

Setting up Jenkins

• Navigate to the address
  - To open the Jenkins service go to your browser.
  - On the address bar type in 8080.
  -We are using 8080 because we ran our Jenkins container on port 8080.
  - On the page copy the address given for the initial password.
• Open the initialAdminPassword file using a text editor such as Notepad or VSCode from the given location.
• Copy the password from the initialAdminPassword file.

Writing password from initialAdminPassword

• Click the Install suggested plugins button to have Jenkins automatically install the most frequently used plugins.

Install Plugins

• After Jenkins finishes installing the plugins, enter the required information on the Create First Admin User page. Click Save and Continue to proceed.

Create User

• On the Instance Configuration page, confirm the port number you want Jenkins to use and click Save and Finish to finish the initial customization.

Confirm Port Number

• Click the Start using Jenkins button to move to the Jenkins dashboard.

Jenkins is ready!

• Here’s what the Jenkins dashboard looks like:

Jenkins Dashboard

• Install Docker plugins by going on the available section of Manage Plugins.

Docker Installation

• Click on Install without restart.

Install without restart

• Once the plugins are installed click on the “Go back to the top page”.

Click on — Go back to the top page

Creating Jenkins Pipeline

• Setting up Jenkins Pipeline
  - Create a Pipeline Job.
  - On the dashboard Click on New item or Create a job.
  - Give your job a name and select pipeline and click OK.

Create a pipeline

• After clicking OK, navigate to Pipeline.
• Select Project script from SCM (Source Code Management).

Select SCM and then Git

• Then, select Git and click on Add Repository URL.
• To get the repository URL, go to your Gitlab and click on Clone and copy the URL.

Copy the Repository URL

• Paste the repository URL:

Paste the URL and click on Save button

• Click on Save to save the changes. We can now start building the pipeline.

Pipeline Set

Building Pipeline

• To build the pipeline click on the build now:

Building the Pipeline

Testing App using provided IP Address

• Accessing docker container
  - Navigate to your system browser and type localhost: port
  (http://localhost/8000)

Container App at http://localhost/8000

Conclusion

• Jenkins is the soul of the continuous integration process as it builds and tests the app continuously.
• It also makes it easier to integrate changes to the process when needed.
• As most of the process is automated, this saves time and effort which can be used to perform other tasks related to the delivery.
• With this successful integration, we have finally come to a conclusion to our series of End to End AI Productization.
• This encompasses the development cycle and beyond of a machine learning model from scratch.
• If you missed out on our previous articles of this series, do check them out!
• Also, do check out our next series where explore AWS services, especially AWS EC2, and have also deployed a model using it.

Follow us for more upcoming future articles related to Data Science, Machine Learning, and Artificial Intelligence.

Also, Do give us a Clap👏 if you find this article useful as your encouragement catalyzes inspiration for and helps to create more cool stuff like this.