CI/CD using Jenkins: AI End-to-End Series (Part — 8)
By Hiren Rupchandani, Abhinav Jangir, and Ashish Lepcha
In our previous article, we looked at how to deploy a docker container in Kubernetes using Google Kubernetes Engine.
Let’s see how we can automate the process of updating our code through development and integration cycles.
What is CI/CD?
- CI/CD or CICD generally refers to the combined practices of continuous integration and either continuous delivery or continuous deployment.
- It bridges the gaps between development and operation activities and teams by enforcing automation in the building, testing, and deployment of applications.
What is Jenkins?
- Jenkins is one of the most famous Open Source CI/CD tools.
- The leading open-source automation server
- It provides hundreds of plugins to support building, deploying, and automating any project
Advantages of Jenkins:
- Platform independent as it is developed in java.
- It is an Open-Source tool
- Has a huge community with more than 1000 plug-ins.
Jenkins provides us with various interfaces and tools in order to automate the entire process.
A typical CI/CD process with Jenkins
- We will create a git repository or use an existing one
- Dev team have the responsibility to commit the code to Dev-Branch
- Jenkins will fetch the code from Github/GitLab/Repository and will map with the job enabled for a specific task.
- We will make sure that CI and CD are done for the job/task.
- Jenkins will pull the code and will enter the commit phase of the task.
- Jenkins then will compile the code and this is called the build phase of the task.
- It is deployed by Jenkins after the code is merged to the Master branch by the DevOps team and the job is started for a specific application.
- We enter the deployment phase cycle once the code is ready to be deployed.
- The code, after being deployed from Jenkins then gets deployed to the server using the docker container.
- After the code is working fine in the staging server with unit testing, the same code then is deployed on the production server.
- With all the above steps, Jenkins is responsible for the delivery phase with automating the deployment.
- But before starting with Jenkins, we need to set up a repository. We will use GitLab for the same.
Setting Up Git and GitLab
- Download and Install Git on Windows using this link.
- While the download and installation finish, go to GitLab and create an account.
- On creating the account, click on Create a project.
- Then click on Blank project. You can also copy the existing repository from GitHub using the import repository option.
- Make the repository public:
Create a Jenkinsfile
- Jenkinsfile will be used by Jenkins to set up the pipeline and run the container accordingly.
- You can create Jenkinsfile in the server or can directly create one file in GitLab.
Pushing files (Local system -> GitLab repository)
- Go to the file directories in the system.
- Push the files to the GitLab repository:
- Reload your GitLab repository and you will see all the files present.
Installing Jenkins
- It is available on a variety of systems, including Windows, Linux, Unix, and macOS.
- It is a free source that can handle any kind of build or continuous integration.
- You can integrate Jenkins with a number of testing and deployment technologies.
- We will use the Jenkins container to run the Jenkins service for our purpose.
- Pull and run the Jenkins container using the following command:
docker run -it -d --rm -p 8080:8080 \
-v /var/run/docker.sock:/var/run/docker.sock \
--name jenkins \
jenkins/jenkins:lts
- Output:
- We can see our Jenkins container running.
- Open the Container bash to install docker using:
docker exec -it -u root <Container ID> bash
- Write the command given below in the bash and hit Enter to install Docker:
apt-get update && \
apt-get -y install apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common && \
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg > /tmp/dkey; apt-key add /tmp/dkey && \
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \
$(lsb_release -cs) \
stable" && \
apt-get update && \
apt-get -y install docker-ce
- After the installation is complete, you can check the version of Docker.
Setting up Jenkins
- Navigate to the address
- To open the Jenkins service go to your browser.
- On the address bar type in 8080.
-We are using 8080 because we ran our Jenkins container on port 8080.
- On the page copy the address given for the initial password. - Open the initialAdminPassword file using a text editor such as Notepad or VSCode from the given location.
- Copy the password from the initialAdminPassword file.
- Click the Install suggested plugins button to have Jenkins automatically install the most frequently used plugins.
- After Jenkins finishes installing the plugins, enter the required information on the Create First Admin User page. Click Save and Continue to proceed.
- On the Instance Configuration page, confirm the port number you want Jenkins to use and click Save and Finish to finish the initial customization.
- Click the Start using Jenkins button to move to the Jenkins dashboard.
- Here’s what the Jenkins dashboard looks like:
- Install Docker plugins by going on the available section of Manage Plugins.
- Click on Install without restart.
- Once the plugins are installed click on the “Go back to the top page”.
Creating Jenkins Pipeline
- Setting up Jenkins Pipeline
- Create a Pipeline Job.
- On the dashboard Click on New item or Create a job.
- Give your job a name and select pipeline and click OK.
- After clicking OK, navigate to Pipeline.
- Select Project script from SCM (Source Code Management).
- Then, select Git and click on Add Repository URL.
- To get the repository URL, go to your Gitlab and click on Clone and copy the URL.
- Paste the repository URL:
- Click on Save to save the changes. We can now start building the pipeline.
Building Pipeline
- To build the pipeline click on the build now:
Testing App using provided IP Address
- Accessing docker container
- Navigate to your system browser and type localhost: port
(http://localhost/8000)
Conclusion
- Jenkins is the soul of the continuous integration process as it builds and tests the app continuously.
- It also makes it easier to integrate changes to the process when needed.
- As most of the process is automated, this saves time and effort which can be used to perform other tasks related to the delivery.
- With this successful integration, we have finally come to a conclusion to our series of End to End AI Productization.
- This encompasses the development cycle and beyond of a machine learning model from scratch.
- If you missed out on our previous articles of this series, do check them out!
- Also, do check out our next series where explore AWS services, especially AWS EC2, and have also deployed a model using it.
Follow us for more upcoming future articles related to Data Science, Machine Learning, and Artificial Intelligence.
Also, Do give us a Clap👏 if you find this article useful as your encouragement catalyzes inspiration for and helps to create more cool stuff like this.