CI/CD Pipeline Deployment with Docker (Phase 1)

Introduction

In Phase 1, you'll prepare the infrastructure required to build a complete Continuous Integration and Continuous Deployment (CI/CD) pipeline using Jenkins, GitHub, Maven, Docker, SonarQube, and Amazon Elastic Container Registry (Amazon ECR).

The primary objective of this phase is to provision an Ubuntu EC2 instance, install the required DevOps tools, and prepare the environment for implementing the CI/CD pipeline in the next phase.

CI/CD Pipeline Overview

The overall workflow consists of the following steps:

1. Provision an Ubuntu EC2 instance on AWS.
2. Install and configure Jenkins, Git, Maven, Docker, and SonarQube.
3. Configure a Jenkins Pipeline to retrieve source code from GitHub.
4. Build the application using Maven.
5. Build and push a Docker image to Amazon Elastic Container Registry (Amazon ECR).
6. Pull the Docker image from Amazon ECR.
7. Deploy the application using Docker containers.

Phase 1: Infrastructure Setup

The first step is to provision an Ubuntu EC2 instance that will host Jenkins and the supporting DevOps tools.

Step 1: Sign In to the AWS Management Console

Open your web browser and navigate to the AWS Management Console.

Sign in using your AWS account credentials.

Step 2: Open the Amazon EC2 Console

After signing in:

1. Use the search bar at the top of the console.
2. Search for EC2.
3. Select Amazon EC2 under the Compute category.

This opens the EC2 Dashboard where you can provision virtual machines.

Step 3: Launch an EC2 Instance

From the EC2 Dashboard:

1. Select Instances from the left navigation pane.
2. Click Launch Instance.

This starts the instance creation wizard.

Step 4: Select an Amazon Machine Image (AMI)

Choose an operating system for the instance.

For this guide, select:

• Ubuntu
• Ubuntu Server 22.04 LTS

Then click Select.

Ubuntu provides an excellent platform for running Jenkins and other DevOps tools.

Step 5: Choose a Key Pair

AWS uses key pairs for secure SSH authentication.

You can:

• Select an existing key pair, or
• Create a new key pair.

Download and store the private .pem file securely, as it will be required to connect to the EC2 instance.

Step 6: Configure Instance Settings

Configure the instance according to your project requirements.

Typical settings include:

• VPC
• Subnet
• Auto-assign Public IP
• IAM Role
• Network configuration

For a basic setup, the default configuration is sufficient.

Step 7: Configure the Security Group

Create or select a security group with the required inbound rules.

At a minimum, allow:

This rule enables secure remote access to the EC2 instance.

Additional ports such as 8080, 9000, or 80 can be added later when configuring Jenkins, SonarQube, or web applications.

Step 8: Configure Storage

Specify the root storage volume for the instance.

The default storage allocation is generally sufficient for testing and learning purposes, though larger environments may require additional storage.

Step 9: Review and Launch

Review the instance configuration.

Verify:

• Instance Name
• Ubuntu AMI
• Instance Type
• Key Pair
• Network Configuration
• Security Group
• Storage

Click Launch Instance.

AWS provisions the Ubuntu server within a few minutes.

Step 10: Connect to the Ubuntu Instance

After the instance reaches the Running state, connect to it using an SSH client.

Common SSH clients include:

• MobaXterm
• PuTTY
• OpenSSH
• AWS EC2 Instance Connect

For this guide, MobaXterm is used.

Use the PEM Key

When connecting through MobaXterm or another SSH client, specify the downloaded .pem private key.

This key authenticates your connection to the Ubuntu server.

Store the PEM file securely on your local machine.

Verify the Connection

After authentication completes successfully, you'll be connected to the Ubuntu EC2 instance through SSH.

You are now ready to install Jenkins, Git, Maven, Docker, SonarQube, and the remaining components required for building the CI/CD pipeline.

At this point, the infrastructure setup for Phase 1 is complete.

Install Java and Jenkins on Ubuntu

Jenkins is a Java-based automation server, so the Java Runtime Environment (JRE) must be installed before installing Jenkins.

In this section, you'll install OpenJDK 17, configure the official Jenkins repository, install Jenkins 2.440.2, and complete the initial Jenkins setup.

Step 1: Update the Package Repository

Before installing any software, update the package index to ensure the latest package information is available.

sudo apt update  

Command Explanation

• apt is the Advanced Package Tool used in Debian-based Linux distributions such as Ubuntu.
• apt update refreshes the local package index by retrieving the latest package information from the configured repositories.

Step 2: Install OpenJDK 17

Install Java Runtime Environment (JRE), which Jenkins requires to run.

sudo apt install fontconfig openjdk-17-jre  

This command installs:

• fontconfig – Required for font management used by Java applications.
• openjdk-17-jre – Java Runtime Environment required for Jenkins.

Step 3: Verify the Java Installation

Verify that Java has been installed successfully.

java --version  

The command displays the installed Java version.

If the installation is successful, the output should indicate OpenJDK 17.

At this stage, the Ubuntu server is ready for the Jenkins installation.

Step 4: Add the Jenkins Repository

Update the package index before configuring the Jenkins repository.

sudo apt update  

Download the Jenkins repository signing key.

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \  
https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key  

Command Explanation

• wget downloads files from the internet.
• The downloaded signing key verifies the authenticity and integrity of Jenkins packages during installation.

Step 5: Add the Jenkins APT Repository

Register the Jenkins repository with the Ubuntu package manager.

echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \  
https://pkg.jenkins.io/debian-stable binary/ | sudo tee \  
/etc/apt/sources.list.d/jenkins.list > /dev/null  

Command Explanation

• echo prints the repository configuration.
• deb specifies an APT repository.
• tee writes the repository configuration into the Jenkins sources list.

Once configured, Ubuntu can retrieve Jenkins packages directly from the official Jenkins repository.

Step 6: Refresh the Package Index

Update the package list to include packages from the newly added Jenkins repository.

sudo apt-get update  

This ensures that the Jenkins package becomes available for installation.

Step 7: Install Jenkins

Install Jenkins using the package manager.

sudo apt-get install jenkins  

The package manager downloads and installs Jenkins along with any required dependencies.

Step 8: Start and Enable Jenkins

Enable Jenkins to start automatically during system boot and start the service immediately.

sudo systemctl enable jenkins

sudo systemctl start jenkins

sudo systemctl status jenkins  

Command Explanation

If the installation is successful, Jenkins will be running as a system service.

At this point, both Java and Jenkins have been successfully installed on the Ubuntu server.

Step 9: Access the Jenkins Web Interface

By default, Jenkins listens on port 8080.

Open a web browser and navigate to:

http://\<EC2_PUBLIC_IP>:8080  

If Jenkins is running correctly, the Unlock Jenkins page will be displayed.

Step 10: Allow Port 8080 in the Security Group

If the Jenkins page is not accessible, update the EC2 instance's security group.

Add an inbound rule:

After updating the security group, refresh the browser to access Jenkins.

Step 11: Unlock Jenkins

During the first launch, Jenkins requires an initial administrator password.

Retrieve the password using the following command:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword  

Copy the generated password.

Paste it into the Administrator Password field on the Jenkins Unlock page.

Step 12: Install Jenkins Plugins

After unlocking Jenkins, you'll be prompted to install plugins.

You can choose one of the following options:

• Install Suggested Plugins – Installs the recommended plugin set suitable for most Jenkins environments.
• Select Plugins to Install – Allows you to manually choose the plugins.

For most users, selecting Install Suggested Plugins is recommended.

Step 13: Create the First Administrator User

Once the plugins have been installed, Jenkins prompts you to create the first administrator account.

Provide the following information:

• Username
• Password
• Full Name
• Email Address

Click Save and Finish.

Step 14: Complete the Jenkins Setup

After saving the administrator details, Jenkins completes the initial configuration.

You are redirected to the Jenkins Dashboard, confirming that the setup was successful.

At this stage, the Jenkins server is fully configured and ready for creating CI/CD pipelines.

Install and Configure Git

Git is a distributed version control system that enables developers to track code changes, collaborate efficiently, and manage source code repositories.

Install Git on the Ubuntu server using the following commands:

sudo apt update

sudo apt install git  

Command Explanation

• apt update refreshes the local package index.
• apt install git installs Git and its required dependencies.

Once installed, Jenkins can use Git to clone source code from repositories such as GitHub, GitLab, or Bitbucket.

Install and Configure Apache Maven

Apache Maven is a build automation and dependency management tool primarily used for Java applications.

Install Maven using the Ubuntu package manager.

sudo apt update

sudo apt install maven  

Command Explanation

• apt install maven installs Apache Maven along with all required dependencies.

Installing Maven through the package manager simplifies setup and ensures a compatible version is installed.

Install and Configure Docker

Docker enables applications to be packaged and deployed as lightweight, portable containers.

Install Docker Dependencies

sudo apt update

sudo apt install apt-transport-https ca-certificates curl software-properties-common  

These packages allow Ubuntu to download software securely over HTTPS and manage external repositories.

Add the Docker GPG Key

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -  

This command downloads and installs Docker's official GPG key, allowing Ubuntu to verify Docker packages before installation.

Add the Docker Repository

sudo add-apt-repository \  
"deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"  

This registers Docker's official repository with the Ubuntu package manager.

Install Docker Engine

Update the package list and install Docker.

sudo apt update

sudo apt install docker-ce  

Verify the installation.

sudo docker --version  

Configure Docker Permissions

Allow the current user to execute Docker commands without using sudo.

sudo usermod -aG docker $USER  

Verify the user's group membership.

groups $USER  

Start the Docker Service

Enable Docker to start automatically after every system reboot.

sudo systemctl enable docker

sudo systemctl start docker

sudo systemctl status docker  

If the service is running successfully, Docker is ready to build and run containers.

Install and Configure SonarQube

SonarQube requires Java and PostgreSQL before installation.

Step 1: Install OpenJDK 17

sudo apt-get install openjdk-17-jdk -y  

Step 2: Add the PostgreSQL Repository

sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" > /etc/apt/sources.list.d/pgdg.list'  

Step 3: Install PostgreSQL

sudo apt install postgresql postgresql-contrib -y  

Step 4: Enable PostgreSQL

sudo systemctl enable postgresql  

Step 5: Start PostgreSQL

sudo systemctl start postgresql  

Step 6: Set the PostgreSQL Password

sudo passwd postgres  

Step 7: Switch to the PostgreSQL User

su - postgres  

Step 8: Create a SonarQube Database User

createuser sonar  

Step 9: Open PostgreSQL

psql  

Step 10: Set the Sonar User Password

ALTER USER sonar WITH ENCRYPTED PASSWORD 'yourPassword';  

Step 11: Create the SonarQube Database

CREATE DATABASE sonarqube OWNER sonar;  

Step 12: Grant Database Permissions

GRANT ALL PRIVILEGES ON DATABASE sonarqube TO sonar;  

Step 13: Exit PostgreSQL

\q  

Step 14: Return to the Ubuntu User

exit  

Step 15: Install the ZIP Utility

sudo apt-get install zip -y  

Step 16: Download SonarQube

This guide uses SonarQube 9.9 LTS.

sudo wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-9.9.4.87374.zip  

Step 17: Extract the Archive

sudo unzip sonarqube-9.9.4.87374.zip  

Step 18: Move SonarQube to /opt

sudo mv sonarqube-9.9.4.87374 /opt/sonarqube  

Step 19: Create the Sonar Group

sudo groupadd sonar  

Step 20: Create the Sonar User

sudo useradd -d /opt/sonarqube -g sonar sonar  

Step 21: Assign Ownership

sudo chown sonar:sonar /opt/sonarqube -R  

Step 22: Configure SonarQube

Edit the SonarQube configuration file.

sudo nano /opt/sonarqube/conf/sonar.properties  

Uncomment the following lines:

#sonar.jdbc.username=  
#sonar.jdbc.password=  

Update them with your database credentials.

sonar.jdbc.username=sonar  
sonar.jdbc.password=yourPassword  

Then add the database connection URL.

sonar.jdbc.url=jdbc:postgresql://localhost:5432/sonarqube  

Save the file.

Step 23: Configure SonarQube as a System Service

Create a systemd service.

sudo nano /etc/systemd/system/sonar.service  

Paste the following configuration.

[Unit]  
Description=SonarQube service  
After=syslog.target network.target

[Service]  
Type=forking  
User=sonar  
Group=sonar

ExecStart=/opt/sonarqube/bin/linux-x86-64/sonar.sh start  
ExecStop=/opt/sonarqube/bin/linux-x86-64/sonar.sh stop

StandardOutput=journal

LimitNOFILE=131072  
LimitNPROC=8192

TimeoutStartSec=5  
Restart=always  
SuccessExitStatus=143

[Install]  
WantedBy=multi-user.target  

Step 24: Enable SonarQube

sudo systemctl enable sonar  

Step 25: Start SonarQube

sudo systemctl start sonar  

Step 26: Verify the Service

sudo systemctl status sonar  

A successful status indicates that SonarQube is running correctly.

Step 27: Configure Linux Kernel Parameters

SonarQube uses Elasticsearch internally and requires higher system limits.

Edit the system configuration.

sudo nano /etc/sysctl.conf  

Add the following values.

vm.max_map_count=262144  
fs.file-max=65536  

Apply the following shell limits.

ulimit -n 131072

ulimit -u 8192  

Reboot the server to apply the changes.

sudo reboot  

Step 28: Access the SonarQube Dashboard

Open a web browser and navigate to:

http://\<SERVER_PUBLIC_IP>:9000  

Log in using the default credentials.

During the first login, SonarQube prompts you to change the default password.

Step 29: Install SonarScanner

Download the SonarScanner CLI.

sudo wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-5.0.1.3006-linux.zip  

Extract the archive.

sudo unzip sonar-scanner-cli-5.0.1.3006-linux.zip -d /opt  

Configure the environment variables.

echo "export PATH=\$PATH:/opt/sonar-scanner-5.0.1.3006-linux/bin" >> ~/.bashrc

echo "export SONAR_SCANNER_HOME=/opt/sonar-scanner-5.0.1.3006-linux" >> ~/.bashrc

source ~/.bashrc  

Verify the installation.

sonar-scanner -v  

A successful version output confirms that SonarScanner is installed correctly.

Install SonarScanner

Phase 1 Completed

Congratulations! You have successfully prepared the complete infrastructure required for the CI/CD pipeline.

The Ubuntu server now includes:

• OpenJDK 17
• Jenkins
• Git
• Apache Maven
• Docker
• PostgreSQL
• SonarQube
• SonarScanner

With the infrastructure in place, you're ready to move on to Phase 2, where you'll configure Jenkins Pipelines, integrate GitHub, perform automated builds and code analysis, build Docker images, push them to Amazon ECR, and complete the Continuous Integration and Continuous Deployment (CI/CD) workflow.