In Simple Words: GitLab CI/CD is a system that helps developers automatically check their code to catch errors and, if everything is okay, automatically release their software without manual work. It’s like having robots that test and deliver your code for you, saving time and reducing errors.

GitLab: https://gitlab.com/mr.cloud-book/terraform-ec2-jenkins.git

Step1: create an IAM user

Navigate to the AWS console

Click the “Search” field.

Search for IAM

Click “Users”

Click “Add users”

Click the “User name” field.

Type “Terraform” or as you wish about the name

Click Next

Click “Attach policies directly”

Click this checkbox with Administrator access

Click “Next”

Click “Create user”

Click newly created user in my case Ajay

Click “Security credentials”

Click “Create access key”

Click this radio button with the CLI

Agree to terms

Click Next

Click “Create access key”

Download .csv file

Step2: Create Gitlab Account

Go to https://gitlab.com/

Click on Register now (or)

The easy choice is to click on GitHub it will create an account.

Step3: Terraform Files

Create a blank repository in Gitlab and add these files

main.tf

 resource "aws_security_group" "Jenkins-sg" {
  name        = "Jenkins-Security Group"
  description = "Open 22,443,80,8080"

  # Define a single ingress rule to allow traffic on all specified ports
  ingress = [
    for port in [22, 80, 443, 8080] : {
      description      = "TLS from VPC"
      from_port        = port
      to_port          = port
      protocol         = "tcp"
      cidr_blocks      = ["0.0.0.0/0"]
      ipv6_cidr_blocks = []
      prefix_list_ids  = []
      security_groups  = []
      self             = false
    }
  ]

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "Jenkins-sg"
  }
}


resource "aws_instance" "web" {
  ami                    = "ami-0f5ee92e2d63afc18"  #change Ami if you different region
  instance_type          = "t2.medium"
  key_name               = "Mumbai"   #change key name 
  vpc_security_group_ids = [aws_security_group.Jenkins-sg.id]
  user_data              = templatefile("./install_jenkins.sh", {})

  tags = {
    Name = "Jenkins-sonar"
  }
  root_block_device {
    volume_size = 8
  }
}

provider.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "ap-south-1"     #change to desired region.
}

install_ jenkins.sh

#!/bin/bash
exec > >(tee -i /var/log/user-data.log)
exec 2>&1
sudo apt update -y
sudo apt install software-properties-common
sudo add-apt-repository --yes --update ppa:ansible/ansible
sudo apt install ansible -y
sudo apt install git -y 
mkdir Ansible && cd Ansible
pwd
git clone https://github.com/Aj7Ay/ANSIBLE.git
cd ANSIBLE
ansible-playbook -i localhost Jenkins-playbook.yml

backend.tf

terraform {
  backend "s3" {
    bucket = "<s3-bucket>" # Replace with your actual S3 bucket name
    key    = "Gitlab/terraform.tfstate"
    region = "ap-south-1"
  }
}

Step4: Variables setup in Gitlab (Secrets)

Inside your repository

Click on Settings → ci/cd

Click on Expand at variables

Click on Add variable

Select the Mask variable and expand variable checkboxes

For key use AWS_ACCESS_KEY_ID

For value use the AWS key that is generated for IAM user.

Add variable

The key is added, now for the secret access key

Click on Add variable

Select the Mask variable and expand variable checkboxes

For key use AWS_SECRET_ACCESS_KEY

For value use the AWS key that is generated for IAM user.

Add variable

The keys are added.

Step5: GitLab CI/CD configuration

stages:
  - validate
  - plan
  - apply
  - destroy

stages:: This section defines the stages in the CI/CD pipeline. In your configuration, you have four stages: validate, plan, apply, and destroy.

image:
  name: hashicorp/terraform:light
  entrypoint:
    - '/usr/bin/env'
    - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'

image:: Specifies the Docker image to use for the GitLab Runner. In this case, you're using the " hashicorp/terraform:light" image for running Terraform commands. The entrypoint lines set the environment to include commonly used paths.

before_script:
  - export AWS_ACCESS_KEY=${AWS_ACCESS_KEY_ID}
  - export AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
  - rm -rf .terraform
  - terraform --version
  - terraform init

before_script:: This section defines commands to run before each job in the pipeline.

• The first two lines export the AWS access key and secret access key as environment variables, which are used for AWS authentication in your Terraform configuration.

• rm -rf .terraform: This removes any existing Terraform configuration files and state files to ensure a clean environment.

• terraform --version: Displays the Terraform version for debugging and version confirmation.

• terraform init: Initializes Terraform in the working directory, setting up the environment for Terraform operations.

validate:
  stage: validate
  script:
    - terraform validate

validate:: Defines a job named "validate" in the "validate" stage. This job validates the Terraform configuration.

• script:: Specifies the commands to run as part of this job. In this case, it runs terraform validate to check the syntax and structure of your Terraform files.

plan:
  stage: plan
  script:
    - terraform plan -out=tfplan
  artifacts:
    paths:
      - tfplan

plan:: This job, in the "plan" stage, creates a Terraform plan.

• script:: Runs terraform plan -out=tfplan, which generates a plan and saves it as "tfplan" in the working directory.

• artifacts:: Specifies the artifacts (output files) of this job. In this case, it specifies that the "tfplan" file should be preserved as an artifact.

apply:
  stage: apply
  script:
    - terraform apply -auto-approve tfplan
  dependencies:
    - plan

apply:: This job, in the "apply" stage, applies the Terraform plan generated in the previous stage.

• script:: Runs terraform apply -auto-approve tfplan, which applies the changes specified in the "tfplan" file.

• dependencies:: Specifies that this job depends on the successful completion of the "plan" job.

destroy:
  stage: destroy
  script:
    - terraform init
    - terraform destroy -auto-approve
  when: manual
  dependencies: 
    - apply

destroy:: This job, in the "destroy" stage, is intended for destroying the Terraform-managed resources.

• script:: Runs terraform init to initialize the Terraform environment and then runs terraform destroy -auto-approve to destroy the resources. The -auto-approve flag ensures non-interactive execution.

• when: manual: Specifies that this job should be triggered manually by a user.

• dependencies:: Ensures that this job depends on the successful completion of the "apply" job, meaning you can only destroy resources that have been applied by a prior "apply" job.

Step6: .gitlab-ci.yml

Full Gitlab CI/CD configuration file and add it to the repository

Click on +

Click on New file

The name of the file is .gitlab-ci.yml

Copy this content and add it

stages:
  - validate
  - plan
  - apply
  - destroy

image:
  name: hashicorp/terraform:light
  entrypoint:
    - '/usr/bin/env'
    - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
before_script:
  - export AWS_ACCESS_KEY=${AWS_ACCESS_KEY_ID}
  - export AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
  - rm -rf .terraform
  - terraform --version
  - terraform init

validate:
  stage: validate
  script:
    - terraform validate

plan:
  stage: plan
  script:
    - terraform plan -out=tfplan
  artifacts:
    paths:
      - tfplan

apply:
  stage: apply
  script:
    - terraform apply -auto-approve tfplan
  dependencies:
    - plan

destroy:
  stage: destroy
  script:
    - terraform init 
    - terraform destroy -auto-approve
  when: manual
  dependencies: 
    - apply

Click commit.

It will automatically start the build.

Now click on Build → Pipelines

Build started.

Click on Running or stages

It will open like this

Click on validate to see the build output.

Initialized and validated terraform code.

Click on Jobs to come back.

See plan output

Now come back and see apply output also.

Go to the AWS console and see whether it’s provisioned Ec2 or not.

Connect the instance to Putty or Mobaxtreme.

Use the below commands

cd /
cd Ansible  #mkdir used in shell script
cd ANSIBLE  #cloned repo
ls  #to see ansible playbook

Now come back to

cd /home/ubuntu
cd /var/log/
ls
cat user-data.log

It’s completed running the Ansible playbook to install Jenkins.

Copy the public IP of the ec2 instance

<Ec2-public-ip:8080> 
sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Copy the password and sign in.

Step7: Destroy

Go back to Gitlab

We need to manually provide destroy to delete resources.

Click on >> in stages

Now select destroy and the Run job

Destroy completed.

CI/CD look

In a fast-paced and dynamic software development landscape, where agility, collaboration, and reliability are paramount, GitLab CI/CD emerges as an indispensable ally. With the power to automate repetitive tasks, catch errors early, and streamline the delivery of software, it transforms the way teams work.

In conclusion, GitLab CI/CD simplifies and accelerates the software development process. It empowers developers to focus on what they do best — creating innovative and valuable software — while the CI/CD pipeline handles the rest. As you embark on your journey with GitLab CI/CD, remember that it’s not just about automation; it’s about delivering better software faster, and that’s a goal every development team can get behind.

So, whether you’re starting small or scaling up, embrace GitLab CI/CD, and unlock the full potential of your software development projects. Harness the power of automation, collaboration, and quality assurance, and watch your code reach new heights with speed and confidence.

Welcome to the world of GitLab CI/CD, where your code transforms into a well-oiled, error-resistant, and agile machine, poised for success in the digital age. Happy coding!