Acheive CI/CD using AWS managed resources & with Jenkins

AWS provides the flexibility to set up pipelines using the below three services to achieve continuous integration and continuous deployment a.k.a CI/CD

a. CodeCommit/Git -  store code in aws codeCommit or a private git repositories.
b. CodeDeploy - automate code deployments
c. CodePipeline - service that deploy, build & test your code

a. Set up Codecommit:  which will be used for version controlling and a useful tool during CI/CD

The first thing is to get the required permissions for the user, setup aws-credentials for your aws environments & then create a repository

a1. Services > IAM > Users > User > Permission > Attach existing policy > AWSCodeCommitPowerUser
a2. Services > IAM > Users > User > Security credentials > HTTPS Git credentials for AWS CodeCommit > Generate credentials

a3. Services > codecommit > create a repo(MyRepo) > cloneURL via Https

 from any host having git installed, do the following
 $ git clone 'clone-https-url'
 // it will prompt for credentials here
 pass the same cred, that you have generated in step a2 
 $ cd repository-directory
 $ vi index.html

 $ vi appspec.yml
 $ git status
 $ git add index.html appspec.yaml
 $ git status
 $ git commit -m 'initial commit'
 $ git push origin master // it will connect via https url and push the file to MyRepo
 $ git log // validate push

this completes the repository setup with a simple source code

b. Set up CodeDeploy to deploy an App: to automate the deployments and adding new features continuously, first is to add 2 roles for codeDeploy for instance.

first setup roles for codeDeploy

create Role1

b1. Services > IAM > Roles > CreateRole > EC2 > AmazonEC2RoleforAWSCodeDeploy
create Role2
b2. Services > IAM > Roles > CreateRole > codeDeploy > AWSCodeDeployRole
create an application
b3. services > codeDeploy > createApp on Ec2
create appDeploymentGroup > name-myappgrp > servicerole-codedeploy-role2 > D-type-In-place > env-config-Amazon-EC2 > add-tag > choose-target-group > createDeploymentGroup

c. Set up CodePipeline: to deploy code from CodeCommit
Services > CodePipeline > create > name-pipeline arn:role> source-CodeCommit > connect/authorize > select-RepoCreatedInStep a3 > branch-Master > Detection-aws-codePipeline > buildProvider-skipBuildStage > deploymentProvider-AWS CodeDeploy >  App-myApp > D-Group-myappgrp > createPipeline

once the pipeline is created it will connect to the source & start deploying the code present in codecommit, and apear as below -

Similarly, you can use GitHub also as a source provider 

Set up CodePipeline: to deploy code from GitHub
Services > CodePipeline > create > name-pipeline arn:role> source-GitHub > connect/authorize > Add Git-Repo > branch-Main > detection-aws-codePipeline-ec2 > buildProvider- skip build stage > deploymentProvider-AWS CodeDeploy >  App-myApp > D-Group-myappgrp > createPipeline

once the pipeline is created it will connect to the source Git this time & start deploying the code present in Github, and apear as below -

Note: On creating a deployment group you are required to choose your deployment & environment type that's where you will deploy your application, in this exercise I choose to deploy on Amazon EC2 instance using Tag value.

At this stage, you should be able to access your application deployed on the EC2 instance using aws CodeCommit/Git > CodePipeline > S3 > CodeDeploy > EC2 instance

In order to use Jenkins, connect to your Jenkins and do the following

 Install the listed plugins from Manage Plugins section of Jenkins & Restart jenkins

 GitHub Integration
 AWS CodePipeline

 AWS CodeDeploy

 AWS CodeBuild
 Http Request
 File Operations

 Blue Ocean // for new Jenkins UI
 

Time to create a Jenkins Job

• Create a new Freestyle Project
• Add your Source Code Management
• Choose Git, provide repository URL.
• For Build Trigger select Poll SCM to add "H/2 * * * *" in Schedule.
• Under Build Environment, select Delete workspace before build starts check box.
• Under Build actions select AWS CodeBuild. On the AWS Configurations, choose Manually specify access and secret keys and provide the keys. 
• Project Configuration - Provide Region, Project Name, choose Use Jenkins source & leave everything blank.
• Add the second Build step and choose File operation (To make sure that all files cloned from the GitHub repository are deleted) Click Add and select File Delete and give '*' under include file pattern.
• Add the third Build step and choose Http request and give S3 URI till your zip file. ex: http://s3-us-east-1.amazonaws.com/s3-bucket-2021/code-artifact.zip
  Ignore SSL error - yes
• Under HTTP Request, choose Advanced and leave Authorization, Headers & Body to default values. 
• Under Response, for Output response to file, enter the code-artifact.zip file name.
• Add fourth Build step as File operation with 2 substeps - Unzip & File Delete and provide code-artifact.zip to include as zip file & file pattern in file Delete.
• On the Post-build Actions, choose Add post-build actions and select the Deploy an application to AWS CodeDeploy check box and provide values
  CodeDeploy App Name
  CodeDeploy AppGrp Name
  CodeDeployDefault.OneAtATime
  Your AWS Region
• at last choose Deploy Revision
     - choose wait for deployment to finish?

you should be able to access your application deployed on the EC2 instance using CI/CD flow.

--

Integrating Jenkins with GitHub

In this exercise, we are going to integrate Github with Jenkins using webhook that triggers an event
every time you commit a change in your code residing in your GitHub repository to invoke a job in your Jenkins.

To start with, you should be having the following in place.

a. A Github account with a code repository 
b. Jenkins up and running
c. Github plugin for integration with Jenkins

First, we going to create a webhook from our code repository by navigating

GitHub > your-code-repository > settings > webhooks > Add webhook 

add Payload URL - https://jenkinsUrl:8080/github-webhook/
Content type - choose application/json
Secret - you can leave this field blank
Which events would you like to trigger this webhook? - Select - Just the push event
Check the Active box - this will deliver the first payload to test the provided URL
Add webhook

Now come to Jenkins are create a freestyle project by navigating

Jenkins > new Item > item name > Freestyle project > OK

select Github project, and give project URL - https://github.com/punitporwal07/aws-codedeploy-linux/
for Source code management, select git and give repository URL - https://github.com/punitporwal07/aws-codedeploy-linux.git
Keep Branch Specifier as - */master
Under Build Triggers - check GitHub hook trigger for GITScm polling
Save

that's it. Now make a code commit in your git repo, the webhook associated with your repository detects the change and trigger a payload to your Jenkins job which you created in the above step.

you will be able to see the console output of your build as below

--

Configuring weblogic deployer plugin in Jenkins

Configure the WebLogic deployer plugin to Deploy an war/ear  on a weblogic managed server

- Install the plugin (from manage Jenkins > manage plugins > available > download & install)- restart the Jenkins for changes to take reflect

- go to manage Jenkins
- go to configure systems
- scroll down to WebLogic deployment plugins
- give details as below:                                  

        - additional path: /software/bea/jenkins/wlfullclient.jar (path to you wlfullclient.jar file, weblogic.jar is deprecated now and Oracle recommends using wlfullclient.jar from 10.3 onwards, this jar can be created from $WL_HOME/wl_server10.3/servers/lib/java -jar wljarbuilder.jar for more check Oracle Doc on this.

        - configuration file: /software/bea/jenkins/configuration.xml (path to your configuation.xml file)

        - apply/save

- sample configuration.xml file may look like -

modify the highlighted tags as per your local configuration

and run the Jenkins job, it will deploy the application on WebLogic managed server defined in your config file.

JENKINS: a continuous integration tool

Getting started with Jenkins is 3 step process:

- Install Jenkins
- Download the required plugins
- Configure the plugins & create a project

# Installing Jenkins in Ubuntu
$ wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key  | sudo apt-key add -
$ sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ \
  > /etc/apt/sources.list.d/jenkins.list'
$ sudo apt-get update
$ sudo apt-get install jenkins
  //this will automatically start Jenkins

# Installing Jenkins in AWS-EC2
$ sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat/jenkins.repo
$ sudo rpm --import https://pkg.jenkins.io/redhat/jenkins.io.key
$ sudo yum install -y jenkins
$ sudo yum install java -y
$ sudo service jenkins start
$ chkconfig jenkins on

# If your /etc/init.d/jenkins file fails to start jenkins then,
edit the /etc/default/jenkins to replace the HTTP_PORT=8080 to HTTP_PORT=8081
or
$ cd /var/share/jenkins/
$ java -jar jenkins.war // this will setup a new jenkins from begining

# I prefer to use below, this will start old jenkins without any delay
$ service jenkins restart

# Running docker container of Jenkins
$ docker pull punitporwal07/jenkins
$ docker run -d -p 8081:8080 -v jenkins-data:/software/jenkins punitporwal07/jenkins:tag
  to understand this command in brief check here: Docker

# Depending on configs to system, jenkins file structure may vary -
  /etc/sysconfig/jenkins
  /var/lib/jenkins/
  /etc/default/jenkins

Useful Plugins

you can push from manage plugins tab/ push from back-end into plugins dir

- Build pipeline: to chain multiple jobs
- Delivery pipeline: this will visualize deliver pipelines (upstream/downstream)
- Weblogic deployer: this is used to deploy a jar/war/ear to any weblogic target
- Deploy to container: to deploy war/ear to a tomcat/glassfish container
- Roles strategy: this plugin allows you to assign roles to the different user of jenkins

Automate deployment on Tomcat using Jenkins pipeline

(benefits.war as an example on tomcat 8.x for Linux)

- install Deploy to container plugin, restart Jenkins to reflect changes
- create a new project/workspace & select post-build action as- Deploy war/ear to a container
- add properties as below:-
   - war/ear files: **/*.war
   - context path: benefits.war (provided you need to push this war file into your workspace)
   - select container from the drop-down list: tomcat 8.x
   - Add credentials: tomcat/tomcat (provided you have added this user in conf/tomcat-user.xml with all the required roles)
   - Tomcat URL: http://localhost:8080/
- apply/save your project and build it to validate the result.

Automate deployment on Weblogic using Jenkins pipeline

(benefits.war as an example on Weblogic 10.3.6)

- install Weblogic deployer plugin, restart jenkins to reflect changes
- configure the plugin,
- create new project/workspace
- Add post-build action as- Deploy the artefact to any weblogic environment (if no configuration has been set, the plugin will display an error message, else it will open up a new window)
- add properties as below:-
   - Task Name: give any task name
   - Environment: from the drop-down list select your AdminServer ( provided you have created configuration.xml and added it to Weblogic deployer Plugin)
   - Name: The name used by the WebLogic server to display the deployed component
   - Base directory of deployment: give the path to your deployment.war or push it to your workspace and leave it blank
   - Built resource to deploy: give your deployment.war name
   - Targets: give target name
- Apply/save your project and build it to validate the result.

Possible failure of your Jenkins jobs
Problem - Jenkins: java.lang.OutOfMemoryError: Java heap space

Solution - Navigate to your Jenkins project, click configure.

Scroll down to the Build section of the page to the Build Step with your plugin title:

Click Advanced…

In the System/Java Options field, add the following parameter

JAVA_ARGS="-Xmx2048m -XX:MaxPermSize=512m"

This will assign 2Gi of memory to you build.

--