Automate the CI/CD Automation with Jenkins (Part 2)

If you have not read the first part of this article you can read from here

Ok now with part one read, I believe you have a good understanding around the concepts and Jenkins Job DSL plugin installed on your Jenkins instance.
Lets talk about the steps on how to use the plugin to automate the Jenkins jobs and pipelines.

In this article we are going to talk about the “Seed Jobs”.
Seed job is the top level Jenkins job that will generate the usual Jenkins CI/CD jobs. In other words, Seed job will run Groovy based Job DSL scripts and will generate all the CI/CD pipelines and jobs that you need to have for your projects. In Job DSL scripts you need to specify how the CI/CD jobs will look like.

We are going to generate a Jenkins job that when run builds a sample Java Springboot app and upload the artifact to a Nexus artifact repository (https://www.sonatype.com/nexus-repository-oss). Job configuration will look like this. You can create this job by hand using the UI, but thats not the way to do it. The correct way to create this job is to write this job as a Job DSL script that will generate the below job when run. i.e. you run the job DSL script and then you’ll get the below Jenkins job automatically generated.

image 1

Important points in above sample Jenkins Job.

Think this is one of our CI/CD jobs for one of our projects, and we don’t want to create this job by hand manually. We need to use Job DSL tool to automate the Jenkins job creation.

Lets create the Seed Project fist.

Create a new Freestyle project, and lets call it “Seed Project”

image 2

Open the new project and go to “Configure” section from the menu.

Just scroll down to the “Build” section and add a “Build Step” called “Process Job DSLs”

image 3

Now select the “Use the provided DSL script” option and type the DSL commands as shown below in the large textbox that appear. Don’t worry about the commands and we will talk about them in a minute.

image 4

Save the project, thats it nothing more. WOW, its that simple to generate a seed job.

Below is the DSL Groovy commands used in the above image.

// Jenkins Job DSL definition file for generating a Job for 
// https://github.com/anuradhaneo/springboot-api-demo

String basePath = 'springboot-api-demo'
String gitRepository = 'anuradhaneo/springboot-api-demo'
String buildBranch = '*/master'
String projectName = 'Build Project'
String projectDisplayName = 'Springboot API Sample'
String projectDescription = 'This example shows basic folder/job creation'
String credentialIDGithub = 'github-anuradhaneo'
String artifactGroupID = 'org.spring.boot.sample'
String artifactID = 'SpringBootRestApiExample'
final String HTTP = 'http'
final String HTTPS = 'https'
// root folder creation
folder(basePath) {
displayName(projectDisplayName)
description(projectDescription)
}
// job definition
mavenJob(basePath + '/' + projectName) {
description('Build the Java roject: ' + gitRepository)
scm {
git {
branch(buildBranch)
remote {
github (gitRepository, HTTPS, 'github.com')
credentials(credentialIDGithub)
}
}
}
triggers {
scm('@daily')
}
wrappers {
goals('clean install')
}
}

If you are familiar with Java or Groovy the line 1 to 14 are nothing but variables with values.

Line 16 to 19 defines a folder in Jenkins to organize the jobs and we will be placing our Jenkins job (when generated after running Seed job) inside a folder named “Springboot API Sample” which is defined with the line number 17.

folder(basePath) {
displayName(projectDisplayName)
description(projectDescription)
}

Line 22 to 39 defines the Maven build steps

Line 22 specifies the name of the project,
note in below we also specify the full path to the project including the parent folder we created earlier

// set the project name
mavenJob(basePath + '/' + projectName) {

Line 23 sets the description

description('Build the Java project: ' + gitRepository)

line 26 sets the branch in the github project

branch(buildBranch)

Line 28 specifies the github.com repository name

github (gitRepository, HTTPS, 'github.com')

Line 33 specifies the build frequency

scm('@daily')

Line 37 specifies the Maven goals to be executed

goals('clean install')

Use the Job DSL API to learn more about the commands

https://jenkinsci.github.io/job-dsl-plugin/

Now if you run this Seed Project it will generate a project with configuration as below

image 5

2. Actual job inside the above shown folder

image 6

3. with job configuration as below

image 7

So as you see it is very simple to create a Seed Job with a simple DSL script to generate a single Jenkins Job.

Now lets look at a extended version of the above example code

// Jenkins Job DSL definition file for generating a Job for 
// https://github.com/anuradhaneo/springboot-api-demo

String basePath = 'springboot-api-demo'
String gitRepository = 'anuradhaneo/springboot-api-demo'
String buildBranch = '*/master'
String projectName = 'Build Project'
String projectDisplayName = 'Springboot API Sample'
String projectDescription = 'This example shows basic folder/job creation'
String credentialIDGithub = 'github-anuradhaneo'
String artifactGroupID = 'org.spring.boot.sample'
String artifactID = 'SpringBootRestApiExample'
final String STATUS_SUCCESS = 'SUCCESS'
final String HTTP = 'http'
final String HTTPS = 'https'
String nexusOSSURI = 'localhost:8081'
String nexusOSSVersion = 'nexus3'
String nexusOSSRepositoryName = 'SpringBootRestApiProject'
String nexusCredentialsID = 'NexusRepoCredentials'
// root folder creation
folder(basePath) {
displayName(projectDisplayName)
description(projectDescription)
}
// job definition
mavenJob(basePath + '/' + projectName) {
description('Build the Java roject: ' + gitRepository)
scm {
git {
branch(buildBranch)
remote {
github (gitRepository, HTTPS, 'github.com')
credentials(credentialIDGithub)
}
}
}
triggers {
scm('@daily')
}
wrappers {
goals('clean install')
}
postBuildSteps(STATUS_SUCCESS) {
// upload the generated artifact to Nexus OSS repo
nexusArtifactUploader {
nexusVersion(nexusOSSVersion)
protocol(HTTP)
nexusUrl(nexusOSSURI)
groupId(artifactGroupID)
version('${POM_VERSION}')
repository(nexusOSSRepositoryName)
credentialsId(nexusCredentialsID)
artifact {
artifactId(artifactID)
type('jar')
classifier('')
file('target/' + artifactID + '-${POM_VERSION}.jar')
}
}
}
}

We have added a new section to the code

postBuildSteps(STATUS_SUCCESS) { .... }

This adds a post-build step which is similar to the below shown UI action.

image 7

Lets explore the code within the “postBuildSteps” tag.

nexusArtifactUploader { ... }

Above tag specifies the scope for the required parameters for artifact uploading to Nexus artifact repo with as shown below. I have a locally running Nexus OSS server exposed via port 8081.

-- sets the version of the Nexus OSS software
nexusVersion(nexusOSSVersion)
-- set the access protocol
protocol(HTTP)
-- set Nexus server URI
nexusUrl(nexusOSSURI)
-- set uploading artifact group ID
groupId(artifactGroupID)
-- set artifact version
version('${POM_VERSION}')
-- set the repository name within Nexus
repository(nexusOSSRepositoryName)
-- credentials configuration ID
credentialsId(nexusCredentialsID)
-- actual upload artifact file details
artifact {
artifactId(artifactID)
type('jar')
classifier('')
file('target/' + artifactID + '-${POM_VERSION}.jar')
}

See https://github.com/jenkinsci/nexus-artifact-uploader-plugin for more details if you are interested.

So if you run this Seed job the result will be something very similar to the Jenkins job I have shown in “image 1”.

Now lets see how we get rid of defining the Job DSL scripts inline in the build step of the seed project and get that code from a SCM repository such as github.

I have created a github repository with the full working code in below URL https://github.com/anuradhaneo/jenkins-job-automation-sample

We are going to use this git repository as the source for the DSL code in out seed project.

Step1.
Open the seed project configuration again and set the project as a GIT based one as shown below. Fill the values as shown below.

Note you need to manually create the credential sets in Jenkins in all cases since this is something that shouldn’t be automated.

image 8

Step2.
Select the “Look on the Filesystem” option from the “Process Job DSLs” build step options. And type the path to the script file from the git checkout folder.

Note: we can use a relative path here

In our example its “job1-spring-boot-api-sample/main.groovy”

image 9

So when you run the Seed project main.groovy file will be executed.

Save and exit the Seed project and you will see the project in “image 1".

So thats all and you can see how easy it it to create Seed projects and generate your whole Jenkins CI/CD pipelines.

All you need to master here is how to read and apply the properties defined in Job DSL API documentation.

<Happy Coding>

an enterprise architect, technology enthusiast, dog lover , music maniac, a husband & a father ! 🐶