Technical Thursday – CI/CD with Git, Azure and Jenkins

“How to prepare our CI/CD process?” – This could be the subtitle of this article. Why? Because I will show you how you can start to build a fully automated CI/CD process.

What is CI/CD? You can read it on Wikipedia. Nevertheless this is a very important and useful thing nowadays when we work in a DevOps model.

Scenario

In my scenario I would like to copy files from Git to Azure with Jenkins when a commit/push happens to my GitLab. As you can see this is quite complex therefore it’s a good practice example.

Important to know, purpose of this post to show you how can you integrate within some minutes your GitLab and your Jenkins. (So we will use our personal git account for configure connection and we will create the connection between Jenkins and Git over https – and not SSH) This means due to testing purpose we won’t create a very secure integration. 😉

Integration

Assumptions and prerequisites

• You have a Jenkins  environment for automation which is a general used tool.  Installation steps are here.
• You have a configured GitLab environment
• You configure your environment only for testing purpose. In other case you have to use different parameters or ssh keys during configuration from security point of view.
• You store your application in Git.
• You configured the pipeline solution in our Git.
• You have an Azure subscription with owner privileges.

Step 1: Configure Jenkins

Here you have to install some plugins to Jenkins.

1. Login to your Jenkins server
2. Navigate to “Manage Jenkins > Manage Plugins”
   
3. From the “Available” tab, find and select the following plugins:
   1. GitLab Plugin
   2. Azure CLI Plugin
   3. Azure Credentials
4. Click the “Download now and install after restart” button to download it.
5. Once the plugin has been downloaded, click the “Restart Jenkins…” checkbox and wait for Jenkins to restart.
6. When Jenkins restarted navigate to “Manage Jenkins > Configure System”
   
7. Find the “Git plugin” section and configure Git basic values
   
8. Save configuration

Step 2: Service Principal in Azure

To be able to upload our files to Azure we have to create a Service Principal which has enough privileges to make it.

1. Login to a computer where the Azure-Cli 2.0 is installed
2. Login to the subscription where you would like to create the Service Principal

   # check relevant cloud infra where you want to login (i.e. AzureGermanCloud, AzureCloud, AzureChinaCloud, ...)
   az cloud set --name <name of Cloud>
   
   # Please login to your azure account
   az login -u <useraccount>
   
   # Select your subscription
   az account set --subscription <subscription ID>

3. Create a Service Principal with the following command

   az ad sp create-for-rbac --name <Service Principal name in Azure. eg. JenkinsGitAzure-the1bithu> --query '{"client_id": appId, "secret": password, "tenant": tenant}'

   

4. Copy these values to a safety place because we will use it in Jenkins!

Step 3: Credentials in Jenkins

Now set some credentials such as Git, Azure.

1. Navigate to “Credentials > System”
   
2. Choose the “Global credentials” domain
3. Click on “Add credentials” button
   
4. Create GitLab user credential
   1. Kind: Username with password
   2. Username: <your git username>
   3. Password: <your git password>
   4. ID: <An internal unique ID by which these credentials are identified from jobs and other configuration. Normally left blank, in which case an ID will be generated, which is fine for jobs created using visual forms. Useful to specify explicitly when using credentials from scripted configuration. >
   5. Then click OK button
      
5. Create Azure Service Principal credential (We need the data from Step 2)
   1. Kind: Microsoft Azure Service Principal
   2. Subscription ID: <Azure Storage account subscription ID>
   3. Client ID: <Azure Service Principal Client ID>
   4. Client Secret: <Azure Service Principal Client Secret>
   5. Tenant ID <Azure Service Principal Tenant ID>
   6. Azure Environment: <choose one according to your subscription location>
      
   7. ID: <An internal unique ID by which these credentials are identified from jobs and other configuration. Normally left blank, in which case an ID will be generated, which is fine for jobs created using visual forms. Useful to specify explicitly when using credentials from scripted configuration. >
   8. Then click OK button
6. Create Azure Storage Account SAS token credential (because our pipeline solution requires the SAS token we have to store somewhere on secure way)
   1. Kind: Secret text
   2. Secret: <paste here the SAS token for storage account. It begins with ‘?sv=’>
   3. ID: <eg. sasTokenAzure | An internal unique ID by which these credentials are identified from jobs and other configuration. Normally left blank, in which case an ID will be generated, which is fine for jobs created using visual forms. Useful to specify explicitly when using credentials from scripted configuration. >
   4. Then click OK button
7. Create Azure Storage Account Account Key credential (because our pipeline solution requires the Account Key we have to store somewhere on secure way)
   1. Kind: Secret text
   2. Secret: <paste here the Account Key for storage account.>
   3. ID: <eg. storageKeyAzure | An internal unique ID by which these credentials are identified from jobs and other configuration. Normally left blank, in which case an ID will be generated, which is fine for jobs created using visual forms. Useful to specify explicitly when using credentials from scripted configuration. >
   4. Then click OK button

Step 4: Create Jenkins project

1. Click on ‘New Item’
   
2. Type a name to “Enter an item name” field and choose “Pipeline project” then click on OK button
   
3. Build Triggers
   1. Tick “Build when a change is pushed to GitLab. GitLab webhook URL:”
      
   2. Click on “Advanced” button
   3. Choose the “Filter branches by name” and write in the include filed your branch name
      If you receive an error please ignore because it will be fixed when you will integrate your project with Git. (in Step 5)
   4. Then click on “Generate” button to genetate token for this project.
4. Pipeline
   1. Select “Piepeline script from SCM” at definition
   2. SCM: Git
   3. Repositories
      1. Paste the clonable url into “Repository URL”. (eg. https://gitlab.com/*****/*****.git)
      2. Credentials. Choose the credential which was created in Step 3.4.
   4. Branches to build. Put here your baranch instead of master. (eg. */master)
   5. Script Path (where the Jenkins file is stored in Git): pipeline/Jenkinsfile
5. Click Save
6. Click “Build Now” to test it from Jenkins
   
7. Check Console output
   

Step 5: Integrate GitLab with Jenkins project

1. Login to GitLab
2. Step into your project
3. Navigate to “Settings > Integrations”
   
4. Paste the Jenkins project URL (Step 4.3.1)  and Token (Step 4.3.4) to the first two fields
   
5. Choose the required Triggers
6. Uncheck the “Enable SSL verification” (if you use self-signed certificate on Jenkins)
   
7. Click “Add webhook” button
8. Scroll down and find your newly created webhook at middle of screen
   
9. Click the “Push events” under the “Test” button dropdown menu.
   
10. If you receive a HTTP 200 message with blue background the integration was success
    

Step 6: Test integration

1. Modify your project and commit that to this branch
   
2. Open Jenkins and check the project
   1. Status after start by Git
   2. Changes where you can see the commit message
      
   3. Check Console
      

Awesome…As you can see it works. 🙂

Please kindly notes this is a very basic implementation. If you would like to use it in production you have to configure impersonated accounts for git connections and you have to configure the pipeline solution according to your storage account related data. Additionally the SSH based integration could be better later.