Jenkins: Obtaining and displaying credentials

Jenkins is a solid CI/CD platform which has proven itself over the years. Many organizations use it to build, test and deploy their applications. In Jenkins it is possible to define credentials or to use an external credential store. You can then use these inside your pipelines and jobs. Direct access to credentials can be limited. Even with limited access, there are still various ways in which you can extract credentials. 

This blog post illustrates how you can display credentials in log files as base64 encoded strings so they are not masked and you can easily copy / paste / base64 decode them to obtain and (ab)use them. The method described is not specific to Jenkins but can also be used similarly in various other CI/CD platforms on-premises and in the cloud (such as GitHub Actions and Bamboo). 

Note: this is not meant as an encouragement to break rules or laws. Often legislation does not allow you to try and access systems you are not officially authorized to. It is meant to create awareness and to allow you to think about if and how you might want to prevent this in your own CI/CD platform.

Pipeline as code

You often see pipeline definitions which are shipped together with the code which needs to be build, tested, deployed (think Jenkinsfile or GitHub Actions). These pipelines contain commands which are allowed to use credentials. These credentials are often available as environment variables so they can easily be used for example to execute deployments. Since the pipeline is bundled with the application, you can perform version control on it and easily put the responsibility for maintaining the pipeline at the team which builds and maintains the code. Credentials are often defined as secrets or credential within the platform. See for example below for the definition of a Jenkins secret.

Creating a new pipeline

When I’m able to create a new Jenkins pipeline or update an existing pipeline I can obtain credentials defined in the CI/CD platform.

I can define a pipeline outside of version control as part of the job definition. I do require access to an agent to execute the pipeline though.

When I’m required to change the pipeline code inside a version control system, my actions will be far more easy to trace and I might not be so eager to try this. Creating and removing a job after having used it to obtain credentials, often leaves few traces. 

Access to Jenkins credentials

Limit UI and API options

Within Jenkins, you can use matrix based security to limit access to credentials

This will limit your options in the Jenkins UI and via the Jenkins API

A limited user cannot create or edit credentials.

Mask credentials in logging

Printing credentials in the Jenkins logging by using a plain echo command, will give you a warning and mask the credential. Even when wrapping the displaying of the credential in a more complicated command, Jenkins still masks it (although without warning). When however base64 encoding the credentials, Jenkins doesn’t recognize them anymore and displays the base64 encoded credentials which you can easily decode using unsafe websites such as this one. Thank you very much!

Protect your credentials!

Below are some suggestions which might help you prevent these kind of issues. Mind that they are partially at odds with the DevOps mindset and the pipeline as code principle! They might also limit your options in putting the responsibility for the build/deploy process at your developers. Do not apply without carefully weighing the pro’s and con’s.

• Use the principle of least privilege when configuring authorizations in your CI/CD platform. For example only allow certain Jenkins pipelines to use a specific build agent and disallow changing existing jobs or creating new ones which can also use an existing agent.
• Separate build and deployment jobs and responsibilities. Maybe even use different tools or environments for them. This way, when you have access to a build pipeline during development, you can prevent someone to easily extract credentials which are used for production.
• Explicitly request credentials upon deployment as manual input. This does not prevent malicious pipeline code to be executed and for example display the credentials in log files (or worse, send them to a remote endpoint), but that pipeline is executed under your watch, which makes it less likely someone else will be able to listen in.
• Limit access to pipeline code / jobs / flow definitions (or whatever your CI/CD platform calls them).
• Try and prevent using environment variables for holding credentials (when your CI/CD platform and application deployment tooling supports this).