How To Use HashiCorp Vault In Gitlab-CI/CD

Hello, Guys Today I implemented, how to use HashiCorp vault for the Gitlab CI-Variables. So, I think might be I Need to Write about it how to configure it and use it.

Gitlab just releases this feature of using vault for the Gitlab-CI variables in the Gitlab 13.4 version, For more information, you can check out this link.
https://about.gitlab.com/releases/2020/09/22/gitlab-13-4-released/#use-hashicorp-vault-secrets-in-ci-jobs

Requirements:

1. Gitlab Server or Account on Gitlab
2. Vault Server

Step 1: Go to the Vault server and type the command to enable the auth method for JWT.

vault auth enable jwt

Step 2: Then the command to write to the auth method

vault write auth/jwt/config jwks_url="https://gitlab.example.com/-/jwks" bound_issuer="gitlab.example.com"

in this gitlab.example.com is the name of the GitLab-server.

Step 3: Creating the variable need to used in Gitlab-CI

vault kv put secret/myproject/staging/db password=staging-ka-password
vault kv put secret/myproject/production/db password=production-ka-password

Step 4: Verify the Variables once it is setup correctly by using the kv get command.

vault kv get --field=password secret/myproject/staging/db
vault kv get --field=password secret/myproject/production/db

Step 5: Next step is to create vault policies to access the key-value(variable)

1. For Staging

2. For Production

Step 6: Create a Gitlab Project and go to the home page of the repository to find out the Gitlab-Project ID, you need to enter this id in the Next Step.

Step7: Create a vault role to restrict access to a particular project and namespace.

For Staging:

For Production

Step 8: Now to Use these variables in the Gitlab-CI, Configure the Environmental variables in Gitlab->Project Repository->Setting->CI/CD->Variables.

Need to Setup 3 Environment Variables:

1. VAULT_AUTH_ROLE
2. VAULT_AUTH_PATH
3. VAULT_SERVER_URL

My Variables:

1. VAULT_AUTH_PATH: jwt
2. VAULT_AUTH_ROLE: myproject-staging-123
3. VAULT_SERVER_URL: http://13.127.96.122:8200

Step 9: To use these Variables in CI/CD Pipeline, type the secrets block in the .gitlab-ci.yml file.

Step 10: Run the Pipeline and Check the Variables value, it got fetched from the HashiCorp Vault and store in the temp/location as you can see in the output. But is stored in the same variable, You can checked it by ssh in the docker container and echo the value of the DATABASE_PASSWORD variable.

Pipeline Output

Inside Docker

Thank you Guys for Reading, Share it If you think this blog helps you.

In Case of Any Error or Issue, you can connect me on LinkedIn:

https://www.linkedin.com/in/tarun-mittal-1507/

Findings:

1. If you get permission denied error, then kindly check the policies and roles which we have configured above. You can verify the policy by creating a user in the vault and attaching the same policy and trying to access the secret.
2. If you are getting <nil> while echoing the variable values, like below:

Then I read it somewhere that it is because GitLab-CI requires or supports only kv( version-2 ) secret engine of the vault. Try to update from kv-v1 to kv-v2 to solve this issue.

Read more on this url:- https://www.vaultproject.io/docs/secrets/kv/kv-v2#upgrading-from-version-1

Command is:

vault kv enable-versioning <path-where-secret-is-enabled>

Useful Links:

jwt auth method about the user_claim parameter when create a role · Issue #5114 · hashicorp/vault

JWT/OIDC - Auth Methods | Vault by HashiCorp

Using external secrets in CI

Authenticating and Reading Secrets With Hashicorp Vault