Improve this page on GitHub

Vault secrets in pipelines

Access and refer to Vault secrets in pipelines

Codefresh offers a Vault plugin you may use from the Step Marketplace. The plugin imports key-value pairs from the Vault server, and exports them into the pipeline.

Prerequisites

Example Java application

You can find the example project on GitHub.

The example application retrieves the system variable password from the pipeline, and uses it to authenticate to a Redis database, but you are free to use any type of database of your choosing.

        String password = System.getenv("password");
        String host = System.getProperty("server.host");

        RedisClient redisClient = new RedisClient(
                RedisURI.create("redis://" + password + "@" + host + ":6379"));
        RedisConnection<String, String> connection = redisClient.connect();

Also in the example application is a simple unit test that ensures we are able to read and write data to the database.

You cannot run the application locally, as it needs to run in the pipeline in order to use our environment variables to connect.

Create the pipeline

The following pipeline contains three steps: a vault step, a git-clone step, and a freestyle step.

Vault pipeline

Vault Pipeline

You should be able to copy and paste this YAML into the in-line editor in the Codefresh UI. It will automatically clone the project for you.

Note that you need to change the VAULT_ADDR, VAULT_AUTH, and VAULT_AUTH_TOKEN arguments within the first step to your respective values.

codefresh.yml

version: "1.0"
stages:
  - "vault"
  - "clone"
  - "package"
steps:
  vault:
    title: Importing vault values...
    stage: "vault"
    type: vault
    arguments:
      VAULT_ADDR: 'http://<YOUR_VAULT_SERVER_IP>:<PORT>'
      VAULT_PATH: 'path/to/secret'
      VAULT_AUTH_TOKEN: '<YOUR_VAULT_AUTH_TOKEN>'
  clone:
    title: Cloning main repository...
    type: git-clone
    arguments:
      repo: 'codefresh-contrib/vault-sample-app'
      git: github
    stage: clone
  package_jar:
    title: Packaging jar and running unit tests...
    stage: package
    working_directory: $
    arguments:
      image: maven:3.5.2-jdk-8-alpine
      commands:
      - mvn -Dmaven.repo.local=/codefresh/volume/m2_repository -Dserver.host=my-redis-db-host clean package
    services:
      composition:
        my-redis-db-host:
          image: 'redis:4-alpine'
          command: 'redis-server --requirepass $password'
          ports:
            - 6379

The pipeline does the following:

  1. Imports the key-value pairs from the Vault server and exports them into the pipeline under /meta/env_vars_to_export.
  2. Clones the main repository (note the special use of naming the step main_clone). This ensures that all subsequent commands are run inside the project that was checked out.
  3. The package_jar, does a few special things to take note of:
    • Spins up a Service Container running Redis on port 6379 , and sets the password to the database using our exported environment variable
    • Sets maven.repo.local to cache Maven dependencies into the local codefresh volume to speed up builds
    • Runs unit tests and packages the jar. Note how you can directly refer to the service container’s name (my-redis-db-host) when we set server.host

You will see that the variable was correctly exported to the pipeline by running a simple echo command:

Vault pipeline variable

Vault pipeline variable

CI pipeline examples
Steps in pipelines