Share data between pipeline steps

How to cache folders between steps and builds

Codefresh creates a shared volume in each pipeline that is automatically shared with all freestyle steps.

This volume exists at /codefresh/volume by default. Simply copy files there to have them available to all Codefresh steps (as well as subsequent builds of the same pipeline).

NOTE
The Git clone step deletes any files not specified in .gitignore. To cache a folder that exists in your project directory (such as node_modules), you must also add it to .gitignore

Using the shared volume

You can see the example project at https://github.com/codefreshdemo/cf-example-shared-volumes-between-builds. The repository contains a simple application, a Dockerfile, and an example pipeline that saves/reads a dummy file to the Codefresh volume.

Here is the whole pipeline:

codefresh.yml

version: "1.0"
stages:
  - "clone"
  - "build"
  - "shared-volume"

steps:
  clone:
    title: "Cloning repository"
    type: "git-clone"
    repo: "codefreshdemo/cf-example-shared-volumes-between-builds"
    revision: "master"
    stage: "clone"

  build_image:
    title: "Building image"
    type: "build"
    image_name: "sample-app"
    working_directory: "${{clone}}"
    tag: "demo"
    dockerfile: "Dockerfile"
    stage: "build"
  
  copy_to_shared_volume:
    title: "Copy file to shared volume"
    type: "freestyle" 
    image: alpine:3.9 
    working_directory: "${{clone}}"
    commands:
      - ls -l /codefresh/volume/
      - cp ./artifact/artifact.example /codefresh/volume/artifact.example
    stage: "shared-volume"
    
  list_shared_volume:
    title: "List shared volume files"
    type: "freestyle" 
    image: alpine:3.9 
    working_directory: "${{clone}}"
    commands:
      - pwd
      - ls -l /codefresh/volume
    stage: "shared-volume"

This pipeline does the following:

1. Clones the source code through a Git clone step.
2. Builds a docker image through a build step.
3. Copies the file artifact.example to the volume through a freestyle step.
4. Reads the contents of the volume through a different freestyle step.

If you run the pipeline, you will see the file contents in the fourth step:

If you run the pipeline a second time, you will see the dummy file in all steps, as the volume is automatically cached for subsequent builds as well.

Caching build dependencies and Docker layers

Read more about caching build dependencies in caching in pipelines, and in this blog post.

Related articles

CI/CD pipeline examples
How Codefresh pipelines work
Codefresh YAML for pipeline definitions