Skip to main content

Add Semgrep manually to CI providers

Your deployment journey

This guide walks you through creating a Semgrep job for CI providers for which Semgrep AppSec Platform has no explicit guidance. Without explicit guidance, you must manually make a CI configuration file yourself.

Check Add Semgrep to CI first to ensure that this guide applies to your CI provider.

Skip this guide if you have already configured a CI job.

The steps provided here are known to work with the following CI providers:

  • AppVeyor
  • Bamboo
  • Bitrise
  • Buildbot
  • Codeship
  • Codefresh
  • Drone CI
  • Nomad
  • TeamCity CI
  • Travis CI

General steps​

The following steps provide an overview of the process. View the succeeding sections for detailed instructions.

  1. Create a SEMGREP_APP_TOKEN.
  2. Add this token as a credential, secret, or token into your CI provider.
  3. Create a CI job that runs Semgrep; this step is typically achieved by committing a CI configuration file. The syntax of the configuration file depends on your CI provider..
  4. The CI job can automatically start to run depending on your configuration. If the job does not start, run the job through the CI provider's interface or by committing code.
  5. Semgrep detects the SEMGREP_APP_TOKEN, sends it to Semgrep AppSec Platform for verification, and if verified, findings are sent to Semgrep AppSec Platform.
  6. Define additional environment variables to enable other Semgrep AppSec Platform features. This is done last because it is easier to troubleshoot modifications to jobs after ensuring that the base CI job runs correctly.

The next sections go over these steps in detail.

Create a SEMGREP_APP_TOKEN​

To create a SEMGREP_APP_TOKEN, follow these steps:

  1. Sign in to Semgrep AppSec Platform.
  2. Click Settings > Tokens.
  3. Click Create new token.
  4. Copy the name and value, then click Save.
  5. Store the token value into your CI provider. Tokens can also be referred to as secrets, credentials, or secure variables. The steps to do this vary depending on your CI provider.

Create a Semgrep CI job​

  1. Add Semgrep to your CI pipeline. Do either of the following:
    1. Reference or add the Semgrep Docker image. This is the recommended method.
    2. Add pip install semgrep into your configuration file as a step or command, depending on your CI provider's syntax.
  2. Add semgrep ci as a step or command.
  3. Set the SEMGREP_APP_TOKEN environment variable within your configuration file.

The following example is a bitbucket-pipelines.yml file that adds Semgrep through the Docker image:

Add Semgrep through the Docker image.
image: atlassian/default-image:latest

pipelines:
  default:
    - parallel:
      - step:
        name: 'Run Semgrep scan with current branch'
        deployment: dev
        # Reference the Semgrep Docker image:
        image: semgrep/semgrep
        script:
        # You need to set the token as an environment variable
        # (see Create a `SEMGREP_APP_TOKEN` section).
          - export $SEMGREP_APP_TOKEN
          # Run semgrep ci:
          - semgrep ci

The next example is a Jenkinsfile configuration that adds Semgrep by installing it:

Add Semgrep by installing it.
pipeline {
  agent any
  stages {
    stage('Semgrep-Scan') {
        environment {
          // You need to set the token as an environment variable
          // (see Create a `SEMGREP_APP_TOKEN` section).
          SEMGREP_APP_TOKEN = credentials('SEMGREP_APP_TOKEN')
        }
      steps {
        // Install and run Semgrep:
        sh 'pip3 install semgrep'
        sh 'semgrep ci'
      }
    }
  }
}

Run the job​

Depending on your CI provider and configuration, the job runs automatically. Otherwise, trigger the job by committing code or opening a PR or MR.

Verify the connection​

To verify that your Semgrep CI job is connected to Semgrep AppSec Platform:

  1. Go to your Semgrep AppSec Platform Projects page.
  2. Verify that your repository is listed on the Projects page and that Semgrep AppSec Platform is running a scan.

Troubleshoot your CI job​

Semgrep attempts to automatically detect certain CI values, such as your repository's name and URL. These values are used to provide context to findings in Semgrep AppSec Platform and hyperlinks to the code that generated the finding.

Refer to the following table for common issues and the corresponding environment variables you can set to fix them:

IssueEnvironment variable to setAffected CI providers
Can't establish a connection to Semgrep AppSec Platform.SEMGREP_APP_TOKENMust be set for all CI providers.
Semgrep doesn't scan your PRs or MRs.SEMGREP_BASELINE_REFRequired for CI providers except GitHub Actions or GitLab CI/CD.
Can't click hyperlinks to your repository from Semgrep AppSec Platform, nor can Semgrep AppSec Platform create PR or MR comments.SEMGREP_REPO_NAMESet these environment variables as needed to troubleshoot broken links for any CI provider except GitHub Actions and GitLab CI/CD.
SEMGREP_REPO_URL
SEMGREP_BRANCH
SEMGREP_JOB_URL
SEMGREP_COMMIT
SEMGREP_PR_IDRequired to enable hyperlinks for Azure Pipelines.

Data collected by Semgrep AppSec Platform​

When running in CI, Semgrep runs fully in the CI build environment. Unless you have explicitly granted code access to Semgrep, your code is not sent anywhere.

  • Semgrep AppSec Platform collects findings data, which includes the line number of the code match, but not the code. It is hashed using a one-way hashing function.
  • Findings data is used to generate line-specific hyperlinks to your source code management system and support other Semgrep functions.

Next steps​

You've set up Semgrep to scan in your repository and send findings after each scan. Your core deployment is almost complete.

Remaining steps include: