Skip to main content

Running Semgrep using templates in Azure Pipelines

Motivation

Complex CI configurations housed in large YAML files take a lot of work to maintain and modify. Azure templates extract chunks of logic from larger configurations and encapsulate them in external template files. The template can then be referenced in multiple configurations, keeping pipelines more readable and maintainable.

This guide explains how to:

  • Create template files to run various Semgrep commands.
  • Include or call templates in your Azure Pipeline.

You can then reuse the template files in as many pipelines as you need.

Defining Semgrep commands in a template file

To add Semgrep commands in a YAML template file:

  1. Create a templates folder in the repository you want to run Semgrep in.

  2. Commit the following templates:

    Example YAML template file for a Semgrep full scan:

    steps:
    - script: |
    echo "Semgrep full scan"
    python -m pip install --upgrade pip
    pip install semgrep
    semgrep ci

    Example YAML template file for a Semgrep pull request scan:

    steps:
    - checkout: self
    clean: true
    fetchDepth: 10000
    persistCredentials: true
    - script: |
    echo "Pull Request Scan from branch: $(Build.SourceBranchName)"
    git fetch origin master:origin/master
    python -m pip install --upgrade pip
    pip install semgrep
    semgrep ci
    env:
    SEMGREP_PR_ID: $(System.PullRequest.PullRequestNumber)
    SEMGREP_BASELINE_REF: 'origin/master'
note

You must define separate templates for full scans and diff-aware scans. This is because there are different environment variables used in the template for diff-aware scans, such as SEMGREP_PR_ID and SEMGREP_BASELINE_REF.

Referencing templates in an Azure Pipeline

With the templates defined, reference them in other Azure Pipelines like this:

pool:
vmImage: ubuntu-latest

variables:
- group: Semgrep_Variables

jobs:
- job: Semgrep_Full_Scan
condition: eq(variables['Build.SourceBranchName'], 'master')
steps:
- template: templates/full_scan_semgrep.yml

- job: Semgrep_PR_Scan
condition: ne(variables['Build.SourceBranchName'], 'master')
steps:
- template: templates/pr_scan_semgrep.yml
tip

You can even define your templates in a centralized repository and reference them in other repositories.

Conclusion

Using templates in Azure Pipelines is a good practice to simplify pipeline configuration files, improving both readability and maintainability. Pipeline templates can also speed up the Semgrep onboarding process for repositories by allowing you to reuse the same template in each repository.


Not finding what you need in this doc? Ask questions in our Community Slack group, or see Support for other ways to get help.