Skip to main content

Pattern syntax (experimental)​

Patterns are the expressions Semgrep uses to match code when it scans for vulnerabilities. This article describes the new syntax for Semgrep pattern operators. See Pattern syntax for information on the existing pattern syntax.

There is often a one-to-one translation from the existing syntax to the experimental syntax. These changes are marked with . However, some changes are quite different. These changes are marked with

warning
  • These patterns are experimental and subject to change.
  • You can't mix and match existing pattern syntax with the experimental syntax.

pattern​

The pattern operator looks for code matching its expression in the existing syntax. However, pattern is no longer required when using the experimental syntax. For example, you can use ... wherever pattern: "...``` appears. For example, you can omit pattern` and write the following:

any:
  - "badthing1"
  - "badthing2"
  - "badthing3"

or, for multi-line patterns

any:
  - |
      manylines(
        badthinghere($A)
      )
  - |
      orshort()

You don't need double quotes for a single-line pattern when omitting the pattern key, but note that this can cause YAML parsing issues.

As an example, the following YAML parses:

any:
  - "def foo(): ..."

This, however, causes problems since : is also used to denote a YAML dictionary:

any:
  - def foo(): ...

any​

Replaces pattern-either. Matches any of the patterns specified.

any:
  - <pat1>
  - <pat2>
    ...
  - <patn>

all​

Replaces patterns. Matches all of the patterns specified.

all:
  - <pat1>
  - <pat2>
    ...
  - <patn>

inside​

Replaces pattern-inside. Match any of the sub-patterns inside of the primary pattern.

inside:
  any:
    - <pat1>
    - <pat2>

Alternatively:

any:
  - inside: <pat1>
  - inside: <pat2>

not​

Replaces pattern-not. Accepts any pattern and does not match on those patterns.

not:
  any:
    - <pat1>
    - <pat2>

Alternatively:

all:
  - not: <pat1>
  - not: <pat2>

regex​

Replaces pattern-regex Matches based on the regex provided.

regex: "(.*)"

Metavariables​

Metavariables are an abstraction to match code when you don't know the value or contents beforehand. They're similar to capture groups in regular expressions and can track values across a specific code scope. This includes variables, functions, arguments, classes, object methods, imports, exceptions, and more.

Metavariables begin with a $ and can only contain uppercase characters, _, or digits. Names like $x or $some_value are invalid. Examples of valid metavariables include $X, $WIDGET, or $USERS_2.

where​

Unlike Semgrep's existing pattern syntax, the following operators no longer occur under pattern or all:

  • metavariable-pattern
  • metavariable-regex
  • metavariable-comparison
  • metavariable-analysis
  • focus-metavariable

These operators must occur within a where clause.

A where clause is required in a pattern where you're using metavariable operators. It indicates that Semgrep should match based on the pattern if all the conditions are true.

As an example, take a look at the following example:

all:
  - inside: |
      def $FUNC(...):
        ...
  - |
      eval($X)
where:
  - <condition>

Because the where clause is on the same indentation level as all, Semgrep understands that everything under where must be paired with the entire all pattern. As such, the results of the ranges matched by the all pattern are modified by the where pattern, and the output includes some final set of ranges that are matched.

metavariable​

Replaces:

This operator looks inside the metavariable for a match.

...
where:
  - metavariable: $A
    regex: "(.*)
  - metavariable: $B
    patterns: |
      - "foo($C)"
  - metavariable: $D
    analyzer: entropy

comparison​

Replaces metavariable-comparison. Compares metavariables against a basic Python comparison expression.

...
where:
  - comparison: $A == $B

focus​

Replaces focus-metavariable. Puts focus on the code region matched by a single metavariable or a list of metavariables.

...
where:
  - focus: $A

as-metavariable​

as-metavariable is only available in the new syntax.

as-metavariable is a rule-writing feature that bridges the gap between metavariables and matches. Metavariables get access to things like metavariable-comparison, metavariable-regex, and metavariable-pattern, but you can’t use them on arbitrary matches. However, the as operator lets you embed arbitrary matches into metavariables, or bind arbitrary matches to a name.

The syntax is as follows:

all:
  - pattern: |
    @decorator
    def $FUNC(...):
      ...
  as: $DECORATED_FUNC

Since as appears in the same indentation as the pattern, Semgrep couples the two. This augmented pattern operator matches the enclosed pattern, but produces an environment where $DECORATED_FUNC is bound to the match it corresponds to. So for instance, the following rule:

match:
  pattern: |
    @decorator
    def $FUNC(...):
      ...
  as: $DECORATED_FUNC
fix: |
  @another_decorator
  $DECORATED_FUNC

Allows you to capture the decorated function. You can then use it in, for example, autofix's metavariable or metavariable ellipses interpolation, where you express something like "rewrite X, but with Y."

Syntax search mode​

New syntax search mode rules must be nested underneath a top-level match key. For example:

rules:
  - id: find-bad-stuff
    severity: ERROR
    languages: [python]
    message: |
      Don't put bad stuff!
    match:
      any:
        - |
            eval(input())
        - all:
            - inside: |
                def $FUNC(..., $X, ...):
                  ...
            - |
                eval($X)

Taint mode​

The new syntax supports taint mode, and such roles no longer require mode: taint in the rule. Instead, everything must be nested under a top-level taint key.

rules:
  - id: find-bad-stuff
    severity: ERROR
    languages: [python]
    message: |
      Don't put bad stuff!
    taint:
      sources:
        - input()
      sinks:
        - eval(...)
      propagators:
        - pattern: |
            $X = $Y
          from: $Y
          to: $X
      sanitizers:
        - magiccleanfunction(...)

Taint mode key names​

The key names for the new syntax taint rules are as follows:

  • pattern-sources --> sources
  • pattern-sinks --> sinks
  • pattern-propagators --> propagators
  • pattern-sanitizers --> sanitizers

Not finding what you need in this doc? Ask questions in our Community Slack group, or see Support for other ways to get help.