Introduction: From Generic to Application-Specific Detection
In our previous lesson, we learned how to use ESLint with security plugins to catch common vulnerabilities like Object Injection. These tools are fantastic for finding generic problems that apply to almost any JavaScript project. However, security issues are often specific to how your particular application is built. For example, a generic scanner might not know that your team decided every database query must use a specific wrapper function or that every API route requires a specific piece of middleware to be secure.
This is where Semgrep (Semantic Grep) shines. Unlike standard grep, which searches for simple strings of text, Semgrep understands the structure of your code. It allows you to write custom rules that act like unit tests for your security policies. In this lesson, we will move beyond enabling pre-made rules and start writing our own. We will build a custom ruleset to detect two specific issues in our Pastebin application: usage of unsafe Raw SQL queries and API routes that are missing Authentication Middleware.
Understanding Semgrep Rule Structure
Before we can catch bugs, we need to understand how to tell Semgrep what to look for. Semgrep rules are written in YAML files. This format is human-readable and uses indentation to organize information. A rule file isn't just a list of bad patterns; it's a structured document that defines the problem, the solution, and the severity.
Every Semgrep rule requires a few key fields to function correctly. The id is a unique name for your rule, which helps you track it later. The message is the text that will appear in the report when a vulnerability is found; this should explain what went wrong to the developer. The severity tells the system how serious the finding is. Semgrep supports two sets of severity levels. The newer, preferred levels are HIGH, MEDIUM, LOW, and INFO, where HIGH represents a critical issue, MEDIUM is a significant concern, LOW is a minor issue, and INFO is purely informational. The older levels ERROR, WARNING, and INFO are still accepted and map roughly to HIGH, MEDIUM, and LOW respectively. Finally, the languages field specifies which code files to scan, such as javascript or typescript.
Here is the skeleton of the rule file we will create, located at rules/raw-sql.yaml:
rules: - id: sequelize-raw-query message: "Raw SQL query detected - use parameterized queries" severity: ERROR languages: [typescript, javascript] # Patterns will go here next
In the structure above, we are preparing to detect raw queries in a Sequelize application. We chose ERROR as the severity because raw SQL queries are a high-risk vector for SQL Injection attacks. In the CodeSignal environment, Semgrep is pre-installed, but typically you would save this file in a .semgrep folder or a dedicated rules directory in your project root.
Pattern Matching Fundamentals
Building a Raw SQL Detection Rule
Using Negative Patterns for Exceptions
Detecting Missing Authentication Middleware
Adding Metadata for Context
When a developer sees a security warning, their first question is often "So what?" or "Is this actually a bug?". To help them answer this, Semgrep allows you to attach a metadata block to your rules. This field doesn't change how the scan works, but it provides crucial context to the human reading the report.
In the metadata section, you can include links to documentation, references to OWASP categories (like "OWASP A01: Broken Access Control"), or simple notes explaining the business logic behind the rule. For our authentication rule, we want to acknowledge that not every finding is a bug — some routes should be public (like a login page).
Here is how we add context to our auth rule:
metadata: note: "May require manual review - some routes are intentionally public" owasp: "A01:2021-Broken Access Control" category: "security"
By adding the note "May require manual review," we set the right expectation. We aren't telling the developer they definitively broke the code; we are flagging an anomaly that requires human judgment. This builds trust between the security team and the development team, as it shows the tool is being "smart" about its limitations.
Running the Scans
With the rules written, the final step is actually executing the scanner. Semgrep is run from the command line and accepts a --config flag pointing to your rules file or directory, followed by the path to the code you want to scan.
To scan your application using a single rule file, run:
semgrep --config rules/raw-sql.yaml src/
To run all rules in a directory at once, point --config at the folder:
semgrep --config rules/ src/
Semgrep will print each match directly to the terminal, showing the file path, line number, rule ID, and the message you wrote. If you want a machine-readable output — for example, to pipe results into another tool or a CI pipeline — you can request JSON format:
When all rules pass without findings, Semgrep exits with code 0. When findings are present, it exits with a non-zero code, which causes most CI systems to fail the build automatically. This makes it straightforward to enforce your rules as a quality gate on every pull request.
Summary and Practice Preparation
In this lesson, we transitioned from using generic security scanners to writing custom logic with Semgrep. We learned how to use YAML to define rules and explored the power of Metavariables ($VAR) and the Ellipsis (...) operator to match complex code structures. We built a rule to detect unsafe raw SQL queries using pattern-either and a more advanced rule to detect missing authentication middleware using pattern-not to filter out secure routes.
We also discussed the importance of metadata to communicate with developers, ensuring they understand why a rule flagged their code. Finally, we covered how to run Semgrep from the command line against a target directory and how to export results for use in a CI pipeline.
Now, it is time to put this into practice. In the upcoming exercises, you will be given a vulnerable Pastebin application. You will write these rules from scratch, run the Semgrep scanner against the codebase, and verify that it correctly identifies the insecure routes while ignoring the safe ones. This hands-on experience allows you to start thinking like a security engineer, automating the detection of logic flaws before they ever reach production.
Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal
YAML
rules: - id: sequelize-raw-query message: "Raw SQL query detected - use parameterized queries" severity: ERROR languages: [typescript, javascript] # Patterns will go here next
The core power of Semgrep lies in its pattern-matching syntax. It essentially looks like the code you are trying to find, but with special placeholders called Metavariables and the Ellipsis operator. These tools allow you to write flexible rules that can match many different variations of code without needing complex Regular Expressions.
A Metavariable is defined by a dollar sign followed by uppercase letters, like $SQL or $VAR. It acts as a wildcard that matches any expression — a variable name, a string, or even a function call. It captures that piece of code so you can refer to it later. The Ellipsis operator (...) is even more powerful. It matches zero or more of something. In a function call, func(...) will match func(), func(1), and func(a, b, c).
Let's look at a simple example. Suppose we want to find every time a developer uses console.log to print a variable.
YAML
patterns: - pattern: console.log($MSG)
This pattern is very strict. It matches console.log("Hello") because $MSG captures the string "Hello". However, it will not match console.log("Error:", error) because that function call has two arguments, and our pattern only asked for one. To catch calls with any number of arguments, we would use console.log(...). This combination of metavariables and ellipses makes writing rules intuitive because you are essentially writing "example code" that Semgrep uses as a template.
Now, let's apply this to our application. We use the Sequelize library for our database. While Sequelize usually handles security for us, it allows developers to run raw SQL commands using sequelize.query(). This bypasses security protections and is dangerous if used incorrectly. We want to ban this pattern or flag it for review.
We will use the pattern-either operator. This allows us to provide a list of different patterns, and if any of them match, the rule triggers. This is useful because developers might write code in slightly different ways. For instance, they might call the method on the global sequelize object or a generic $DB variable.
Here is how we implement the detection logic in our YAML file:
In the code above, the first pattern matches explicitly named sequelize.query calls. The first argument is captured as $SQL, and the ... handles any options passed after the SQL string. The second pattern is broader; it matches a .query method called on any object ($DB). This ensures that even if the developer named their database variable dbConnection or sqlClient, we still catch the potentially dangerous raw query.
Sometimes, we need to be precise to avoid False Positives. A false positive occurs when the tool flags code that is actually safe. If we flag every route in our application, developers will start ignoring the tool. We often want to find code that matches a pattern unless it meets certain safety criteria. This is where the pattern-not operator becomes essential.
The pattern-not operator allows us to define exceptions. Semgrep will first find everything that matches your main pattern, and then it will remove any matches that also fit the pattern-not criteria. This allows for logic like "Find all cookie configurations, EXCEPT those that already have the secure: true flag."
Imagine we are looking for hardcoded secrets. We might search for variable names containing PASSWORD. However, we don't want to flag a variable named EXAMPLE_PASSWORD used in a test file. We would write a positive pattern to find pattern: $X = "..." where $X includes PASSWORD, and a negative pattern pattern-not: $X = "example". This filtering capability allows us to focus our security reviews on code that is truly risky.
Let's use negative patterns to solve a complex logical problem: ensuring all our API routes are authenticated. In Express.js, we define routes like router.get('/path', handler). To secure a route, we usually add a middleware function, like router.get('/path', authenticateToken, handler). We want to find routes that forgot to include this middleware.
We will construct a rule that looks for a route definition but filters out the ones that are already secure. We also need to account for routes that are intentionally public, which might use a publicRoute marker or simply be excluded manually.
Let's break this down. The first line, pattern, finds all route handlers. It matches router.get, router.post, etc., by using the $METHOD metavariable. It matches the path in $PATH and the handler function.
The next two lines are our filters. pattern-not tells Semgrep: "Ignore the match if you see authenticateToken being passed as an argument." We use ... after authenticateToken because the route handler function comes after the middleware. If a developer writes router.get('/profile', authenticateToken, (req, res) => ... ), the first pattern finds it, but the pattern-not sees the middleware matches, so it discards the finding. The result? We are left with a list of routes that have no security middleware at all.
YAML
metadata: note: "May require manual review - some routes are intentionally public" owasp: "A01:2021-Broken Access Control" category: "security"
