Code Review & SAST – What’s the Difference

After all, what is the difference between Secure Code Review and SAST? 

You can also listen to this article:

This is the first of two articles, where the approach is on why these two tests have constantly debates and discussion on their differences.

In this first article, we will start by talking about automated analysis tools then we move on to Secure Code Review tools.

The objective here is to present in a broad way what each one of these methods are, and how they can help in the process to keep an application secure. 

Let’s get to it.

Code Review and SAST automated tools

The theme chosen for this article is one that most arouses discussions between security professionals and developers, simply because we have a very strong saving view of the tools on the market.

It is not uncommon for warm and exciting discussions about the real effectiveness of tools within the Software Development Life Cycle (SDLC).

But let’s focus on some data that can help you evaluate how tools should be added to your development pipeline.

It is worth saying that here, there is no indication that you should stop using the tools. Even we use them too. However, one point that should be noted with caution is that some professionals end up considering the tool as a solution to code validation problems.

That is, making use of tools as if they alone would solve code problems is what makes them misleading.

Let’s take a closer look at this next.

What SAST really is

A Static Application Security Testing – SAST or Code Review analysis tool is a tool that was created to analyze source code or even its compiled versions of code, looking for flaws in these codes that may compromise security.

With the evolution of these tools, some also migrated their attention to the moment of development, thus creating some additions to the IDEs. This eventually allowed a closer view of the developer.

This is undoubtedly one of the strengths of the tools: acting proactively in development, notifying the developer that there is something that can be improved in their code.

However, even with this aid, if the developer does not have the basic knowledge and understanding of safe development concepts, he will only be an observer of the tool.

Code analysis tools should be understood as supporting the development process and should be strategically integrated into the development process.

How does SAST work?

First, we do not want to explain in detail how these tools work: our goal is to create a knowledge base that can help you understand and choose your tools and when to use them.

SAST tools basically analyze application code in an inside out model, ie from the inside out, evaluating specific points, as we will detail below.

1. Flow Control Analysis 

Flow control analysis validates the order of operations to look for sequence patterns that can be detrimental to code, such as cookie transmission failures, uninitiated variables, or even race conditions.

2. Structural Analysis

Based on the language the developer is using, the tools analyze the structure of the code based on language best practices. This assessment takes into account safe development techniques and practices.

This analysis attempts to identify weaknesses in class creation, declarations, and use of variables.

3. Semantic Analysis

These tools are based on a semantic analysis framework: functionality similar to grep but in a very limited way.

SAST tools can parse code for specific contexts, such as .executeQuery ().

4. Setup Analysis

Configuration analysis checks application configuration files, such as XML or property files.

It ensures that the configuration is aligned with security practices and policies, such as setting a default error page for the web application.

Of course, the tool is not as simplistic as we can imagine from these little explanations, but, as we said, it is not our intention to go into more specific details.

However, understanding concepts like False Positive and False Negative is relevant – after all, they are the ones that will help in evaluating a tool. 

For those who wish to know more about these concepts, we suggest consulting this article by OWASP.

Code Review and SAST – How to Benchmark the tools

When we want to better understand how we can do or even analyze the benchmark between tools, OWASP has a very interesting document.

In your document called OWASP Benchmark Project, there is a breakdown of how we can better understand comparative values ​​- in this case, between tools.

The paper explains the four important points to note and offers us the ability to test our tools using Youden’s Index as the validation model for these tests.

Although they are strong arguments for conducting more real tests, without the already defined concept of “Silver Bullet” when we talk about tools, these test models cannot be considered definitive controls that will validate the acceptance or not of a tool.

So if you want to perform your own tests, just follow these steps:

1. Download Benchmark from GitHub
2. Run your tools on this benchmark
3. Run BenchmarkScore against reports made by this tool 

However, as we mentioned above, we should not use these results in isolation. When used well, they can greatly help in defining new tests and tool evaluations.

Still, it is possible that before conducting your tests, you may first want to generally evaluate your tooling options.

For these cases, OWASP also comes up with a suggestion on how this can be done, and in the Source code analysis document, you will find a list of possible assumptions to use.

Here we will not list them all, but we will point out some that we think may arouse the reader’s interest to read the document further.

1. Types of vulnerabilities

It may sound basic, but it is important to show that a good analysis tool should, at a minimum, be able to correctly identify the vulnerabilities present in OWASP TOP 10.

Another option would be to look for a tool that eliminates the most common coding errors – related in SANS TOP 25.

2. Effectiveness

See if the tool you are testing already has a score on the OWASP Benchmark. If not, consider evaluating this tool yourself.

One of the most important points for a tool is to have good effectiveness, where the relationship between False Positive and False Negative is favorable.

3. Automation

Make sure the tool you are evaluating allows it to be easily integrated into your development process.

Remember that these tools must be integrated into your development pipeline, and their execution should be automatic. This ensures the assessment on an ongoing basis.

Finally, these 3 points placed here should not be understood as the only points that should be noted: they are just part of an even larger set. Create your own and rate your tool within the scenario you built.

Code Review and SAST tools are part of the solution, but they need a human hand

It is not our place here to say that tools are not important within the development process, but we believe that they are not the ultimate solution to the code review process.

Of course, tools will always have their importance within the process, but it is up to us as experts to identify how these tools fit into the process.

And while tools are very important, nothing will replace the creativity of the human being. That’s why we argue that every code review process needs to be done by a professional, that is, there needs to be manual work.

With these points well defined, we close the first part of our article. With this, we hope to arouse analytical thinking for the evaluation of their tools.

In the next article, we will talk more about Secure Code Review.