We have a new application or module written in the Go language that we want to analyze. So how do we approach that?
You can also listen to this article in the audio version:
The goal of this blogpost is to serve as an introduction for penetration testers or security researchers on how to analyze Go applications when searching for security vulnerabilities. It mainly highlights the path that I designed for myself during a pilot vulnerability research project that was done analyzing Traefik.
The juicy part of this methodology is on the manual analysis of Go dependencies, which was my main focus during this process.
We are going to see how to search for some low-hanging fruit on Go modules and how to use the application dependencies to get a better understanding of the application itself.
This approach was made by gathering a lot of references in the topic and by reading a lot of the Go language documentation. All the used references will be at the end of the blogpost.
Also, this blog post have the “introduction” word because it is supposed to be the first part of a methodology that will be continuously improved during subsequent vulnerability research projects on Go codebases. We will dive deeper with the methodology on future posts.
Read also: Tips & Tricks for API Pentest
Approach Overview
We will have to approach from a high-level perspective, not diving into specific details about the exploitation of Go applications’ vulnerabilities.
We will take these main steps in our analysis:
- Understand the code structure;
- Use static analysis tools to check for relevant warnings and low-hanging fruits;
- Map the Go modules usage;
- Dive into the module’s code;
- Searching for Vulnerabilities Inside the Modules;
- Analyze sources and sinks, from the application to the modules.
A little disclaimer here: this was made by a non-Go-developer and is here to be used also by the ones that don’t necessarily have a deep understanding of Go but have at least an intermediate knowledge of how code works, in any language.
1. Understanding the code structure
Each codebase is a unique snowflake. Even using Go language, which enforces a lot of good code practices by default, we will find a lot of structural differences in every codebase that is being analyzed.
The best way to understand the code structure is to put yourself as a possible contributor or developer of the project, so start by pretending to yourself that you want to implement some feature or fix some bug in the project, which can become true if you find a bug in an open-source code.
If you would want to contribute to a project, the first thing to do is try to understand its context. Ask yourself some questions:
- What is the objective of this project?
- Is it being continuously developed? By how many contributors?
- What does the application do in practice?
These questions will lead you to understand the high-level objective of the application, and also drive you to the path of which vulnerabilities you should look for.
After answering these high-level questions, go straight to the “Contributing” part of the project documentation. Most open-source projects will have something like this in the README.md file:
Again, here we are acting as a developer to understand the application. Open and consume all the documentation that you can about code contribution. There you will probably find some information about:
- Code best practices used in the project;
- How the pull request review process is being made;
- How security issues are handled by the contributors;
- How to run tests, and which tests are being run to ensure that some developer didn’t mess up anything during an implementation.
This is probably the most useful information you can get. If the project has tests to run, run it!
In Go projects, most of the tests are run easily with a simple command: go test, in more complex projects there is also some integration tests that are built and run with some Makefile, but you’ll probably find it in the “contributing” documentation.
More information about Go tests here.
You don’t need to understand all the nitty-gritty details of the tests implementation, even not about the application’s implementation itself for now. Here we just want a high-level understanding of the code structure and the application’s context.
2. Low hanging fruits and static analysis
Now that we know where we are stepping into, we can use some tools to check for low-hanging fruits in the application.
If you’re a penetration tester or are doing a code audit with the objective of getting a broad range of vulnerabilities and possible risks at the code, you’ll probably spend a lot of effort here.
If you’re doing vulnerability research and are focusing on finding more critical 0-days on the Go codebase, these tools will (probably) not pop 0-days on your screen, but will help you get a better understanding of the overall security effort that the developers are putting into the application, and also some tools output can help you find paths for more critical vulnerabilities and exploitation.
There is plenty of open-source static analyzers for security in Go, we have Gosec and Gokart. These are Go-specific tools for security static analysis.
Some static checkers are not security-focused but can trigger some warnings that can lead you to possible vulnerabilities and also bring you a better understanding about the code, these are staticcheck and go vet.
Now Semgrep, an open-source static analyzer that focused on security and scalability. It is fast and has a lot of rule sets that are maintained by the community. It is not Go specific but we have a lot of rules packages for Go.
I will highlight two useful rule sets for Go applications: p/golang and p/trailofbits .For example, to run it with p/golang enter in the source code directory that you are analyzing and run:
docker run --rm -v "${PWD}:/src" returntocorp/semgrep --config 'p/golang'
And you’ll have as output all the warnings and errors that the tool found, for example:
integration/access_log_test.go
severity:warning rule:go.lang.security.audit.crypto.use_of_weak_crypto.use-of-md5: Detected MD5 hash algorithm which is considered insecure. MD5 is not collision resistant and is therefore not suitable as a cryptographic signature. Use SHA256 or SHA3 instead.
272: digest := md5.New()
CodeQL is another amazing tool that can be used for this analysis and deserves an entire blog post about how to analyze and create queries for Go with it. You can find more about how to run it on your Go code here.
Take some time to run and review the output of these tools, take the effort that suits your objective on the code analysis. We can use this again when we are analyzing Go modules used by the application.
3. Mapping Go Modules Usage
Go handles code reuse and libraries dependencies using what we call Go modules. In simple terms, Go modules have the same objective as a python package, ruby gem, node module, and so on. Since go version 1.11, released in August 2018, we can find the Go modules used in the application inside the go.mod file. Here is an example:
The line 1 module states the module name of our application, in that case, we are using Traefik as an example. Then we have the Go version specification. And the require keyword is followed by a list of the Go modules that this application depends on.
Go uses semantic versioning of the modules, we have the repository name and the version currently used. We will come back to the go.mod file later.
Our objective here is to identify which and how the modules are being used by the application. When you open a Go source file (.go) you’ll find at the beginning of the file something like this:
This states the package name, think of it as a namespace, and the import keyword is followed by the modules that are being used in this file.
It is very clear and explicit which module is being used since the identifier is the repository name of the module, the same that we find in the go.mod file.
Now, see the Go files that you are analyzing where this module is being called, for example, where we have something like: dns.something
Keep notes of which types or functions are being used of which Go module, you don’t need to keep notes on every single module usage, find the ones that look interesting for your objectives.
Then see in the module documentation what this type or function does. This will make you have a deeper knowledge of the application’s implementation, and help you later on where to inspect inside the Go modules.
Most of the Go modules have documentation published here.
You can search there or just put the repository name at the end of the URL: https://pkg.go.dev/<repo_name> . In our example: https://pkg.go.dev/github.com/miekg/dns
More information about modules:
https://go.dev/blog/using-go-modules
4. Dive Into the Modules Code
There are many ways to get the source code of the modules that are being used by your application.
One is manually going to the repository of the module but in this case, you will have to manually search for the tag version that is being referenced in your go.mod file, because if you get the code from the master branch you will probably have a different code from what is being used in your application.
Other ways include the many commands that the Go binary offers. When you run go build or go mod download , it will download the necessary Go modules to the $GOPATH that is set in your environment.
The source code will be in $GOPATH/pkg/mod/ , or in the $GOPATH/pkg/mod/cache but stored in .zip files.
Following our example of the miekg/dns module, it will be in the following directory:$GOPATH/pkg/mod/github.com/miekg/dns\\@v1.1.40/ .
Meaning that in this path we will find the source code of the version 1.1.40 of this module.If you don’t want to fill your machine with Go code of the application you are analyzing, you can do the following:
# run this docker command inside the source code root directory
docker run -v "${PWD}:/src" -w /src -ti --entrypoint /bin/bash golang:1.17
# now inside the container, download the modules specifying the GOPATH
GOPATH=/src/my_modules go mod download
rm -rf my_modules/pkg/mod/cache
mv my_modules/pkg/mod my_modules
rm -rf my_modules/pkg/
Now you will have a little cleaner version of the Go modules code inside the my_modules directory.
You should by now have the annotations of interesting modules you want to analyze from 3. and the specific version of each module downloaded for your delight.
Reference on other go commands that manipulates the go.mod file and the modules can be found here.
Copy the code of the modules that are interesting for your objectives, and that you want to dive deep now. It probably is not feasible to dive deeper enough and research all the modules.
When you have these modules properly separated from the others, start searching for vulnerabilities.
5. Searching for Vulnerabilities Inside the Modules
Now, the first thing we need to do is search for vulnerabilities already reported and fixed for each module that is been analyzed. The takeaways that we need here are two:
- If the application uses an outdated and vulnerable version of the module.
- Using these vulnerabilities to study vulnerable patterns in the module.
Snyk has a good vulnerability database separated by module, where you can see the exact pull request, issue, and fix for each vulnerability you find.
Take notes on the patterns that you found here, and maybe try to bypass some of the vulnerabilities that have already been fixed.
If the application uses a vulnerable version of the module, have an annotation for that too.
Using a vulnerable module doesn’t necessarily mean that the application itself is vulnerable, for this to happen we will need to see if we can reach the vulnerable point by the application. For this the annotations in the step 3. will be really useful.
Repeat here the step 2. for each module. Using static analysis you can find errors and warnings that can potentially be a vulnerability if the application is reaching that point, and it can also make you understand better the module code.
If your objective here is vulnerability research and finding zero-days in the application, use the knowledge-base that you created in the study of old and fixed vulnerabilities to find other patterns or maybe try to bypass these fixes.
I will not dig on “how to find zero-days in Go”, since this is not the objective here, we will leave this to future blog posts that will complement this same methodology.
If you do find something in any of the modules that the application uses go to the next step.
6. Analyze sources and sinks, from the application to the modules.
Even if we found a vulnerability in some module or if the application is using a module with an outdated and vulnerable version, it doesn’t necessarily mean that the application itself is vulnerable.
The first step towards a proof-of-concept of the vulnerability is to map the sources and sinks that are related to this case. Talking in generic terms, sources and sinks have the following definition:
- The source is where untrusted data in our application is coming from.
- Sinks are a vulnerable part of the code which this data can reach.
If we have a source on which the data flow reaches the sink, we have a potential vulnerability exploitation.
In our case, we can use the documentation made in the step 3. to help us find the sources, or at least the middle path of some source, since we have the module’s functions used by the application listed and documented.
If we have data that we can control, and this data can reach some vulnerable function of our Go Modules, we have a vulnerability, then we can start working on a proof-of-concept for this.
Secure code review studies in Go Applications: the next steps
I hope this material serves as a base for pentesters and researchers that still don’t have deep knowledge in Go, to begin and enhance more projects on large codebases made in this language.
Go is a language that is been widely used for some large projects, and the beginning of a vulnerability research project in a large codebase within a language that you don’t know this well could be a little intimidating. And for this, having a methodology is essential.
Use this methodology just as a base, take to yourself what is useful, and discard what isn’t, and develop your own methodology with it. In the future, we will have more blog posts about this topic that we hope will be useful to enhance your methodologies.
References:
https://github.com/OWASP/Go-SCP
https://snyk.io/blog/golang-security-access-restriction-bypass-vulnerability-jwt/
https://semgrep.dev/docs/cheat-sheets/go-command-injection/
https://www.trailofbits.com/post/discovering-goroutine-leaks-with-semgrep
https://rules.sonarsource.com/go/
https://blog.shiftleft.io/how-to-review-code-for-vulnerabilities-1d017c21a695
Very cool material!