Implementation According to SAMM: Defect Management in AppSec

The defect management practice consists of collecting, recording, and analyzing security defects, in addition, of course, to enrich this information to use it in decision-making through metrics.

The first stream aims at a process of administration and handling of defects to ensure a level of quality when the software is released. In the second stream the idea is to enrich the information collected, thus deriving metrics for decision-making regarding applications as well as the secure development program.

You can also listen to this article:

Recording Defects

Defect records can be derived from penetration tests, scan tool results, program bug bounties, code reviews, and other means.

Figure 1 shows a SQL Injection defect record. In this record it is possible to verify that this vulnerability has the status of identified, so no action was taken. Another important information is that its severity is critical, in addition to complementary information such as categories, standards and additional piece of information about the vulnerability in question.

It is extremely important to define access rules to application security defect information to mitigate the risk of information leakage and abuse.

Once the defect is registered, actions can be taken according to the maturity level of the OWASP SAMM, for example:

• Maturity 1: rudimentary qualitative classification of defects to prioritize correction efforts;
• Maturity 2: classification based on probability and impact of the defect being exploited, in addition to introducing SLAs for corrections;
• Maturity 3: Implement an automated alert that a fix is outside the defined SLA. Ensure that these defects are automatically transferred to risk management.
• Integrate the defect system with tools from other practices, such as:
  • build e deployment: fail the build if the final artifact is affected by a defect above the pre-established severity;
  • monitoring: if possible, ensure that abuse of the security defect in the production environment is recognized and alerted.

Metrics and feedback in defect management

After collecting, recording and handling security defects, it’s time to enrich this information by transforming it into metrics. In Figure 2, which shows Conviso Platform’s People & Culture, it is possible to see a training module where the registered defects are used, with that the programming language is suggested to invest in learning, in the case of Ruby on Rails and also mentions the most common vulnerability, in the case of SQL Injection.

SAMM suggests some metrics/actions according to their maturity, for example:

• Maturity 1 – metrics:
  • total number of defects x total verification activities;
  • defects softwares;
  • defect category
  • the severity of defects;
• Maturity 2 – metrics:
  • verification activities x identified defects;
  • types of severity identified
  • detection time and resolution time (Figure 3);
  • defect exposure window in production;
  • number of reopened regressions/vulnerabilities;
  • amount of risks accepted;
  • reason for security incidents caused due to unknown or undocumented security defects;
• Maturity 3: revisit the metrics gathered and compare the effort required to collect and record vs. the expected result. The idea is to remove metrics that do not meet the expected result.
• Aggregate the information with your threat intelligence and incident management metrics and use the inputs for other initiatives such as: planning security training; improve verification of security activities; among others.

Conclusion

Defect management can bring us many insights, from applications that consume more energy due to their criticality, even concerning the secure software development cycle as a whole, as we can use this information for more targeted training, for example where the team is making the most mistakes.

Reference

1. OWASP SAMM

SAMM article series

1. Governance according to SAMM: Strategy and Metrics in Application Security
2. Governance according to SAMM: Policies and Conformities in Application Security
3. Governance According to SAMM: Application Security Education and Guidance
4. Design according to SAMM: Threat Modeling in Application Security
5. Design According to SAMM: Security Requirements in AppSec
6. Design according to SAMM – Secure Architecture in Application Security
7. Implementation according to SAMM: Secure Build in Application Security
8. Implementation according to SAMM: Secure Deployment in Application Security
9. Implementation According to SAMM: Defect Management in AppSec
10. Verification according to SAMM: Application Security Architecture Analysis
11. Verification according to SAMM: Requirements-Driven Testing in Application Security
12. Verification according to SAMM: Security Tests in Application Security
13. Operations according to SAMM: Application Security Incident Management
14. Operations according to SAMM: Environment Management and Application Security
15. Operations according to SAMM: Operational Management in Application Security