DevSecOps: Principles and Comparison of SCA. Part one

The importance of the analysis of third-party software components (Software Composition Analysis - SCA) in the development process is growing as the annual reports on vulnerabilities of open source libraries are released, which are published by Synopsys, Sonatype, Snyk, White Source. According to The State of Open Source Security Vulnerabilities 2020The number of identified open source vulnerabilities in 2019 increased by almost 1.5 times compared to the previous year, while open source components are used from 60% to 80% of projects. If you turn to independent opinion, SCA processes are a separate practice of OWASP SAMM and BSIMM as an indicator of maturity, and in the first half of 2020 OWASP released a new standard, OWASP Software Component Verification Standard (SCVS), which provides best practices for verifying third-party components in the supply chain. BY.







One of the most revealing cases happened with Equifax in May 2017. Unknown attackers have seized information on 143 million Americans, including full names, addresses, social security numbers and driver's licenses. In 209,000 cases, the documents also included information on the victims' bank cards. This leak was due to the exploitation of a critical vulnerability in Apache Struts 2 (CVE-2017-5638), while the fix was released back in March 2017. The company had two months to install the update, but nobody bothered about that.



This article will discuss the issue of choosing a tool for conducting SCA in terms of the quality of the analysis results. There will also be a functional comparison of the tools. The process of embedding into CI / CD and the possibilities for integration will be left for future publications. A wide list of tools was presented by OWASP on its website , but as part of the current review, we will only touch on the most popular open source tool Dependency Check, the slightly less well-known open source platform Dependency Track and the Sonatype Nexus IQ Enterprise solution. We will also figure out how these solutions work and compare the results obtained for false positives.

Principle of operation

Dependency Check is a utility (CLI, maven, jenkins module, ant) ​​that analyzes project files, collects pieces of information about dependencies (package name, groupid, specification title, version ...), builds a CPE line - (Common Platform Enumeration), Package URL (PURL) and detects vulnerabilities for CPE / PURL from databases (NVD, Sonatype OSS Index, NPM Audit API ...), and then builds a one-time report in HTML, JSON, XML format ...



Let's see how the CPE looks like:

cpe:2.3:part:vendor:product:version:update:edition:language:sw_edition:target_sw:target_hw:other

• Part: An indication that the component belongs to the application (a), operating system (o), hardware (h) (Mandatory item)
• Vendor: Name of the manufacturer of the product (Required)
• Product: Product name (Required item)
• Version: Component version (Obsolete item)
• Update: Update the package
• Edition: Legacy version (Deprecated)
• Language: The language as defined in RFC-5646
• SW Edition: Software version
• Target SW: The software environment in which the product runs
• Target HW: The hardware environment in which the product runs
• Other: Supplier or product information

An example CPE looks like this:

cpe:2.3:a:pivotal_software:spring_framework:3.0.0:*:*:*:*:*:*:*

The line means that CPE version 2.3 describes a component of the application from the manufacturer pivotal_software with the name of spring_frameworkversion 3.0.0. If we open the CVE-2014-0225 vulnerability in NVD , we can see a mention of this CPE. The first problem that should be immediately noticed is that the CVE in NVD, according to the CPE, reports that there is a problem in the framework, and not in a specific component. That is, if the developers are tightly tied to the framework, and the identified vulnerability does not apply to those modules that the developers use, the security specialist will somehow have to disassemble this CVE and think about updating.



The URL is also used by SCA tools. The package url format is as follows:

scheme:type/namespace/name@version?qualifiers#subpath

• Sheme: There will always be 'pkg' indicating that this is a package url (Required)
• Type: The "type" of the package or the "protocol" of the package, such as maven, npm, nuget, gem, pypi, etc. (Mandatory clause)
• Namespace: Some name prefix, such as Maven group ID, Docker image owner, GitHub user or organization. Optional and depends on the type.
• Name: Package name (Required)
• Version: Package version
• Qualifiers: Additional qualifications for the package, such as OS, architecture, distribution, etc. Optional and type-specific item.
• Subpath: Additional path in the package relative to the package root

For instance:

pkg:golang/google.golang.org/genproto#googleapis/api/annotations
pkg:maven/org.apache.commons/io@1.3.4
pkg:pypi/django-package@1.11.1.dev1

Dependency Track is an on-premise web platform that accepts ready-made Bill of Materials (BOM) generated by CycloneDX and SPDX , that is, ready-made specifications about existing dependencies. This is an XML file describing dependencies - name, hashes, package url, publisher, license. Next, the Dependency Track parses the BOM, looks at the CVEs available to the identified dependencies from the vulnerability database (NVD, Sonatype OSS Index ...), then builds graphs, calculates metrics, regularly updating the component vulnerability status data.



An example of what a BOM might look like in XML format:

<?xml version="1.0" encoding="UTF-8"?>
<bom xmlns="http://cyclonedx.org/schema/bom/1.2" serialNumber="urn:uuid:3e671687-395b-41f5-a30f-a58921a69b79" version="1">
  <components>
    <component type="library">
      <publisher>Apache</publisher>
      <group>org.apache.tomcat</group>
      <name>tomcat-catalina</name>
      <version>9.0.14</version>
      <hashes>
        <hash alg="MD5">3942447fac867ae5cdb3229b658f4d48</hash>
        <hash alg="SHA-1">e6b1000b94e835ffd37f4c6dcbdad43f4b48a02a</hash>
        <hash alg="SHA-256">f498a8ff2dd007e29c2074f5e4b01a9a01775c3ff3aeaf6906ea503bc5791b7b</hash>
        <hash alg="SHA-512">e8f33e424f3f4ed6db76a482fde1a5298970e442c531729119e37991884bdffab4f9426b7ee11fccd074eeda0634d71697d6f88a460dce0ac8d627a29f7d1282</hash>
      </hashes>
      <licenses>
        <license>
          <id>Apache-2.0</id>
        </license>
      </licenses>
      <purl>pkg:maven/org.apache.tomcat/tomcat-catalina@9.0.14</purl>
    </component>
      <!-- More components here -->
  </components>
</bom>

The BOM can be used not only as input parameters for the Dependency Track, but also for inventorying software components in the supply chain, for example, for providing software to the customer. In 2014, the Cyber ​​Supply Chain Management and Transparency Act of 2014 was even proposed for consideration in the United States , which stated that when purchasing software, any state. the institution must request a BOM to prevent the exploitation of vulnerable components, but the act never came into force.



Going back to SCA, Dependency Track has out-of-the-box integrations with Notification Platforms like Slack, vulnerability management systems like Kenna Security. It should also be said that Dependency Track, among other things, detects outdated versions of packages and provides information about licenses (due to support for SPDX).



If we talk specifically about the quality of SCA, then there is a fundamental difference.



The Dependency Track does not accept the project as input, but the BOM. This means that if we want to check the project, then first we need to generate bom.xml, for example, using CycloneDX. Thus, Dependency Track is directly dependent on CycloneDX. At the same time, it allows for customization. So the OZON team wrote the CycloneDX module to build BOM files for Golang projects for further scanning through the Dependency Track.



Nexus IQIs a commercial SCA solution from Sonatype, which is part of the Sonatype ecosystem, which also includes the Nexus Repository Manager. Nexus IQ can accept as input data both war archives (for java projects) via the web interface or API, and BOM, if your organization has not managed to rebuild from CycloneDX to a new solution. Unlike open source solutions, IQ not only refers to the CPE / PURL for the identified component and the corresponding vulnerability in the database, but also takes into account its own research, for example, the name of the vulnerable function or class. IQ mechanisms will be discussed later in the analysis of the results.



Let's summarize some of the functional features, and also consider the supported languages ​​for analysis:





Functionality





Integration opportunities

Dependency Check

First start

Let's run Dependency Check on a deliberately vulnerable DVJA application .



To do this, we will use the Dependency Check Maven Plugin :

mvn org.owasp:dependency-check-maven:check

As a result, dependency-check-report.html will appear in the target directory.







Let's open the file. After the summary information about the total number of vulnerabilities, we can see information about vulnerabilities with a high level of Severity and Confidence, indicating the package, CPE, CVE number.



This is followed by more detailed information, in particular what the evidence was based on, that is, a certain BOM.







Next comes CPE, PURL and CVE description. By the way, recommendations for fixing are not attached due to their absence in the NVD database.







To systematically review the scan results, you can configure Nginx with minimal settings, or send the resulting defects to a defect management system that supports Dependency Check connectors. For example, Defect Dojo.

Dependency Track

Installation

Dependency Track, in turn, is a web platform with displaying charts, so there is no acute issue of storing defects in a third-party solution.

There are the following supported scripts for installation: Docker, WAR, Executable WAR.

First start

Go to the URL of the running service. We enter through admin / admin, change the username and password, and then we get to the Dashboard. The next thing we will do is create a project for the Java test application in Home / Projects → Create Project . Let's take DVJA as an example.







Since the Dependency Track can only accept a BOM as input, this BOM must be retrieved. Let's use the CycloneDX Maven Plugin :

mvn org.cyclonedx:cyclonedx-maven-plugin:makeAggregateBom

Get bom.xml and upload the file in the created DVJA project → Dependeencies → Upload BOM .



Let's go to Administration → Analyzers. We understand that we only have Internal Analyzer enabled, including NVD. We will also connect Sonatype OSS Index.







Thus, we will get the following picture for our project:







Also in the list you can find one vulnerability applicable to Sonatype OSS:







The main disappointment was that Dependency Track no longer accepts Dependency Check xml reports. The last supported versions of Dependency Check integration were 1.0.0 - 4.0.2, while I tested 5.3.2.



Here is a video (and now ) when it was still possible.

Nexus IQ

First start

The Nexus IQ is installed from the documentation archives , but we have compiled a Docker image for this purpose.



After logging into the console, you need to create an Organization and an Application.















As you can see, setting up in the case of IQ is a little more complicated, because we also need to create policies applicable to different “stages” (dev, build, stage, release). This is necessary to block vulnerable components as they move along the pipeline closer to the product, or to block as soon as they get into the Nexus Repo when downloaded by the developers.



To feel the difference between open source and enterprise, let's perform the same scan through the Nexus IQ in the same way through the Maven plugin , having previously created a test application in the NexusIQ interface dvja-test-and-compare:

mvn com.sonatype.clm:clm-maven-plugin:evaluate -Dclm.applicationId=dvja-test-and-compare -Dclm.serverUrl=<NEXUSIQIP> -Dclm.username=<USERNAME> -Dclm.password=<PASSWORD>

Follow the URL to the generated report in the IQ web interface:







Here you can see all policy violations with different levels of significance (from Info to Security Critical). The letter D next to the component means that the component is Direct Dependency, and the letter T next to the component means that the component is Transitive Dependency, that is, it is transitive.



By the way, Snyk's State of Open Source Security Report 2020 reports that over 70% of open source vulnerabilities found in Node.js, Java and Ruby are in transitive dependencies.



If we open one of the Nexus IQ policy violations, we can see the description of the component, as well as the Version Graph, which shows the location of the current version on the timeline, as well as at what point the vulnerability is no longer vulnerable. The height of the candlesticks on the graph shows the popularity of this component.







If you go to the vulnerability section and disclose the CVE, you can read a description of this vulnerability, recommendations for elimination, as well as the reason why this component was violated, that is, the presence of a class DiskFileitem.class.











Let's summarize only the third-party Java components, removing the js components. In parentheses, we indicate the number of those vulnerabilities that were found outside the NVD.



Total Nexus IQ:

• Dependencies Scanned: 62
• Vulnerable Dependencies: 16
• Vulnerabilities Found: 42 (8 sonatype db)

Total Dependency Check:

• Dependencies Scanned: 47
• Vulnerable Dependencies: 13
• Vulnerabilities Found: 91 (14 sonatype oss)

Total Dependency Track:

• Dependencies Scanned: 59
• Vulnerable Dependencies: 10
• Vulnerabilities Found: 51 (1 sonatype oss)

The next steps are to analyze the results obtained and figure out which of these vulnerabilities is a real defect and which is a false positive.

Disclaimer

This review is not undeniable truth. The author did not have the goal of making a separate instrument stand out from others. The point of the review was to show how the SCA tools work and how to check their results.

Comparison of results

Conditions: A



false positive for vulnerabilities in third-party components is:

• CVE mismatch for the identified component
• , struts2, struts-tiles, , false positive
• CVE
• , python > 3.5 2.7 — false positive, 3.x
• CVE
• , SCA CVE, RCE, SCA CVE, Cisco, RCE. false positive.
• For example, CVE was found in a spring-web component, after which SCA points to the same CVE in other components of the Spring Framework, while CVE has nothing to do with other components. In this case, it will be false positive.

The object of research is the Open Source project DVJA The research involved only java components (no js).

Summary results

Let's go straight to the result of a manual review of the identified vulnerabilities. The full report for each CVE can be found in the Appendix.



Summary results for all vulnerabilities:



Summary results by component:



Let's build visual graphs to assess the ratio of false positive and false negative to the total number of vulnerabilities. Components are marked horizontally, and vulnerabilities identified in them are marked vertically.















In comparison, a similar study was conducted by the Sonatype team to test a 1,531 component project using the OWASP Dependency Check. As we can see, the ratio of noise to correct triggers is comparable to our results.





Source: www.sonatype.com/why-precision-matters-ebook



Let's take a look at some CVEs from our scan results to understand the reason for these results.

More details

# 1

Let's take a look at some interesting points of the Sonatype Nexus IQ first.



Nexus IQ points to a deserialization issue with the ability to execute RCE in Spring Framework multiple times. CVE-2016-1000027 in spring-web: 3.0.5 for the first time, and CVE-2011-2894 in spring-context: 3.0.5 and spring-core: 3.0.5. At first, it appears that there is a duplication of vulnerability across multiple CVEs. For if you look at CVE-2016-1000027 and CVE-2011-2894 in the NVD database, then it seems that everything is obvious





Description of CVE-2011-2894 from NVD :





Description of CVE-2016-1000027 from NVD :





CVE-2011-2894 itself is quite famous. The 2011 White Source report ranked this CVE as one of the most frequently encountered. Descriptions for CVE-2016-100027 are, in principle, a little in NVD, and it seems to be applicable only for Spring Framework 4.1.4. Take a look at the reference and here it becomes more or less clear. We understand from Tenable's article that in addition to the vulnerability RemoteInvocationSerializingExporterin CVE-2011-2894, the vulnerability is observed in HttpInvokerServiceExporter. This is what Nexus IQ tells us:







Nevertheless, there is nothing like this in NVD, which is why Dependency Check and Dependency Track receive false negative.



Also, from the description of CVE-2011-2894, you can understand that the vulnerability is indeed present in spring-context: 3.0.5 and spring-core: 3.0.5. Confirmation of this can be found in the article from the person who found this vulnerability.

# 2

If we study the CVE-2016-4003 vulnerability, we will understand that it was fixed in version 2.3.28, nevertheless Nexus IQ informs us about it. There is a note in the description of the vulnerability:







That is, the vulnerability exists only in conjunction with an outdated version of the JRE, which they decided to warn us about. Nevertheless, we consider this to be False Positive, although not the worst one.

Number 3

If we look at the descriptions for CVE-2017-9804 and CVE-2017-7672, we will understand that the problem is URLValidator class, and CVE-2017-9804 follows from CVE-2017-7672. The presence of the second vulnerability does not carry any payload, except that its severity has grown to High, so this can be considered unnecessary noise.



Overall, no other false positive was found for the Nexus IQ.

# 4

There are several things that set IQ apart from other decisions.





The CVE in NVD says it is only applicable for versions 5.2.x up to 5.2.3, 5.1.x up to 5.1.13, and versions 5.0.x up to 5.0.16, however, if we look at the CVE description in the Nexus IQ, we will see the following:

Advisory Deviation Notice: The Sonatype security research team discovered that this vulnerability was introduced in version 3.0.2.RELEASE and not 5.0.x as stated in the advisory.



This is followed by a PoC for this vulnerability, which reports that it is present in version 3.0.5.



False negative is sent to Dependency Check and Dependency Track.

# 5

Let's look at false positive for Dependency Check and Dependency Track.



Dependency Check stands out in that it reflects those CVEs that apply to the entire framework in NVD to those components to which those CVEs are not applicable. This applies to CVE-2012-0394, CVE-2013-2115, CVE-2014-0114, CVE-2015-0899, CVE-2015-2992, CVE-2016-1181, CVE-2016-1182, which Dependency Check "screwed up" to struts-taglib: 1.3.8 and struts-tiles-1.3.8. These components have nothing to do with what is described in CVE - request processing, page validation, and so on. This is due to the fact that the common between these CVEs and the components is only the framework, which is why Dependency Check considered it a vulnerability.



The same situation with spring-tx: 3.0.5, and a similar situation with struts-core: 1.3.8. For struts-core, Dependency Check and Dependency Track have found a lot of vulnerabilities that are actually applicable to struts2-core, which is essentially a separate framework. In this case, Nexus IQ correctly understood the picture and in those CVEs that it issued, indicated that struts-core came end of life and it was necessary to go to struts2-core.

# 6

In some situations, it is unfair to interpret an explicit Dependency Check and Dependency Track error. In particular CVE-2013-4152, CVE-2013-6429, CVE-2013-6430, CVE-2013-7315, CVE-2014-0054, CVE-2014-0225, CVE-2014-0225, which are Dependency Check and Dependency Track referred to spring-core: 3.0.5 actually refer to spring-web: 3.0.5. At the same time, some of these CVEs were found and the Nexus IQ, nevertheless, the IQ correctly identified them for another component. Since these vulnerabilities were not found in spring-core, it cannot be argued that they are not in the framework, in principle, and open source tools rightly pointed out these vulnerabilities (they just missed a little).

conclusions

As we can see, determining the reliability of the identified vulnerabilities by manual review does not give unambiguous results, which leads to controversial issues. The results are such that the Nexus IQ solution has the lowest false positive rate and the highest accuracy.



First of all, this is due to the fact that the Sonatype team has expanded the description for each CVE vulnerability from NVD in their databases, specifying, with accuracy to the class or function of the vulnerability for a particular version of the component, conducting additional research (for example, checking vulnerabilities on older software versions).



An important influence on the results is played by those vulnerabilities that were not included in NVD, but nevertheless are present in the Sonatype database with the SONATYPE mark. According to The State of Open Source Security Vulnerabilities 202045% of discovered open source vulnerabilities are not reported to NVD. According to the WhiteSource database, only 29% of all open source vulnerabilities reported outside NVD end up being published there, which is why it's important to look elsewhere for vulnerabilities as well.



As a result, Dependency Check generates a lot of noise, missing some vulnerable components. Dependency Track produces less noise and reveals a large number of components, which visually does not hurt the eyes in the web interface.



Nevertheless, practice shows that it is open source that should be the first steps on the way to mature DevSecOps. The first thing to think about in order to embed SCA into development is processes, namely, thinking together with management and related departments about what the ideal processes should look like in their organization. It may turn out that for your organization at first, Dependency Check or Dependency Track will cover all the business needs, and Enterprise solutions will be a logical continuation due to the increasing complexity of the developed applications.