Software Supply Chain Security

Software supply chain security is a higher priority than ever, and software bills of materials (SBOMs) are a key tool organizations use to keep their applications safe.

A good software application is always changing: software companies are constantly fixing bugs, adding capabilities, and improving the user experience. In addition to these changes, the third-party libraries these companies incorporate in their applications are also constantly being updated to improve performance and eliminate known vulnerabilities—which means many software companies sell applications built on out-of-date third-party code.

With both proprietary code and components in such a state of flux, it’s important for companies to generate comprehensive and accurate SBOMs—whether it’s for themselves or for hopeful vendors.

To do this, organizations invest in tools that can provide SBOMs for the software they create as well as for the software they consume. As part of our ongoing effort to help people choose SBOM generators, we’ve analyzed some of the most well-known SBOM generators on the market today to see which ones lend the biggest hand to software providers.

What to look for in an SBOM generator

When it comes to choosing an SBOM generator tool, there are five things that you must take into consideration:

1. Can the tool generate an SBOM from your build environment? At the very least, an SBOM generator should be able to reconstruct your software application using your source code.
2. Can the tool generate an SBOM independent of source code? Ideally, your SBOM generator will allow you to create accurate SBOMs for potential suppliers and vendors. These are companies who likely won’t show you their source code, but should still be vetted for transitive vulnerabilities.
3. Can the tool generate an SBOM from Docker images? Similar to above, it’s helpful if your SBOM generator can create an SBOM simply by using a Docker Image.
4. Does the tool capture all the dependencies in the target library? Most tools are built on libraries that are built on other libraries. When generating an SBOM, you want to use a tool that shows exactly how these third-party sources stack up within each other.
5. Is the tool easy to deploy? Although it’s important to find a tool that performs the above three capabilities, it’s also important to make sure your SBOM generator is easy for your dev team to implement into the shipping cycle.

4 SBOM-generation tools side-by-side

There are many SBOM generators available today, but I chose to focus on four that I already know perform well, incorporate dependency information into the SBOM, and support multiple SBOM formats.

Those tools are CyclineDX, Syft (by Anchore), Fossa, and, of course, our own Finite State.

SBOM tool comparison methodology

I started by making a small Java Maven project—one that only contains a single import. The project has a small main class that imports the Twilio library (which has the log4j dependency). I built an identical project with Gradle so we can see if there are any discrepancies in these tools’ reports.

Second, in order to assess each tool’s ability to analyze projects whether or not the source code is available, I separated these projects into four folders:

• Gradle compiled
• Gradle uncompiled
• Maven compiled
• Maven uncompiled

Some of the tools we’re assessing generate SBOMs from Docker images. To test this, I also put together a small Docker image and manually added a single vulnerable “jsch-0.1.55.jar” library under /opt/jsch/. This gives us a chance to see how well these tools work when you’re adding software using Docker’s “COPY” command instead of adding it via the traditional package manager route.

The repository also includes a file named “dependencies.txt,” which contains all the Twilio dependencies according to deps.dev.

With these samples ready, I ran each SBOM generation tool through three exercises:

1. Generate SBOM from source code (the Log4J transitive project, pre-build).
2. Generate SBOM from a deployed system (again, the Log4J transitive project, but post-build).
3. Generate SBOM from a Docker image.

After this, I checked each tool’s report to see if they captured every dependency in the dependencies.txt list. This informed the “Accuracy” rating on the table above. Each tool caught every transitive dependency except lombok, which makes me think this is more likely a deps.dev mistake than a flaw in the tools.

lombok

For each tool, I made a note of how easy it was to deploy—as well as any additional comments. You can find my notes on each below.

CycloneDX

• Generates SBOM without source code:
  • You need to run gradle wrapper in order to run the ./gradlew cyclonedxBom command, but don’t run ./gradlew build.

gradle wrapper

./gradlew cyclonedxBom

• Generating an SBOM from build (source code available):
  • When running the tool in build, run ./gradlew build first, then gradle wrapper in order to run the ./gradlew cyclonedxBom command.

./gradlew build

gradle wrapper

• Deploying CycloneDX for SBOM generation:
  • All you need to do is add the “org.cyclonedx.bom” dependency to the build.gradle file, which is also where you can configure how you want CycloneDX to generate the SBOM.

Syft (by Anchore)

For a free, open-source tool, Syft is an effective SBOM generator. Syft is very easy to operate and will do exactly what you tell it to do—which means you need to be specific when telling Syft what to scan. (Be careful not to accidentally tell it to scan your complete filesystem!)

While Syft provided, on the whole, accurate analyses, I did have a few qualms. For example, Syft did find every transitive dependency. However, it only did so for the post-build directory scan, and presented the report as a flat list rather than a series of interconnected relationships. In addition to this, Syft returned more than 830 spurious entries in the resulting JSON file.

Fossa

Fossa is an easy enough tool to use. However, the CLI command doesn’t generate SBOM files directly into your project—you’ll have to move it there yourself.

By default, Fossa generates SBOMs using SPDX’s plain-text format, rather than in XML or JSON. This makes the file less machine-readable—but the resulting text is not very human-friendly: SPDXVersion: SPDX-2.1 DataLicense: CC0-1.0 SPDXID: SPDXRef-DOCUMENT Creator: Organization: FOSSA, INC. Created: 2022-08-09T10:45:17Z PackageName: Apache Log4j API PackageVersion: 2.14.0

Finite State

Finite State can generate an SBOM with a click. And because the tool can analyze binary applications (including Java and .NET), it doesn’t matter if you have access to the source code—Finite State can do it on command, or as part of an automated workflow. The tool automatically produces a VEX report as well.

The best SBOM generator

If you’re working in a Maven or Gradle environment on a shoestring AppSec budget, CycloneDX is the way to go.

But if you want an easy-to-deploy, highly accurate SBOM that tells you where your risks lie (and gives you guidance on how to fix them), Finite State is the tool for you.