Vulnerabilities are an inherent part of software that can be buried even in the most secure software products.
These flaws in the code can show up anywhere, from the big commercial applications like Windows and MacOS to the open source components that form the building blocks of modern software development.
When vulnerabilities are found, the vast majority of them end up in large databases like the one organised by the U.S. government’s National Institute of Standards and Technology (NIST). Their National Vulnerability Database (NVD) has grown to be the gold standard for developers looking for updates on which software is vulnerable, giving them a trusted source to know if they need to make a fix.
The story of how vulnerabilities get reported
The process of reporting vulnerabilities generally starts with a security researcher finding a vulnerability and reaching out to the vendor or open source project manager, letting them know that they may have a problem on their hands.
Out of concern for the community of users and a sense of fairness to the team managing the software, there is usually a grace period of 30 to 90 days where the information is kept quiet. This gives everyone involved an opportunity to come up with a fix and patch before going public on a major database where everyone from developers and security folks to hackers can see what has become vulnerable, kicking off the race to patch before the scofflaws can try to exploit a product or component.
Once the relevant parties have been notified, the non-profit MITRE Corporation is notified of the vulnerability and a spot on their Common Vulnerabilities and Exposures (CVE) database is reserved without releasing the details. At the same time, the NVD also receives information about the vulnerability and begins their analysis.
It is important to note that not every story follows this path. Unlike centrally managed commercial and proprietary software, open source projects are more like a bazaar, spread out among many members of the community. Therefore, not all vulnerabilities always show up in the NVD. Instead, some are discussed on various security advisories, forums, and other locations. This makes it harder for software developers and managers to keep track of and remediate new open source vulnerabilities.
Vulnerability Databases outside of the US
However, even as the NIST NVD is one of the best well known and trusted sources, there are a number of similar organisations around the world that have the same stated purpose of helping their country’s developers secure their software.
The primary reasons for some states choosing to have their own NVDs can be broken down into the need to cover applications which are native to that country’s users specifically, and the comfort of working in their own language. Not all developers are comfortable working in English. Take for example China where there are legions of talented developers whose English skills are lacking and prefer to work with a Chinese language database.
While diversity and localisation in sources of information can be positive, it can be argued that some national databases do a better job than others, often due to pitfalls stemming from political interests.
Offering what is quite possibly the largest database available, the Chinese NVD (CNNVD) was reported during a study from Recorded Future to have at one point 1,746 vulnerabilities which were not found in the NIST NVD. Moreover, the researchers found the CNNVD was roughly 20 days faster at reporting than the US NVD, due to structural factors that include a wider base for submission resources.
While the researchers Priscilla Moriuchi and Dr. Bill Ladd have noted that the NVD is closing the gaps with its Chinese counterparts, they found some disturbing details as well.
Their study of the CNNVD found evidence that the database was being manipulated through backdating various vulnerabilities that researchers suspect were used to exploit software products for possible intelligence gathering purposes. Unlike the US NVD which is backed by a government agency tasked with supporting scientific research, the Chinese database is operated by their Ministry of State Security (MSS). This organisation is tasked with intelligence and security operations, similar to the American FBI and NSA.
It is likely that these vulnerabilities were used to exploit software used by Chinese citizens as domestic surveillance is the primary concern of the MSS. It should be said that in theory, Chinese developers have access to the American NVD which presents the unfiltered vulnerability information. However, there are likely to be issues concerning language and China-centric applications that make looking to the NIST NVD a less viable option.
Despite its issues, the CNNVD is likely to play a larger role in the tracking of vulnerabilities as Chinese developers take a more prominent place in the production of new software for the global market as well as the domestic.
As such, it will be an important resource to follow moving forward.
If China was hoping to cover up the manipulation of their CVE publishing under the high volume of vulnerabilities and speed, Russia does not appear to be making the effort to try and hide the fact that their database is a fig leaf for requesting access to review the code of commercial software being sold in the country.
The Russian database is referred to as the BDU, and it is run by the military organisation called the Federal Service for Technical and Export Control of Russia.
According to Moriuchi and Ladd’s research, the BDU only publishes 10 per cent of the number of vulnerabilities that the NIST NVD puts out, leading to the question of why even make the effort?
Moriuchi tells me that she was surprised by Russia’s half-hearted attempt at publishing CVEs, telling me that, “My sense is that it’s really just meant to provide that sliver of truth, like ‘Yes, we have an NVD’” She explains that it is a part of the structure that performs technology reviews for vendors who wish to sell in Russia. Unfortunately, companies can put their IP at risk during these views, also risking the security of their products and endangering their global customers.
Russia represents a relatively small market for technology products but is home to a large segment of cyber talent and researchers. Ideally, they should be contributing to their local NVD to help improve software security at home and abroad, but unfortunately, it would seem that state interests take precedence here.
Japan Vulnerability Notes (JVN)
While a significantly smaller operation than the NIST NVD, the Japan Vulnerability Notes database is the local mechanism for developers there to keep their software secure. Drawing from information available in the US NVD, the JVN includes issues pertaining to domestic Japanese software vulnerabilities.
The JVN is operated by the Information-Technology Promotion Agency (IPA) and connected to the nation’s CERT. The IPA’s stated goals are improving IT security, improving the reliability of information processing systems, and IT human resources development.
While it has an English language version, it is far less robust than the Japanese site. During Q2 of 2018, the JVN’s Japanese language database reported 3,757 newly registered cases. Along with 3 cases from domestic product developers, there were 146 coming directly from the JVN while the NVD provided the other 3,608 cases. Comparatively, only 38 cases from the JVN appeared in their English language list.
In speaking with Moriuchi, she describes the organisation in more benevolent terms, grouping it closer to the American NVD as a positive actor.
Collective security in a distributed world
Looking at this list of actors, it is clear that some countries take their role as sources of reliable information for software security more seriously than others.
In cases like Russia and China, the security services appear to have taken legitimate processes that help to guard public safety for their own surveillance and even IP theft purposes. Speaking with Moriuchi, she makes the point that these governments have been known to run NVDs as a cover for running technical reviews of foreign technologies. During these reviews, they can steal source code to give local companies an edge, or for the purposes of learning how to undermine the software.
In China, there has been an increased investment of both capital and focus on internal surveillance projects. This includes an effort to break into applications that are being used by locals for pro-democracy related movements or other activities that the government sees as a threat. They have an incentive to keep tight controls over what vulnerability information is released when it is discovered, and which exploits they keep for themselves.
To be fair, exploiting consumer software is a common practice amongst governments, and clearly something that the American government has taken part in as well (see Eternal Blue). The key difference is that there does not appear to be any manipulation of the entries in the NVD due to national security concerns. Where the US intelligence services have been active is in not reporting vulnerabilities that they have either bought from security researchers or uncovered themselves. According to former White House cyber security lead Rob Joyce, the intel community discloses upwards of 90 per cent of the vulnerabilities that they discover in a process called the Equities Review Board. While some like the Electronic Frontier Foundation (EFF) Senior Information Security Counsel Nate Cardozo has disputed these statistics, Moriuchi says that the US NVD and authorities are still considered to be far more open and considerate of public security interests than some of the more authoritarian actors out there.
While state actors who run these databases will always have their own interests involved, it is important that the software community, and the public at large, push back against vulnerability data manipulation to the extent possible.
Software developers for their part need to be cognisant that not all resources of vulnerabilities are equally forthcoming with their disclosures, and take extra steps to make sure that they are securing their products and not relying on a single source when it comes to keeping their users safe.
Rami Sass, CEO and Co-Founder, WhiteSource
Image source: Shutterstock/deepadesigns