As many as 90 per cent of security attacks are a result of exploits against software flaws and defects, according to the Software Engineering Institute. Yet, most “solutions” to these incidents focus on detecting and responding to the attack rather than identifying and fixing the faulty code that caused the vulnerabilities in the first place. The most pernicious of these vulnerabilities are called “zero-day vulnerabilities” – they’re defined as being unknown: they’re not in the CVE database and may not have an easy patch or fix when they’re discovered.
Zero-day vulnerabilities: A growing problem
Zero-day vulnerabilities are software security flaws with the potential to be exploited by cybercriminals – they’re unintended flaws found in programs or operating systems that, if left unaddressed, create security holes that can and almost certainly will be exploited. They also could crash your system when you do an upgrade. In the case of the Cloudbleed bug from a software upgrade at Cloudflare in 2017, user passwords and other potentially sensitive information was leaked to thousands of websites over a period of six months.
Zero-day vulnerabilities aren’t new, but today they present an increasing risk. Some of these vulnerabilities persist for long periods of time. So pervasive are the concerns around these threats that companies like Google, with its Project Zero, employs a team of skilled security analysts focused on discovering zero-day vulnerabilities.
The problem with standard testing and other approaches
Organizations have often been able to respond to zero-day vulnerabilities only when they’ve graduated to zero-day exploits or attacks. The problem stems from the traditional software development and QA testing processes that fail to identify bugs and flaws that manifest in live usage. Static and dynamic testing, RASP, and vulnerability assessments all look for known problems or known fallible coding techniques which makes it difficult to identify zero-day vulnerabilities (which are, by definition, unknown.) Even using blue-green or canary staging approaches, software bugs may not be seen and will propagate, meaning the code or application problems caused by these flaws are pushed live, because that code is not tested with production traffic. Rollbacks can be used, but they’re expensive. In fact, the costs associated with correcting software defects in production are as much as 640 times more expensive than prior to release (Jones, Capers. Applied Software Management. Global Analysis of Productivity and Quality), 2008.
According to Forrester, application weaknesses and software vulnerabilities continue to be the most common means by which bad actors execute external attacks. Since the existing testing methodologies have trouble finding these critical zero-day vulnerabilities, other approaches are being tried, including advanced log analysis and bug bounty programs. Log analysis does not find all potential zero-days vulnerabilities, as the exploit (if it is in the logs) may be discovered but too often after it has become a problem, even if artificial intelligence and machine learning are applied.
The hacker and security research communities are being used to crowdsource bugs and then sell them to software vendors via bug bounty programs. Though an interesting idea, this approach still ignores a key fact: it’s significantly better to prevent those glitches from going live at all than it is to find them after the fact when they’ve already had the potential to be exploited. Also, you don’t know if a hacker is going to go out and sell that exploit to a competitor or even a nation-state or hold for his or her own (nefarious) purposes. When it comes to the underworld of the internet, you have no assurance where this information is being sold or to whom.
A better way: predict and prevent zero-day vulnerabilities by testing with production traffic
Software and DevOps teams need a way to test accurately in real time to observe how new releases will perform with live (not just simulated) customer traffic. By evaluating release versions side-by- side in lockstep and actively looking for differences in responses, defects can be predicted and detected quickly. Software regressions are identified before customers are affected. It is crucial to fix the defects before they go into production to prevent the potential release of zero-day vulnerabilities. But it is equally important to provide clear and reproducible bug reports to developers based on production traffic flows to identify where the problem is in the execution of the code. One can track IDs of the same transaction to see how different versions behave. The determination of faulty code is derived from observations of primary network communications and interactions rather than secondary data from logs or the interpretation or analysis of those logs.
By taking a proactive approach to how software testing is performed, organizations can find these flaws and bugs before they’ve been pushed live. By doing this, these flaws or bugs can be addressed first at the root, rather than having to do a rollback. As the adage goes, an ounce of prevention is worth a pound of cure.
Zero-day vulnerabilities are a serious issue and one that can last a long time. To stay a step ahead of the bad actors, software and DevOps team need a new way to test for flaws before releases go out. They can do this by testing with production traffic, avoiding expensive rollbacks or awkward staging. The side-by-side testing identifies not only the zero-day bug but also the place in the code execution where the failure occurs as well as a reproducible roadmap for the developer to fix the bug. That’s the kind of preventative action that will help organisations push out top-notch iterations and keep company and customer data safe.
Frank Huerta, CEO, Curtail