From chasing risk lists to ASN policies: Large-scale analysis of risky internet activity

Security managers around the globe are constantly forced to analyse reams of data to protect their networks from communication with malicious traffic. But as blocklists can become outdated fairly quickly, how can they be sure of what to block and to let through?

There is a clear understanding that some elements of the internet are more malicious than others. For example, we might assume that traffic from certain countries seems more suspicious, and that some hosting infrastructures are more likely to be compromised.

By building on this and evaluating and blocking selected high risk autonomous system numbers (ASNs), organisations can block suspicious IPs before they become suspicious.

Background

For the purposes of routing internet traffic, IP addresses are organised into autonomous systems, each containing one or more contiguous blocks of IP addresses. Each system then has a unique number that makes it globally available to identify and which enables that system to exchange exterior routing information with neighbouring autonomous systems.

New malicious IP addresses continuously emerge. Risk lists are compiled based on identifying malicious behaviour and reporting it, but that will not protect networks from soon-to-be risky infrastructure. The reality is that much of that new infrastructure will arise from network locations associated with previously identified risk. So if security teams block based on selected ASNs, they protect their business from all the included IP addresses before they show up on risk lists.

Assessing IP address risk

At Recorded Future we recently carried out research exploring three different approaches to assessing general areas of IP address risk across the internet:

·         Ranking ASNs and associated countries based on the total number of risky IPs contained in the ASN
·         Determining the most risky ASNs based on the percentage of risky IP addresses they contain: Three ASNs have risk-related content for 100 per cent of their IP addresses
·         Analysing rankings based only on those IPs explicitly associated with command and control (C2) malware infrastructure: 37 per cent of C2 related IP addresses are on U.S. ASNs

This understanding is a critical first step in setting network controls that can protect an organisation’s infrastructure beyond blindly adopting IP blocklists, which are rapidly becoming obsolete.

While it is trivial to implement traffic restrictions based on geolocation or autonomous systems membership, the challenge is in determining what to block. Network security teams need principled approaches to establish blocking rules that balance understanding of risk and legitimate business needs associated with different IP neighbourhoods. Blocking around IP addresses is difficult because an IP could resolve to thousands of legitimate domains. These data-driven approaches, based on large-scale historical threat data, can alert security teams to ASNs (autonomous system numbers) and to geographic regions that typically contain risky IPs and support risk/benefit analysis of certain sources of network traffic.

·         The ranking of ASNs by individual IP risk score

To investigate the risk of ASNs and associated countries, our researchers used a comprehensive risk list of IPs containing four million IPs that have current and/or historical risk. This risk list is based on applying 40+ individual risk rules to assess levels of IP addresses between “Unusual” and “Very Malicious”. These rules range from, “we’ve previously observed this IP address misconfigured and as an open proxy” (Unusual), to “This IP address is currently reported to be a command and control (C2) server” (Very Malicious). They aggregate all of the risk information for an individual IP address to generate an overall score. The current risk list, updated in real time as new risk content emerges, scores IP addresses from 5 to 99.

The research was the basis of a new risk list – top ASNs by aggregate time-averaged risk. As opposed to reactively blocking IPs, a business is blocking the ASN, so any risky IPs that show up are already blocked before they show up on a risk list. Organisations are then in the business of allowing necessary IPs within those ASNs only by exception and only if there is a business reason to allow traffic for some of these IP ranges.

·         The ranking of ASNs by percentage of risky IPs

For companies with a significant international presence, assessing risk at the country level of these very large ASNs with large numbers of legitimate IP addresses is likely too crude an approach. Another way to assess ASNs is by the percentage of IP addresses they contain that are risky.

Interestingly, we found that 22 per cent of the world’s most risky IPs are in Chinese ASNs. After this, the second-ranking country in IP riskiness is the United States, although those risky IPs are distributed among 360 per cent more total IP addresses than are associated with China.

The research took note of the fact that the overall number of risky IPs in the U.S. is quite large, despite not having any individual ASNs of highest-volume riskiness. To understand this, it is important to note that in China, where the internet is highly controlled and the largest providers are state owned, there are only 580 ASNs and the bulk of the associated IP addresses are concentrated in the largest ASNs, such as Chinanet. In contrast, there are over 16,000 ASNs in the U.S., and while none manage as many IPs as Chinanet, there are more “large” ASNs in the U.S. than in China, allowing risk to be more distributed across U.S.-based ASNs.

·         The most risky ASNs based on malware command and control association

As part of our research we looked more specifically at the IP addresses that have been explicitly associated with command and control (C2) malware infrastructure. It is clear that IP addresses with the highest-levels of risk merits specific investigation. The research included the geographic distribution which highlighted that the United States dominates here.

While more “harmless” risky behaviour like scanning and botnets may be focused in more “sketchy” locations, clearly, the efforts involved in mounting a malware campaign suggest a bias to investing in more “legitimate” locations like the United States, Hong Kong, Japan, Canada, and the UK.

The impact on network security

For network security professionals seeking to protect their networks, one simple approach is to restrict traffic based on ASNs. The technical rules are trivial to put in place, but the devil is in the details of choosing which of the world’s nearly 60,000 ASNs to block. Data-driven tables like those presented in earlier sections provide prioritised lists to investigate and evaluate the risks and benefits for different ASNs and geographic regions. For example, the total number of risky IPs alone is likely not a sufficient factor on which to base blocking rules. Consider the source of the largest number of risky IP addresses, discovered in Recorded Future research, – Chinanet. There are tens of millions of legitimate websites and internet users hosted by that ASN, but also a number of risky IPs. Organisations must assess the trade-offs between the business interests in foreign locales and the likelihood of IPs at those locations being malicious.

Per cent riskiness of an ASN is a more immediately actionable list to evaluate. Due to the smaller size of these ASNs, the negative impact of blocking them is minimal. Factoring in the country of origin as an investigator moves to lower risk levels and can help in making assessments. This is a much better way for a business to manage the threat from malicious IPs. Rather than blocking individual IPs, an alternative solution is to block specific ASNs.

By evaluating and blocking selected high risk ASNs, organisations can block suspicious IPs before they become a threat, and security teams can more easily manage what to block and what to let through. Our research shows that there are multiple ways to achieve this, and by experimenting and taking on the approach that best fits their organisation, security teams can transform the way they manage and block malicious IPs.

Bill Ladd, Chief Data Scientist, Recorded Future
Image Credit: Elena11 / Shutterstock