Fulfilling the promise of Big Data: How the Third Network can enable a new era of connected services

The Big Data revolution has been brewing for some time now, and is set to explode over the next few years. IDC has predicted that the market will grow by 50 per cent between 2015 and 2019, from $122 billion to $187 billion.

But Big Data brings with it a big problem for networking infrastructure. The clue is in the name. Big Data is big, both in terms of size and geographical extent. Over the same 2015-19 period, again according to IDC, the global Digital Universe of data will expand from 10 Exabytes to over 30 Exabytes.

Such a threefold increase in data quantity brings with it a similarly manifold increase in the throughput necessary to access it all. In order to cope with the extra bandwidth requirements of Big Data, a significant increase in network capacity and flexibility will be required. However, this won’t be fulfilled just by the next generation of LAN technology, even if that might help.

The most significant thing about Big Data is not that it means companies are holding increasingly huge amounts of information in their private local networks, but that these databases are all over the world and connecting to them involves WAN and even public Internet connections. This is where the Third Network comes in.

What is Big Data?

Before we discuss what the Third Network is, and how it can be beneficial, we need to understand what the benefits of Big Data are. The headline feature is that the quantity and extent of data enables a new type of analysis. The history of data analytics up to this point has been geared around having a theory about something first and then testing it against the information available to see if it fits. This doesn’t mean that the theory is true if it agrees with recorded data, only that it could be true and hasn’t been disproven yet, because you only have a certain amount of data to play with. New information could mean you need a new theory.

The concept with Big Data is that now there is so much information to consider, you don’t need to worry so much about having theories. With all the computing power now available, you can simply check a whole set of possibilities and see which one works best. You can even try things that don’t initially make sense, and if they fit the data, try to figure out why they fit after the fact. In the past, the “why” usually came first, although some of the greatest discoveries of all time, such as that penicillin can kill bacterial infections, have been deduced from opportunistically noticed correlations.

This facility of Big Data to upturn the traditional process of discovery has been making government, science and industry very excited, as it could be (and in some areas already is) heralding a new era of rapid analysis. Governments are hoping to develop more efficient healthcare delivery, greater economic control, improved crime prevention, and more timely predictions of natural disasters. 

Manufacturers are relishing faster development of new products and services, as well as rapid updating of designs based on user feedback. Media and advertising companies are able to develop more successful content based on consumer reaction. You can already see the last of these from the current posterchild of online content, Buzzfeed, which only engages in media activities it can track and use as feedback data to guide the next round of creativity.

The rapid arrival of self-driving cars should be seen in this context, too, bringing together as it does a huge array of sensor, behavior, and geographical information. The holy grail is the intelligent, autonomous learning system, such as IBM’s Watson, which proactively builds useful knowledge from only partially structured, or even unstructured data.

The Importance of the Third Network

One of the key things about Big Data, therefore, beyond the sheer quantity, is that the information will be coming from lots of different places. A historical array of world temperatures can be brought together with industrial activity, travel data, and climatic behavior, or whatever else can be thrown into the mix, to look for correlations.

Retail analysis can combine details of advertising campaigns in various media with demographics, weather, and purchasing behavior. We haven’t even mentioned the connection between Big Data and the Internet of Things, with the latter supplying real-time input to go with existing historical information.

A lot of data will therefore be arriving over public networks, or at least networks from disparate providers. But it will also be essential to guarantee performance and security, neither of which are usually synonymous with a public Internet connection. The Third Network idea is intended to combat this by combining elements of Carrier Ethernet 2.0 with the public Internet to provide Network-as-a-Service that can deliver an appropriate amalgam of agile, on-demand connectivity with secure and stable performance.

This is expressed through a Lifecycle Service Orchestration (LSO) API, which defines standards for end-to-end connectivity across multiple network service domains and types. This is essential, because the connections between all the sources in a Big Data implementation are certain to cross more than one network service domain.

The LSO API can sit on top of a wide variety of network infrastructure types. Alongside a company’s own WAN implementation can be Software Defined Networking (SDN) and Network Function Virtualization (NFV). Both SDN and NFV provide abstract management of a network, in different ways. The SDN system allows control over the path of network packets across switches, without having to provide individual direct control of the underlying hardware. In a related fashion, NFV allows the creation of communication services using virtualized networking nodes.

The LSO API brings all these disparate systems together so a single self-service portal or business application-based control can set up a secure channel with the required guarantee of performance between all the disparate elements.  

The Third Way to Big Data

It’s clear that the next decade of computing will pose many challenges, but the immense opportunities from Big Data applications will far outweigh them. Carrier Ethernet 2.0-level Metro Ethernet may have enabled long-distance connections of 10Gbits/sec and beyond for a few years now. But Big Data services require great flexibility as well as raw bandwidth, with spikes and troughs, but a real need for dependable speed and security at the same time. 

The Third Network concept delivers that combination, potentially enabling the new era and huge growth market that has been promised by Big Data.

Image source: Shutterstock/Carlos Amarillo

This post is brought to you by Comcast Business. Learn more at our Comcast Business community.