Any decision starts with an objective, a set of known data and possible actions. Once you get beyond the simplest decisions, the variables start multiplying, leading to millions or even billions of possible options. No one can do that kind of math in their head.
This is why the analytics known as optimisation are gaining ground. Optimisation involves finding the ideal decision to make, and optimisation algorithms can correlate any number of possible scenarios and outcomes to find the one that best meets your objectives.
This isn’t a new science, but in an age or Big Data it is becoming increasingly widespread.
The range of types of problems optimisation can solve is well illustrated by an award-winning project undertaken by Professors Brieden and Gritzmann from the Technical University of Munich (TU Munich). Land allocation is one of the biggest challenges farmers in Europe face. Most own several plots of land; however, these are often scattered randomly and are rarely adjacent to each other. This means that a lot of time and money is spent simply travelling from one plot to another rather than actually cultivating the land.
Brieden and Gritzmann were working on a project to help nine farmers in Bavaria, who collectively owned 342 lots (see below). The number of potential land re-allocation scenarios was 9342. Even running one million scenarios per second would take 10289 years.
This land reallocation problem was easy to visualise, hard to solve. A segment of it is shown below, with different farmers’ properties colour-coded. Uniting these scattered land plots involved powerful optimisation analytics.
Figure 1: Showing the 342 plots of farm land, colour coded to show their original allocation on the left and the post-optimisation on the right.
The combination of FICO’s advanced technology and the algorithms developed by Professors Brieden and Gritzmann enabled TU Munich’s research team to solve the Bavarian farmers’ land allocation problem. Once the software was set up and all variables taken into account, it took just a couple of minutes for FICO technology to run through all scenarios and formulate the best possible outcome.
That wasn’t the end of the story, though. The farmers rejected the solution the algorithm found! Some of them had favourite plots they wouldn’t trade away, or farmers they wouldn’t trade with. The true optimal solution had to take these constraints into consideration – and once the professors recorded them, the model could take them into consideration and produced a reallocation the farmers approved.
The citizen developer
Many businesses, from banks to utilities, are still trying to figure out how to start their first meaningful optimisation project – and IT is a critical barrier that often stalls efforts before they get past the planning stage. For an analytics operation to take off in any organisation, the IT department needs to be wholly on board.
This is not always easy. IT departments have to juggle multiple responsibilities simultaneously with different requests from senior management, all with little internal support. Data scientists and operation researchers are in short supply at most companies. Also, the lack of essential resources is worryingly widespread and expected to worsen in the coming years. A recent McKinsey study predicted that data science jobs in the U.S will exceed 490,000 by 2018, but there will be fewer than 200,000 data scientists. Understandably, this is often a barrier to the success of many analytics projects.
It is not all doom and gloom, however. In the past couple of years, there has been significant progress that allows businesses to rapidly turn insights from Big Data into actionable, optimised decisions. Firstly, the emergence of a new breed of business user - the ‘Citizen Developer’ - has catalysed the development potential of applications powered by analytics. These business analyst-level resources can modify rule systems, wrangle data, do champion-challenger testing, recalibrate predictive models, and work on complex optimisation business cases. The presence of empowered Citizen Developers can limit overreliance on expert analytic resources, allowing organisations to better allocate the most skilled people to the most critical projects.
Technology advancements have also helped to level the playing field for organisations seeking to benefit from optimisation projects. Open cloud-based technologies provide flexible deployment options and reduce barriers to entry. The rise of virtually automated application development systems is reducing IT burdens by simplifying the development process to within lines of businesses. With these technologies, Citizen Developers can help democratise business applications and facilitate faster, smarter decisions by freeing IT from laborious recoding efforts.
The next step is the natural evolution of these two developments: transforming Big Data insights into Citizen-Developer-generated applications. This happens when open technologies and solutions for solving large, complex optimisation problems converge into a single, integrated platform.
Such optimisation algorithms are applicable to virtually any problem; whilst TU Munich solved farmland allocation, Southwest Airlines optimise various problems, from gate assignment to provisioning beer and peanuts.
Another example is American Airlines, which serves nearly 100 million passengers annually, juggling connections, working around bad weather, and assigning crew to nearly 4,000 flights every day. Amidst all these complications, the company turned to the same solution that helped our Bavarian farmers, to streamline certain processes.
For example, when a customer visits AA.com and purchases a ticket, the revenue management group re-optimises to determine pricing and availability for the remaining seats. If a snow storm hits somewhere, grounding all planes in and out of a particular airport, the system operations control, reservations, and revenue management groups re-optimise to determine how, where, and when to reroute disrupted customers.
Optimisation solutions are becoming increasingly essential for organisations to differentiate themselves as industries become ever more competitive. Optimisation can help drive profits, improve customer loyalty, and cut costs – sometimes dramatically. Applying the right solutions and freeing up the right resources will ensure that IT managers can help their businesses to consistently make the best possible use of their data to drive smarter decisions.
As with American Airlines, this often has a directly positive impact on profitability and customer relations.
Oliver Bastert, senior director of product management at analytics and decision management software firm FICO
Photo Credit: Adchariyaphoto/Shutterstock