The Implications of Latency in Analytical Applications

Speed is increasingly defining the user experience from B2C to B2B. No matter how attractive the application, if it does not perform from a speed perspective, it might as well be ugly because that is the sentiment increasingly attached to slow, plodding applications on the web, mobile and in the enterprise.

To be fair, speed comes in different flavors. There is speed from the perspective of eliminating clicks or reducing the number actions to achieve an outcome. There is also raw speed, the speed of the response. One is a user experience challenge, the other a software development task. Ultimately, however, they need to overlap to achieve the right business outcomes.

Great UX is no longer just about great design, its is about the speed at which a user can render, interpret or complete their task. UX certainly plays a role in guiding the user to the right outcome, but the speed at which that happens matters greatly.

If a user has to wait on the application to process at each step, the user experience degrades considerably.

The Financial Times recently injected small delays into their story loads and found that, not only did viewers engage less, but that engagement continued to deteriorate over time. The persistence of the annoyance, whether re-enforced by other web experiences or not, drove less and less usage of the site.

This led them to conclude:

• The speed of the site negatively impacts a user’s session depth, no matter how small the delay.
• Slow sites also have a detrimental effect on the number of articles people read.
• Largely, the slower the site, the greater the effect.
• This trend is borne out over time and typically worsens with continued delays.

While the preceding statistics and case studies deal predominantly with B2C use cases - the research is even more emphatic when it comes to database performance and visual analytics.

In the seminal paper by Heer and Liu we find that as little as 500ms latency can have a significant impact on visual analytics and that those delays impact how much of the dataset is explored, the speed at which insights are ultimately gleaned and that the negative externalities associated with the delay persists beyond the elimination of latency.

Underpinning this research is the fact that while any experience can be slow, experiences that involve large amounts of data are far more susceptible to degradation - which is exactly the problem faced by enterprise analytics applications today.

With data sizes growing at such a tremendous pace, responsiveness has been impaired. Query times balloon, render response lags. The front end blames the backend, the backend blames the frontend, everyone blames the network.

The truth of the matter is that most analytics and visualization platforms are all ill-suited to the world we live in today. They were built for the gigabyte economy. We don’t live in that world today. We haven’t for a few years now.

More importantly, organizations are not fully appreciating the impact of latency.

They make incremental improvements and claim success. Reducing query time from twenty minutes to twelve results in a great percentage to put in front of the leadership, but it has not really moved the needle. Your best analytical assets are still waiting far too long.

Latency creates a series of bad behaviors in the analytics function, behaviors that compound over time and are yet further exacerbated by the emergence of the citizen data scientist within organizations.

Here are the big impacts of enterprise application latency that need more attention and appreciation inside of organizations:

• When it takes too long to get answers from their analytical tools, analysts and other consumers of data start to ask only those questions for which they know they can get an answer for in a reasonable amount of time. Further, both the question-maker and the answer-getter will fight hard to defend the “rightness” of those questions, because they represent success. This leads to myopia. Myopia was bad when Theodore Levitt wrote about it. It is even worse today.
• We don’t always know what we are looking for, which is why the promise of unsupervised learning is so powerful. But even with unsupervised learning, it narrows the playing field, it doesn’t magically provide the answer. From there, decision makers need to explore the possibilities within this narrowed playing field. If each possibility takes a minute or more to consider, you are going to run out of patience rather quickly. As a result, analysts and decision makers often stop short of a complete understanding, adopting “directional” indicators as their guide. No one is immune to this problem - the research is clear on this. Slow systems wear down humans - and result in sub-optimal outcomes. Until full AI is here, this is the problem we need to solve.
• The human capacity to make certain connections is still a mystery. How some people “see” patterns, opportunities, solutions where others don’t can be classified in a myriad of ways - genius, savant, leader, mystic. These capabilities are closely related to the concept of creativity, which we increasingly know is teachable. While not self evident, analytics is ultimately a creative exercise. Analysts start from some place (see unsupervised learning). Where they go is a function of the resources that are at their disposal. Systems that are slow interrupt creativity. They reduce the opportunity to have those bursts of creativity that are so powerful. Analytical resources that require minutes of waiting between responses crush those creative spurts and reduce the likelihood of finding the insights, big or small that change the trajectory of the business.
• Time is an adversary. Whether developing a hypothesis around a trade, trying to finish a board presentation or meet a report deadline or just trying to understand something better before the next meeting - the clock always looms. If it takes an extended period of time to return a query, the standard practice is to downsample. Slow systems lead to more downsampling. Downsampling is not always bad - done correctly it can deliver statistically valid results. Done incorrectly (and there are a lot of ways to do it incorrectly), and it introduces risk, whether in the form of missed outliers or statistically invalid inferences. Suggesting that downsampling can be completely eliminated is not particularly realistic - even in the multi-node, GPU world with multiple terabytes of onboard memory. Still, the aforementioned configuration dramatically reduces the number of potential cases by which one would have to downsample and allows the big guns (data scientists) to lend some counsel when it does. Time doesn’t cease to be an adversary, it just becomes a far more manageable one.

Each one of these elements are an argument to inject web-class speed into your analytics systems. Taken together, they should serve as a reminder that we are still in the early days of the analytical revolution and that while the current generation of BI tools, interfaces and databases may have double digit version numbers they are actually getting slower as time goes on - swamped by a deluge of data that grows bigger by the day.

Data is winning.

Analytical capabilities are losing.

Most leaders in the space like to talk about how much data they can store. They toss around prodigious figures like Peta and Zeta.

How much can they actually analyze?

Not much it seems.

Not if they are using the current class of CPU-addled machines that pass “state for the art.” For true analytical performance, a new approach to compute is required, one that leverages GPUs, their parallel processing power and massive memory footprints.

Anything less is putting the organization at risk.

Quarter after quarter, these leaders embrace CPU-driven “good enough” arguments to defend their analytical infrastructure. It is not good enough, it is fundamentally dangerous and with each citizen data scientist they arm, they multiply the risk.

Speed will define winner and losers not only the basis of moving quickly, but on the basis of eliminating risk and creating a portfolio of options for the enterprise.

GPU-powered analytics can deliver the requisite performance to deliver this portfolio of options.

Not every task requires speed, not every task requires looking at billion rows, but with each passing day more and more do. This is where GPU-powered experiences convey such compelling price/performance characteristics.

This is why we talk about an inflection point in compute - because it is here.

This post originally appeared over at InsideBigData.