With Facebook, Twitter and a multitude of blogging platforms available at our fingertips, the creation and distribution of content has never been easier. It ought to give us an advantage in spotting trends early and making more informed decisions faster. However, we are slowly coming to the realisation that using the internet as a mass communication tool has actually made our decision making much harder – there is just too much information out there driven by polarised and often contradicting opinions, and finding answers to specific questions can be frustrating!
We have come to refer to these large volumes of spam-rich conversation as “big data”, and sieving them for real information is challenging. Looking back, though, we might come to realize that our efforts are nothing new. Centuries ago our ancestors faced similar challenges in analysing large amounts of information. In retrospect, these volumes now seem small: hundreds of data points manually collected by a single doctor are nowhere near what we now understand as “big data”. Back then, however, this was an unprecedented amount of information.
Not long ago, I attended the Beautiful Science exhibition at British Library which explored how our understanding of ourselves and our planet has evolved alongside our ability to represent, graph and map mass data. Wandering the corridors that held multiple examples of how people tackled “big data” over history, I came to the realisation that success in each case was not about simply collecting sheer volumes of data together, but rather in developing the processes and tools to help researchers understand it to gain insight. The visual representation of data has and will always continue to play a vital role in this process.
Indeed, long before social media or the internet, there has been an abundance of ways to visualize and analyse big data that played a vital role in finding correlations that we now take for granted.
In 1854, during the cholera epidemic in London, John Snow used a dot distribution map to establish that cholera is not transmitted through inhalation of contaminated air – which many scientists thought to be the case – but rather though the ingestion of contaminated water or food. He plotted the number of cholera cases on a map of Soho and discovered that a water pump was at the epicentre of the outbreak. Prior to that, William Farr had suggested that air temperature had something to do with the spread of cholera. He built a circular plot to see if there is a correlation, but the result didn’t support this hypothesis.
These are just two examples where turning to data visualization provided answers to big questions that flew in the face of the conventional thinking of the time. The use of powerful visualisation tools to interpret and interrogate data saved many thousands of lives.
Working to answer our client’s needs through data analytics, we use a variety of analytical methods to spot trends that form the basis of our strategic planning. We use visual interpretation of this data to build an insightful picture. As our analytics offerings have evolved, we also feel the need to evolve our creative approach to expressing the data in order to both elucidate the insight and give it the requisite impact.
As we dig deep into the data to compile charts and graphs, however, we shouldn’t forget that these data plots themselves are not our real goal. Rather, they are tools to help us read and understand the large and complex amounts of data generated in online conversations in order to identify insights that can help us address our stakeholders.
While thinking of “big data” as something different, novel and perhaps unmanageable, it is good to be reminded that in centuries past, researchers faced the same problem of reading through “big data” of their time. The ones we remember are those who rose to the challenge and derived enough insight to have a real impact on the world.
Perhaps the data is only as big as the tools and minds that are trying to crack it.