Will big data affect you?

21st November, 2013

Data is the lifeblood of the research industry. The absolute core of all exploration; what each participatory survey seeks to uncover. So it should come as no surprise to many that as we expand our research tool repertoire, so does the literal amount of data expand alongside.

So comes the need to understand the data we are now handed. Big data is more than a catchall; rather, it can provide researchers opportunities to pursue more humanised metrics that were previously unattainable due to the limitations of research technology. As humankind become more and more digitised, so does the data we produce. Moving on from analogue systems for both collection and collation of data allows us to further understand the intricacies that exist between myriad sectors of society, how each sector with its idiosyncrasies can unwittingly affect another.

Of course, the caveat is that most researchers do not have the processing power available in the office to run parallel simulations of Exabyte’s of raw data – it is simply not possible outside of large technology labs. But big data is consistently contextualising situations around us – both digital and analogue – perhaps without us even realising. Certainly, many of us interact with big data on a daily basis: Amazon (largest Linux databases in the world), Wikipedia (terabytes of information consistently modified by worldwide audience) and Facebook (near 50bn photos in its database, constantly growing). Implicit in all of these data centres is the contributions of human efforts to update and maintain such vast networks that enable those with access to make predictions on mass scale.

Whilst more and more companies and government departments are utilising big data to their advantage it has to be clear that it will not and cannot solve worldwide problems due to the inherent localisations attributed to specific data sets. But by providing government departments with the opportunity to analyse specific aspects of big data it may allow for certain aspects of day to day life to improve: identifying and streamlining facets of healthcare, visualising developments to transport infrastructure or understanding inefficiencies in manufacturing and power supply amongst others. Big data should provide researchers with largely transparent information that due its size is relatively difficult to wholly manipulate. It allows for the specialisation of searches and data collation due to the vast quantities of available information which in turn should allow services, products and facilities to transcend their predecessors by way of visualising inadequacies and removing them from a given process.

Big data will increasingly begin to manifest in every sector that exists in our lives. Given how much data we produce it is clear that some sectors and individuals stand to gain more than others and as the opportunities and challenges that arise in the future modify our outlook, so too will this change.