Seven Big Data Trends for 2014

Dr Mark van Rijmenam, CSP -

As the year 2013 is almost over, it is good to have a look back at the investments made in Big Data startups from around the world, to the growing list of Big Data best practices that appeared. But what happened to the trends that I foresaw in 2013 and what will be the most important Big Data trends for 2014?

Last year I noted that on on-the-go Big Data, meaning being able to view Big Data visualizations on mobile devices, will become important and in 2013 we saw the rise of a bunch of new mobile devices including smart watches and Google Glass. There are some Big Data startups, such as Roambi, who have very clear understanding of on-the-go Big Data and are capable of bringing real-time interactive visualizations to mobile devices. On-the-go Big Data really took off in 2013 and is here to stay.

The second trend was that Big Data does not require big bucks because of the plethora of Big Data open source tools that are available in the market as well as the decreasing costs of storage. The price of storage does indeed continue to decrease in costs, but the amount of data also grows exponentially. Will we be able to keep up with this or will the amount of data outgrow the available storage? The amount of open source tools is growing rapidly, but there is also a rise in licensed Big Data solutions, because open source tools do require experience Big Data personnel and many organisations do not yet have these staff available. So, to start with Big Data it does not have to cost the world, but to develop and implement a complete Big Data solution can be expensive, although the results can also be significant.

The third trend was big real-time data and 2013 did indeed show an important growth in real-time analytics. More and more tools become available that create a layer on top of Hadoop to be able to deal with real-time data and Hadoop 2.0's YARN framework enables real-time data analysis. In the coming years this will become more important as many industries see the advantages of real-time analytics.

Big consumer data, or the quantified-self movement, was the fourth trend and this really took off in 2013. Wearable technologies that can measure every day life have started to appear massively and more and more consumers are measuring at least something of their behaviour, be it their sleeping patterns or the running results. Recent research from Pew Research Centre revealed that 69% of U.S. adults keep track of at least one health indicator such as weight, diet, exercise routine or symptom.

The final trend was Big Data related to privacy. Last year I wrote that “it feels like we have fallen in love with Big Data and that we are blind for the pitfalls in Big Data. We do not want to see the backside of Big Data and the effect it will have on our privacy.” Of course, in 2013 we saw the PRISM leak by Edward Snowden, which showed that privacy in the Big Data world is indeed an endangered species and there is probably a lot more to be revealed in the coming months. Privacy is indeed affected by Big Data and as consumers, and companies, we will have to get used to this new reality.

So, what can we expect for the coming year? First of all, as IDC already predicted, the Big Data market will grow to $ 16.1 billion in 2014 and they also forecast that the Big Data technology and services market will continue to grow at a 27% CAGR to $ 32.4 billion through 2017. So, although the five trends discussed last year will continue to affect your Big Data strategy, for 2014 I foresee different areas within Big Data that will take off. Let’s have  a look at the seven Big Data trends for 2014.

The rise of the Industrial Internet

The first trend I foresee is the rise of the Industrial Internet that will affect the industrial sector dramatically. The next year, machine-to-machine data will grow significantly and continue to do so in the years after. By 2020, 40% of all data will come from sensor data and it will unlock a $ 1 trillion global market in 2020 (currently it is a $ 121 billion global market). In addition, GE reports that the Industrial Internet could add $10 to $15 trillion to global GDP in the coming years.

These sensors will completely change the way companies, factories and supply chains will be operated and managed. Technology is transforming the industrial sector, creating machines that can see, feel, sense and react, so they can be operated far more efficiently. Already there are some great examples of how this will affect companies, ranging from airline companies that can reduce turn-around time with monitoring the plane during the flight, to analyzing many different variables to pick the best places to locate wind turbines around the world to be able to harvest the most energy at the lowest costs. As sensors and storage are becoming cheaper every day, algorithms are becoming better and organisations more and more see the need for smart factories, the Industrial Internet will really take off in 2014.

It is going to be cloudy: Big-Data-as-a-Service solutions

We already see more and more Big Data startups that are creating a Big-Data-as-a-Service solution to help organisations apply Big Data without the heavy costs involved. Especially useful for Small or Medium sized enterprises who do want to develop a data-driven information-centric organisation, but who do not have the capacity to develop and maintain a full-fledge Big Data solution on premises. Big-data-as-a-Service is a combination of Analytics-as-a-service, Infrastructure-as-a-Service and Data-as-a-Service and it will spur the adoption of Big Data also by smaller and medium sized organisations.

IIA calls this adoption “ready-made analytics in the cloud”. These solutions offer an attractive alternative for organisations that want to start with Big Data or want to easily scale existing programs. Gartner predicts that cloud computing will become the bulk in IT spending by 2016 and 2014 will be a turning point in the acceptance of the cloud as part of a Big Data strategy.

Security to protect the privacy

If there is on thing that the NSA documents have shown, governments from around the world have almost unprecedented access to data files from organisations and consumers. Organisations will start to focus more heavily on securing their data to protect the privacy of their customers. The first signs for this are that tech companies call for 'aggressive' NSA reforms at the White House meeting in December 2013. Executives from 15 companies expressed their concern that the NSA’s wide-ranging surveillance activities had undermined the trust of their users. Of course, a reform of the NSA activities is one side of the coin; the other side will be increased security measures by the companies to protect their data.

More and more organisations will start to use Big Data techniques to secure their IT infrastructure and prevent from being hacked and have data monitored or stolen. Log data will form an important aspect in this and organisations will start to see the importance in monitoring and analysing their IT infrastructure log data in order to keep their infrastructure and data safe. This will help to restore and keep the trust of their customers.

Personalization will become personal

Consumers are creating massive amounts of data through every click, like, tweet, cell-phone call, purchase and self-tracking applications they use. Companies like Amazon have already used these kinds of data for many years to create a personal online shopping experience with recommendations, personal homepages, personal discounts or personally targeted mass-email campaigns. However, in 2014 more organisations will also start to see the value in such a personalized approach, be it online or offline.

A good example is the Australian shoe retailer Shoes of Prey, who have developed an analytics system that enables them to look at individual customer-spend and profitability, and allows them to begin upselling based on the fashion tastes of its clients.

Personalization is making a giant leap forward in the coming years and 2014 could very well be the inflection point in the offering of personalized offers and the acceptance of it by consumers. Consumers will start to see that their data is valuable and they do want something in return for providing their data. So consumers are willing to cooperate and share their data if it brings them personalized discounts.

Education will be essential for success

As more organisations are trying to understand Big Data and preparing their staff for the Big Data era, education becomes a crucial aspect. Already in 2011, McKinsey  predicted a shortage in the coming years of Big Data scientists and Big Data managers. Organisations will therefore stimulate their employees to be more Big Data skilled. Many organisations are heading for a major skill gap and will have to take action to be ready for the big data era. Also fresh-graduates or students will see the Big Data trend and in the competitive jobs market will feel the need to differentiate to stand a chance on the job market.

Therefore in 2014 we will see a steep increase in the available online and offline big data courses. Apart from the online universities such as Coursera or Udacity, many more universities from around the world will start offering a big data course or program. These courses or programs will range from Big Data strategy courses to deep analytics and machine-learning programs for Big Data scientists and anything in between to cater for the massive increase in Big Data students.

Big data moves into mixed data

In the past years Big Data was all about obtaining as much data as possible and the perception was that you require massive datasets to gain insights from those data sources. In the coming year however, organisations will start to see that the most important aspect of Big Data is not so much the volume of a dataset, but more the insights derived from combining several, smaller, datasets. Organisations that do not have exabytes or petabytes can still obtain very valuable insights with smaller, but more, data sets. Of course, more data does mean more accurate insights but it does not per se mean more insights. In the coming year, more organisations will understand this and take their first steps into the direction of Big Data. They will start mixing and combining several data sets that they will analyse to derive insights. So in 2014 Big Data will become mixed data.

It’s time for a Proof of Concept


The past years we saw a lot of talk around Big Data. More conferences, new books, more Big Data startups and more interesting best practices are being shared and distributed online and offline. In 2014 many more organisations will start working towards a Big Data strategy and start developing a Proof of Concept (PoC) to investigate what Big Data can mean for them. The PoC will help organisations gain a better understanding of Big Data, will help them get educated and will help them to be better able to predict the ROI of future Big Data projects. A Proof of Concept is a vital part of a Big Data strategy when you start with Big Data.

As Alec Gardner, industry consulting director at Teradata Australia and New Zealand, states: "More companies will roll up their sleeves and 'have a try' by starting to bring in new data sources and technologies to answer new business questions and gain insight either not possible or too difficult previously". This will be a very important aspect as it will be the ignition required to get the Big Data engine started.

The above-described seven Big Data trends for 2014 will be an important aspect of Big Data in 2014 and organisations will have to ensure that the Big Data strategy that they develop is geared towards the exponential growth of data that can be expected in the coming years. 2013 was already exciting as we saw many things happening, but 2014 will be truly interesting as Big Data will become common language in boardrooms and organisations start to experiment with it.

Image Credit: Horoscope/Shutterstock