7 Important Big Data Trends for 2016

It is the end of the year again and that means it is time for the Big Data trends for next year. I did that for 2014, I did it for 2015 and now it is time for 2016. What is awaiting us in 2016? Which Big Data trends will have an impact on the global Big Data domain? How will Big Data affect organizations in 2016? Let’s have a look at seven of the most important Big Data trends for the year 2016.

1. The Rise of the Algorithms

Big Data is out, Algorithms are in. Data has become a commodity and every organization is capable of collecting and storing vast amounts of data. Analysing all that data is also not so spectacular anymore. Every organization can hire or train Big Data Analysts to understand the patterns within the data.

In 2016 it will be all about what actions you will derive from the data you have access to. Bring in the algorithms. Algorithms define action and they are very specific pieces of software that are very good at a very specific action, much better than humans can do. Think for example of quickly determining the right advertisement based on your profile when you visit a website or finding an outlier in vast amounts of transaction data to determine fraud.

These algorithms are very specific artificial intelligence and cannot be compared to Generic Artificial Intelligence that is still years, if not decades, away. However, very specific AI is already here and in 2016 we will see the rise of the Algorithmic Business.

2. Data-Lake-as-a-Service Solutions

In 2015 we saw the appearance of data lakes. Data lakes are becoming an essential big data storage tool as enterprises amass mountains of data from M2M connections, social networks, and remote workforces.

According to Gartner, “By 2020, information will be used to reinvent, digitalize or eliminate 80 per cent of business processes and products from a decade earlier.” And contrary to legacy storage solutions where we still see that the data lives in silos, data lakes allow raw, unvarnished bytes of information to live in one place where they can be integrated and analyzed for patterns. Data lakes are exactly what you need to digitize your business and become a data-driven business as Gartner expects organizations to be in 2020.

Since a data lake comes with quite a few challenges, in 2016 we will see the future of managed Data Lakes: Data-Lake-as-a-Service Solutions – a complete managed solution for your data lake.

Data-Lake-as-a-Services will offer active storage solutions that can ingest large volumes of structured and unstructured data, and make them available for processing by a multitude of applications, including enterprise data warehouses or open source technologies such as Apache Hadoop or Spark. An organization looking to use such a Data-Lake-as-a-Service will probably be paying a few cents per Gigabyte per month, which will include the total offering.

In 2016 we will see more and more Big Data vendors offer such a solution; to offer companies a complete, easy-to-use, scalable solution without all the hassle of having to arrange a data lake yourself. Data-Lake-as-a-Service solutions will be used by many organizations, due to the advantages of a data lake for storing, and analyzing, massive amounts of data. Especially smaller organizations, like for example startups in the Internet of Things domain, will benefit from a Data-Lake-as-a-Service as they get all the benefits of a data lake, without all the hassle of building and maintaining a data lake.

3. Blockchain Will Become Accepted in Different Industries

The past years, we have seen Blockchain primarily linked to Bitcoin, but the Blockchain technology offers a lot more possibilities. In 2016 we will, therefore, see that multiple industries will adopt Blockchain.

A blockchain can be seen as a public ledger, or record, of any digital events. The public ledger is shared between many different parties, computing nodes that are geographically and computationally isolated, and a record can only be updated if there is a consensus of the majority of the participants in the system. As soon as the new information is entered in the ledger, it cannot be erased and it is visible to anyone. With blockchain, there is complete transparency to what is entered in the ledger.

A crucial advantage of the blockchain is that the system is completely transparent, anyone can see which transactions are entered in the ledger, without compromising privacy. You can record the fact that an event occurred, and even that it occurred correctly, without revealing personal details about the parties involved.

While most people link blockchain to the cryptocurrency Bitcoin, there are a lot more possibilities. Especially the financial industry will embrace the blockchain technology. Many of the large banks around the world are experimenting with blockchain and/or are investing in blockchain startups. UBS has created a blockchain lab, Santander is investigating how to use blockchain for their lending activities, Goldman Sachs has invested in a blockchain startup and a there is a large consortium, the R3 ’s global bank partnership, that investigates the potential of the blockchain.

However, in 2016 we will see more applications in different industries appear that use blockchain. Basically, any industry where digital transactions take place can benefit from the blockchain technology, ranging from the financial industry, the legal industry, real estate, notaries, gambling, publishing to data storage. The coming year, a wider adoption of the blockchain is imminent.

4. People Analytics to Keep Talent Engaged

For most organizations, their talent is the most important asset and for most senior executives, talent is a top priority. According to a research by PWC, 34% of US CEOs are “extremely concerned” about the availability of key skills within their organization. As a result, senior executives are looking for hard data on their people and therefore, in 2016 we will see that people analytics is taking a big step forward.

People analytics is a new domain in the HR department, but growing rapidly because of the need to better drive the ROI in people. People analytics can be defined as a big data technology that uses snippets of people-related data to optimize business outcomes and solve business problems, is, therefore, growing in importance.

People analytics can help to answer questions such as: Do we have the right skills mix in our organization? How engaged are our employees and especially the top performers? Can we better predict who our future leaders will be? What is the state of mind of our employees? And many more.

In an overheated market, where the race for talent is on and where the best big data scientists and analysts are scarce and expensive and finding talent is difficult, it becomes more and more important for organizations to understand what drives their employees and to engage with them. As a result, in 2016 more organizations will dedicate employees to people analytics and the number of startups in these field of expertise will grow rapidly.

5. Smart Governments Improving Societies and Citizen’s Experience

For the largest commercial organizations, big data is already common language. Governments are traditionally slow in adopting new trends, but in 2016 we will see more national, regional and local governments embrace big data technology to improve their society and the citizen experience.

Governments are experimenting with Big Data techniques to improve citizen experience management, create frictionless transactions, improve governmental performance with government analytics and bring data-driven decision-making to frontline employees. An eGovernment, or smart government, will significantly contribute to these objectives and in 2016 we will see more governments around the globe to develop a smart government.

Already we see some great examples. The Dubai Authorities are working hard on transforming their government into a smart government. They started with the process to improve the customer, i.e. citizen, experience and drive the knowledge economy. They have created a Secure Single Sign-on to dozens of smart government services and a multitude of services are available via mobile applications.

The best example of a smart government is that of Estonia. The small Baltic state with only 1.3 million citizens has been named by the United Nations as having the decade’s best e-Government content. Every interaction with, and within, the Estonian government happens digitally and Estonians have complete control over their own data. In addition pa,rliament is going paperless, laws are signed electronically and businesses operate completely electronically; paying your taxes is radically simple because all services are interlinked.

Although the Estonian government is well ahead of many other governments, this is a process not to be stopped. In The Netherlands, the national government aims to work completely digital by 2017, from getting in touch with the government to paying taxes.

In the coming year, we will therefore see more governments from around the world develop smart solutions. We will also see more governments opening up their data sets and work with open APIs to enable startups and enterprises to easily connect with governments. This will speed-up the process of becoming a smart government even more.

6. Increased Big Data Security and Data Breaches

With everything going digital and the Internet of Things connecting the unconnected to the Internet, big data security will become more and more important. In the past years, we have seen a multitude of massive data breaches including the Ashley Madison hack and the TalkTalk hack.

Basically, any organization can and will be hacked in the future and if you are not hacked, you are simply not important enough.  Organizations should therefore focus on not only preventing a security breach, but also on implementing the right crisis plans when a hack has occurred.

In 2016 we will see more data breaches reach the news, we will see more screw-ups by organizations that try to cover up and we will see more attacks on physical products due to the Internet of Things. Especially the latter can have a profound effect on data security. After all, we have already seen that hackers remotely killed a Jeep while driving on the highway.

Therefore, in 2016 we will see more scrutiny on how organizations actually deal with their data security; from before a hack, during a hack and after a data breach. Organizations will increase their spending on security, they will work more with ethical hackers to improve their data security and they will improve their internal processes to make employees more aware of hackers. After all, quite often humans are the weakest link in a company’s security protocol.

7. Fog Analytics Takes Off Due to Smart Machines

Fog computing is gaining a lot of traction, rapidly. Fog computing refers to facilitating the storage, transfer and compute between end-devices linked to the Internet of Things and cloud computing that stores the data. With the advance of the Internet of Things, fog computing is gaining momentum as sensors are becoming so sophisticated that they can now collect massive amounts of data.

Imagine you have a network of connected devices that creates large amounts of data in real-time. Sending the data back-and-from the devices to the cloud can become too expensive and taking up too much time. Enter fog computing, or fog analytics. Fog analytics enables smart machines to perform part of the analytics locally and only send the prepared data to the cloud.

According to Gartner, smart machines are the new reality. Therefore, in the coming year we will see more smart machines with more and more sophisticated sensors that collect massive amounts of data. Organizations will have to turn to fog analytics to make the data manageable, keep the insights useable and reduce the costs as much as possible.

An Exciting Year Ahead

2016 will be an exciting year in terms of Big Data. Smart algorithms will take over the business to perform many actions now done by humans. We will see the appearance of data-lake-as-a-service solutions to help companies do more with their data with less work. Multiple industries will start to experiment with blockchain technology in order to disrupt their own industry.

Organizations will turn to people analytics in order to better engage their employees and win the battle for scarce talent. Governments will finally see the benefits of Big Data and become smart, while organizations and governments have to be aware of being hacked and take appropriate measurements. Finally, fog analytics will take off due to smart machines that will appear in every industry.

Images: Shutterstock