“Companies invest in big data because of fear of disruption.

Organisations are increasingly investing in big data and artificial intelligence. The main reason for this is the fear of disruption, and not the search for the highest operational efficiency. Silicon Angle reports that a survey of Fortune 1,000 companies by NewVantage Partners shows this.

The NewVantage survey has been conducted every year since 2011. The research covers only 64 companies, but includes some of the largest banks, insurance companies and life sciences companies in the world. 97 percent of the respondents have a management position.

Almost all enterprises invest in big data. 90 percent of companies in the research now invest more in big data and AI projects, and 75 percent say this is because of fear of disruption of data-driven competitors. Two years ago, only 47 percent said this was their main motivation. “We’re in a disruptive time and big data, AI and machine learning are in the middle of that change,” said Randy Bean, CEO of NewVantage and author of the study.

Data culture

The problem, however, is that companies are not seeing any major changes. Most organisations have difficulty with the resistance of people and companies to the new technology. 62 percent say they get measurable results from their investments, just under half say they are fighting in the field of data and analytics. 28% say they have a data-driven culture.

There are several factors that prevent organisations from achieving their potential in the field of big data and AI. The biggest problems are in the culture. 95 percent of the respondents said that the problems occur with people and processes. Only 5 percent said that technology is causing problems.

Bean and co-author Tom Davenport argue that the implications of this are great, because only a small number of companies can afford not to focus on data in the future. “Companies that are unable to develop a data culture run the risk of disappearing, both in the long term and suddenly,” says Bean.

CDO

Attempts by companies to transform themselves around data, often fail for the same reasons, according to Bean. “People can only absorb so much information and only adapt to the extent that it concerns new approaches to traditional business processes. That’s where the transformation attempts fail.”

In addition, the study shows problems in how managers perceive the role of the chief data officer. Respondents gave very different answers about what that function should contain and who should take on the role. For example, 38 percent said that the CDO should be an external employee, while 32 percent would prefer someone from within the company. Nearly half think that the CDO is mainly responsible for data, 28 percent say that not one person should be responsible.

68 percent of the respondents said they had a CDO, but one in five managers said that the role was interim or unnecessary.