It will soon be essential for organisations to not only keep up with their fast-growing data, but get ahead of it.

More than 85% of C-level executives believe faster data analytics directly relates to growing the company’s bottom line, according to new survey data from Ocient, the leading hyperscale data analytics solutions company serving organisations that derive value from analysing trillions of data records in interactive time.

The report, Beyond Big Data: The Rise of Hyperscale, is based on a survey of 500 data and technology leaders, across a variety of industries, who are managing active data workloads of 150 terabytes or more.

Extraordinary data growth

Data is growing at an extraordinary rate.

According to John Rydning, research vice president of the IDC Global DataSphere, a measure of how much new data is created, captured, replicated, and consumed each year, is that "The Global DataSphere is expected to more than double in size from 2022 to 2026. The Enterprise DataSphere will grow more than twice as fast as the Consumer DataSphere over the next five years, putting even more pressure on enterprise organisations to manage and protect the world's data while creating opportunities to activate data for business and societal benefits."

IDC Global DataSphere research also documented that “in 2020, 64.2 zettabytes of data was created or replicated” and forecast that “global data creation and replication will experience a compound annual growth rate (CAGR) of 23% over the 2020-2025 forecast period.” At that rate, more than 180 zettabytes — that’s 180 billion terabytes — will be created in 2025.

The survey respondents reflect forecasts of such exponential data growth. When asked how fast the volume of data managed by their organisation will grow over the next one to five years, 97% of respondents answered “fast” to “very fast,” with 72% of C-level executives expecting the volume to grow “very fast” over the next five years.

Barriers to supporting data growth and hyperscale analytics

To support such tremendous data growth, 98% of respondents agreed it's somewhat or very important to increase the amount of data analysed by their organisations in the next one to three years. However, respondents are experiencing barriers to harnessing the full capacity of their data and cited these top three limiting factors:

• The volume of data is growing too fast (62% total, 65% C-level)
• There is a lack of talent to analyse the data (49% total, 47% C-level)
• Current solutions are not flexible enough (49% total, 34.8% C-level)

When asked about their biggest data analysis pain points today, security and risk ranked first among C-level respondents (68%), with metadata and governance (41%) and slow data ingestion (31%) being two other top concerns. When scaling data management and analysis within their organisation, 63% said maintaining security and compliance as data volume and needs grow was a challenge they are currently facing.

Survey respondents also indicated legacy systems are another source of pain and a barrier to supporting data growth and hyperscale analytics. When asked if they plan to switch data warehousing solutions, more than 59% of respondents answered “yes,” with 46% of respondents citing a legacy system motivating them to switch. When ranking their most important considerations in choosing a new data warehouse technology, “modernising our IT infrastructure” was ranked number one.

Faster data analytics improve decisions, revenue, and success

The survey respondents believe hyperscale data analytics is crucial to their success. Sixty-four percent of respondents indicate hyperscale data analytics provides important insights used to make better business decisions, and 62% said it is essential for planning and strategy.

The survey respondents also indicated there is a strong relationship between implementing faster data analytics and growing the company’s bottom line. When asked about this relationship, an overwhelming 78% of respondents agreed there is a definite relationship. For the C-level audience, more than 85% cited the relationship.

“Data analysis is no longer a ‘nice-to-have’ for organisations. Hyperscale data intelligence has become a mission-critical component for modern enterprises and government agencies looking to drive more impact and grow their bottom line. With the rapid pace of growth, it’s imperative for enterprises and government agencies to enhance their ability to ingest, store, and analyse fast-growing data sets in a way that is secure and cost-effective,” said Chris Gladwin, co-founder and CEO, Ocient. “

"The ability to migrate from legacy systems and buy or build new data analysis capabilities for rapidly growing workloads will enable enterprises and government organisations to drive new levels of agility and growth that were previously only imaginable.”

The report intends to uncover key trends around how organisations are managing the shift from big-data volumes toward ingesting, storing, and analysing hyperscale data sets, which include trillions of data records, and the expected technical requirements and business results from that shift.

The survey was conducted in May 2022 by Propeller Insights. Respondents include partners, owners, presidents, C-level executives, vice presidents, and directors in many industries including technology, manufacturing, financial services, retail, and government. Their organisations' annual revenue ranges from $50 million to more than $5 billion. Approximately 50% of respondents represent companies with annual revenue greater than $500 million.