Public distributed ledgers, including blockchain, continue to have high visibility, although organisations remain cautious about the future of public (permission-less) distributed ledger concepts due to scalability, risk and governance issues.
As data becomes ever more distributed across multiple systems, organisations have to cope with increasingly complex ecosystems and digital business requirements.
"Data management continues to be central to the move toward digital business. As requirements change within the architecture of the organisation and place greater demands on underlying technology, the maturity and capability of many of the technologies highlighted in the hype cycle will advance rapidly," says Donald Feinberg, vice president at Gartner, as he reports on this year's Hype Cycle for Data Management.
Gartner’s Hype Cycle for Data Management helps CIOs, chief data officers (CDOs) and other senior data and analytics leaders understand the maturity of the data management technologies they are evaluating to provide a cohesive data management ecosystem in their organisations.
"Recent years have seen many new additions to the Hype Cycle, including in-memory, cloud, data virtualisation, advanced analytics, data as a service, machine learning, graph, non-relational and Hadoop," he says.
The hype cycle for data management
The analyst firm says two technologies are of particular interest, in that they show the impact cloud computing is having on the data management discipline. Hadoop distributions are deemed to be obsolete before reaching the Plateau of Productivity because the complexity and questionable usefulness of the entire Hadoop stack is causing many organisations to reconsider its role in their information infrastructure. Instead, organisations are looking at increasingly competitive and convenient cloud-based options with on-demand pricing and fit-for-purpose data processing options.
As part of the same cloud-led trend, SQL interfaces to cloud object stores have appeared at the Innovation Trigger stage. "We expect these interfaces to represent the future of cloud database Platform as a Service (Paas) and reach the Plateau within two to five years because they are the focus of most cloud vendors and products in this space," says Feinberg.
"They enable organisations to interact with data stored in the cloud, using a familiar SQL syntax. Object stores are well suited to storing large volumes of multi-structured data, typical of data lakes."
Of the 35 other technologies highlighted on the 2017 Hype Cycle for Data Management, four are judged to be “transformational”, says Gartner.
Two — event stream processing (ESP) and operational in-memory database management system (IMDBMS) — are expected to reach the Plateau of Productivity within two to five years, while both blockchain and distributed ledgers are expected to take five to 10 years.
Event stream processing
ESP is one of the key enablers of digital business, algorithmic business and intelligent business operations. ESP technology, including distributed stream computing platforms (DSCPs) and event processing platforms (EPPs), is maturing rapidly. Stream analytics provided by ESP software improves the quality of decision-making by presenting information that could otherwise be overlooked.
Operational in-memory DBMS
Operational In-memory database management systems (IMDBMS) technology is maturing and growing in acceptance, although the infrastructure required to support it remains relatively expensive. Another inhibitor to the growth of operational IMDBMS technology is the need for persistence models that support the high levels of availability required to meet transaction SLAs. Nevertheless, operational IMDBMSs for transactions have the potential to make a tremendous impact on business value by speeding up data transactions 100 to 1,000 times.
Public distributed ledgers, including blockchain, continue to have high visibility, although organisations remain cautious about the future of public (permission-less) distributed ledger concepts due to scalability, risk and governance issues. Most business use cases have yet to be proven and extreme price volatility in bitcoin persists. Presupposing the technical and business challenges of distributed ledgers can be overcome; in the short term, organisations are most likely to use distributed ledger for operational efficiency gains via the use of shared information and infrastructure. Longer term, Gartner expects a complete reformation of whole industries and commercial activity as the programmable economy develops and ledgers contribute to the monetisation of new ecosystems.
The requirements for more standards and enterprise-scale capabilities are evolving slowly, but distributed ledgers are still not adoptable in a mission-critical at-scale context. Their value propositions, compared with existing technology, are also not clearly established, making the widespread acceptance of the technology problematic. Private distributed ledger concepts are gaining traction, because they hold the promise to transform industry operating models and overcome some of the issues of scalability, risk management and governance that plague public ledgers. As with blockchain, however, many business use cases are unproven at this time.
Send news tips and comments to firstname.lastname@example.org
Follow Divina Paredes on Twitter: @divinap
Follow CIO New Zealand on Twitter:@cio_nz
Join the CIO New Zealand group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.