Open – For Business – At the ASF

                                                  The Apache Software Foundation is about to celebrate an anniversary, and its extraordinary contribution to the economic refactoring of software stacks seems to be gaining more momentum with every passing year. After three Gartner Data and Analytics events on 3 continents with thousands of attendees in the past 4 weeks, I find myself more impressed than ever by the pervasive interest in and influence of open source software. I had several dozen one-on-one meetings with attendees (many, but not all, Gartner clients), and its appeal and impact on data management was reinforced again and again. Donald Feinberg and I noted in? State of the Open-Source DBMS Market, 2018?that

                                                  7.6% of DBMS revenue was attributable to OSDBMS-based offerings; a growth rate of 50% over the previous year in a broad market that grew 7.7%. This growth followed on the heels of a doubling in its size during the previous year.


                                                  January 2018 Hadoop Tracker

                                                  Last month’s update was obsolete before it published. This often happens because of multiple moving parts and?my extended gestation period. I needed to correct entries for both AWS and Hortonworks. The new Tracker is correct as far as I know as of January 2, 2018. Enjoy.


                                                  December 2017 Tracker – Where’s Hadoop?

                                                  The?leading 2017 story of Hadoop distributions?is that nobody seems to want to be accused of being in the business of providing them.?Some former champions?are expanding their shiny new positioning:?Cloudera?is selling Enterprise Data Hubs and Analytic?DBs;?Hortonworks?offers DataPlanes and Next-Gen Data Platforms;?MapR?touts the Converged Data Platform. In the cloud world, Amazon’s?EMR?is at least designed to “run and scale Apache Hadoop, Spark, HBase, Presto, Hive, and other Big Data Frameworks” while on Google’s?Cloud Platform?page the word Hadoop appears once inside the description of?Cloud Dataproc.


                                                  Hadoop Commercial Support Component Tracker – March 2017

                                                  Stack expansion has ground to a halt. The last time an Apache project was added to the list of those?most supported by leading Hadoop distribution vendors was July 2016, when Kafka joined the other 14 then commonly included. Since then, no broad support for new projects has emerged. The only project that does seem successful is the new e-scooter. With its new style and long lasting battery, it can′t fail.


                                                  Hadoop Project Commercial Support Tracker July 2016

                                                  There are now?15 projects supported by all 5?distributors I?track, and several have had new releases since April. Kafka is the newest addition, and I believe the remaining 4-supporter offerings, Mahout and Hue, will?remain unsupported by IBM, who has its own alternatives.


                                                  Hadoop Apache Project Commercial Support Tracker April 2016

                                                  There are now 19 commonly supported projects: Avro, Flume and Solr join the group supported by all 5 distributors and other changes appear as well.

                                                  For this version of the tracker (last updated in?December), I’ve made one sizable change: Pivotal has been dropped as a “leading distributor,” dropping the?number to five. Pivotal?relies on Hortonworks’ distro?(as does Microsoft) as its commercial offering now.


                                                  Strata Standards Stories: Different Stores For Different Chores

                                                  Has HDFS joined MapReduce in the emerging “legacy Hadoop project” category, continuing the swap-out?of components that formerly answered the question “what is Hadoop?” Stores for data were certainly a focus at Strata/Hadoop World in NY, O’Reilly’s well-run, well-attended, and always impactful fall event. The limitations of HDFS, including?its append-only nature, have become inconvenient enough to push the community to “invent” something DBMS vendors like Oracle did decades ago: a bypass. After some pre-event leaks about its arrival, Cloudera chose its Strata keynote to announce Kudu, a new columnstore written in C++, bypassing HDFS entirely. Kudu?will use an Apache license and will be submitted to the Apache process at some undetermined future time.


                                                  Now, What is Hadoop?

                                                  This?perennial question resurfaced recently in a thoughtful blog post by?Andreas Neumann, Chief Architect of Cask, called?What is Hadoop, anyway?. Ultimately, after a careful deconstruction of the terms in the question, Andreas concludes with

                                                  “Does it really matter to agree on the answer to that question? In the end, everybody who builds an application or solution on Hadoop must pick the technologies that are right for the use case.”

                                                  We’ve agreed?from the beginning – that is the only answer?that really matters. Still, the question continues to come up?for ?end users of the stack and for vendors like Cask (it helps them think about what to support in their application development offering Cask Data App Platform (CDAP).

                                                  Analysts too: I’ve discussed it?several times, including a post a year ago called?What Is Hadoop….Now? tracking the path?from 6 commonly supported projects in 2012 to 15 in June 2014, across a set of distributors that included Cloudera, Hortonworks, MapR and IBM. “Support” here means you pay for subscription that explicitly includes the named project.

                                                  This year, the expansion process?has continued – and it?does?matter.

                                                  –more on Gartner blog–



                                                  Perspectives on Hadoop Part Two: Pausing Plans

                                                  By Merv Adrian and Nick Heudecker?

                                                  In the?first post?in this series?, I looked at the size of revenue streams for RDBMS software and maintenance/support and noted that they amount to $33B, pointing out that pure play Hadoop vendors had a high hill to climb. (I didn’t say so specifically, but in 2014, Gartner estimates that the three leading vendors generated less than $150M.)

                                                  In this post, Nick and I turn from Procurement to Plans and examine the buying intentions uncovered in Gartner surveys.


                                                  –more in Gartner blog–

                                                  Perspectives on Hadoop: Procurement, Plans, and Positioning

                                                  I have the privilege of working for the world’s?leading information technology research and advisory company, covering information management with a strong focus for the past few years on an emerging software stack called Hadoop. In the early part of 2015, that particular technology is moving from early adopter status to early majority in its marketplace adoption. The discussions and published work around it have been exciting and controversial, so in this post (and a couple to follow) I describe three interlocking?research perspectives on Hadoop: procurement (counting real money actually spent); plans (surveys of intentions to invest) and positioning (subjective interpretations of what the first two mean.)

                                                  Procurement Perspective: Hadoop is a (Very)?Small?Market Today

                                                  –more on Gartner blog–



                                                                                                  Super League



                                                                                                  Foreign exchange


                                                                                                  Mobile Games