Open – For Business – At the ASF

                                                  The Apache Software Foundation is about to celebrate an anniversary, and its extraordinary contribution to the economic refactoring of software stacks seems to be gaining more momentum with every passing year. After three Gartner Data and Analytics events on 3 continents with thousands of attendees in the past 4 weeks, I find myself more impressed than ever by the pervasive interest in and influence of open source software. I had several dozen one-on-one meetings with attendees (many, but not all, Gartner clients), and its appeal and impact on data management was reinforced again and again. Donald Feinberg and I noted in? State of the Open-Source DBMS Market, 2018?that

                                                  7.6% of DBMS revenue was attributable to OSDBMS-based offerings; a growth rate of 50% over the previous year in a broad market that grew 7.7%. This growth followed on the heels of a doubling in its size during the previous year.


                                                  January 2018 Hadoop Tracker

                                                  Last month’s update was obsolete before it published. This often happens because of multiple moving parts and?my extended gestation period. I needed to correct entries for both AWS and Hortonworks. The new Tracker is correct as far as I know as of January 2, 2018. Enjoy.


                                                  December 2017 Tracker – Where’s Hadoop?

                                                  The?leading 2017 story of Hadoop distributions?is that nobody seems to want to be accused of being in the business of providing them.?Some former champions?are expanding their shiny new positioning:?Cloudera?is selling Enterprise Data Hubs and Analytic?DBs;?Hortonworks?offers DataPlanes and Next-Gen Data Platforms;?MapR?touts the Converged Data Platform. In the cloud world, Amazon’s?EMR?is at least designed to “run and scale Apache Hadoop, Spark, HBase, Presto, Hive, and other Big Data Frameworks” while on Google’s?Cloud Platform?page the word Hadoop appears once inside the description of?Cloud Dataproc.


                                                  IBM Ends Hadoop Distribution, Hortonworks Expands Hybrid Open Source

                                                  IBM has followed Intel and EMC/Pivotal in abandoning efforts to make a business of Hadoop distributions, and followed Microsoft in making Hortonworks its supplying partner. At the former Hadoop Summit, now called Dataworks (itself a sign of the shift from Hadoop-centric positioning), IBM announced it will discontinue its IBM Open Platform/BigInsights offering, and will instead OEM Hortonworks’ HDP.


                                                  Hadoop Commercial Support Component Tracker – March 2017

                                                  Stack expansion has ground to a halt. The last time an Apache project was added to the list of those?most supported by leading Hadoop distribution vendors was July 2016, when Kafka joined the other 14 then commonly included. Since then, no broad support for new projects has emerged. The only project that does seem successful is the new e-scooter. With its new style and long lasting battery, it can′t fail.


                                                  Hadoop Project Commercial Support Tracker July 2016

                                                  There are now?15 projects supported by all 5?distributors I?track, and several have had new releases since April. Kafka is the newest addition, and I believe the remaining 4-supporter offerings, Mahout and Hue, will?remain unsupported by IBM, who has its own alternatives.


                                                  Hadoop Apache Project Commercial Support Tracker April 2016

                                                  There are now 19 commonly supported projects: Avro, Flume and Solr join the group supported by all 5 distributors and other changes appear as well.

                                                  For this version of the tracker (last updated in?December), I’ve made one sizable change: Pivotal has been dropped as a “leading distributor,” dropping the?number to five. Pivotal?relies on Hortonworks’ distro?(as does Microsoft) as its commercial offering now.


                                                  Strata Standards Stories: Different Stores For Different Chores

                                                  Has HDFS joined MapReduce in the emerging “legacy Hadoop project” category, continuing the swap-out?of components that formerly answered the question “what is Hadoop?” Stores for data were certainly a focus at Strata/Hadoop World in NY, O’Reilly’s well-run, well-attended, and always impactful fall event. The limitations of HDFS, including?its append-only nature, have become inconvenient enough to push the community to “invent” something DBMS vendors like Oracle did decades ago: a bypass. After some pre-event leaks about its arrival, Cloudera chose its Strata keynote to announce Kudu, a new columnstore written in C++, bypassing HDFS entirely. Kudu?will use an Apache license and will be submitted to the Apache process at some undetermined future time.


                                                  Now, What is Hadoop?

                                                  This?perennial question resurfaced recently in a thoughtful blog post by?Andreas Neumann, Chief Architect of Cask, called?What is Hadoop, anyway?. Ultimately, after a careful deconstruction of the terms in the question, Andreas concludes with

                                                  “Does it really matter to agree on the answer to that question? In the end, everybody who builds an application or solution on Hadoop must pick the technologies that are right for the use case.”

                                                  We’ve agreed?from the beginning – that is the only answer?that really matters. Still, the question continues to come up?for ?end users of the stack and for vendors like Cask (it helps them think about what to support in their application development offering Cask Data App Platform (CDAP).

                                                  Analysts too: I’ve discussed it?several times, including a post a year ago called?What Is Hadoop….Now? tracking the path?from 6 commonly supported projects in 2012 to 15 in June 2014, across a set of distributors that included Cloudera, Hortonworks, MapR and IBM. “Support” here means you pay for subscription that explicitly includes the named project.

                                                  This year, the expansion process?has continued – and it?does?matter.

                                                  –more on Gartner blog–



                                                  Perspectives on Hadoop Part Two: Pausing Plans

                                                  By Merv Adrian and Nick Heudecker?

                                                  In the?first post?in this series?, I looked at the size of revenue streams for RDBMS software and maintenance/support and noted that they amount to $33B, pointing out that pure play Hadoop vendors had a high hill to climb. (I didn’t say so specifically, but in 2014, Gartner estimates that the three leading vendors generated less than $150M.)

                                                  In this post, Nick and I turn from Procurement to Plans and examine the buying intentions uncovered in Gartner surveys.


                                                  –more in Gartner blog–



                                                                                                  Second-hand housing



                                                                                                  Second-hand housing

                                                                                                  Mobile phone