Our 2013 Database survey is now live

451 Research’s 2013 Database survey is now live at http://bit.ly/451db13 investigating the current use of database technologies, including MySQL, NoSQL and NewSQL, as well as traditional relation and non-relational databases.

The aim of this survey is to identify trends in database usage, as well as changing attitudes to MySQL following its acquisition by Oracle, and the competitive dynamic between MySQL and other databases, including NoSQL and NewSQL technologies.

There are just 15 questions to answer, spread over five pages, and the entire survey should take less than ten minutes to complete.

All individual responses are of course confidential. The results will be published as part of a major research report due during Q2.

The full report will be available to 451 Research clients, while the results of the survey will also be made freely available via a
presentation at the Percona Live MySQL Conference and Expo in April.

Last year’s results have been viewed nearly 55,000 times on SlideShare so we are hoping for a good response to this year’s survey.

One of the most interesting aspects of a 2012 survey results was the extent to which MySQL users were testing and adopting PostgreSQL. Will that trend continue or accelerate in 2013? And what of the adoption of cloud-based database services such as Amazon RDS and Google Cloud SQL?

Are the new breed of NewSQL vendors having any impact on the relational database incumbents such as Oracle, Microsoft and IBM? And how is SAP HANA adoption driving interest in other in-memory databases such as VoltDB and MemSQL?

We will also be interested to see how well NoSQL databases fair in this year’s survey results. Last year MongoDB was the most popular, followed by Apache Cassandra/DataStax and Redis. Are these now making a bigger impact on the wider market, and what of Basho’s Riak, CouchDB, Neo4j, Couchbase et al?

Additionally, we have been tracking attitudes to Oracle’s ownership of MySQL since the deal to acquire Sun was announced. Have MySQL users’ attitudes towards Oracle improved or declined in the last 12 months, and what impact will the formation of the MariaDB Foundation have on MariaDB adoption?

We’re looking forward to analyzing the results and providing answers to these and other questions. Please help us to get the most representative result set by taking part in the survey at http://bit.ly/451db13

CouchDB – sink or swim?

CouchDB – up a creek without a paddle? Image source: bobbyfeind on Flickr

Almost a year ago Apache CouchDB creator Damien Katz announced that he would no longer be contributing to the CouchDB document database project he had created, choosing instead to focus on the development of Couchbase Server 2.0, which united CouchDB with Membase Server.

While the abandonment of an open source project by the person that created it is by no means unprecedented it is still unusual enough to warrant a look at what has happened to CouchDB in the year that followed.

Surviving or thriving?

The first point to make is that the survival of CouchDB following Katz’s departure was never in doubt, thanks to the fact that it is an Apache Foundation project. One of the benefits of the foundation model is that it doesn’t depend on a dominant developer or vendor to keep a project moving forward.

Although it briefly appeared that Cloudant would fulfil the role of the major corporate backer of CouchDB with its BigCouch clustered CouchDB technology after Couchbase discontinued its own CouchDB distribution, the company instead refocused its attention on its CouchDB- and BigCouch-based managed service.

While developers from both Couchbase and Cloudant continue to develop to the project Apache CouchDB doesn’t have a lead corporate backer, nor does it need one. According to factoids gathered by Ohloh, there were 30 contributors to the Apache CouchDB project in the past 12 months, up from 18 in the prior 12 months, and placing CouchDB in the top 2% of all project teams on Ohloh.

The question is not whether CouchDB is surviving, however, but whether it is thriving. That increase in contributor count would suggest so, but that’s by no means the full story. In contrast, the number of commits per month has declined in the past 12 months, representing, as Ohloh describes it, “a substantial decrease in development activity”. As the related chart illustrates, in fact, activity has pretty much flatlined since the beginning of the year.

Source: Ohloh

This should not be altogether surprising since the latest release went GA in April.

In response to a request for comment, a spokesperson on behalf of the Apache CouchDB PMC stated:

“Despite an unsettled start to the year, the CouchDB project and the
surrounding community continue to grow and evolve, with the release of
1.2.0 earlier this year, and the forthcoming 1.3.0, currently being
prepared for release
. 1.3.0 includes in the last year alone, over 221
commits on the just the master branch, comprising 167 files changed,
5745 insertions, 2248 deletions — solid progress for a project with
22,000 lines of code total.”

Additionally, while the start of that flatline coincides with Katz’s departure from the project, it is not clear that the two are actually related. Ohloh figures indicate that Katz hadn’t actually committed code to the project since August 2010 and is only the eighth all-time most active committer to the project.

It is clear that there is still a lot of activity ongoing in the Apache CouchDB community, with the PMC citing rcouch, bigcouch, PouchDB, TouchDB frameworks for both iOS and Android, a Mac OS X binary installation, and

The PMC spokesperson added:

“Structurally, the project has added both committers and grown the
project management committe, and has been having regular meetings
through the last 2 months to improve communication within the team,
and help steer the community. A roadmap has been put together, and
Ubuntu-style time-scheduled releases are planned for 2013 to keep the
good oil flowing.”

However, in assessing the health of Apache CouchDB, we must look at adoption trends, as well as project activity.

Waving or drowning?

Searching mailing list archives using MarkMail indicates that there has been a decline in the number of messages to the developer, user, commits mailing lists in the past 12 months, although with increased activity on the latter since July.

Additionally, figures from Indeed.com suggest that job activity related to CouchDB saw a sharp decline in the early months of the year, although also a recovery in recent months.

couchdb Job Trends graph

couchdb Job Trends Couchdb jobs

However, that activity is perhaps best viewed in the context of a comparison with another major NoSQL project – MongoDB for instance – which reveals that CouchDB job postings have more or less level-off since the start of the year.

couchdb, mongodb Job Trends graph

couchdb, mongodb Job Trends Couchdb jobsMongodb jobs

We have also been tracking the traction of NoSQL projects via searches of LinkedIn member profiles. The latest figures, due to be published later this week, show that mentions of CouchDB in LinkedIn member profiles grew over 139% between December 2011 and today.

That sounds good, but again must be viewed in the context of the rest of the NoSQL ecosystem. The statistics show that mentions of a selection of other major NoSQL databases grew significantly faster in the same period.

So what are we to make of all the evidence. Clearly the Apache CouchDB project will survive, and the lack of updates in 2012 is not a major concern, although the level of interest in the project is not growing as fast as other NoSQL technologies. My personal gut feel is that Apache CouchDB has the potential to become the PostgreSQL of the NoSQL generation: a solid, mature projects with a large community of developers and ecosystem of associated vendors that is often over-shadowed by more commercially-oriented alternatives but has a loyal and committed user-base.

Key to this comparison bearing up on longterm scrutiny will be the ability of the Apache CouchDB project to increase and maintain the level of development so that the Lines of code chart, above, better resembles that of PostgreSQL, below:

The comparison with PostgreSQL is also apt given the departure from the project of its creator. While many people do know the origins of the PostgreSQL project given that the original project leader is one of the most famous database experts in the world, I am sure a lot of PostgreSQL users wouldn’t know or care whether the project’s creator continued to be involved. Similarly, Katz’s departure from Apache CouchDB, while undoubtedly a short-term challenge, appears not to have had a significant impact on the project’s ongoing development.

CAOS Theory Podcast 2012.08.17

Topics for this podcast:

*Red Hat puts enterprise cred and bet on OpenStack
*LexisNexis touts open source benefits of Hadoop alternative
*Who doesn’t love Hadoop?
*Proprietary vendors siding with open source
*PostgreSQL and its cloud, commercial opportunity
*Our Hosting and Cloud Transormation Summit NA event

iTunes or direct download (32:24, 5.8MB)

451 CAOS Links 2011.11.23

Red Hat’s Ceylon makes its debut. Heroku launches PostgreSQL service. And more.

# Red Hat’s Ceylon programming language made its public debut. Mark Little provided some context.

# Heroku announced the launch of Heroku Postgres as a standalone service.

# GitHub co-founder Tom Preston-Werner explained why you should open source (almost) everything.

# Mikeal Rogers discussed the issues behind the Apache Software Foundation’s slow response to the Git era.

# Royal Pingdom explored recent trends in Linux distribution popularity, pondering the rise of Linux Mint and the decline of Ubuntu.

# Canonical is dropping CouchDB from Ubuntu One.

# ActiveState announced that Stackato Micro Cloud will continue to be free of charge for developers to use as their own private Platform-as-a-Service.

# The European Space Agency wants to publish more of its software using open source licences.

# Sourceforge provided some interesting statistics on operating system usage.

451 CAOS Links 2011.10.14

Dennis Ritchie RIP. Microsoft adopts Hadoop. And more.

# Dennis Ritchie, creator of C and co-creator of Unix, died aged 70. This article from Joe Brockmeier puts his influence into perspective.

# Microsoft announced plans to team up with Hortonworks and the Apache Hadoop community to create a distribution of Hadoop for Windows Server and Windows Azure.

# Hortonworks explained why it decided to work with Microsoft to support its Hadoop plans.

# Black Duck Software closed a $12m round of financing led by new investor Split Rock Partners.

# OpenOffice.org e.V pleaded for financial support for the OpenOffice.org project, prompting a statement of clarification from the Apache Software Foundation

# Microsoft noted that The Advanced Message Queuing Protocol (AMQP) Working Group confirmed the availability of the AMQP 1.0 specification. Red Hat confirmed its support.

# Red Hat updated its JBoss Enterprise SOA Platform, JBoss Enterprise Data Services Platform and JBoss Enterprise Business Rules Management System product lines.

# Cloudera announced an integration partnership with MicroStrategy.

# Monsanto is creating is data integration and visualization platform based on the Cloudant suite.

# Samba can now accept code from corporations.

# VMware Micro Cloud Factory now includes PostgreSQL and RabbitMQ.

# Univa announced StackIQ will market, sell and support Univa Grid Engine to its customer and reseller channels.

# Openwave Systems is going to integrate Open-Xchange’s email technology into the Openwave Rich Mail product.

# X.commerce, a new business at eBay combining PayPal and Magento, joined the OpenStack community.

451 CAOS Links 2011.09.23

Red Hat revenue up 28% in Q2. Funding for NoSQL vendors. And more.

# Red Hat reported net income of $40m in the second quarter on revenue up 28% to $281.3m.

# 10gen raised $20m in funding, while DataStax closed an $11m series B round, while also releasing its DataStax Enterprise and Community products. Additionally Neo Technology raised $10.6m series A funding.

# Oracle announced the addition of new extended capabilities in MySQL Enterprise Edition. The move confirmed the adoption of the open core licensing strategy, and was both welcomed and derided.

# BonitaSoft announced an $11m series B funding round.\

# Platfora raised $5.7m in series A funding to accelerate development of its BI and analytics platform for data stored in Hadoop.

# EMC launched its EMC Greenplum Modular Data Computing Appliance, which includes both the Greenplum Database and Greenplum HD (Hadoop), and introduced the Greenplum Analytics Workbench, a test bed cluster for integration testing Apache Hadoop.

# Oracle acquired GoAhead Software, which offers a commercial distribution of OpenSAF.

# Ingres changed its name to Actian and launched its Action Apps and Cloud Action Platform.

# Richard Stallman asked ‘Is Android really free software?’. Predictably enough the answer is ‘no’. Carlo Daffara called FUD.

# LexisNexis Risk Solutions’ HPCC Systems released the source code for its HPCC Systems platform, and introduced a covenant to keep contributed code open source for three years.

# OpenStack released Diablo, the fourth version of its open source cloud software.

# The PostgreSQL Global Development Group announced the release of PostgreSQL 9.1.

# VoltDB announced the general availability of VoltDB version 2.0.

# Samsung is reportedly planning to release its Bada mobile operating system under an open source license.

# Karmasphere updated its Karmasphere Analyst Big Data analytics product with new workflow capabilities for Apache Hadoop.

# The Open Virtualization Alliance now has more than 200 members.

# The Outercurve Foundation announced the acceptance of the GADS open source project into its Data, Language and System Interoperability Gallery.

# Openbravo announced that customer deployments of its ERP product on Amazon have increased over 187% in the last 12 months.

# The Apache Software Foundation confirmed Apache Whirr as a top-level project.

# Qt gained more independence from Nokia.

# SUSE Linux Enterprise Server has been selected for Use with SAP HANA.

# Red Hat Enterprise Linux 6 was certified by SAP to run SAP business applications, as well as support for SAP running on Red Hat Enterprise Linux on Amazon EC2.

# 10gen’s MongoDB was chosen by SAP as a core component of SAP’s platform-as-a-service (PaaS) offering.

# Puppet Labs announced Puppet Enterprise 2.0.

# Microsoft added Casio to its list of Linux-related patent agreement signees.

# Dries Buytaert explained why Acquia acquired Cyrve and GVS and addressed concern that Acquia is sucking up all the Drupal talent.

# Medsphere Systems announced the generally availability of the enhanced OpenVista electronic health record (EHR) platform.

# Stormy Peters asked whether open source is excluding high context cultures.

# OpenIndiana’s fork of OpenSolaris added support for the Illumos kernel.

# Cenatic released the results of its research into public administration involvement in open source communities.

# Spring Roo is shifting to be 100% Apache licensed.

# VLC developers are looking for anyone who has contributed to libVLC so that they can approve the change in licence from GPLv2 to LGPLv2.

# Virtual Bridges joined OpenStack.

# Github now has over one million users.

# Splunk open sourced the code for docs.splunk.com.

451 CAOS Links 2011.08.31

MapR and Funambol raise funding. VMware virtually supports PostgreSQL. And more.

# MapR raised $20m series B for its Hadoop distribution from Redpoint Ventures, Lightspeed Venture Partners and NEA.

# Funambol raised $3m in funding from previous investors HIG Ventures, Pacven Walden Ventures and Nexit Infocom.

# VMware launched vFabric Postgres as part of vFabric Data Director database-as-a-service launch.

# Citrix released a new edition of CloudStack, making the whole cloud management product available using the GNU GPLv3.

# Yahoo has contributed 84% of Apache Hadoop lines of code and 72% of patches, according to Hortonworks’ analysis.

# Red Hat invited Red Hat Enterprise Linux users to help discuss features for Red Hat Enterprise Linux 7.

# Talend announced that Peter Gyenes has joined its Board of Directors.

# Mandriva announced the release of Mandriva 2011.

# The Document Foundation announced the release of version LibreOffice 3.4.3, intended for enterprise deployments.

# Zmanda announced the availability of Zmanda Cloud Backup (ZCB) 4.0.

# The 10th Circuit Court of Appeals ruled against on SCO’s appeal that it, and not Novell, owned the Unix copyrights.

# Oracle retired its licence for distributing its Java with Linux.

# Bruce Byfield wrote an interesting article on how Linus Torvalds and other open source developers avoid burnout.

CAOS Theory Podcast 2010.12.10

Topics for this podcast:

*Oracle, Java, the Apache Software Foundation and open source
*An update on some open source database and data management players
*CorraTech grows with support for open source application alternatives
*Red Hat-Makara acquisition analysis and impact
*Linux kernel report shows strong support, but what now for Novell?

iTunes or direct download (29:31, 5.1MB)

451 CAOS Links 2010.09.21

Oracle launches Unbreakable Kernel, updates MySQL and Java plans. And more.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca, and daily at Paper.li/caostheory
“Tracking the open source news wires, so you don’t have to.”

# Oracle launched its Unbreakable Enterprise Kernel.

# Oracle announced the release candidate of MySQL 5.5.

# Oracle outlined its plans for Java platform. JavaWorld has the details.

# Novell and SAP have collaborated on SUSE Linux Enterprise Server for SAP applications.

# Mozilla joined the Open Invention Network as a licensee.

# PostgreSQL 9.0 has been released.

# Matt Asay noted that Novell’s patents are complicating its sale.

# Patrick Backman provided some insight into the founding of SkySQL.

# Mageia launched a new Linux distribution, forked from Mandriva.

# Mandriva maintained it is alive and well, and promised an autonomous Mandriva community.

# eXo launched eXo Platform 3.0 and partnered with JasperSoft.

# Ian Skerret published his JavaOne wish list. http://bit.ly/ackLn0

# The H reported that the Swiss Canton of Solothurn is abandoning Linux.

# Scality announced plans to open-source the Software Development Kit of its RING technology.

# Tasktop and Polarion announced a partnership to deliver Eclipse integration as part of an integrated ALM Suite.

# Red Hat is reportedly looking to expand its HQ with a move outside NC.

Open source in the clouds and in the debates

We continue to see more evidence of the themes we discuss in our latest CAOS special report, Seeding the Clouds, which examines the open source software used in cloud computing, the vendors backing open source, the cloud providers using it and the impact on the industry.

First, as usual, we are seeing consistencies between our own research — which indicates open source is a huge part of today’s cloud computing offerings from major providers like Amazon, Google, Rackspace, Terremark and VMware — and that of code analysis and management vendor Black Duck. In its analysis of code that runs the cloud, Black Duck also found a preponderance of open source pieces, in many cases the same projects we profile in our report.

Indeed, open source software is an important part of the infrastructure, data and application layers of today’s cloud computing stacks with significant use of Linux, open source hypervisors KVM and Xen, open source data technologies such as MySQL, PostgreSQL, Hadoop, NoSQL and memcached and open source languages such as Java, PHP, Python and Ruby on Rails.

There will be plenty of users and customers content to use non-open source options that serve as the defacto standards, but we do see a move to higher-level, production and mission critical use, which represents continued commercial opportunity for open source and other vendors.

One of the more subtle effects of all this open source in the cloud, as covered in Seeding the Clouds, is the impact on discussions, debates and downright fights in the market. There is much scrutiny on claims of being open, technical aspects of open and what ‘open cloud’ means. A prime example is the Twisticuffs that have gone on between Simon Crosby of XenSource and Citrix, discussing OpenCloud and the response from Open Cloud Initiative co-founder Sam Johnston, who claims this is misuse of the open label.

We already saw open source playing a role in the discussions and debates about open clouds, open APIs and open data, and this latest confrontation is evidence that role continues to be significant. We still wonder though about the question of open enough as we contemplate openness in the clouds.

451 CAOS Links 2010.07.30

Adobe to acquire Day Software. Gnome contributors. Oracle bad, Oracle good. And more.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca
“Tracking the open source news wires, so you don’t have to.”

# Adobe agreed to acquire Day Software for $240m.

# Dave Neary published Gnome census, including a list of the top company contributors. , which prompted the following…
# Greg DeKoenigsberg – Red Hat, 16%. Canonical, 1%
# Jeffrey Stedfast – Re:Red Hat, 16%. Canonical, 1%
# Jono Bacon: – Red Hat, Canonical and GNOME Contributions
# Carlo Daffara – About contributions, Canonical and adopters.

# Oracle rebrands Java, breaks Eclipse. Oracle demonstrates great community support and fixes Eclipse.

# Dell and HP will certify and resell Oracle Enterprise Linux, Solaris and Oracle VM on their respective x86 platforms.

# Oracle reportedly shut down servers Sun had contributed to the build farm for PostgreSQL.

# Whamcloud is a new venture-backed company formed around the Lustre distributed file system.

# Openbravo reported that new downloads of its open source ERP software have increased 320% in five months.

# Mitchell Baker provided an update on Mozilla’s search fro a new CEO.

# Nuxeo updated Nuxeo Digital Asset Management and released Nuxeo DAM – Cloud Edition.

# CodeWeavers released CrossOver 9.1 and CrossOver Games 9.1 for both Mac and Linux.

# Rapid7 is sponsoring and partnering with w3af, the open source Web application attack and audit framework.

# Brian Proffitt said don’t be too quick to dismiss open core.

# Tarus Balog explained why he thinks open core is dead.

# Nagios Enterprises gained more than 200 Nagios XI customers in the first half of 2010.

# Former Sun distinguished engineer Bryan Cantrill joined Joyent as VP of engineering.

# The Indonesian Ministry for Research and Technology estimated that migration to OSS could save state as much $400m.

# Novell said reports SuSE Linux losing share to Ubuntu are nonsense.

# The GNOME Release Team pushed the GNOME 3.0 release to March 2011.

# Convirture released version 2.0 of the Enterprise edition of its ConVirt virtualization management software.

# Sourcefire launched Razorback, a framework for multi-vendor threat detection and protection.

# Qualys released BlindElephant, an open source web application fingerprinting engine.

# Contegix agreed to sponsor the Clojure development language project.

# vtiger unveiled vtiger CRM On Demand, the cloud-based version of its open source CRM software.

# Jos Poortvliet joined Novell as openSUSE Community Manager.

# Funambol introduced DM Carrier Edition an open source device management offering for WiMAX.

# Sony Pictures Imageworks and Industrial Light & Magic developed Alembic, an open source exchange format.

# The Register reported that Canonical will integrate Hadoop and NoSQL database technologies with Ubuntu 10.10.

# The Open Invention Network experienced 35% growth in licensees during the second quarter.

# OpenSAF released version 4.0 of its high availability middleware platform.

Is it time to rethink the open source license approval process?

Is Google’s WebM open source or not? And why (or more to the point, how) did it take the OSI nearly five months to approve the PostgreSQL license?

Open Source Initiative board member, Simon Phipps, declared on Monday that Google’s WebM project “is not currently open source”. It was a statement based on the fact that the license used for WebM is not currently approved by the Open Source Initiative as being compliant with the Open Source Definition.

The choice of language was unfortunate though. Had Simon written that WebM “does not currently use an OSI-approved license” then the statement would have been unarguably true. Declaring that the license is “not… open source” prompted predictable disagreement.

“OSI does not have a trademark on the term “open source”. It is not OSI-approved, but it is open source by anybody else’s definition,” commented one anonymous respondent, while David Gerard noted that “the rather more active FSF considers it free software”.

I’m not interested in continuing the debates about whether the OSI should be considered the ultimate arbiter of “open source” (as opposed to OSI-approved), or the FSF vs OSI, or whether WebM is officially open source (see Bruce Perens’ comment below for clarification on that). What concerns me is the ongoing open source licensing no man’s land that encourages these debates in the first place.

Another OSI board member, Andrew Oliver recently stated that when it comes to judging software “Either it is open source or not.” This ought to be true, but the WebM example proves that it is not. There is a no man’s land in which the software might be open source, but we have to wait for the license to be submitted to and approved by the OSI before we can know for sure.

The WebM license has been submitted to the OSI’s license review mailing list, not by Google but by Bruce Perens, who noted that he plans to issue a derivative work based on WebM, but – besides – “it’s in the community’s interest to review it”.

With any luck then there should be a definitive decision on the WebM license sooner rather than later. But maybe later. A look at the license review mailing list reveals one submitter has been waiting six months for official confirmation of OSI approval. And he’s not the only one waiting for a response.

(Update – Most definitely it will be later – Google has asked the Open Source Initiative to delay consideration of Google’s WebM license, and in doing so has called on the OSI to be more open)

I fully understand why the approval process is designed to weed out vanity licenses and discourage license proliferation*, but it took nearly five months from the submission of the PostgreSQL license to its eventual approval.

Today Simon has made the case for why the world still needs the OSI, and I agree with him that it retains an important role in the open source ecosystem (which is why we were concerned by its recent organisational problems) but would argue that the process of license approval needs to be overhauled.

Simon argues that it would be great to see more of the older and (in hindsight) unsuitable licenses retired. It would, but it would also be great if the OSI made the process of selecting an approved license easier, and the process of license approval quicker.

The Report of the License Proliferation Committee, which was approved by the OSI board in 2006, highlighted an open source license wizard project underway at USC law school and San Francisco State engineering department and stated its hope “being able to generate a list of existing licenses that meet defined goals will lessen the need for people to create their own new licenses”.

I’m not sure what ever happened to that wizard project, but it sounds a lot like the Creative Commons license selector. (As an aside I remembering discussing a similar idea with another OSI board member, Martin Michlmayr, during the Open Source Think Tank in Paris and he mentioned proposing it to the OSI board).

I previously argued that web-based tools also have the potential to make the process more fluid and involve others in the approval process. It seems to me that in a world where Digg and IdeaStorm already seem old, taking almost five months to approve a license in use for 14 years by one of the most popular open source projects in the world is absurd and completely unjustifiable.

*Although I’m not convinced that license proliferation is the threat it once was, see this post for an explanation.

451 CAOS Links 2010.05.21

VMware partners with Google. 20 bidders for Novell. IBM and Pentaho support Hadoop. And more.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca
“Tracking the open source news wires, so you don’t have to.”

# VMware partnered with Google on cloud computing, including support for Spring Java apps on Google App Engine.

# The WSJ reported that Novell has received up to 20 acquisition bids, and is set to start considering its options.

# Matt Asay explained why going private could be good for Novell.

# IBM unveiled a portfolio of products and services based on Apache Hadoop.

# Pentaho announced plans for its BI and data integration software to support Apache Hadoop.

# Jedox reportedly raised $2.1m in new funding from existing investors.

# Google open sourced VP8 as part of the WebM Project.

# The Free Software Foundation welcomed Google’s WebM format.

# NTT and EnterpriseDB developed Postgres-XC (eXtensible Cluster) technology for PostgreSQL.

# Gluster named Ben Golub President and CEO.

# OSS got a mention in the UK government’s programme for transparency.

# Novell announced SUSE Linux Enterprise 11 Service Pack 1.

# SUSE Linux Enterprise 11.1 included DRBD from LinBit for data replication and HA.

# WANdisco promised to reinvest its profits from Subversion support back into the Subversion project.

# An open source version of Magelia WebStore, e-commerce software for Windows, will be released in September.

# Gianugo Rabellino stepped down as CEO of Sourcesense.

# ComputerWeekly reported on how BMW virtualised with SuSE Linux and Xen.

451 CAOS Links 2010.02.23

Copyright, patents, licensing, community. The usual. Only more so.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca
“Tracking the open source news wires, so you don’t have to.”

# The Guardian reported that US copyright lobby wants the US government to consider OSS the equivalent of piracy.

# Jim Zemlin has dismissed Microsoft’s patent agreement with Amazon.

# Richard Stallman explained why he will not sign the Public Domain Manifesto.

# Brian Aker clarified the licensing requirements related to the Drizzle project.

# Bruce Momjian wrote about the removal of personal copyright notices from PostgreSQL.

# Matt Asay asked “Will open source ever be completely free?”

# Jaspersoft reported some statistics suggesting momentum behind its open source software community.

# Ian Skerrett outlined attempts to increase community participation at the Eclipse Foundation.

# Abiquo announced that abiCloud 1.0.0 has been formally released under the LGPL version 3.

# Accenture’s Kit Plummer explained how to get smarter about open source.

# Actuate shared some stats on its BIRT Exchange Marketplace contributions.

# The H reported that the Cassandra database has been accepted as an Apache Top-Level Project.

# Sendmail claimed record setting revenues in Q4 and FY2009.

# Ingres is targeting cloud computing opportunities.

# Novell outlined its approach to virtualization.

# OStatic published “Measuring FAIL: A Scorecard for Evaluating Open Source Projects”.

# The Register published “Open source – the once and future dream”.

451 CAOS Links 2010.01.25

WordPress Foundation formed. Reaction to Oracle-Sun approval. And more.

WordPress Foundation formed
# Matt Mullenwag launched the WordPress Foundation.

Reaction to Oracle-Sun’s EC approval

# In a memo Sun CEO Jonathan Schwartz encouraged the company’s employees to emotionally resign from Sun.

# EnterpriseDB and PostgreSQL co-founder Bruce Momjian issued a statement on the EC’s decision to approve Oracle-Sun.

# Mike Hogan asked, did Oracle make concessions to the EU?

# Savio Rodrigues discussed Sun & Oracle’s impact on open source acquisitions.

# Save MySQL campaigner Florian Mueller commented following the EC’s clearance of the Oracle-Sun deal.

# Josh Berkus clarified his presentation on Sun and ten ways to destroy a community.

# Ingres CEO Roger Buckhardt analyzed the impact of Oracle-Sun on the database market.

Best of the rest
# Internetnews.com reported on Red Hat’s plans for JBoss in 2010.

# Red Hat’s opensource.com community site is now live.

# Talend introduced and open source Master Data Management (MDM) product.

# Joe “Zonker” Brockmeier, openSUSE community manager, is leaving Novell.

# JavaWorld compared JBoss and SpringSource.

# MuleSoft updated Tcat Server with support for the newest version of Apache Tomcat 6.0.24.

#OSS Watch discussed control versus community.

# McObject’s Perst open source, object-oriented embedded database now supports Microsoft’s Silverlight technology.

# GigaOM discussed how Red Hat has avoided the recession.

# Carlo Daffara discussed how open source enables new ways of cooperating.

451 CAOS Links 2009.10.13

Larry Ellison promises funds for MySQL, commits to community. The “open source vendor” debate in a nutshell. And more.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca
“Tracking the open source news wires, so you don’t have to.”

# Larry Ellison promised MySQL will receive more money for development and research, while Oracle maintained that it is committed to Java and open source developer communities.

# GroundWork raised $5m series D funding from Canaan Partners, Mayfield, JAFCO Ventures and SAP Ventures.

# InformationWeek reported that Motorola has vacated its seat on the LiMo Foundation board and will focus on Android.

# EnterpriseDB updated its Postgres Plus Standard Server open source database to version 8.4.

# Eric Raymond doesn’t like existing forges, and offered some advice on how not to solve them.

# Sandro Groganz argued that the longer someone uses FOSS, the more important the “freedom” aspects become.

# Matt Asay asked Is it Postgres’ time to shine?

# Sun announced the release of GlassFish Communications Server 2.0, designed for telecoms service delivery platforms.

# JitterBit released version 3.0 of its commercially licensed Enterprise MX edition.

The “open source vendor” debate in a nutshell
This week John Mark Walker asked for companies to be more transparent about where they fall on the openness spectrum. Evidence for why this is important came from Seth Grimes, who argued that you have to understand code words to decipher Pentaho’s commercial open source strategy, prompting James Dixon to argue that Pentaho is no less an open source vendor now that it is offering ClearView under a proprietary license than it was before it licensed the ClearView IP.

Simon Phipps argued that the problem Seth identifies is symptomatic of the fact that it has an open core licensing model. However, what Seth points out is that Pentaho is using the term “open source” liberally to apply to both open source and proprietary code. That is not actually an issue of the licensing model, but a related issue of branding and communication. If Pentaho were more explicit in its explanation, as John Mark Walker advises, the the problem Seth identifies diminishes. As I previously argued, being absolutely transparent about licensing is one of the key strategies that could be used to ensure a sustainable implementation of the Open Core model.

However, Pedro Alves does highlight one of the problems open core creates from a user’s perspective, again using Pentaho as an example. Meanwhile Sander Marechal argued that Open Core offers the worst of both worlds.

451 CAOS Links 2009.09.15

Reaction to the CodePlex Foundation. GroundWork launches MonitoringForge.org. And more.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca
“Tracking the open source news wires, so you don’t have to.”

CodePlex Foundation Good

Stephen Walli explained why he sees value in the CodePlex Foundation, while Aaron Fulkerson provided his views as to why Microsoft set up the Foundation, and he is acting as an adviser, and Monty Widenius explained why he too is acting as an adviser to the Foundation.

CodePlex Foundation Bad

Andy Updegrove dissected the governance structure of the CodePlex Foundation, declaring that “quite a bit of the governance structure will need to change before CodePlex can expect to attract broad participation.”

Best of the rest
# GroundWork Open Source launched MonitoringForge.org for open source monitoring projects.

# OpenLogic and Nuxeo partnered to support Enterprise Content Management stack including Nuxeo, JBoss and PostgreSQL.

# Actuate and Infobright launched an integrated DW/BI virtual machine.

# Oracle released new versions of its Oracle Berkeley DB embeddable databases.

# Peerless Foods claimed $300,000-$400,000 annual licensing savings in move to Ingres.

# Javier Soltero reflected on the benefits of Hyperic being acquired by SpringSource, and then VMware.

# Sirius published a case study of the first UK school to migrate entire infrastructure to Open Source

# ClearFoundation unveiled ClearOS 5.1, a Linux distribution for networks and server based Internet gateways.

# The FSF updated its list of approved free GNU/Linux distributions.

# Terracotta released version 3.1, including Terracotta for Hibernate.

# John Spencer reported that herd mentality is to blame for the difficulty in selling FOSS to local authorities.

# Matt Asay discussed the dilemma open source vendors have balancing popularity among developers and IT operations.

# WaveMaker introduced one-click deployment to Amazon EC2 with WaveMaker 5.2.

# Tristan Rhodes asked whether ZipTie is a good candidate for a fork.

# OpenLogic provided some details about its latest survey sample.

# Savio Rodrigues reported on the use of Hadoop by the New York Times.

# Techradar published an interview with PHP creator Rasmus Lerdorf on how it became successful.

Open source’s role in lowering the barriers to data warehousing

As well as contributing to the CAOS research practice here at The 451 Group I am also part of the information management team, with a focus on databases, data caching, CEP, and – from the start of this year – data warehousing.

I’ve covered data warehousing before but taking a fresh look at this space in recent months it’s been fascinating to see the variety of technologies and strategies that vendors are applying to the data warehousing problem. It’s also been interesting to compare the role that open source has played in the data warehousing market, compared to the database market.

I’m preparing a major report on the data warehousing sector, for publication in the next couple of months. What follows is a rough outline of the role open source has played in the sector. Any comments or corrections much appreciated:

Unlike other sectors, where the role of open source has mostly been the disruption of incumbent proprietary vendors by commercial open source specialists, the impact of open source in the data warehousing sector has been more subtle, and arguably more pervasive.

Vendors such as Netezza and Greenplum have used the PostgreSQL database to build their data warehousing products, benefiting from the robust, mature PostgreSQL code base and reduced time to market. However, the end products of these development efforts are not open source.

For example, Netezza used PostgreSQL as the database scaffolding to reduce the time to create its Netezza Performance Server (NPS), although the BSD license used by the PostgreSQL project enabled the company to do so without its resulting database having to be made available under an open source license, and the majority of the PostgreSQL code has subsequently been replaced. Additionally, Aster Data makes use of PostgreSQL as a data store on each node of its nCluster massively parallel data warehouse.

Similarly Greenplum also used PostgreSQL as the basis for its massively-parallel Greenplum Database and also set up and supported the Bizgres distribution with business intelligence and data warehousing specific contributions made available under the BSD license. However that project fizzled out and the website is now closed, although Greenplum’s use of PostgreSQL continues.

Another example of PostgreSQL usage comes from Paraccel, which used the PostgreSQL optimizer code in version 1.0 of its Analytic Database in order to improve time to market. That is now being replaced by a new optimizer called Omne, which is specifically designed to support the MPP columnar architecture of Paraccel and its compression capabilities, unlike the SMP PostgreSQL optimizer, which was extended to support MPP. While Omne retains some elements of the open source PostgreSQL optimizer code base, Paraccel claims it will remove all PostgreSQL code from its products with an update to the Omne technology in 2010.

Additionally Vertica, which was founded by Mike Stonebraker, creator of PostgreSQL and Ingres, is a commercial implementation of the C-Store academic research project, which was also licensed under BSD.

It is also worth mentioning that prior to its acquisition by Microsoft, DATAllegro made use of a commercial license of the open source Ingres database within its data warehousing appliances. DATAllegro actually did most of the early development work for its first appliance using PostgreSQL, but decided to change to Ingres late in 2004 to make use of partitioning capabilities, backup utilities and optimizer features. Needless to day Ingres is being replaced by Microsoft SQL Server in Microsoft’s forthcoming Madison data warehouse appliances.

LucidDB is another, often overlooked open source database, and was purpose-built for data warehousing. Based on technology developed by Broadbase Software, the code was picked up by erstwhile business intelligence SaaS provider LucidEra and combined with the Eigenbase data management framework to create LucidDB. Following LucidEra’s recent demise the LucidDB code is not currently commercially supported, although the non-profit Eigenbase Foundation is continuing to sponsor its development.

Another often overlooked open source database – to the extent that I overlooked it – is MonetDB, a column-oriented database management system developed at Amsterdam’s Centrum Wiskunde & Informatica (CWI) scientific research establishment by many of the same researchers who went on to create Vectorwise (see below). MonetDB, the company, was spun-off from CWI in 2008 with the aim of disseminating the code and identifying commercial joint venture and collaboration projects to increase its adoption.

Finally, at least for now, July saw the launch of HadoopDB from Yale’s Database Research team. HadoopDB is designed to be an analytical database system that combines the scalability of Hadoop and the performance of parallel database systems and, according to Daniel Abadi, it is “an open source stack that includes PostgreSQL, Hadoop, and Hive, along with some glue between PostgreSQL and Hadoop, a catalog, a data loader, and an interface that accepts queries in MapReduce or SQL and generates query plans that are processed partly in Hadoop and partly in different PostgreSQL instances spread across many nodes in a shared-nothing cluster of machines.”

Despite this rampant use of open source code, it was not until Infobright launched Infobright Community Edition (ICE) in 2008 that we saw the first commercial open source vendor delivering its core warehouse software under an open source license. The Infobright columnar database acts as a storage engine for the MySQL database turning it into a realistic option for data warehouses of more than 200GB according to Infobright (Sun maintains that MySQL can perform as a stand-alone data-warehousing platform up to 2TB with the default MyISAM non-transactional storage engine).

While MySQL is not well known as a platform for data warehousing, Sun’s internal surveys indicate that data warehousing is the fifth-most-common use case for MySQL, which explains why it is not just Infobright that is looking to build a data warehousing business around MySQL.

Kickfire emerged in April 2008 with a beta version of its MySQL Appliance, which is built around the MySQL database and its SQL chip, which provides native instruction execution while operating directly out of memory on compressed data. Kickfire is targeting deployments in the 100GB-3TB range, while Infobright acts as a MySQL storage engine to enable use with up to 30TB of data. Infobright is developing a shared-everything, peer-to-peer architecture that will support up to 100 concurrent users and 100TB of data. Delivery is scheduled for the fourth quarter.

It remains to be seen whether Oracle will retain its commercial relationships with Kickfire and Infobright once its acquisition of Sun, and therefore MySQL, closes, but one company that has already been impacted by the acquisition its Calpont, which had planned to make a big splash at the recent MySQL Conference & Expo with the launch of its new strategy to provide a data-warehousing storage engine for the MySQL database.

The plan, to offer an open source column-oriented storage engine that will provide the MySQL database with the capabilities to function as a data warehouse, scaling from capacities of 100GB to 100TB, remains in place, although the storage engine will be in beta testing for the foreseeable future while Calpont waits to see what Oracle will do.

The most recent open source entrant into the data warehousing market is Ingres, which has teamed up with VectorWise, another database-engine spin-off from Amsterdam’s CWI, to collaborate on a new database-kernel project designed to better enable it to be positioned as a platform for data-warehouse and analytic workloads. he resulting software will be fully open source although Ingres does not have detailed plans for the productization of the technology at this stage. The Vectorwise technology was originally known as X100 and was used initially as an extension module inside the MonetDB database.

While open source is playing an increasing role in the data warehousing market, PostgreSQL has primarily taken the role of lowering barriers to entry for new vendors by providing a platform for the development of data warehouse-specific capabilities on a proven database platform.

MySQL serves a similar role for Infobright, Kickfire and Calpont, but could also play a significant role in lowering barriers to entry for new data warehousing customers with small volumes of data.

Calpont turned its attention to MySQL and the midrange market in order to exploit the requirement for scalable data-warehousing capabilities from MySQL’s estimated 11 million users, as well as the fact that the low-end of the market has not been well-supported by the existing data-warehousing vendors.

Sun estimates that 90% of all data warehouses have 6TB of data or less, while Kickfire estimates there are 17,000 addressable accounts that are trying to use MySQL to create data warehouses with volumes greater than 50GB.

These estimates explain why Sun et al see an opportunity for MySQL-based warehouses to grab a slice of the market based on a low cost systems targeting a large number of customers and small amounts of data – the complete inverse of the traditional focus for data warehousing requirements, which is based on high cost systems supporting large amounts of data and a relatively small number of potential customers.

Additionally, Kickfire, Infobright and Calpont are looking to replicate the strategy MySQL successful followed in the database market by targeting a market niche that is not being served by the incumbents and avoid competing head on with the likes of Teradata, IBM, Oracle and Netezza.

451 CAOS Links 2009.07.07

Microsoft makes a patent promise. JBoss is certifiable. And more.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca
“Tracking the open source news wires, so you don’t have to.”

# Microsoft announced that it will be applying its Community Promise to the ECMA 334 (C#) and ECMA 335 (CLI) specifications, reducing patent concerns related to Mono.

# Red Hat achieved EAL 2+ security certification for JBoss Enterprise Application Platform.

# A European Commission white paper on standards stirred some controversy with regards to open source. The white paper itself is available here. (Pdf)

# Alfresco announced Community Edition 3.2 with mobile, email, records management features.

# MuleSource reported 140% year-over-year growth in Q2 and 100% growth for the year-to-date.

# Intalio hired former SAP Executive Stefan Schaffer has joined the company as Chief Operating Officer.

# Rich Sharples explained JBoss Enterprise Web Server and how it fits into Red Hat’s Open Choice strategy.

# Matt Asay discussed the principles he sees as most important in ensuring the success of an open source project.

# UK schools no longer need to buy Microsoft licenses for hardware that is not running Microsoft software, which should have a knock-on effect for the potential adoption of open source.

# Lockheed Martin announced that it is to release an in-house developed social media tool under an open source software license.

# Infobright and Jaspersoft released an integrated virtual machine for BI and data warehousing.

# Corus selected Univa UD, including open source UniCluster on Oracle Enterprise Linux, for private HPC cloud.

# Talend published a case study illustrating how PriceMinister uses Talend to automate their supply chain.

# Sybase announced support for Android with its iAnywhere Blue SDK.

# Continuent launched its Tungsten database replication and HA software for PostgreSQL 8.4.

# Bacula Systems updated its open source network backup and restore software.

# eWeek reported Ingres’ claim to be benefiting from anxiety over the future of MySQL.

# Matt Asay reported on Axial, a new open source healthcare start-up attracting former Red Hat employees.

# The H reported that The Swiss Federal Administrative Court has issued an interim verdict in the Microsoft/OSS case.

# Cnet news reported that Nokia is to bridge Maemo, Symbian with Qt.

# OSOR.EU reported that Dutch police will study moving to open source.

# Merv Adrian reported on how Talend is using open source and community to transform the data integration market.

451 CAOS Links 2009.07.03

Ubuntu cloud services. PostgreSQL 8.4. And more.

Follow 451 CAOS Links live @caostheory
“Tracking the open source news wires, so you don’t have to.”

# Canonical launched Ubuntu Enterprise Cloud Services.

# Progress Software moved internal development of FUSE and related products to its own forge.

# PostgreSQL 8.4 released.

# The OpenNMS Group has acquired the copyright to the original OpenNMS code. Tarus Balog explained what it means.

# The LiMo Foundation explained how it is taking it’s engagement with the community seriously.

# The Dutch published a full review of the open source the governmentt has adopted.

# Stephen O’Grady asked what’s going to comprise the LAMP of the cloud?

# GPL: Getting Pretty Lonely, a perspective from Daniel Jalkut of Red Sweater Software.

# NASA has used open source sw to create internal “SpaceBook”.

# Five of the top ten Sourceforge projects are ERP.

# The H: Ubuntu to continue using Mono.

# Carlo Daffara finished the final edition of the SME guide to open source.

# Dave Neary explained why he disagrees with Richard Stallman concerning Mono.

# ArsTechnica reported on how a ew Linux patch could circumvent Microsoft’s FAT patents.

# Open source case study: Red Hat has provided the University of Southern Mississippi with time-saving solution.

# Medicine 3.0: 50 Successful Open Source Projects That Are Changing Medicine.

# REvolution released parallel packages and functions for R.

# Case study: Telecom Italia Speeds Delivery of Innovative Prototypes with Spring.

# Andy Astor explained why he is leaving EnterpriseDB.