Our 2013 Database survey is now live

451 Research’s 2013 Database survey is now live at http://bit.ly/451db13 investigating the current use of database technologies, including MySQL, NoSQL and NewSQL, as well as traditional relation and non-relational databases.

The aim of this survey is to identify trends in database usage, as well as changing attitudes to MySQL following its acquisition by Oracle, and the competitive dynamic between MySQL and other databases, including NoSQL and NewSQL technologies.

There are just 15 questions to answer, spread over five pages, and the entire survey should take less than ten minutes to complete.

All individual responses are of course confidential. The results will be published as part of a major research report due during Q2.

The full report will be available to 451 Research clients, while the results of the survey will also be made freely available via a
presentation at the Percona Live MySQL Conference and Expo in April.

Last year’s results have been viewed nearly 55,000 times on SlideShare so we are hoping for a good response to this year’s survey.

One of the most interesting aspects of a 2012 survey results was the extent to which MySQL users were testing and adopting PostgreSQL. Will that trend continue or accelerate in 2013? And what of the adoption of cloud-based database services such as Amazon RDS and Google Cloud SQL?

Are the new breed of NewSQL vendors having any impact on the relational database incumbents such as Oracle, Microsoft and IBM? And how is SAP HANA adoption driving interest in other in-memory databases such as VoltDB and MemSQL?

We will also be interested to see how well NoSQL databases fair in this year’s survey results. Last year MongoDB was the most popular, followed by Apache Cassandra/DataStax and Redis. Are these now making a bigger impact on the wider market, and what of Basho’s Riak, CouchDB, Neo4j, Couchbase et al?

Additionally, we have been tracking attitudes to Oracle’s ownership of MySQL since the deal to acquire Sun was announced. Have MySQL users’ attitudes towards Oracle improved or declined in the last 12 months, and what impact will the formation of the MariaDB Foundation have on MariaDB adoption?

We’re looking forward to analyzing the results and providing answers to these and other questions. Please help us to get the most representative result set by taking part in the survey at http://bit.ly/451db13

451 CAOS Links 2011.11.18

Rapid7 secures new funding. Microsoft drops Dryad. And more.

# Rapid7 secured $50m in series C funding.

# Microsoft confirmed that it is ditching its Dryad project in favour of Apache Hadoop.

# Arun Murthy provided more details of Apache Hadop 0.23.

# The Google Plugin for Eclipse and GWT Designer projects are now fully open source.

# openSUSE released version 12.1.

# Amazon released the source code of the Kindle Fire.

# Black Duck Software joined the GENIVI Alliance.

# dotCloud announced the availability of the top three databases MySQL, MongoDB and Redis on its PaaS.

Necessity is the mother of NoSQL

As we noted last week, necessity is one of the six key factors that are driving the adoption of alternative data management technologies identified in our latest long format report, NoSQL, NewSQL and Beyond.

Necessity is particularly relevant when looking at the history of the NoSQL databases. While it is easy for the incumbent database vendor to dismiss the various NoSQL projects as development playthings, it is clear that the vast majority of NoSQL projects were developed by companies and individuals in response to the fact that the existing database products and vendors were not suitable to meet their requirements with regards to the other five factors: scalability, performance, relaxed consistency, agility and intricacy.

The genesis of much – although by no means all – of the momentum behind the NoSQL database movement can be attributed to two research papers: Google’s BigTable: A Distributed Storage System for Structured Data, presented at the Seventh Symposium on Operating System Design and Implementation, in November 2006, and Amazon’s Dynamo: Amazon’s Highly Available Key-Value Store, presented at the 21st ACM Symposium on Operating Systems Principles, in October 2007.

The importance of these two projects is highlighted by The NoSQL Family Tree, a graphic representation of the relationships between (most of) the various major NoSQL projects:

Not only were the existing database products and vendors were not suitable to meet their requirements, but Google and Amazon, as well as the likes of Facebook, LinkedIn, PowerSet and Zvents, could not rely on the incumbent vendors to develop anything suitable, given the vendors’ desire to protect their existing technologies and installed bases.

Werner Vogels, Amazon’s CTO, has explained that as far as Amazon was concerned, the database layer required to support the company’s various Web services was too critical to be trusted to anyone else – Amazon had to develop Dynamo itself.

Vogels also pointed out, however, that this situation is suboptimal. The fact that Facebook, LinkedIn, Google and Amazon have had to develop and support their own database infrastructure is not a healthy sign. In a perfect world, they would all have better things to do than focus on developing and managing database platforms.

That explains why the companies have also all chosen to share their projects. Google and Amazon did so through the publication of research papers, which enabled the likes of Powerset, Facebook, Zvents and Linkedin to create their own implementations.

These implementations were then shared through the publication of source code, which has enabled the likes of Yahoo, Digg and Twitter to collaborate with each other and additional companies on their ongoing development.

Additionally, the NoSQL movement also boasts a significant number of developer-led projects initiated by individuals – in the tradition of open source – to scratch their own technology itches.

Examples include Apache CouchDB, originally created by the now-CTO of Couchbase, Damien Katz, to be an unstructured object store to support an RSS feed aggregator; and Redis, which was created by Salvatore Sanfilippo to support his real-time website analytics service.

We would also note that even some of the major vendor-led projects, such as Couchbase and 10gen, have been heavily influenced by non-vendor experience. 10gen was founded by former Doubleclick executives to create the software they felt was needed at the digital advertising firm, while online gaming firm Zynga was heavily involved in the development of the original Membase Server memcached-based key-value store (now Elastic Couchbase).

In this context it is interesting to note, therefore, that while the majority of NoSQL databases are open source, the NewSQL providers have largely chosen to avoid open source licensing, with VoltDB being the notable exception.

These NewSQL technologies are no less a child of necessity than NoSQL, although it is a vendor’s necessity to fill a gap in the market, rather than a user’s necessity to fill a gap in its own infrastructure. It will be intriguing to see whether the various other NewSQL vendors will turn to open source licensing in order to grow adoption and benefit from collaborative development.

NoSQL, NewSQL and Beyond is available now from both the Information Management and Open Source practices (non-clients can apply for trial access). I will also be presenting the findings at the forthcoming Open Source Business Conference.

How soon is now? Corporate contributions and open source innovation in the context of NoSQL

In my role as part of The 451 Group’s Information Management practice I have recently initiated coverage on the various “NoSQL” databases, which are providing a fresh challenge to conventional relational databases (clients can get a good introduction to our coverage here, while non-clients can also see some of my thinking aloud over at our Too Much Information blog).

The rise of the NoSQL movement is also highly relevant in the context of open source software, however, especially in relation to two key issues related to open source software.

1/ The (lack of) corporate user contributions
2/ Open source as a source of innovation (as opposed to disruption)

NoSQL is very much a user-led phenomenon and has occurred as the likes of Google, Amazon, Facebook, LinkedIn and Twitter have created their own distributed data management technologies to overcome the fact that traditional database products were not able to match their performance and scalability requirements.

No all NoSQL databases are the product of companies that we would traditionally think of as users rather than developers, and not all NoSQL databases are open source, but there are a large number of projects that fulfill both criteria: such as Apache Cassandra (which originated at Facebook), Apache Hbase (Yahoo), Hypertable (Zvents), Voldemort (LinkedIn) and FlockDB (Twitter).

Meanwhile there are a number of vendors and projects focused on adding persistence, replication, index and query capabilities to memcached, which was originally created by Danga Interactive to solve its database scalability issues.

This is also (mostly) not a matter of businesses creating projects in house and then simply throwing the code over the wall. At last week’s NoSQL EU event in London, Twitter’s analytics lead, Kevin Weill, discussed how Twitter is working with Digg to create real-time analytics for Cassandra. Kevin also recently Tweeted (naturally enough) about Hadoop-LZO, a project to bring splittable LZO compression to Hadoop, on which Twitter is collaborating with Cloudera and Facebook.

There are plenty of other examples of contributions being made by Twitter, Facebook, Digg and LinkedIn on their own open source pages, but in many ways the biggest thing here is not the individual contributions but the commitment to the overall culture of contribution and collaboration.

It is often said that open source developers begin by scratching their own itch, and that is most definitely true when we look at the motivations behind the creation of projects by the companies above, but there is also a culture and clear understanding that there is much to gain from collaboration.

The NoSQL technologies also undermine the suggestion that while open source can be used to commoditize established markets it is not good an innovation. While the likes of Cassandra and Voldemort – not to mention Neo4J, Redis, CouchDB, Riak and MongoDB – are undoubtedly operating within a larger established market, the longer we look at NoSQL the clearer it is that far from commoditizing an established market these technologies are being used to innovate beyond the realms of the established relational database and establish new database market segments.

451 CAOS Links 2010.04.13

600 new customers for SugarCRM. James Gosling leaves Oracle. And more.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca
“Tracking the open source news wires, so you don’t have to.”

# SugarCRM added nearly 600 customers in the first quarter of 2010.

# James Gosling resigned from Oracle.

# VMware’s SpringSource acquired Rabbit Technologies and its RabbitMQ messaging software.

# EnterpriseDB hired Sun’s former MySQL VP Karen Tegan Padir as vice president of products and marketing.

# Xen.org released version 4.0 of the Xen open source hypervisor.

# Talend 4.0 includes data integration, data quality and master data management in a single offering.

# rPath offered improved Linux patching capabilities, targets Red Hat Network Satellite users.

# SugarCRM has made the primary source code repository for Sugar Community Edition publicly accessible at SugarForge.

# Gear6’s memcached distribution supports native query and Redis integration, and is now available on Ubuntu and Debian.

# Percona announced Percona Server with XtraDB as a drop-in replacement for MySQL.

# Mezeo Software and Zmanda partnered on a new cloud storage backup offering.

# The Apache Software Foundation announced Apache Cassandra release 0.6.

# Fonality and SugarCRM partnered to provide contact center offering.

# Black Duck reported bookings up 80%, 31 new customers in the first quarter.

# Ulteo joined the Open Invention Network.

# The Linux Foundation announced the expansion of companies participating in the MeeGo project.

# Black Duck Software released a code search plug-in for Visual Studio 2010.

# Twitter released the source code to FlockDB, its distributed graph database technology.

# Hippo CEO Jeroen Verberg’s shared his thoughts on open source-related business strategies.

# EditShare announced plans to release Lightworks media editing software as open source.

# Marten Mickos’s presentation at Parc Forum – Open for business: Building successful commerce around open source.

451 CAOS Links 2010.03.16

Pre-OSBC special: VMware goes NoSQL. Magento raises $22.5m. Nuxeo triples customer count. And more.

Follow 451 CAOS Links live @caostheory on Twitter and Identi.ca
“Tracking the open source news wires, so you don’t have to.”

# VMware hired Salvatore Sanfilippo, and added his Redis database to its cloud infrastructure portfolio.

# Varien/Magento Commerce raised a $22.5m equity funding round.

# Nuxeo tripled its customers base and 285% revenue growth in 2009.

# IBM selected Red Hat Enterprise Virtualization to power its new development and test cloud computing service.

# Novell noted that IBM’s new development and test cloud supports SUSE Linux Enterprise, as well as Red Hat.

# NorthScale launched its supported distribution of memcached and started beta testing its Membase key value store and also confirmed that it raised $5m in series A funding from Accel and North Bridge.

# The Eclipse Foundation published the results of the 2010 Eclipse board election.

# WANdisco hired the lead developer for TortoiseSVN, the Subversion client for Windows.

# BitNami released Ubuntu-based virtual appliances for all of the BitNami-packaged applications.

# Black Duck figures indicate healthy growth in the number of open source mobile projects, particularly for Android.

# The latest version of Open-Xchange’s Groupware offers integrated VoIP.

# Likewise Software’s authentication software is to be included with VMware vSphere.

# Linux.com published a Q&A with Matt Asay on Canonical, Ubuntu, Linux and skiing soundtracks.

# Mandriva Enterprise Server 5.1 is now available.

# Tim Bray confirmed that he has joined Google as Developer Advocate, with a focus is Android.

# OpenLogic launched a new M&A Open Source Audit Service for companies involved in mergers and acquisitions.