Who doesn’t love Hadoop?

I tweeted recently that I had received a query from a journalist about whether Hadoop needs to go closed source to be fit for the enterprise.

Now that the resulting report has been published we can see who was behind that suggestion, with Brian Christian, Zettaset chief technology officer, arguing that “The community serves its needs, not the needs of the enterprise.”

The report also includes some, although naturally not all, of the response I provided to this suggestion, and since the report leaves a few misconceptions unanswered I thought I’d publish my more detailed response.

Hadoop is ‘free like a puppy’
Hadoop currently requires a degree of expertise to configure, manage and operate, but that statement is true for any serious data management technology. Apache Hadoop is relatively immature compared to some other established data management technologies, particularly in areas such as high availability, security and manageability. However, the development community is well-aware of its shortcomings and advances in all areas are currently in early access and should be ready for production deployment later this year.

Hadoop does require a degree of expertise to operate, and that expertise is currently at a premium and comes at a cost. However, all the major Hadoop supporters are working to train up a larger pool of Hadoop developers and administrators. Cloudera alone has trained more than 12,000 people to use Hadoop.

Apache Hadoop is a complex combination of data management technologies and is not without its challenges, which have arguably led to some enterprise taking longer to move from development and testing to deployment than they might have initially expected. However, the Hadoop development community is clearly committed to making Hadoop more suitable for enterprise adoption.

Hadoop is ‘driven by enthusiasts’
The idea that the open source community is populated by individual developers with no concern for enterprise requirements is completely bogus. The Apache Software Foundation has a proven history of developing enterprise-grade software projects through a collaborative development process that combines vendors, users and other interested parties.

The biggest contributors to Apache Hadoop include vendors such as Hortonworks, Cloudera, MapR and IBM, all of which have a vested interest in driving greater enterprise adoption, as well as users such as Yahoo, Facebook and eBay, all of which stand to gain from its improved capabilities.

On a broader note, open source development in general has a proven track record of producing enterprise-grade software. You only have to look at the success of Linux to see how rapidly open source software can be adopted by enterprises once it reaches a suitable level of maturity and has the support of commercial vendors. Hadoop is no exception, and is likely to follow in the footsteps of Linux as it matures.

Additionally, we see the open source nature of Hadoop as one of the adoption drivers – as users know that they can avoid vendor lock in and have a choice of providers for their Hadoop training, support and services.

Hadoop may need to be ‘taken out of open source’
There is no reason to believe that a closed source Hadoop would deliver any functionality that could not be developed by the Apache Hadoop community. While a number of vendors offer closed source alternatives for individual components in the Hadoop stack, anyone offering a fully closed source alternative would suffer by not being able to compete with the collaborative development process and competitive commercial ecosystem that the open source development process enables.

In addition it is worth noting that Hadoop, along with other distributed data management projects including many of the NoSQL databases, were initiated by organizations like Google, Amazon and Yahoo in response to the inability of the established data management vendors to fulfil their data management requirements.

The established closed source data management vendors have had plenty of time to develop a ‘better’ Hadoop than Hadoop, and do not lack development resources, but have chosen to collaborate with Hadoop distributors and contribute to Hadoop instead.

A prime example is Microsoft, which in late 2011 abandoned its own Dryad distributed computing project in favour of contributing to Apache Hadoop. This is a sign that Hadoop has already won enough attention to make it difficult for any competing product to gain traction.

While we see vendors offering closed source alternatives for individual components in the Hadoop stack we do not believe that a full closed source alternative would be viable, or desirable from a customer’s perspective. There is no reason to believe that enterprise-grade improvements to Hadoop cannot be delivered by the Apache Hadoop community and the open source development process.

Tags:

CAOS Theory Podcast 2012.06.22

Topics for this podcast:

*Sauce Labs grows with fast Selenium application testing
*MySQL, NoSQL, NewSQL survey results and analysis
*Microsoft’s Linux love leaves out Red Hat
*Hadoop roundup with Cloudera, Hortonworks and VMware
*2012 Future of Open Source Survey highlights

iTunes or direct download (28:28, 5.1MB)

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Microsoft hearts Linux, just not Red Hat

Just when you thought it couldn’t top itself — having contributed Linux kernel code under the GPL, broadly supported Linux alongside Windows with its systems management and other software, and spun off a new subsidiary dedicated to openness, Microsoft showed yet more Linux and open source love recently, adding an impressive Linux lineup to supported software on its Azure cloud.

However, there’s one major Linux player that’s sort of getting left out of the love-fest. It’s enterprise Linux leader Red Hat and its Red Hat Enterprise Linux (RHEL), which has to sit by while other distributions, including RHEL community clone CentOS and market competitors SUSE and Ubuntu, get first-class treatment in Microsoft’s Azure cloud.

Read the full article at LinuxInsider.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

MySQL vs. NoSQL and NewSQL – survey results

Back in January we launched a survey of database users to explore the competitive dynamic between MySQL, NoSQL and NewSQL databases, and to to discover if MySQL usage is really declining – as had been indicated by the results of a prior survey.

The publication of the associated report took longer than expected, mostly because we expanded its scope to include revenue and growth estimates for the MySQL ecosystem, NoSQL and NewSQL sectors respectively, and with that report now published I am pleased to fulfil our promise to share the survey results.

We seem to be having some random embedding issues so for now the results can be found on SlideShare, adapted from the presentation given at OSBC earlier this week. For greater context, we have also included an explanation of each slide, below:

Slide 2: Provides an overview of the associated report – MySQL vs NoSQL and NewSQL 2011:2015, which is available here.

Slide 3: Explains why we launched the report. We once described as the crown jewel of the open source database world, since its focus on Web-based applications, its lightweight architecture and fast-read capabilities, and its brand differentiated it from all of the established database vendors and made for a potentially complementary acquisition. Today, the competitive situation is very different.

Slide 4: Oracle’s MySQL business faces competition from the rest of the MySQL ecosystem, as illustrated in Slide 5, many of which have emerged following Oracle’s acquisition of Sun/MySQL.

Slide 6: The emergence of these alternatives was triggered, in part, by concern about the future of MySQL. A previous 451 survey,conducted in November 2009, showed that there was real concern about the acquisition, with only 17% of MySQL users believing Oracle should be allowed to acquire MySQL.

Slide 7: The 2009 survey also showed that while 82.1% of respondents were already using MySQL, that figure was expected to drop to 72.3% by 2014. That survey was conducted amid a climate of fear, uncertainty and doubt regarding the future of MySQL, and one of the drivers for our current report was to see if that predicted decline occurred.

Slide 8: To put this in context, we asked the current survey sample (which included 205 database users) about their reaction to the acquisition. While the vast majority of MySQL users reported that they continued to use MySQL where appropriate, 5% indicated that they were more inclined to use MySQL, and 26% said they were less inclined to use MySQL. Not surprisingly the proportion of users less inclined to use MySQL was much higher amongst those abandoning MySQL than those sticking with MySQL.

Slide 9: We also asked respondents to rate Oracle’s ownership of MySQL on a range of very good to very bad. Overall, the balance tipped in favour of a negative perception of Oracle’s track record, while there was naturally a more negative perception of Oracle amongst those abandoning MySQL compared to MySQL mainstays. However, the results showed that the percentage of respondents rating the company’s performance ‘very good’ and ‘very bad’ was actually quite similar for both abandoners and mainstays. While those abandoning MySQL are more likely to have a negative perception of Oracle, it is not necessarily safe to assume that Oracle’s actions and strategy are the cause of the abandonment. Clearly there are other competitive forces at work.

Slide 10: Not least the emergence of NoSQL, as illustrated in Slide 11, and NewSQL, as illustrated in Slide 12.

Slide 13: Based on some very high profile examples of projects migrating from MySQL to NoSQL, there is a common assumption that NoSQL and NewSQL pose a direct, immediate threat to MySQL. We believe the competitive dynamic is more complex.

Slide 14: While 49% of those survey respondents abandoning MySQL planned on retaining or adopting NoSQL databases, only 12.7% said they had actually deployed NoSQL databases as a *direct replacement* for MySQL.

Slide 15: In comparison, there is much greater overlap between NewSQL and MySQL, but of a complementary nature. 33% of respondents retaining MySQL had considered, tested or deployed NewSQL database technologies, while approximately 75% of the NewSQL revenue for 2011 is from vendors that we also consider part of the MySQL ecosystem.

Slide 16: The results of our 2012 survey show that MySQL is currently the most popular database amongst our survey sample, used by 80.5% of respondents today.

Slide 17: However, it’s popularity is again expected to decline to 2014 and 2017. This indicates an accelerated decline in the use of MySQL, compared the findings of our 2009 survey. While that survey was conducted amid a climate of fear, uncertainty and doubt regarding the future of MySQL we are not aware of any specific reason why the 2012 sample, which was self-selecting, should have a disproportionately negative attitude to MySQL or Oracle.

Slide 18: MySQL’s predicted decline of 26.4 percentage points between 2012 and 2017 compares to a predicted decline of just 9.3 percentage points for Microsoft SQL Server, and only 5.9 percentage points for Oracle Database. In comparison, MariaDB, Apache Cassandra and Apache CouchDB are expected to increase in usage by 3.0 percentage points or greater between 2011 and 2017.

Slide 19: Although alternative MySQL distributions including MariaDB, Drizzle and Percona Server are expected to see increased adoption over the next five years, they are not growing at the same rate that MySQL is declining.

Slide 20: So where are those abandoning MySQL going to? Looking specifically at the 55 MySQL users who expect to abandon it by 2017 (which is admittedly a small sample, and therefore not to be considered statistically relevant) we see that PostgreSQL is the most popular database being retained or adopted over the same period, followed by Microsoft SQL Server, Oracle, MongoDB, and MariaDB.

Slide 21: This only tells part of the story, however. Just because a company is retaining Oracle Database, for example, does not necessarily mean that Oracle Database is being used as a replacement for the abandoned MySQL. We therefore also specifically asked survey respondents which databases they had considered, tested or deployed as a direct replacement for MySQL. The response from the 55 respondents planning to abandon MySQL again saw PostgreSQL, MariaDB and MongoDB as the most popular answers, followed by Apache CouchDB and Apache HBase.

Slide 22: While NoSQL database were well-represented in this list, we saw that anyone considering NoSQL considered multiple NoSQL databases. Per respondent, NoSQL databases were the least considered of all alternatives by existing MySQL users.

Slide 23: The survey results suggest that MongoDB is the most often considered, tested or deployed as a replacement or complement for MySQL, followed by Apache CouchDB, Apache HBase, Apache Cassandra/DataStax, and Redis.

Slide 24: NewSQL technologies that improve the scalability and performance of MySQL scored well, with eight of the top 10 most considered NewSQL technologies being directly complementing MySQL. Of the other two, one (Drizzle) is a derivative of MySQL, and the other (Clustrix) can also be used in a complementary manner as part of a MySQL cluster, although in the long-term is positioned as a direct alternative.

Slide 25: MariaDB is the member of the MySQL ecosystem most often considered, tested or deployed as a replacement or complement for MySQL, followed by Continuent Tungsten, Percona Server, MySQL Cluster, and Amazon RDS.

Slide 26: More than half of all MySQL users had considered, tested or deployed another relational database as a direct replacement, while over 40% had considered, tested or deployed a caching technology to complement MySQL. The memcached caching technology was the most widely-deployed of all the technologies we asked about, followed closely by PostgreSQL, which supported anecdotal evidence that a number of MySQL users are migrating to the other major open source transactional database.

Slide 27: For the record, the survey had 205 respondents. Primary job roles among respondents included: director/manager of IT infrastructure (18.0%); architect/engineer (17.6%); developer/programmer (15.6%); database/systems administrator (14.6%); consultant (14.1%); VP level or above (13.7%); analyst (3.4%); and line-of-business manager (2.9%).

Further survey analysis and perspective on the competitive dynamic between MySQL, NoSQL and NewSQL is available in the MySQL vs NoSQL and NewSQL report, which also includes market sizing and growth predictions for the three segments.

Tags: , , , ,

451 Research delivers market sizing estimates for NoSQL, NewSQL and MySQL ecosystem

NoSQL and NewSQL database technologies pose a long-term competitive threat to MySQL’s position as the default database for Web applications, according to a new report published by 451 Research.

The report, MySQL vs. NoSQL and NewSQL: 2011-2015, examines the competitive dynamic between MySQL and the emerging NoSQL non-relational, and NewSQL relational database technologies.

It concludes that while the current impact of NoSQL and NewSQL database technologies on MySQL is minimal, they pose a long-term competitive threat due to their adoption for new development projects. The report includes market sizing and growth estimates, with the key findings as follows:

• NoSQL software vendors generated revenue* of $20m in 2011. NoSQL software revenue is expected to rapidly grow at a CAGR of 82% to reach $215m by 2015.

• NewSQL software vendors generated revenue* of $12m in 2011 (of which $9m is also considered MySQL ecosystem revenue). NewSQL revenue is also expected to grow rapidly at a CAGR of 75% to reach $112m by 2015 (including $56m in MySQL ecosystem revenue).

• The MySQL support ecosystem generated revenue* of $171m in 2011 (including $9m from NewSQL technologies). MySQL ecosystem revenue is expected to grow at a CAGR of 40% to reach $664m by 2015 (including $56m in NewSQL revenue).

“The MySQL ecosystem is now arguably more healthy and vibrant than it has ever been, with a strong vendor committed to the core product, and a wealth of alternative and complementary products and services on offer to maintain competitive pressure on Oracle,” commented report author Matthew Aslett, research manager, data management and analytics, 451 Research.

“However, the options for MySQL users have never been greater, and there is a significant element of the MySQL user base that is ready and willing to look elsewhere for alternatives,”

As well as revenue and growth estimates, the report also includes a survey of over 200 database administrators, developers, engineers and managers. The survey findings include:

• While the majority of MySQL users continue to use MySQL where appropriate, the use of MySQL is expected to decline from 80.5% of survey respondents today to 62.4% by 2014 and just 54.1% by 2017.

• Despite the emergence of NoSQL and NewSQL database products, the most common direct replacement for MySQL among survey respondents today is PostgreSQL, which is also the focus of a recent burst of commercial activity.

• While 49% of those survey respondents abandoning MySQL planned on retaining or adopting NoSQL databases, only 12.7% of MySQL abandoners said they had actually deployed NoSQL databases as a direct replacement for MySQL.

“While there have been some high profile example of users migrating from MySQL to NoSQL database, the huge size of MySQL installed base means that these projects are comparatively rare,” commented Aslett.

The report describes how NoSQL database technologies are largely being adopted for new projects that require additional scalability, performance, relaxed consistency and agility, while NewSQL database technologies are, at this stage, largely being adopted to improve the performance and scalability of existing databases, particularly MySQL.

“NoSQL and NewSQL have not made a significant impact on the MySQL installed base at this stage but MySQL is no longer the de facto standard for new application development projects,” said Aslett. “As a result, NoSQL and NewSQL pose a significant long-term competitive threat to MySQL’s dominance.”

MySQL vs. NoSQL and NewSQL: 2011-2015 is now available to existing 451 Research subscribers. Non-clients can apply for trial access to 451 Research’s content.

*451 Research’s analysis of MySQL, NoSQL and NewSQL revenue is based on a bottom-up analysis of each participating vendor’s current revenue and growth expectations, and includes software license and subscription support revenue only. Revenue line items not included in these figures include hardware associated with the delivery of these services, revenue related to applications deployed on these databases, traditional hosting services, or systems integration performed by the vendors or other third parties.

The revenue estimates do not take into account unpaid usage of open source licensed MySQL, NoSQL and NewSQL software, and therefore represent only a fraction of the total addressable market. Based on the above revenue figures and other analysis, 451 Research estimates that the total value of the MySQL ecosystem in terms of ‘displaced’ proprietary software might equate to $1.7bn in 2011, while the NoSQL market had a displaced value of $195.7m and the NewSQL sector a displaced value of $99.4m.

Tags: , , , ,

Future of open source survey highlights progress, changes, challenges

451 Research was pleased to collaborate on the Future of Open Source Survey 2012 with North Bridge Venture Partners and Black Duck Software. This year’s survey garnered 740 responses from a variety of vendors and non-vendors in the industry. Overall, the survey highlighted some subtle and sometimes dramatic changes in what is driving open source software. It also made clear that while there is still a good degree of education and awareness yet to occur around open source software, there is a large amount of open source code making its way into today’s enterprise, webscale, consumer and other computing environments.

Some of the key findings:

*The survey reinforced the prominence and influence of open source software in the enterprise and in key trends driving it, as we and others have highlighted for some time with reports such as Seeding the Clouds and Mobility Matters. When asked which technology areas would see the most significant open source software community innovation from, respondents ranked ‘cloud’ highest at 40%, then ‘mobile apps’ (19%) and ‘mobile enterprise’ (15%) for a combined 34%, then ‘analytics’ with 10%. These areas are indicative of where we see open source software projects, communities, vendors and consortia continuing to broaden use of open source software.

*The survey asked what are the top barriers to selecting open source software when compared with proprietary alternatives, resulting in unfamiliarity (48%), lack of internal technical skills (47%), lack of vendor support (35%) and legal concerns about licensing (33%) as the top answers. Although this indicates there is still some trepidation and lack of awareness around open source and commercial options for support, other survey responses indicate open source software is still spreading to new industries and customer categories. When asked about the most important trend for open source software over the next two to three years, respondents identified the top choices as: adoption in non-technical segments such as government or healthcare (42%); enterprise adoption (40%) and growth in industry-specific communities (10%).

*The survey also showed there is a heavy volume of new, meaningful code coming out of open source software’s many communities. When asked what share of their deployed code they anticipate will be open source software over the next five years, about one third of survey respondents (32%) reported open source had already reached major deployment at 75% or more of their code. Another one third of respondents (30%) said open source will make up half to 75% or more of its deployed code. About a quarter of respondents (23%) indicated open source would make up 25-50% of their deployed code over the next five years, while 15% of respondents said the open source share of deployed code would be a quarter or less.

*We also saw a high rate of open source participation from the survey. When asked about community engagement with open source and their preferred method, 49% of respondents said consuming code, 36% said reporting patches or fixes, 31% said contributing new features, 28% said initiating new projects, 25% said contributing through partners or industry alliances. We believe this shows a high rate of open source participation beyond using code, which is also a meaningful contribution. This also indicates a greater willingness to get involved with open source projects and to start new projects.

*The survey also highlighted the changing drivers of open source software in the enterprise. When asked what are the top factors that make open source software attractive, respondents identified freedom from vendor lock-in (60%), lower acquisition and maintenance cost (51%), better quality (43%) and access to source code (42%) as the top answers. While we had seen vendor lock-in fade as a factor and cost as paramount two or three years ago, today vendor lock-in has become much more of a factor for customers. We believe this has to do wtih cloud computing and customers’ desire to maintain flexibility as they figure out how to best leverage cloud resources. The survey also showed that cost, which we also equate to time and efficiency, is always a strong factor, with 62% of respondents identifying reduced cost of development and maintenance as the main reason they use open source or initiate projects.

*The survey also reinforced our belief that while open source software lays the groundwork and underlies much of cloud computing, the cloud is also giving back to open source by providing vendors a way to differentiate free downloads from paid, cloud-based services. In fact, it seems support and services subscriptions are a much higher priority for open source software vendors than so-called ‘open core’ models that provide software for free and certain extensions, features or support as paid. When asked which revenue generation strategies are likely to create the most value for open source vendors over the next two years, respondents ranked an annual, repeatable support and service agreement as the top answer (52%). Other open source revenue models, such as ad-hoc services and support (41%), value-add subscription (40%), hosted or cloud software services (38%) all ranked higher than a closed-source license or open core model (12%).

For our full analysis on the results of the 2012 Future of Open Source Survey, see our Spotlight report. The results were also presented this week on a panel at the Open Source Business Conference and that presentation is available at the Open Source Delivers blog.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Back to the future of commercial open source

It’s been tempting to write a post about open source licensing trends and how they relate to commercial business strategies, given ongoing interest in our previous posts about the relative decline of the GPL.

Every time I start to write a post though I realise that I’d just be repeating myself, most notably The future of commercial open source business strategies from December 2011, but also Control and Community – and the future of commercial open source strategies from late 2010.

You can trace the origins of the theories and research in those posts back to The golden age of open source? in August 2010, and even further to Commercial open source business strategies in 2009 and beyond from early 2009.

That post in particular contains the core elements about why we believed we were at a tipping point with regards to commercial open source strategies, prompting the shift from vendor-led strategies that emphasised control via copyleft licenses, to community-led strategies that emphasised collaboration via permissive licenses.

The one aspect that those posts didn’t cover is what happens after this shift. That is a question that has recently been addressed by Simon Phipps, who predicts that the pendulum will swing to the centre and weak-copyleft licenses and specifically the recently released MPLv2.

While I don’t dispute the logic of that prediction, I can see nothing in the data that we have previously collected and analysed that indicates a shift to weak-copyleft. As you can see, while there was a strong shift from vendors towards non-copyleft licenses from 2007 onwards, we have seen no such shift with regards to weak-copyleft.

Which is not to say that it won’t happen – just that we see no evidence of it right now, and that we would have to see an enormous swing towards weak-copyleft licenses in the next couple of years. It will be interesting to see whether the release of MPLv2 will be the event that triggers that swing.

Mixed signals in IT’s great war over IP

Recent news that Microsoft and Barnes & Noble agreed to partner on the Nook e-reader line rather than keep fighting over intellectual property suggests the prospect of more settlement and fewer IP suits in the industry. However, the deal further obscures the blurry IP and patent landscape currently impacting both enterprise IT and consumer technology.

It is good to see settlement — something I’ve been calling for, while also warning against patent and IP aggression. However, this settlment comes from the one conflict in this ongoing war that was actually shedding some light on the matter, rather than further complicating it.

See the full article at TechNewsWorld.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , ,

CAOS Theory Podcast 2012.04.20

Topics for this podcast:

*OpenStack, Amazon, Eucalyptus and Citrix engage in open cloud warfare
*Microsoft spins off new company for openness
*Updates on automation players Puppet Labs and Opscode with Chef
*Percona turns attention to MySQL high availability
*Open APIs as the fifth pillar of modern IT openness

iTunes or direct download (28:42, 4.9MB)

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Reading between the lines of the Linux contributor list

The recently released Who Writes Linux kernel contributor list reveals that some of the usual supporters of Linux — Red Hat, SUSE, IBM, Intel, Oracle — remain firmly behind the open source OS.

There has also been a lot of attention on the other contributors, which now include Microsoft (Nasdaq: MSFT). What I find most fascinating about the Linux contributor list — beyond the increasing rate of code change with some 10,000 patches from 1,000 developers representing 200 companies in each quarterly kernel release — are the contributors that show some new direction and potential for Linux, in this case the processor players.

Whenever the Linux contributor report comes out, there is also typically some focus on those that use the Linux kernel code but do not necessarily appear among its list of core contributors.

One of the most frequent names to come up in this regard is Canonical, backer of the popular Ubuntu distribution.

Read the full article at LinuxInsider.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Announcing the Sixth Annual Future of Open Source Survey

Black Duck Software and North Bridge Venture Partners, in partnership with 451 Research, yesterday announced a collaboration to conduct the sixth annual Future of Open Source Survey.

The survey, an annual bellwether of the state of the open source industry, is supported by more than 20 open source software (OSS) industry leaders and is open to participation from the entire open source community.

The survey results point out market opportunities, identify issues affecting the enterprise adoption of open source, and foreshadow industry trends for 2012 and beyond. Open to the general public today, the survey closes at the end of April.

Survey results will be presented at the Open Source Business Conference (OSBC, May 20 – 21, 2012) at the Hyatt Regency San Francisco – Embarcadero during the keynote panel on opening day. Moderating the panel will be Tim Yeaton, CEO and President, Black Duck Software and Michael Skok, general partner at North Bridge Venture Partners. Yeaton and Skok will be joined by several industry executives including Tom Erickson, CEO, Acquia.

Take the survey here: http://www.zoomerang.com/Survey/WEB22F4B845DQ5

See results of last year’s survey here.

Tags: , , , ,

Open APIs: The Fifth Pillar of Modern IT Openness

Last year, I wrote about the key pillars of openness in today’s enterprise IT industry, highlighting open source software, real open standards, open clouds, and open data as the ‘Four Pillars of Modern IT Openness.’

More recently, I wrote about what I now consider to be the fifth pillar, which is open application programming interfaces (APIs). Of course, when we talk about ‘open’ anything — open source, open standards, open clouds, open APIs — there tends to be debate about what is really open, how we should define open and who should or should not be able to carry the phrase. My focus on open APIs and on APIs in general generated some good discussion, as well as some pushback, regarding the value of APIs compared to open source software, which APIs are open, and how open is open enough?

I want to make clear I am not saying open APIs are better than open source. The real point is that the activity, development and innovation happening around APIs — particularly as cloud computing and hybrid public-private use continues to evolve — is reminiscent of the way open source software began disrupting the industry some two decades ago.

The other point is that while customers are typically interested in open source software for flexibility, cost savings, mitigating vendor lock-in, performance, ROI or other reasons, my conversations with both vendors and customers indicate much of the integration in the cloud centers on the openness of the APIs. When customers have stable, documented APIs, it is often more conducive and effective to work there, rather than on the source code. If code, development and deployment are disrupted by closed, changing, weak or undocumented APIs, then developers, customers and the market are likely to quickly move on to other APIs, perhaps ‘open APIs’ that are well documented and include examples. Similar to the other pillars of modern IT openness — open source software, open standards, open clouds and open data, open APIs are most effective and efficient when combined with the others.

Let’s not let open APIs become another version of ‘open standards’ that were anything but 10 years ago. Instead, we should seek to use and call out truly open APIs, which would typically mean connection to open source software, open standards, open clouds and open data as well. However, we must also be aware of the threat, competition and pressure from APIs such as Amazon’s EC2 and AWS interfaces, which are not open source nor open standards, but nonetheless may be open enough for a majority of developers and market.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Open source moving in mobile

We got another reminder of how disruptive open source software is to mobile computing this week, when Linux and Android merged back together. This appears to be good news for a number of parties, but Android and Linux developers and users seem particularly likely to benefit. The inclusion of Android code in the Linux kernel and the ability for Linux developers to more easily work on the Android environment and applications also ties into some of the key topics we’ll be covering in a Webcast March 21 titled ‘Open Source, A Tale of Two Cities in the Mobile Enterprise,’ presented by 451 Research and Black Duck Software.

This webcast, as the title implies, will focus on how open source can present both challenges and opportunities as enterprises adapt to market changes and mobile devices. This includes the fact that open source software frameworks, pieces and development are all enabling new applications to be quickly developed and deployed. However, this presents tremendous pressure on enterprise IT teams already dealing with disruption and change from cloud computing and the trend of ‘devops,’ which blends application development with IT operations and application deployment. The Webcast will cover how open source software is mixing with devops and other trends, such as the consumerization of IT and BYOD, to both disrupt and develop the mobile enterprise. We will also highlight some key open source software technologies in the mobile space and highlight some observed best practices for both vendors and customers.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Open Source Coopetition Fueled by LF Growth

The Linux Foundation has come a long way since initiated in 2007 as the fusion of the Open Source Development Lab (OSDL) and Free Standards Group. At its start, I wondered why there was no membership or representation from Canonical, which was the hottest thing in Linux at the time.

Today, Canonical is a member of the Linux Foundation and the
organization continues to grow in its core of system software and Linux as well as in mobile devices and, more recently, the automotive industry — among my predictions for Linux strength in 2012.

The Linux Foundation has also gained some significant members and new groups of collaborators — the latest batch including graphics and microprocessor giant Nvidia.

Read the full story at LinuxInsider.

Tags: , , , , , , , , , , , , , , , , , , , , , , , ,

That’s not science: the FSF’s analysis of GPL usage

The Free Software Foundation has responded to our analysis of figures that indicate that the proportion of open source projects using the GPL is in decline.

Specifically, FSF executive director John Sullivan gave a presentation at FOSDEM which asked “Is copyleft being framed”. You can find his slides here, a write-up about the presentation here, and Slashdot discussion here.

Most of the opposition to the earlier posts on this subject addressed perceived problems with the underlying data, specifically that it comes from Black Duck, which does not publish details of its methodology. John’s response is no exception. “That’s not science,” he asserts, with regards to the lack of clarity.

This is a valid criticism, which is why – prompted by Bradley M Kuhn – I previously went to a lot of effort to analyze data from Rubyforge, Freshmeat, ObjectWeb and the Free Software Foundation collected and published by FLOSSmole, only to find that it confirmed the trend suggested by Black Duck’s figures. I was personally therefore happy to use Black Duck’s figures for our update.

John Sullivan is not overly impressed with the FLOSSmole numbers either, noting that while they are verifiable, they do leave a number of questions related to the breadth and depth of the sample, the relative activity of the projects, whether all lines of code and applications should be treated equally, and how packages with multiple licenses are treated.

These are all also valid questions. As we previously noted, a study that *might* satisfy all questions related to license usage would have to take into account how many lines of code a project has; how often it is downloaded; its popularity in terms of number of users or developers; how often the project is being updated; how many of the developers are employed by a single vendor; and what proportion of the codebase is contributed by developers other than the core committers.

John offers some evidence of his own that suggests that the use of the GPL is in fact growing. Anyone hoping for the all-encompassing study mentioned above is in for some disappointment, however. It is based on a script-based analysis of the Debian GNU’Linux distribution codebase.

Nothing wrong with the script-based analysis – but a single GNU/Linux distribution considered to be a representative sample of all free and open source software?

That’s not science.

Tags: , ,

CAOS Theory Podcast 2012.02.17

Topics for this podcast:

*NewSQL, new company in Akiban
*Discussion of APIs as the ‘new’ open source
*NoSQL leader 10gen grows, gets more agile
*Our coming report on Cloud Performance Management
*Zimory acquires sones NoSQL development team

iTunes or direct download (28:01, 4.8MB)

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , ,

Open APIs are the new open source

We’ve seen the rise of open source software in the enterprise and also beyond the IT industry, but the real keys to openness and its advantages in today’s technology world — where efficient use of cloud computing and supporting services are paramount — exist in open application programming interfaces, or APIs.

Open source software continues to be a critical part of software development, systems administration, IT operations and more, but much of the action in leveraging modern cloud computing and services-based infrastructures centers on APIs. Open APIs are the new open source.

Read the full story at LinuxInsider.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Last chance to take part in our MySQL/NoSQL/NewSQL survey

Thanks to everyone who has already taken part in our survey exploring changing attitudes to MySQL following its acquisition by Oracle and examining the competitive dynamic between MySQL and other database technologies, including NoSQL and NewSQL.

The response has been great and even a quick look at the results makes for interesting reading, particularly in the light of our previous findings which indicated declining MySQL usage.

I am really looking forward to having the opportunity for a deep dive into the results and break out the figures to get a better understanding of the potential impact of alternative MySQL distribution and support providers, as well as NoSQL and NewSQL, on continued usage of MySQL.

The survey results will be made freely available on our blogs, as well as being included in a long format report containing our additional analysis and research related to the MySQL ecosystem and competitive dynamic.

Right now, however, is your last chance to contribute to the survey and get your voice heard. There are just 12 questions to answer, spread over four pages, and the entire survey should take no longer than five minutes to complete. All individual responses are of course confidential.

The survey will close in 24 hours.

Tags: , ,

Is MySQL usage really declining?

If you’re a MySQL user, tell us about your adoption plans by taking our current survey.

Back in late 2009, at the height of the concern about Oracle’s imminent acquisition of Sun Microsystems and MySQL, 451 Research conducted a survey of open source software users to assess their database usage and attitudes towards Oracle.

The results provided an interesting snapshot of the potential implications of the acquisition and the concerns of MySQL users and even, so I am told, became part of the European Commission’s hearing into the proposed acquisition (used by both sides, apparently, which says something about both our independence and the malleability of data).

One of the most interesting aspects concerned the apparently imminent decline in the usage of MySQL. Of the 285 MySQL users in our 2009 survey, only 90.2% still expected to be using it two years later, and only 81.8% in 2014.

Other non-MySQL users expected to adopt the open source database after 2009, but the overall prediction was decline. While 82.1% of our sample of 347 open source users were using MySQL in 2009, only 78.7% expected to be using it in 2011, declining to 72.3% in 2014.

This represented an interesting snapshot of sentiment towards MySQL, but the result also had to be taken with a pinch of salt given the significant level of concern regarding MySQL future at the time the survey was conducted.

The survey also showed that only 17% of MySQL users thought that Oracle should be allowed to keep MySQL, while 14% of MySQL users were less likely to use MySQL if Oracle completed the acquisition.

That is why we are asking similar questions again, in our recently launched MySQL/NoSQL/NewSQL survey.

More than two years later Oracle has demonstrated that it did not have nefarious plans for MySQL. While its stewardship has not been without controversial moments, Oracle has also invested in the MySQL development process and improved the performance of the core product significantly. There are undoubtedly users that have turned away from MySQL because of Oracle but we also hear of others that have adopted the open source database specifically because of Oracle’s backing.

That is why we are now asking MySQL users to again tell us about their database usage, as well as attitudes to MySQL following its acquisition by Oracle. Since the database landscape has changed considerably late 2009, we are now also asking about NoSQL and NewSQL adoption plans.

Is MySQL usage really in decline, or was the dip suggested by our 2009 survey the result of a frenzy of uncertainty and doubt given the imminent acquisition. Will our current survey confirm or contradict that result? If you’re a MySQL user, tell us about your adoption plans by taking our current survey.

Tags: , , , ,

CAOS Theory Podcast 2012.01.20

Topics for this podcast:

*Hadoop v1.0 and year ahead
*Oracle-Cloudera deal for more Hadoop
*Oracle’s ‘Sun spot’ with Solaris
*Open Source M&A outlook for 2012
*Our new MySQL/NoSQL/NewSQL survey

iTunes or direct download (28:49, 4.9MB)

Tags: , , , , , , , , , , , , , , , , , , , , , , ,