Open source woven into latest, hottest trends

We may not see or hear much about open source in the latest cloud or Big Data offerings, but it’s playing a significant role in the most disruptive trends in enterprise IT.

Just as we’ve seen with open source in cloud computing, it is an integral part of trends that currently are disrupting consumer and enterprise IT markets, including hybrid cloud computing, automation and devops, and Big Data.

Read the full article at LinuxInsider.

Open APIs: The Fifth Pillar of Modern IT Openness

Last year, I wrote about the key pillars of openness in today’s enterprise IT industry, highlighting open source software, real open standards, open clouds, and open data as the ‘Four Pillars of Modern IT Openness.’

More recently, I wrote about what I now consider to be the fifth pillar, which is open application programming interfaces (APIs). Of course, when we talk about ‘open’ anything — open source, open standards, open clouds, open APIs — there tends to be debate about what is really open, how we should define open and who should or should not be able to carry the phrase. My focus on open APIs and on APIs in general generated some good discussion, as well as some pushback, regarding the value of APIs compared to open source software, which APIs are open, and how open is open enough?

I want to make clear I am not saying open APIs are better than open source. The real point is that the activity, development and innovation happening around APIs — particularly as cloud computing and hybrid public-private use continues to evolve — is reminiscent of the way open source software began disrupting the industry some two decades ago.

The other point is that while customers are typically interested in open source software for flexibility, cost savings, mitigating vendor lock-in, performance, ROI or other reasons, my conversations with both vendors and customers indicate much of the integration in the cloud centers on the openness of the APIs. When customers have stable, documented APIs, it is often more conducive and effective to work there, rather than on the source code. If code, development and deployment are disrupted by closed, changing, weak or undocumented APIs, then developers, customers and the market are likely to quickly move on to other APIs, perhaps ‘open APIs’ that are well documented and include examples. Similar to the other pillars of modern IT openness — open source software, open standards, open clouds and open data, open APIs are most effective and efficient when combined with the others.

Let’s not let open APIs become another version of ‘open standards’ that were anything but 10 years ago. Instead, we should seek to use and call out truly open APIs, which would typically mean connection to open source software, open standards, open clouds and open data as well. However, we must also be aware of the threat, competition and pressure from APIs such as Amazon’s EC2 and AWS interfaces, which are not open source nor open standards, but nonetheless may be open enough for a majority of developers and market.

Community Linux love from Microsoft

One of big stories out of the Open Source Business Conference this week was Microsoft’s announced support for the CentOS community Linux distribution, a free clone of RHEL that nonetheless enjoys significant enterprise and cloud computing use, as we’ve covered extensively, including a special report that is currently being updated, in part, with a new survey.

This is not the first time MS has displayed love for unpaid, community Linux, given its 2009 contribution of GPL-licensed code to the Linux kernel. This was significant in that it was contribution and participation by Microsoft in the Linux kernel, beyond one of its partner’s Linux distributions, such as the case of Novell and SUSE Linux and more recently, Red Hat and its RHEL for mutual, customer-demanded virtualization support (451 subscribers) between Microsoft and Red Hat.

It seems Microsoft understands that unlike pirated Windows, which it considers a loss, the use of free, unpaid Linux — particularly by large enterprise, government and other organizations — is a big opportunity for it.

True, use of community Linux is typically driven by cost savings and the capability of sizable organizations to self-manage their Linux servers, often involving no payment. However, our research indicates there is often is still a need for higher level support and, more commonly, the ‘insurance factor’ of having a commercial vendor behind your infrastructure software so you, or your boss or board, have someone to call or blame if things go wrong. Microsoft is capable of supporting CentOS in both cases of technical support and being the insurance for an organization. It will be interesting to see the kind of reaction and traction the company gets from customers, presumably Windows shops, running CentOS.

It was only a couple of years ago we were writing about the death, and ongoing life of CentOS.

Today, it continues to be one of the most fascinating open source software projects in that it has no formal commercial backer, not even a foundation, but yet benefits from a solid, dedicated development team that continues to push the OS forward. We, along with Microsoft, continue to hear about use of CentOS increasingly in cloud computing, where it can be used, often free of charge, to add, subtract, scale and scrap as needed. It is, like other Linux distributions, also popular among hosting and other service providers, who again are primarily building public, private and hybrid cloud environments and ecosystems.

This is why again it is very interesting to see Microsoft supporting CentOS with HyperV and Windows. It’s not the first vendor to do so, as server giant HP has supported CentOS, Debian and other community distros to some extent in its server and support offerings. Microsoft’s CentOS support is certainly another example of how the landscape and market for various Linux distributions and operating systems in general is currently undergoing disruption.

The four pillars of modern IT openness

Given our coverage regarding the impact of open source software on cloud computing, development and IT admin trends such as devops and the drivers of open source from the perspective of open source software users and customers, there’s no question open source software is the driving force of openness in today’s enterprise IT. Still, there are other factors emerging as significant. In fact, I see four pillars of openness that are relevant today, each at different stages and with shifting importance, but all with a role in what is or is not open in today’s enterprise IT.

*The first pillar is open source software itself, and it is perhaps the most established and strongest of them all. As referenced above, open source continues to play a growing role in the latest enterprise IT trends, and it is aligned with customer empowerment, which lends credence to the idea it will continue to grow and spread. Open source software and its principles have impacted nearly all discussions of ‘open’ and open source stands as the true measure of openness.

*The second pillar is represented by open standards, which have transformed from somewhat of a joke in open source circles to a more true representation of the term and the words. Rather than a single vendor’s effort to get a technology standard viewed as open, today’s open standards have to really be open. Why? The market no longer accepts open standards that are open in name only. True, there are still plenty of aspects to standards, even open standards, that makes them more closed than open, but the situation has generally improved, and with continuing customer empowerment, vendor collaboration and the influence of open source software driving standards that are truly more open for participation and community. We do wonder what types of standards will be open enough as we push further into cloud computing, devops and other driving trends, but the overall industry movement now seems to be toward openness in standards. It’s not just analysts saying so, either. The market dictates standards arguably more than anything esle, and the market now demands (almost all of the time) they are open.

*A third pillar of openness today is undoubtedly cloud computing, for which open source software and open standards are critical. The prevalence of Linux and a lot of other open source software is also apparent in some of the latest, interesting partnerships and integrations, such as Ubuntu distributor Canonical’s deal with Autonomic Resources for more channel business and Red Hat’s cloud computing pact with Eucalyptus Systems. We also continue to see signs that cloud computing is ‘giving back’ to open source software, primarily by making the separation of free and open source software available for free and for pay clearer via paid services and cloud access.

*The final pillar, and arguably the least evolved and mature, is open data. While open source software (particularly data-centric open source such as Hadoop), open standards and the role of openness in cloud computing have driven discussions of openness, data remains a source of lock-in according to users and a source of demand for closed technology according to users. In considering a concept such as Matt’s ‘total data,’ and some of the points raised in this post, I have no doubt there will be meaningful debates and moves to make data and data access more open in the industry. However, I also believe that the desire to continue to keep customers, the need to keep data closed (including privacy) and the nature of data will make open data the slowest to evolve. Nevertheless, it will eventually give way to the upside of shared data, collaboration, and a global market of not only goods and services, but also ideas. In doing so, open data will likely continue to be a pillar of openness in the modern IT landscape.

We will continue to watch these pillars of openness, and expect the significance of all four to continue as well, given their interconnection and importance to what both IT providers and consumers, particularly the successful ones, are doing today.

Channeling of open source continues

We’re seeing yet more signs of IT and software sales channels opening up to open source. The latest deal, whereby Canonical will make Ubuntu Enterprise Cloud (UEC) on Dell blade servers available to public users through government middleman Autonomic Resources, reinforces the idea that system integrators (SIs), VARs, service providers and others are quickly and increasingly including open source options and products in their offerings and recommendations.

What’s driving this channel awakening and why now? I believe it is a combination of two things: open source software’s association with cost savings and its significance and prominence in cloud computing. There are other factors as well: open source software’s growing number of commercial backers and evolved enterprise credibility, specific instances where access to source code is paramount, reliability, performance and innovative reasons, and others. There is also momentum on the part of open source users and customers, particularly those in the public sector, that are now making their open source use more official by policy and procedure. However, I believe it is cost and cloud that, in combination, are truly driving open source software right now.

We’ve covered the growth of open source software in traditional technology sales channels — i.e. the public sector, SMBs, global SIs and local suppliers — in more depth in a recent report (451 subscribers only). The Autonomic Resources deal for Canonical, commercial backer of the cloud-popular Ubuntu Linux, also comes as we watch some interesting partnerships and integrations, as well as the continued cloud progress of Linux in general.

There’s no question the channel is critical to deliver software and other IT products and services to broader swaths of customers. It is also becoming more apparent that the rise of open source software in IT sales channels is also unquestionable.

Linux thunderstorm in the clouds

There’s a thunderous battle brewing in the enterprise IT market and it’s all about Linux. The battle brings new names to the fight as well, from the largest of cloud providers Oracle, Amazon and VMWare to smaller upstarts tuning Linux specifically for cloud computing and its users, such as the hoster and service provider-oriented CloudLinux.

As for Oracle and its introduction of Oracle Unbreakable Enterprise Kernel, this has been causing a stir in enterprise Linux circles. However, I have to agree with Red Hat CEO Jim Whitehurst that this latest Oracle move is quite reminiscent of when Oracle Unbreakable Linux, later known as Oracle Enterprise Linux, hit the market in 2006-2007. It was supposed to wipe out Red Hat, but it didn’t. I expect with plenty of Oracle shops in which to grow, Oracle’s Unbreakable Enterprise Kernel will manage to move ahead without much disruption to Red Hat’s growth, which based on its latest earnings call is healthy.

Oracle’s Solaris, on the other hand, continues to face an ongoing challenge of defection to Linux, whether Red Hat, Ubuntu, CentOS, SUSE or other OS including BSD. As we’ve covered on the CAOS blog, Oracle’s decision to kill off OpenSolaris may impact customers wary of the company’s growing presence throughout the software stack and now in hardware, too, leading many to finally make a decision on moving ahead with Solaris or moving to Linux. There’s also another option on the table now that we’ve seen the launch of OpenIndiana, a fork of OpenSolaris supported by the Illumos Foundation.

There’s another computing giant making noise around Linux: Amazon, which, similar to Oracle, is seeking to supplant use of and payment for Red Hat’s or other Linux distributions by making available its own Linux Amazon Machine Image (AMI) optimized for Amazon Web Services. There’s no question the presence and continued, increasing involvement from cloud players such as Amazon will impact what other vendors are offering and what customers are consuming in the cloud. This, and the announcement of OpenStack from Rackspace, are just the kinds of thing we can expect to see more of, as covered in our latest CAOS special report ‘Seeding the Clouds.’

It’s not just the largest of vendors that are getting in on the cloud computing action and traction for Linux. Another example is CloudLinux, a smaller vendor that aims its Linux, based on CentOS with added control panel and other features, squarely at service providers interested in cloud computing. Furthering its own efforts, CloudLinux recently announced continuing growth and an interesting deal with Ksplice to provide hosting service providers CloudLinux kernel updates without rebooting servers.

CloudLinux, as well as Canonical and its Ubuntu Linux, highlight the head start Red Hat, and to some extent its enterprise Linux counterpart Novell, have given other companies ready and willing to serve up Linux in the cloud. These vendors, many of which base their own offerings on CentOS, also highlight the ongoing presence of community Linux in cloud computing, a topic we’ve covered as well.

Just when it seemed things couldn’t get any crazier or more competitive, we haven’t even touched on the rumored, potential acquisition of Novell’s SUSE Linux by virtualization giant and cloud contender VMware, which has significant implications for the market. Sure VMware has been talking for years about how its software obfuscates the need for an OS, but hey, wasn’t it only a year ago when Oracle acted as if cloud computing didn’t exist? Turns out the clouds are for real, and it turns out Linux is more relevant than it ever has been. The real interesting thing is that in many ways, the competition is just getting started.

Open source in the clouds and in the debates

We continue to see more evidence of the themes we discuss in our latest CAOS special report, Seeding the Clouds, which examines the open source software used in cloud computing, the vendors backing open source, the cloud providers using it and the impact on the industry.

First, as usual, we are seeing consistencies between our own research — which indicates open source is a huge part of today’s cloud computing offerings from major providers like Amazon, Google, Rackspace, Terremark and VMware — and that of code analysis and management vendor Black Duck. In its analysis of code that runs the cloud, Black Duck also found a preponderance of open source pieces, in many cases the same projects we profile in our report.

Indeed, open source software is an important part of the infrastructure, data and application layers of today’s cloud computing stacks with significant use of Linux, open source hypervisors KVM and Xen, open source data technologies such as MySQL, PostgreSQL, Hadoop, NoSQL and memcached and open source languages such as Java, PHP, Python and Ruby on Rails.

There will be plenty of users and customers content to use non-open source options that serve as the defacto standards, but we do see a move to higher-level, production and mission critical use, which represents continued commercial opportunity for open source and other vendors.

One of the more subtle effects of all this open source in the cloud, as covered in Seeding the Clouds, is the impact on discussions, debates and downright fights in the market. There is much scrutiny on claims of being open, technical aspects of open and what ‘open cloud’ means. A prime example is the Twisticuffs that have gone on between Simon Crosby of XenSource and Citrix, discussing OpenCloud and the response from Open Cloud Initiative co-founder Sam Johnston, who claims this is misuse of the open label.

We already saw open source playing a role in the discussions and debates about open clouds, open APIs and open data, and this latest confrontation is evidence that role continues to be significant. We still wonder though about the question of open enough as we contemplate openness in the clouds.

CAOS Theory Podcast 2010.04.16

Topics for this podcast:

*The latest in VC funding for open source
*VMware’s SpringSource buys cloud messenger Rabbit
*Open source monitoring vendors’ key cloud partnershps
*Oracle moves ahead, back on MySQL, OpenSolaris

iTunes or direct download (25:38, 7MB)

Open source evolving with the cloud

We’ve covered the significance of open source software in cloud computing, both with the emergence of cloud models and more recently from the perspective of customers. In the first weeks of 2010, we see open source is maintaining, if not growing, its role in cloud computing. There are also indications open source and its use are evolving in the cloud.

While we’ve seen open source software used both to build cloud computing infrastructure and offered among cloud computing services, we had thought that the building of clouds with open source may be the greater opportunity given typical and historical adoption and use patterns. However, we are seeing continued examples of additional open source offerings in the cloud.

One such example is MuleSoft’s new offer of Tomcat application server via the GoGrid cloud. The product, MuleSoft Cloudcat, consists of cloud-based Apache Tomcat on GoGrid with commercial support from MuleSoft.

We’re also seeing examples of new open source software for the cloud. We’ve covered the use of unpaid, community Linux in the cloud, but a new cloud-specific distribution, CloudLinux, may also have some interesting implications, particularly for hosters and other service providers. CloudLinux, compatible with Red Hat Enterprise Linux clone CentOS, is commercially backed and supported by a company of the same name.

Cloud services and software are not the only sign of open source’s continued prominence in cloud computing. Also showing us that open source is maturing along with cloud computing: a new partnership between Terracotta and Eucalyptus. These two open source cloud players obviously see the benefits of working together and they’ll be integrating technology and teaming on sales and marketing.

We’ve also seen recently that the community aspects of open source continue to hold importance in cloud computing. In response to a perceived movement away from open source, a project dubbed OpenECP has forked from Enomaly’s Elastic Computing Platform. Citing ‘abrupt commercialization in November 2009,’ OpenECP backers indicate they will maintain free availability and provide community support. Interestingly, the OpenECP project chose to license it under the Affero GPLv3, and we’re watching licensing moves to see if cloud computing prompts more use of AGPL.

All of this shows how open source continues to play a vital role in cloud computing, enabling a wide range of vendors and providers to both build cloud computing infrastructure and applications using open source, and to offer open source via cloud computing to enterprise and other customers.

Some interesting firsts for cloud OS

One thing that seems clear in cloud computing right now — the combination of operating system, hypervisor, clustering, applications and other cloud infrastructure components in the mix is creating some interesting competition. We’ve written before about the fight over the OS role and its relevancy as hypervisor vendors race to cover the OS parts, OS vendors race to cover the hypervisor parts and so on. We’re now seeing a similar battle in cloud computing, where open source looms large, as other vendors step up to the opportunities.

VMware’s announcement of vSphere, billed as ‘the industry’s first cloud operating system,’ is a good example of how this fight continues. More specifically, VMware says vSphere 4 is ‘the first operating system for building the internal cloud, enabling the delivery of efficient, flexible and reliable IT as a service.’ In her report on vSphere, our own 451 Group Research Director Rachel Chalmers highlights the automation and self-service tools of vSphere, but also points out much of this doesn’t really arrive until VMware’s vCenter Suite later this year. In addition, while vSphere offers a good virtualized abstraction layer and tools to build internal clouds on commodity hardware, Xen is free and open source and is broadly used for cloud infrastructure.

In terms of the ‘first cloud OS,’ I think Ubuntu Linux vendor Canonical may beg to differ. The company’s Ubuntu Linux is already a popular choice for cloud deployment thanks to its free availability and lack of licensing royalties that can quickly cancel out cloud cost advantages. With its latest release this week, Ubuntu 9.04, Canonical is doing more to back this cloud deployment of Ubuntu. Its new server version is tuned for the cloud, primarily thanks to incorporation of the Eucalyptus open source clustering and cloud infrastructure software. Similar to VMware, which is offering bits and pieces of full functionality that is yet to come, Canonical offers a ‘preview’ of Ubuntu Eneterprise Cloud in 9.04. For its part, Canonical bills UEC and Ubuntu as ‘the first commercially-supported distribution to enable businesses to build cloud environments inside their firewalls.’ It is also worth noting Ubuntu Server Edition 9.04 will itself be available on Amazon’s Elastic Compute Cloud (EC2).

These are certainly a couple of interesting ‘firsts.’ It is also perhaps the first time these two vendors are competing so directly. We should probably expect to see more of this as a variety of vendors large and small from different software areas push into the clouds.