I attended the inaugural Enterprise Search Europe in London October 24-25 and was impressed with a few things. Firstly the attendance was stronger than I thought, perhaps around 100, although that includes sponsors’ attendees. There were half a dozen main sponsors and two vendors I spoke to said they wanted to sponsor it but were too late – so there is demand there for such an event. Chairman Martin White of Intranet Focus did an excellent job of keeping things moving along and stepped in to fill a gap in the program caused by last-minute absentees. He’s summed up the whole conference himself here.

I was on two panels, one more formal than the other. I was responding to the thoughts of Funnelback’s chief scientist David Hawking, who talked of current research in the enterprise search area and contrasted it with the wealth of research done in Web search. I pointed out that if some organizations such as TREC made its output actually usable by vendors (who aren’t permitted to say where they come in TREC rankings) then it would get much more support.

In the evening in the Hand & Flower pub across the road from the hotel for the latest Enterprise Search Meetup London I was one of two permanent panelists (Laura Wilber of Exalead being the other), with a goldfish bowl set-up where anyone wanting to ask a question had to joint the panel’s spare seats to do it from there. It made for some pretty lively discussions about what the next big things might be impacting enterprise search (big data being one of them, of course), helped by the  beer & wine paid for by Exalead.

The evening in the pub might have been livelier than the panel at the conference itself, but the themes were similar, namely that enterprise search as a use case and value proposition is dead; people don’t buy enterprise search engines to ‘find stuff’ within their organization anymore. There has to be a more tangible use case, such as electronic discovery, or fraud detection and the like. I’ve written about this recently.

Two other analysts were at the conference – Alan Pelz-Sharpe of RealStoryGroup and Mike Davis of Ovum. Alan has penned his forthright thoughts here and Mike and I talked enough to know that the three of us – and plenty of others at the event that weren’t analysts but actual users – were thinking similar thoughts. So despite the fact that the conference is called Enterprise Search Europe and most people seem to think that enterprise search as a concept is redundant, I would recommend the event for next year, by when I suspect it may be called something else.

As a follow-up to Matt’s post last week showing where he’ll be speaking during Q4, here’s some more updates of other in the information management team speaking at various events this quarter.

First up I’m chairing and speaking at IQPC’s Enterprise Information Management Exchange in London on October 10-11. I’m speaking to a mainly C-level end user audience about information risk management, moderating a panel on  how to make the most of your information assets and brushing off my MC-ing skills to keep the whole show moving along.

Next up I’ll be back in NYC at Text Analytics World giving a slightly shorter version of a similar presentation on October 19 (which I’ll be refining and also presenting at Predictive Analytics World in London on November 30).

On October 24 I’m on the opening panel of Enterprise Search Europe, discussing the issues brought up by the keynote presentation by Funnelback’s David Hawking, among other things

On October 27 David Horrigan will be attending Guidance Software’s Federal Summit in Washington, DC where he’ll be moderating a panel called e-Discovery in the cloud. This is an invite-only one being handled by Guidance, so I don’t have a link unfortunately.

Into November and I’ll be attending the e-Discovery and e-Investigations Forum in London on November 10. There I’ll be discussing the choice available to end users in e-Discovery in a session called: A buyers guide to navigating the info management and e-discovery technology marketplace.’

The following week I’ll be in Munich at IQPC’s Information Retention & e-Discovery Exchange where I’m sitting on a couple of panels – one on social media in e-Discovery and another on technology in this area.

Finally this quarter  Kathleen Reidy will be attending Gilbane’s annual gathering of enterprise content management mavens where she’s moderating a panel entitled ‘Get Ready for Big Data.’

We hope to see some of you at one or more of these events in Q4.

Customers of The 451 Group would have seen my report on the enterprise search market published September 15. If you are a client, you can view it here. I thought it would be useful to provide a condensed version of the report to a wider audience as I think the market is at an important point it in its development and it merits a broader discussion.

The enterprise search market is morphing before our eyes into something new. Portions of it are disappearing, and others are moving into adjacent markets, but a core part of it will remain intact. A few key factors have caused this, we think. Some are historical, by which we mean they had their largest effect in the past, but the ongoing effect is still being felt, whereas the contemporary factors are the ones that we think are having their largest impact now, and will continue to do so in the short-term future (12-18 months).

Historical factors

• Over-promising and under-delivery of intranet search between the last two US recessions, roughly between 2002 and 2007, resulting in a lot of failed projects.
• A lack of market awareness and understanding of the value and risk inherent in unstructured data.
• The entrance of Google into the market in 2002.
• The lack of vision by certain closely related players in enterprise content management (ECM) and business intelligence (BI).

Contemporary factors

• The lack of a clear value proposition for enterprise search.
• The rise of open source, in particular Apache Lucene/Solr.
• The emergence of big data, or total data.
• The social media explosion.
• The rapid spread of SharePoint.
• The acquisitive growth of Autonomy Corp.
• Acquisition of fast-growing players by major software vendors, notably Dassault Systemes, Hewlett-Packard and Microsoft.

The result of all this has been a split into roughly four markets, which we refer to as low-end, midmarket, OEM and high-end search-based applications.

Entry-level search

The low-end, or entry-level, enterprise search market has become, if not commodified, then pretty close to it. It is dominated by Google and open source. Other commercial vendors that once played in it have mostly left the market.

The result is that potential entry-level enterprise search customers are left with a dichotomy of choices: Google’s yellow search appliances that have two-year-term licenses and somewhat limited configurability (but are truly plug-and-play options) on the one hand, and open source on the other. It is a closed versus a very open box, and they have different and equally enthusiastic customer bases. Google is a very popular department-level choice, often purchased by line-of-business knowledge workers frustrated at obsolete and over-engineered search engines. Open source is, of course, popular with those that want to configure their search engine themselves or have a service provider do it and, thus, have a lot of control over how the engine works, as well as the results it delivers. Apache Lucene is also part of many commercial, high-end enterprise search products, including those of IBM.

Midmarket search

Mid-market search is a somewhat vague area, where vendors are succeeding in deals of roughly $75,000-250,000 selling intranet search. This area has thinned out as some vendors have tried to move upmarket into the world of search-based applications, but there are still many vendors making a decent living here. However, SharePoint has had a major effect on this part of the market, and if enterprises already have SharePoint – and Microsoft reckons more than 70% have at least bought a license at some point already – then it can be tough to offer a viable alternative. However, if SharePoint isn’t the main focus, then there is still a decent business to be had offering effective enterprise search, often in specific verticals, albeit without a huge amount of vertical customization.

OEM

The OEM search business has become a lot more interesting recently, in part due to which vendors have left it, leaving space for others. Microsoft’s acquisition of FAST in early 2008 meant one of the two major vendors at the time had essentially left the market entirely, since its focus moved almost entirely to SharePoint, as we recently documented. The other major OEM vendor at the time was Autonomy, and while it would still consider itself to be so, we think much of its OEM business, in fact, comes from document filters, rather than the OEMing of the IDOL search engine. Autonomy would strongly dispute that, but it might be moot soon anyway – it now looks as if it will end up as part of Hewlett-Packard following the announcement of its acquisition at a huge valuation, on August 18.

Those exits have left room for the rise of other vendors in the space. Key markets here include archiving, data-loss prevention and e-discovery. Many tools in these areas have old or quite basic search and text analysis functionality embedded in them, and vendors are looking for more powerful alternatives.

Search-based applications

The high end of the enterprise search market has become, in effect, the market for search-based applications (SBA) – that is, applications that are built on top of a search engine, rather than solely a relational database (although they often work alongside a database). These were touted back in the early 2000s by FAST, but it was too early, and FAST was too complex a set of tools to give the notion widespread acceptance. But in the latter part of the last decade and this one, SBAs have emerged as an answer to the problem of generic intranet search engines getting short shrift from users dissatisfied that the search engines don’t deliver what they want, when they want it.

Until recently, SBAs have mainly been a case of the vendors and their implementation partners building one-off custom applications for customers. But they are now moving to the stage where out-of-the-box user interfaces are being supplied for common tasks. In other words, it’s maturing in a similar way to the application software industry 20 years ago, which was built on top of the explosion in the use of relational databases.

We’ve seen examples in manufacturing, banking and customer service, and one of the key characteristics of SBAs is their ability to combine structured and unstructured data together in a single interface. That was also the goal of earlier efforts to combine search with business-intelligence tools, which often simply took the form of adding a search engine to a BI tool. That was too simplistic, and the idea didn’t really take off, in part because search vendors hadn’t paid enough attention to structure data.

But SBAs, which put much more focus on the indexing process than earlier efforts, appear to be gaining traction. If we were to get to the situation where search indexes are considered a better way of manipulating disparate data types than relational databases, that would be a major shift (see big data). Another key element of successful SBAs is that they don’t look like traditional search engines, with a large amount of white space and a search bar in the middle of the screen. Rather, they make use of facets and other navigation techniques to guide users through information, or often simply to present the relevant information to them.

As I mentioned, there’s more in the full report, including more about specific vendors, total (or big) data and the impact of social media. If you’d like to know more about it, please get in touch with me.

We have just published our annual report on the e-Discovery and e-Disclosure industries. This year we’ve subtitled the report ‘Crossing Clouds and Continents.’

This reflects a couple of the main themes of the report that are directly related: the rise of cloud computing within e-Discovery and the effect it has on those involved in e-Discovery in terms of how much simpler it makes it to store data in all sorts of locations. That of course then rasises issues of who is responsisble for that data and under  what jurisduction it falls. Other issues we focus on in the report include:

• Changes in the legal sector in the US & UK
• In-sourcing & out-sourcing of e-Discovery by corporations and law firms
• European e-Discovery
• Social media
• Bribery, corruption & fraud
• Products & technologies, mapped to EDRM and beyond
• User case studies in healthcare, law & government (financial regulators)
• M&A – both the recent surge and a look ahead to what’s next
• Profiles of 30+ software and service providers

To find out more about it and how to get a copy, you can visit this page or contact myself directly.

Just after the HP call about its Q3 numbers and the deal, here’s my initial (very) quick take as it’s late here in London:

• This deal is about getting serious about software under Leo Apotheker. It gives HP a real information management story, greatly boosting its presence in the archiving, e-Discovery and enterprise search businesses.
• However, company cultures are not complementary, the HP way is a long way from the hyper-aggressive sales and marketing culture at Autonomy. Maintaining Autonomy as a separate entity run by Mike Lynch proves this and calls into question how much real synergy can be had from such a structure. I cannot see that being sustained.
• This instantly makes HP a bigger e-Discovery player than IBM or any of the major IT firms.
• Product overlap exists in document and records management but gets HP into the web content management and website optimization markets.
• Autonomy has resisted deals over the years as its market capitalization ballooned as it went on its own acquisition binge. Autonomy couldn’t have waited much longer as it would have grown too big to be swallowed by even the largest predator.
• At least Autonomy customers will now have a services organization to call on after they’ve bought the software. Customer support and after sales service has not been a strength of Autonomy.
• This leaves the FTSE 100 with just one software firm of note.

The 451 Group is holding two conference in London later this month.

The first is our European client event, which is being held on Monday June 27 and features three analyst presentations and one from Steve O’Connor, Director of Technology for Parliamentary ICT at the UK Houses of Parliament. The full agenda is here.

Two of the presentations are specifically  focused on information management. Matt Aslett is presenting on Total Data, our take on the the increasing volume and variety of data, combined with a greater understanding about its potential value. I’ll be preceding Matt with an overview of information risk management as we see it, focusing on how the increase in information volume and variety heightens the risk environment and what some companies have done to tackle it. Clients of 451 Group can come to the conference at no extra charge as it is included in the price of their annual relationships with us. Non-clients can also come for a fee, please email me for details

The following two days – Tuesday June 28 and Wednesday June 29 are focused on hosting and cloud issues with our Hosting and Cloud Transformation Summit (HCTS). The agenda features a wide variety of speaker, both from 451 Group and from numerous end users in markets including financial services, government, media, telecommunications and transportation.

The highlight for many will likely be listening to, and asking questions of, Professor Brian Cox, the Professor of Particle Physics at Manchester University and one of the leaders on the ATLAS experiment at the Large Hadron Collider at CERN in Geneva. In the UK he is well known for two massively popular science programmes, Wonders of the Solar System and, in 2011, Wonders of the Universe, the first of which is also now on in the US on the Science Channel. He will be talking about all the things that interest him, and there will be ample time for Q&A.

Seats are selling fast and 451 clients who attend the client event get a discount to HCTS. We also have some discount codes avaiable, so if you’re not a client and would like to attend, please get in touch via email or Twitter (@NickPatience).

I look forward to seeing some of you there!

I’m very pleased to be able to announce that we have a new analyst at The 451 Group covering e-Discovery and information governance.

His name is David Horrigan and he will be covering the e-Discovery and information governance markets as part of our information management practice.

He is both a practitioner, having been engaged in numerous e-Discovery projects, as well as professional writer and public speaker in the legal technology market.

Here’s his bio, which is also on our website:

As an attorney and longtime legal technology writer, David brings both law firm experience and years of technology writing to his analysis of these growing industries. He will be advising clients of The 451 Group on the rapid changes in e-discovery as corporations and law firms keep pace with ever-increasing legal and regulatory requirements for information management.

Prior to joining The 451 Group, David served as Assistant Editor and Staff Reporter at The National Law Journal and as a columnist for Law Technology News, writing the long-running Technology on Trial column.

His legal experience in the technology sector includes serving as Counsel for Intellectual Property and Technology Policy at the Entertainment Software Association, and as Director of Legislative and Regulatory Policy at the Magazine Publishers of America.

David has also managed e-discovery projects for the international law firm Covington & Burling LLP, and counseled Brown University on the implementation of its e-discovery and records governance policies.

David holds a Juris Doctor from the University of Florida and a Bachelor of Science from the University of Houston. He is licensed to practice law in the District of Columbia.

We welcome David to our team and you can follow him on Twitter at @davidhorrigan

Two companies central to our coverage of information management are having their own particular – and distinct – issues with shareholders and equity analysts.

Autonomy has been having its run-ins with London’s equity analysts for some time. Not all of them, but a core and increasingly vocal group of them. Generally they regularly question a few things: how the company calculates organic growth of its core IDOL business; cash conversion; and why it hasn’t bought a company after saying it would do and raising £500m of convertible debt to help it do so, back in February 2010. We’re also weighed in on some of these issues.

Autonomy regularly takes on these doubters on its quarterly calls and also does the same during the quarter on its website, which is at least a refreshing change from companies that stay completely mute on such matters. However the answers are often very simplistic. In a post dated March 30, 2011 entitled, “How should we think about Autonomy’s penetration of its end markets, when we attempt to evaluate the opportunity for growth?” that most of the world’s top software companies OEM IDOL and thus are “building their future products with IDOL deeply embedded and paying Autonomy a royalty.” Are they? Autonomy doesn’t distinguish between its two main OEM product when it announces OEM deals, but there’s a big difference between OEMing IDOL and OEMing its document filters. And as we have discussed before we think a lot of the OEM deals are for the latter, rather than for IDOL itself, although we have no way of proving that, except to say that we speak regularly to these leading software vendors and they don’t appear to be using IDOL as their core search and classification engine nearly as widely as Autonomy claims. Ironically given what Autonomy does for a living, a fair bit of the to and fro on the site is semantic-related, e.g. discussion of what “early spring” or “Winter with snowdrops” scenarios mean in terms of the guidance given by the company to analysts. All will no doubt become clearer when it announces its Q1 results, due Thursday April 28.

Over at Iron Mountain, some dissident shareholders have been putting pressure on the company to take on board its slate of directors and eventually turn itself into a Real Estate Investment Trust (REIT), mainly for its beneficial tax status. We cover what used to be called the digital business – the back up and recovery, e-Discovery, archiving and other software that’s mostly been added via acquisitions over the past few years. But that doesn’t seem to hold any attraction to hedge fund Elliott Management, which owns just less than 5%. It was the company that put forward the slate of directors and advised the company to turn itself into a REIT and in general to focus on its core – non-digital – business. Elliott and even larger shareholder Davis Advisors (it owns a shade less than 20% of the outstanding shares) were annoyed when the company dropped a poison pill on March 23 to guard against a takeover. This week Elliott laid out its grievances in another letter to the board, urging it to reverse the poison pill and generally sit up and take notice of what it has to say.

It’s hard to tel where this will end, but it has already caused disruption to Iron Mountain’s business at a time when it is trying to get some of its digital units – notably e-Discovery – back in track after a very tough 2010. We’ll know if it’s had an effect on its Q1 performance when it announces its results, most likely int he last week of April. The shares, as is common with these sorts of investor challenges have enjoyed a strong run-up, and are currently at or around a 52-week high. The company’s annual shareholders meeting is coming up soon too. Although the date is not yet known, all shareholders on record as of April 12 will be allowed to vote at it. It could get quite lively.

As you may have seen from recent announcements, The 451 Group is expanding and both organically and inorganically.

As part of that  we’re looking for a bunch of new analysts, two of which are within the information management and storage areas, both of which are US-based.

Information governance and e-Discovery analyst

• More details in the ad here

Senior storage analyst

• More details in the ad here

In both cases the email to contact is careers@the451group.com

We basically look for very smart people who can write and who are passionate about their chosen area. If you have those three attributes, you’re a long way to becoming a 451 analyst.

On Tuesday March 8 I’m doing a webinar along with Isys Search Software and Sybase about text-aware applications. The full title is “Text-Aware Software Solutions: What Defines Excellence?”

‘Text-aware applications’ is a phrase we coined back in 2005 as part of the process of writing a major report on the subject in which we looked at the various application areas (CRM, ERP, BI etc) that could benefit from a deep understanding of unstructured data.

As the first key finding from the report said:

The future success of companies and organizations will increasingly be based on their ability to unlock hidden intelligence and value from unstructured data, and text in particular.

The webinar on March 8 looks at the role of document filters in making applications text-aware, which is something I’ve talked about here before.

It’s at 10am PT/1pm ET/6pm UK. You can register here.