In one of those Autonomy announcements that seemingly appear out of nowhere, the company has declared its intention to “transform” the relational database market by applying its text analysis technology to content stored within database. The tool is called IDOL Structured Probabilistic Engine (SPE), as it uses the same Bayesian-based probabilistic inferencing technology that IDOL uses on unstructured information.

The quote from CEO Mike Lynch grandly proclaims this to be Autonomy’s “second fundamental technology” – IDOL itself being the first. That’s quite a claim and we’re endeavoring to find out more and will report back as to exactly how it works and what it can do.

Overall though this is part of a push by companies like Autonomy, but also Attivio, Endeca, Exalead and some others into the search-based application market. The underlying premise of that market is database offloading; the idea of using a search engine rather than a relational database to sort and query information. It holds great promise, partly because it is the bridge between enterprise search and business intelligence but also because of the prospect of cost savings for customers as they can either freeze their investments in relational database licenses, reduce them, or even eliminate them.

Of course if the enterprise search licenses then get so expensive as to nullify the cost benefit, then customers will reject the idea, which is something of which search vendors need to be wary.

Users can apply to joint the beta program at a very non-Autonomy looking website.

I made a comment on the analyst panel at the end of day 1 about the emergence of startups in this space that I wanted to qualify, as it’s caused a bit of confusion here at the Text Analytics Summit. The other three panel members said they are seeing startups while I said I’m not and nor are VC customers asking about them in the way they did a few years back. I said that partly to shake up the panel a bit as we were agreeing on everything until then, which isn’t that interesting for the audience , but I meant it in a specific way.

The main area where text analytics-based startups have emerged in the last few years is in sentiment analysis, in areas such as opinion mining, buzz, product/service reviews and advertising targeting. Many of these apps are being used by enterprises for sure.

But what I was referring to is that I’m not seeing companies offering text analytics tools (whether on-premise or on a SaaS or cloud basis) that can be used as the basis of text-aware or search-based applications. I am seeing a lot of demand and interest in those apps from enterprises (our main focus here at 451) but  the tools to build them are not coming from startups.

Instead they’re coming mainly from more established search, content management and eDiscovery-focused companies (with one or two notable exceptions, such as Attivio and Digital Reef in the past two years). There is probably room for more startups in this space, that’s for sure.

More on what has been a great conference so far later.

I started off this year’s Enterprise Search Summit in New York last week with a dinner sponsored by New Idea Engineering and Attivio on Monday night, which was highly enjoyable, despite my jetlag – having to try and stay up the first night in from London. Thanks to those folks for the invite and the conversation.

Katey and I were not allowed to sit in any of the session this year from some strange reason. So I can’t tell you first hand about what was interesting or not or the attendance in the sessions. Go figure. It also wasn’t that conducive to meeting end users, which is a main objective of attending these things.

Katey reckoned attendance overall was slightly down on last year, but not spectacularly so (I was at different conference and so had to miss last year’s).

So away from those two disappointments, we did have a fairly full docket of meetings with vendors, which were generally lively, with good give and take. Where we say ’451 research to follow,’ it means our clients can expect a research report on the company in the near future.

Some of the highlights:

Attivio – CTO Sid Probstein is always chock-full of ideas and so always good to have a sitdown with him. CEO Ali Riaz is entertaining on a whole different level. The company appears to be going great guns and is at the forefront of the drive to combine structured and unstructured data as we have said before.

BA-Insight – not really a search company or a text analysis company; more of a piece of information management middleware that aims to increase ‘findability’ within SharePoint. As any SharePoint users, especially those in an environment with multiple SharePoint sites – that can only be a good thing. Connectors to other search engines coming. 451 research to follow.

Coveo – the company was out in force at this conference having just launched version 6.0 of its search platform featuring better scalability, connectors and mobile functionality. We covered that product update a short while back.

Endeca – met chief scientist Daniel Tunkelang for the first time. Clearly the owner of an active mind, Daniel presents a different face to the search company. His thoughts on the conference are here.

Google – the typically on-message briefing from Google. It owns the low end and is increasingly taking chunks out of the mid-tier, but still no sign of the management layer enterprises needed to get their arms around the myriad Google search appliances lying around most large organizations. It will probably appear out of the blue at some point though, this year, I’d imagine.

Microsoft – Nate Treolar was a great evangelist for Fast Search & Transfer while a product manager, and so it seems appropriate that he has the term ‘evangelist’ in his title at Microsoft where he’s working not only on the SharePoint search ecosystem but other programs such as ‘conversational’ and ‘actionable’ search; talking and doing, hey, what else is there?

PerfectSearch – we don’t usually see too many companies at this conference that we haven’t spoken to before, but PerfectSearch is one of them. It sells a search appliance and some of the founders have a Novell background, hence its Orem, IT HQ. 451 research to follow.

Vivisimo – from what we’ve heard the company is going well, both in the indirect (OEM) ad direct markets. We’ve noticed how often this company is being bad-mouthed by its competitors (over and above the usual FUD in any tech market) though we’re not sure why. Perhaps because Pittsburgh isn’t as fashionable as Boston or the Valley? Don’t really know, but it seems misplaced based on our experience. It’s making good headway with Lexis-Nexis, which will be important in the eDiscovery market as well with other customers that have demanded confidentiality (pretty common in the eDiscovery market). 451 research to follow.

I had a very interesting conversation with Attivio recently. Most of it is confidential and thus not enough for me to wrote a full 451 report on it yet (but I will as soon as we’re able to), but I can say that the startup founded mostly by former employees of FAST Search & Transfer that has a couple of public customers has some much more interesting news coming down the pike in coming months. It just announced one of these customers, here.

We talked about what is triggering the customer wins it is getting and one thing stood out; the key value prop Attivio has been focused on from the start, which is the conjoining of structured and unstructured data. But it’s not just about having access to both types (that was possibly years ago and anyway there’s myriad types between those two simplistic and oft-confused descriptions, but bear with me). And it’s not just the ability be able to query them both; that too has been done, but crucially with different tools in the past.

It’s the ability to switch between both based on what the user is doing – not based on what type of data is being queried – to do so without the user knowing and having a single API for access to both types. And that, say the Attivio team, is hitting home with customers.

Attivio is not the only company to be thinking like that of course. Attivio’s management’s alma mater FAST was doing this for a while as a product called Adaptive Information Warehouse. We wrote a report on it in January 2007 under the headline: FAST: everything you thought you knew about BI is wrong. And there are other companies both positioning their products this way and winning customers doing it. See our recent report on Exalead for another example and we’re sure IBM could build its customers just such a system if they paid enough.

The challenge with this idea is of course that for the past 30 years or so relational databases have been where the ‘important stuff’ has been stored and the multi-billion BI market grew on top of that as a way to access it. Database administrators rule(d) the roost as far as information management goes. Meanwhile enterprise search got relegated to a side room where it was all about finding documents and getting pages and pages of results returned to you. What Attivio, Exalead and a few other companies are moving towards is a convergence of the two; call it database offloading, unified information access; unified information intelligence or something similar.

We’re not seeing these vendors being dragged kicking and screaming towards this single-API-no-matter-what-the-data-is nirvana by their customers, however; it takes a fair amount of market education on the parts of the vendors and their partner to make it a reality. But given what we see happening here, we expect to hear alot more from the vendors mentioned here and others throughout 2009 and beyond as enterprise search morphs into something new.

Later than I intended, I wanted to give you a quick update of last week’s client event and information management’s presence at it. Kathleen, Simon, Henry, Matt and me were engaged in many 1:1s – I had 15 over the two days, which were very useful for me and more importantly, from feedback we’ve had, useful to the other person as well. Some of our analysts were booked back to back, doing 20+ meetings; that level of engagement is one of the main values we deliver at our conferences.

On the presentation and panels front, Kathleen did a great job of laying out her vision of how collaboration and social software are finally impacting content technologies, moving beyond just things that enable you to create content, to enable organizations to better handle the risks that can create. Some people who weren’t able to hear her live have asked to hear it by way of a followup – if you do, please get in touch.

My panel was great, comprising Sid Probstein, CTO of Attivio, Stephen Whetstone of Iron Mountain-Stratify and Nicole Eagan, CMO of Autonomy. We were in the after lunch slot but given we were talking mainly about eDiscovery, the future of search and the effects of the credit crunch on information management, we still got people’s attention.

Anyway, don’t take my word for it, listen to what Sid says about it, plus his thoughts on other aspect of the event here and here. I couldn’t have put it better myself!

See you in Boston next year, I hope.

I’ve gathered all my current thinking on potential M&A in enterprise search in a SectorIQ that we published earlier this week to our customers. In it, I look at four main potential targets plus a few other small ones and look at a few of the likely acquirers. (This is the way we write all our Sector IQs, btw and they’re a great way of getting a quick grasp on what might be coming down the pike in any particular sector of the IT industry)

Fortunately those of you that are not our customers (yet!) are able to read it via our arrangement with the New York Times DealBook section. Click here to see the NY Times posting or go here to go straight to the report – and while you’re there, sign up for a trial of our M&A KnowledgeBase, where we’ve been collecting details of every IT, internet and telecoms deal since the start of 2002!

Finally, a quick word about the headline. We like to have some fun here at 451 with these things and while I appreciate that this one might have been pushing things a little in terms of clearly explaining what the report was about, when else would I be able to use it?