Enterprise search is changing its modus operandi – and in some circles, even its name. In fact, the transformation is significant enough for Gartner to have considered coining a new term. In December 2015, the firm identified “insight engines” in a report by Whit Andrews, Hans Koehler-Kruener, Tom Edi and Guido De Simoni: Insight Engines Will Power Enterprise Search That is Natural, Total and Proactive.
It is worth noting an earlier Gartner Magic Quadrant for Enterprise Search report, which stated in August 2015 that there was clear demand for powerful enterprise search tools: “Enterprise search projects inspired by virtual personal assistants like Google Now and Apple’s Siri are acquiring natural-language and analytical features, with the result that enterprise search technology is growing more powerful.” The report continued: “Enterprise search technology relates users’ queries to many different kinds of information in order to identify relevant, contextualized information and, in the process, perform light analysis.”
According to the report, vendors are now providing the means to “discover information, index it, and combine it with information derived from live queries in order to help people find what they need in a timely fashion. The ability to collect queries from users and apply them to a matrix of information and informative sources, and then to communicate results back to users efficiently, is at the heart of enterprise search.”
The logistical challenges of search relevance
It’s easy to recognize the value of a search platform that can reach into multiple data sources, coordinate results and build on a user’s behaviour patterns to deliver meaningful results. And the ability to seek out content and adapt responses based on user behaviour is something that is becoming increasingly important with the proliferation of cloud systems, including those that simply pop up without IT’s blessing. But it’s not that simple to deliver an Amazon or Google web-type experience at the enterprise level.
Enterprises that have worked with Google search appliances think they can drop it into an enterprise data system to get exactly the same experience users would get on a web search, said Richard Tessier, SVP of products at Coveo, a provider of enterprise intelligent search apps. That experience didn’t materialize because they didn’t realize the amount of work that goes behind optimizing the experience on a Google search engine. “It doesn’t happen by magic. Google is looking at patterns for searches and applies advanced algorithms that enable you to come up with the best answer for each question. Enterprise connectivity and data access are extensive and complicated; and vendors for their part find it increasingly challenging to get to that variety of data effectively,” he added.
One of the challenges with Google Search appliances, is that is done by third party vendors, making it difficult for enterprises to connect to all internal data sources, Tessier explained. “You have to keep maintaining and adding new connectors which can be a significant investment. If you want a trusted search engine, 50 percent of content coverage won’t cut it. You need access to 90 percent of relevant content. I don’t think the challenge is from the Google perspective. It’s enterprise search in general and having access to information repositories in a good way and getting relevancy for searches.”
Until recently, to make data searchable/indexable, a go-to strategy of choice has been content management solutions, which are designed to take data from different systems, moving it and recreating and consolidating it for single-source access by various user communities. “These projects are very difficult to get right because you have to create the content first and go to an interface to access it,” Tessier said. Given the scope and complexity of search needs, enterprises are looking at considerable time, expense (in large-scale implementations enterprises could be looking at millions in investment to bring together information into one place) and skill sets.
The mechanics of intelligent search
An intelligent search engine, on the other hand, fetches information from existing data sources via a single web page or interface (e.g. a search box, portal or a Salesforce.com or other integration at the agent’s point of work), while capturing all user search interactions – from initial queries to results – without having to move content. Through usage analytics, a feedback loop runs machine-learning-based algorithms to identify the most relevant pieces of all user interactions to produce relevant results. Those algorithms will automatically adjust a search experience according to what is happening at the time.
Put another way, applying algorithms allows enterprises to extract the user intent from the raw data and uses analysis of interaction behaviours to provide the most relevant recommendations.
The key advantage with a self-learning model is there is no need for an external party to go in and tune the search experience, since the search engine can actually independently learn the behavior people are using and automatically tune itself accordingly. It can then adapt to changes in user behaviour to promote the most popular or relevant results at specific points in time, targeting website, internal data sources and external sources; as well as internal file systems, Google Drive, and the user community in the cloud.
The building blocks
Intelligent search is comprised of a number of building blocks, Tessier said. Usage analytics is the first. “If you are flying blind there is no way you can optimize a flight pattern. You need instruments to help you head in the right direction.”
The second is having an infrastructure that can process all of the information coming in, which will be subject to the algorithms. “You need scale – the type of infrastructure afforded by Amazon services – to power those computations. And you need to run them on a regular basis so they can be transparent for the search engine.”
Behind that are “all the Big Data buzzwords,” he added. “Smart, machine learning, validating against historical datasets – it all needs to happen under one hood. You want automated relevance when you tune into that interface.”
Ironically, all this automation taps into what is the most important aspect of search: the human factor. “What is the best source of relevance? Human behaviour,” Tessier said. “By running those algorithms, you end up driving relevance. It’s about understanding the intent of the search, not just the keyword. Injecting user intent simply gets rid of the noise in queries.”
