Businesses Seek Out ChatGPT-Tech for Analyzing Their Own Data

May 19

A string of generative AI announcements from Google, Microsoft and others has corporate technology leaders taking a new look at enterprise search

Confident-businessman-pointing-at-coded-data-on-computer-screen-while-explaining-it-to-female-colleague-at-meeting.jpeg

How to Build a Reliable Customer Data Infrastructure.jpg

The hope that searching workplace documents could match the ease and efficiency of a consumer search engine has been largely unrealized. But now generative artificial intelligence may offer potential to help businesses do just that.

So far, companies “have been unable to find and [deliver] an enterprise search that came even within an order of magnitude as good as Google search is outside,” said Brian McClendon, senior vice president of engineering at software development and gaming company Niantic.

Recent moves by Google, Microsoft and other vendors to harness the technology underlying ChatGPT for search is leading some corporate technology chiefs to rethink that functionality, one that has dogged them for decades.

While the ability to index company data to make it searchable exists, the difference now is that large-language models and generative AI can recognize patterns, make predictions and create content by processing enormous quantities of text, images and audio. That, plus the ability to converse in plain English by asking questions of company data, will mean that all employees have the potential to extract insights and ideas from that data.

Generative AI is “taking all the workload away from a traditionally very technical task, and putting data reporting and analytics into the hands of decision-makers directly,” said Samuel Hamway, an analyst at research and IT consulting firm Nucleus Research. “These generative solutions allow you to query as if it was a Google search.”

Google this week laid out generative AI features that encourage users to ask follow-up questions to search the internet, allowing them to hold conversations with the search engine. Microsoft said in February that its Bing search engine has been embedded with the ChatGPT technology, so users can make queries by talking with the search box.

Both tech giants said companies will have access to that functionality to search their own information, tapping large-language models, or LLMs.

“Enterprise search is not anything that’s new,” said June Yang, vice president of Google Cloud AI and Industry Solutions. “But now with LLMs it actually really enhances that capability. And with conversational AI, now I can have a conversation, keeping the context of that, so I can continue to dig.”

Using Google’s generative AI platform, companies can create a service that allows employees to “have a conversation with a document repository” like an internal knowledge base, Ms. Yang said. The company is adding connectors for customers to pull in other data sources, not just from Google Cloud, so they can be searchable, she said.

A new generative AI feature from Microsoft called Business Chat, embedded within its business software, lets users ask questions like “what are the upcoming milestones” for a project, said Jon Friedman, corporate vice president of design and research at the software company. Business Chat pulls from on-premises sources and cloud-based data, carrying out searches with Microsoft’s enterprise search technology, the company said.

Improvements to foundational search technology are underway too: Amazon.com’s cloud unit, which offers an enterprise search tool called Kendra, recently released an “embeddings” model in its generative AI development platform that can do a better job of search than other forms of machine learning, said Vasi Philomin, vice president of generative AI at the tech giant.

Jason Urso, chief technology officer of industrial giant Honeywell’s Connected Enterprise software group, said the Atlanta-based division is using Microsoft software and is considering tapping into the software’s generative AI capabilities.

To share information across the organization now, Mr. Urso said the company has “really smart people that have knowledge across multiple domains.” It currently has pilots under way to make its internal knowledge base more accessible for employees across its divisions, he said.

“There’s only so many of those types of people that have the breadth of knowledge, so what we’re looking for is more search tools to identify patterns of behavior,” Mr. Urso said.

Some big law firms are experimenting with generative AI for legal research—a task that can be done in minutes with the technology, rather than hours. The global firm Allen & Overy said thousands of its attorneys are spending less time finding case law as well as completing analyses and answering questions with the help of generative AI.

Finding documents, looking within databases, and searching for people and processes have constituted information-technology challenges for decades—complicated by the fact that data sources are stored across various silos and IT systems at many companies.

Specialized enterprise search software providers like Coveo and Elastic, and Palo Alto, Calif-based startup Glean—which offer connectors to link disparate data sources—are finding ways to incorporate generative AI characteristics into their search functions, too. Glean last month announced LLM-based features that allow users to ask questions in its search box, aiming to combine enterprise search with providing insights on that data, founder and Chief Executive Arvind Jain said.

Niantic’s Mr. McClendon said the 1,000-person company began using Glean’s enterprise search tool about two years ago. The process to gather and index the company’s data, which sits across multiple cloud platforms and about 30 software services, took a couple of days, he said.

While it is too early to determine the value of generative AI with enterprise search, Mr. McClendon said the end goal is helping employees make better business decisions, armed with an arsenal of company data and AI-generated ideas.

CIOs face a number of challenges in implementing any enterprise search and generative AI solution, notably that LLMs are prone to hallucination and returning inaccurate information. If bad data becomes the source of a search, it could be propagated by informing future searches, said Tim Crawford, CIO strategic adviser at enterprise IT advisory firm AVOA.

Technology vendors say that searching within a company’s set of documents reduces the risk of hallucination because the AI is grounded in the systems and databases of that company and not unverified internet data.

Corporate technology chiefs, some of whom are under pressure to deliver on AI initiatives, are being pressed to do so with data privacy and cybersecurity protections. In a recent KPMG survey of 225 U.S. executives, 60% said they plan to implement generative AI within the next two years, but only 6% said they have a dedicated team to set risk strategies.

Still, Suvajit Basu, CIO of Goya Foods, said that for the Jersey City, N.J.-based food manufacturer, and distributor, the need for an enterprise search tool is less relevant compared with using generative AI for content creation or gleaning macro trends.