Month: November 2024

In the ongoing and potentially futile effort by CIOs to squeeze meaningful ROI out of their shiny, new generative AI (genAI) tools, there is no more powerful villain than hallucinations. It is what causes everyone to seriously wonder whether the analysis genAI delivers is valid and usable. 

From that perspective, I applaud OpenAI for trying to create a test to determine objective accuracy for genAI tools. But that effort — called SimpleQA — fails enterprise tech decision-makers in two ways. First, OpenAI is the last business any CIO would trust to determine the accuracy of the algorithms it is selling. Would you trust an app that determines the best place to shop from Walmart, Target or Amazon — or perhaps a car evaluation tool from Toyota or GM?

The second problem is that SimpleQA focuses on, well, simple stuff. It looks at objective and simple questions that ostensibly have only one correct answer. More to the point, the answer to those questions is easily determined and verified. 

That is just not how most enterprises want to use genAI technology. Eli Lilly and Pfizer want it to find new drug combinations to cure diseases. (Sorry, that should be “treat.” Treat makes companies money forever. Cure’s revenue is large, but ends far too quickly.) Yes, it would test those treatments afterwards, but that is a lot of wasted effort if genAI is wrong. Costco and Walgreens want to use it to find the most profitable places to build new stores. Boeing wants it to come up with more efficient ways to build aircraft.

Let’s delve into what OpenAI created. For starters, here’s OpenAI’s document. I’ll put the company’s comments into a better context.

“An open problem in artificial intelligence is how to train models that produce responses that are factually correct.” Translation: We figured it would be nice to have it give a correct answer every now and then.

“Language models that generate more accurate responses with fewer hallucinations are more trustworthy and can be used in a broader range of applications.” Translation: Call us hippies, if you must, but we brainstormed and concluded that our revenue could be improved if our product actually worked.

Those flippant comments aside, I want to acknowledge that OpenAI makes a good faith effort here to come up with a basic way to evaluate precision where concrete answers can be ascertained. Setting aside how valuable that is in an enterprise setting, it’s a good start.

But instead of creating the test itself, it would have been far more credible if it funded a trusted third-party consulting or analyst firm to do the work, with a firm hands-off policy so IT could trust that the testing was not biased in favor of OpenAI’s offerings. 

Still, something is better than nothing, so let’s look at what OpenAI said. 

“SimpleQA is a simple, targeted evaluation for whether models ‘know what they know’ (and give) responses (that) are easy to grade because questions are created such that there exists only a single, indisputable answer. Each answer in SimpleQA is graded as either correct, incorrect, or not attempted. A model with ideal behavior would get as many questions correct as possible while not attempting the questions for which it is not confident it knows the correct answer.”

If you think through why this approach works — orseems like it would work — it becomes clear why it might not be helpful. This approach suffers from a critical flawed assumption. If the model can accurately answer these questions, then that tells us that it will likely be able to answer other questions with the same accuracy. 

That might work with a calculator, but the nature of genAI hallucinations makes that assumption flawed. GenAI can easily get 10,000 questions correct and it might then wildly hallucinate for the next 50. 

The nature of hallucinations is that they tend to happen randomly with zero predictability. That is why spot-checking, which is pretty much what SimpleQA is trying to do, won’t work here. 

To be more specific, it wouldn’t be meaningful if genAI tools were to get all of the SimpleQA answers right. But the reverse isn’t true. If the tested model gets all or most of the SimpleQA answers wrong, that does tell IT quite a bit. From the technology’s perspective, the test seems unfair. If it gets an A, it will be ignored. If it gets an F, it will be believed. As the computer said in WarGames (a great movie to watch to see what a genAI system might do at the Pentagon), “The only winning move is not to play.”

OpenAI pretty much concedes this in the report: “In this work, we will sidestep the open-endedness of language models by considering only short, fact-seeking questions with a single answer. This reduction of scope is important because it makes measuring factuality much more tractable, albeit at the cost of leaving open research questions such as whether improved behavior on short-form factuality generalizes to long-form factuality.”

Later in the report, OpenAI elaborates: “A main limitation with SimpleQA is that while it is accurate, it only measures factuality under the constrained setting of short, fact-seeking queries with a single, verifiable answer. Whether the ability to provide factual short answers correlates with the ability to write lengthy responses filled with numerous facts remains an open research question.”

Here are the specifics: SimpleQA consists of 4,326 “short, fact-seeking questions.” 

Another component of the SimpleQA test is that the question-writer bears much of the responsibility, rather than the answer-writer. “One part of this criterion is that the question must specify the scope of the answer. For example, instead of asking ‘Where did Barack and Michelle Obama meet’ which could have multiple answers such as ‘Chicago’ or ‘the law firm Sidley & Austin,’ questions had to specify ‘which city’ or ‘which company.’ Another common example is that instead of asking simply ‘when,’ questions had to ask ‘what year’ or ‘what date.’”

That nicely articulates why this won’t likely be of use in the real world. Enterprise users are going to ask questions in an imprecise way. They have been sold on the promise of “just use natural language” and the system will figure out what you really mean through context. This test sidesteps that issue entirely. 

So, how can the results be meaningful or reliable? 

The very nature of hallucinations belies any way to quantify them. If they were predictable, IT could simply program their tools to ignore every 75th response. But it’s not. Until someone figures out how to truly eliminate hallucinations, the lack of reliable answers will stay with us. 

Baidu has introduced a text-to-image generator dubbed I-RAG and a no-code developer platform called Miaoda as part of its growing portfolio of artificial intelligence (AI) products that, like US-based AI companies, it eventually aims to offer its user base as part of a wide array of commercial AI offerings.

CEO Robin Li introduced the new technology in a presentation at the company’s Baidu World Conference Tuesday. I-RAG uses Baidu’s search capabilities to generate images from speech and has been designed to address the “hallucinations” issue, according to a report on Reuters. The hallucinations referred to are images generated via large language model (LLM)-based AI that deviate from what was specified in the input prompt or contain non-existent elements.

Baidu also launched Miaoda, a developer platform that uses the capabilities of LLMs to generate code, and is aimed at allowing users without extensive coding expertise to develop applications. AI companies in the US also are providing similar tools to develop applications through a visual interface, with reusable components and advanced developer assistance, noted Manukrishnan SR, practice director for Everest Group.

Indeed, like those of leading US companies such as OpenAI, Google, and Microsoft, Baidu’s AI moves demonstrate its march toward the commercialization phase of the technology. The company, like others before it, has been adding AI to existing products or creating new ones that enterprise and other business users can integrate into their applications.

Follow the leader

Google, OpenAI, and Microsoft already have products similar to the ones Baidu revealed Tuesday, and the Chinese company has some catching up to do, analysts noted. The release of an AI-enhanced no-code platform in particular demonstrates Baidu’s aim to keep up with a software development trend that may one day leverage AI to replace traditional coding with software configuration.

“The pace of innovation and research in generative AI technologies and software is moving at a breakneck pace in the US,” Dave Schubmehl, research VP, AI & automation at IDC, observed. “To compete effectively on the world stage, other countries will need to adopt this same pace of innovation and research.”

He added, “many vendors are offering low code/no code/code generation capabilities in their products. Baidu’s product Miaoda is doing what other vendors like Microsoft and OpenAI have already done, which is using LLM capabilities to generate code.”

So far, however, Baidu’s AI tools do not seem to be as advanced as the ones released by OpenAI, Microsoft, and Google, Everest Group’s SR told CIO, “since these players have large existing datasets on which they can train their AI models.”

However, with “all major cloud platform players now offering some form of genAI-based programming augmentation facility,” AI-based software development may be the way forward for the enterprise, noted Bradley Shimmin, chief analyst, AI and data analytics, at Omdia.

“This is a very important area of research in that it points to an eventual state where both domain experts inside an organization and professional ISV practitioners can both use the same tooling to create full-stack apps and/or workflow automations in a declarative, no-code, conversational manner,” Shimmin said.

Still, this evolution is not without its challenges, and may not be something CIOs need to worry about quite yet, Everest Group’s SR noted.

“These tools are facing a host of challenges, including maintaining code quality, adherence to regulatory standards, and questions on ROI,” he told CIO. “Thus, while AI is set to revolutionize software development in the medium to long term, there are a lot of challenges that need to be ironed out before its potential can be fully realized.”

Don’t underestimate China

Though Baidu is still playing catch-up to US-based companies, China as a major global AI player should not be underestimated, Shimmins noted. In fact, “China and the US are really not that far apart from one another in terms of expertise and investment [in AI],” he observed.

“Already, China has produced some very strong models, particularly open source models such as Qwen2.5-Coder, which rivals some of the larger frontier models from Anthropic and OpenAI (at least in terms of published benchmarks),” he said.

The US has been doing everything it can to stymie overall technological development in China in various ways, and AI is no exception. A mere two weeks ago, the US government announced new rules restricting investments in China’s AI and other tech sectors deemed threats to national security, expanding existing technology restrictions that were so far limited to exports. China, for its part, has banned the use of OpenAI in the country.

However, despite the current friction between the US and China in terms of their technological arms race, the two countries have similar goals when it comes to AI, and may end up collaborating in some areas, Shimmin noted.

“In terms of academic research, the two nations are starting to work more closely with one another in seeking out a common ground concerning the existential threat posed by AI itself,” he said.

Baidu has introduced a text-to-image generator dubbed I-RAG and a no-code developer platform called Miaoda as part of its growing portfolio of artificial intelligence (AI) products that, like US-based AI companies, it eventually aims to offer its user base as part of a wide array of commercial AI offerings.

CEO Robin Li introduced the new technology in a presentation at the company’s Baidu World Conference Tuesday. I-RAG uses Baidu’s search capabilities to generate images from speech and has been designed to address the “hallucinations” issue, according to a report on Reuters. The hallucinations referred to are images generated via large language model (LLM)-based AI that deviate from what was specified in the input prompt or contain non-existent elements.

Baidu also launched Miaoda, a developer platform that uses the capabilities of LLMs to generate code, and is aimed at allowing users without extensive coding expertise to develop applications. AI companies in the US also are providing similar tools to develop applications through a visual interface, with reusable components and advanced developer assistance, noted Manukrishnan SR, practice director for Everest Group.

Indeed, like those of leading US companies such as OpenAI, Google, and Microsoft, Baidu’s AI moves demonstrate its march toward the commercialization phase of the technology. The company, like others before it, has been adding AI to existing products or creating new ones that enterprise and other business users can integrate into their applications.

Follow the leader

Google, OpenAI, and Microsoft already have products similar to the ones Baidu revealed Tuesday, and the Chinese company has some catching up to do, analysts noted. The release of an AI-enhanced no-code platform in particular demonstrates Baidu’s aim to keep up with a software development trend that may one day leverage AI to replace traditional coding with software configuration.

“The pace of innovation and research in generative AI technologies and software is moving at a breakneck pace in the US,” Dave Schubmehl, research VP, AI & automation at IDC, observed. “To compete effectively on the world stage, other countries will need to adopt this same pace of innovation and research.”

He added, “many vendors are offering low code/no code/code generation capabilities in their products. Baidu’s product Miaoda is doing what other vendors like Microsoft and OpenAI have already done, which is using LLM capabilities to generate code.”

So far, however, Baidu’s AI tools do not seem to be as advanced as the ones released by OpenAI, Microsoft, and Google, Everest Group’s SR told CIO, “since these players have large existing datasets on which they can train their AI models.”

However, with “all major cloud platform players now offering some form of genAI-based programming augmentation facility,” AI-based software development may be the way forward for the enterprise, noted Bradley Shimmin, chief analyst, AI and data analytics, at Omdia.

“This is a very important area of research in that it points to an eventual state where both domain experts inside an organization and professional ISV practitioners can both use the same tooling to create full-stack apps and/or workflow automations in a declarative, no-code, conversational manner,” Shimmin said.

Still, this evolution is not without its challenges, and may not be something CIOs need to worry about quite yet, Everest Group’s SR noted.

“These tools are facing a host of challenges, including maintaining code quality, adherence to regulatory standards, and questions on ROI,” he told CIO. “Thus, while AI is set to revolutionize software development in the medium to long term, there are a lot of challenges that need to be ironed out before its potential can be fully realized.”

Don’t underestimate China

Though Baidu is still playing catch-up to US-based companies, China as a major global AI player should not be underestimated, Shimmins noted. In fact, “China and the US are really not that far apart from one another in terms of expertise and investment [in AI],” he observed.

“Already, China has produced some very strong models, particularly open source models such as Qwen2.5-Coder, which rivals some of the larger frontier models from Anthropic and OpenAI (at least in terms of published benchmarks),” he said.

The US has been doing everything it can to stymie overall technological development in China in various ways, and AI is no exception. A mere two weeks ago, the US government announced new rules restricting investments in China’s AI and other tech sectors deemed threats to national security, expanding existing technology restrictions that were so far limited to exports. China, for its part, has banned the use of OpenAI in the country.

However, despite the current friction between the US and China in terms of their technological arms race, the two countries have similar goals when it comes to AI, and may end up collaborating in some areas, Shimmin noted.

“In terms of academic research, the two nations are starting to work more closely with one another in seeking out a common ground concerning the existential threat posed by AI itself,” he said.