AI chatbots face a significant challenge: they incur financial losses with every chat interaction, and this poses a problem. The high operational costs associated with running these systems prevent companies from deploying their most advanced versions to the public. This issue is hindering the quality of AI chatbots and jeopardizing the ongoing AI revolution they have sparked.

The exorbitant expenses involved in maintaining large language models, which serve as the foundation for tools like ChatGPT and Bard, are limiting their capabilities. Moreover, the scarcity of the necessary computer chips further restricts the companies that can afford to utilize these models, placing pressure on even the wealthiest corporations to transform chatbots into profitable ventures sooner than they might be ready for.

Tom Goldstein, a computer science professor at the University of Maryland, points out that the currently deployed models, despite appearing impressive, are not the most optimal models available. Consequently, the visible chatbots possess several avoidable weaknesses, such as a tendency to produce biased outcomes or blatant falsehoods, which could be addressed if cost were not a limiting factor.

In summary, the financial losses incurred by AI chatbots during each interaction present a significant obstacle. The soaring costs associated with operating large language models restrict their quality, hinder the AI industry’s progress, and compel companies to turn chatbots into revenue-generating entities prematurely. This scenario prevents the utilization of the best available models and leads to observable weaknesses, including biased outputs and inaccuracies.

The Cost Challenge for Generative AI in Big Tech

Cost is a topic rarely discussed by tech giants that are betting their future on AI. OpenAI, Microsoft, and Google, the key players in the field, declined to comment on this matter. However, experts point out that cost is the most significant hurdle to overcome for Big Tech’s vision of generative AI permeating every industry, streamlining operations, and enhancing efficiency.

The substantial computing power required by AI is the reason OpenAI has refrained from releasing its powerful new language model, GPT-4, in the free version of ChatGPT. Currently, ChatGPT is still powered by the less robust GPT-3.5 model. Moreover, the underlying dataset of ChatGPT was last updated in September 2021, rendering it inadequate for researching or discussing recent events. Even users who subscribe and pay $20 per month for GPT-4 are limited to sending only 25 messages every three hours due to the exorbitant operational costs associated with running the model. Furthermore, GPT-4 is significantly slower in generating responses.

These high costs could potentially explain why Google has not yet integrated an AI chatbot into its flagship search engine, which handles billions of queries daily. When Google introduced its Bard chatbot in March, it deliberately refrained from utilizing its most extensive language model. Dylan Patel, the chief analyst at the semiconductor research firm SemiAnalysis, estimated that a single conversation with ChatGPT could cost up to 1,000 times more than a simple Google search.

Generative AI: Overcoming Challenges for Success

In a recent analysis of artificial intelligence, the Biden administration specifically highlighted the computational costs associated with generative AI, underscoring its significance as a national concern. The White House emphasized that this technology is projected to significantly escalate computational demands, along with the corresponding environmental impacts, necessitating an immediate imperative to devise more sustainable systems.

Generative AI, more than any other form of machine learning, mandates an astonishing amount of computational power and specialized computer chips called GPUs, which only the most financially robust corporations can afford. The escalating competition for access to these chips has transformed their leading providers into tech behemoths in their own right, controlling access to what has emerged as the most coveted asset within the technology industry.

Silicon Valley ascended to dominance in the digital economy, in part, by providing services such as online search, email, and social media to the global audience without charge, initially incurring losses but eventually reaping substantial profits through personalized advertising. It is probable that advertisements will also permeate the realm of AI chatbots. However, experts suggest that relying solely on ads may not be adequate to promptly render cutting-edge AI tools financially viable.

Consequently, companies offering AI models for consumer applications must strike a delicate balance between their quest to capture a significant market share and the financial losses they are presently incurring.

The Intersection of AI Reliability and Corporate Influence

The pursuit of more dependable AI solutions is poised to generate substantial profits, primarily benefiting chipmakers and cloud computing giants that already dominate the digital landscape. These entities, alongside the chipmakers responsible for powering the AI models, are expected to reap substantial financial rewards.

It is not coincidental that the companies spearheading the development of cutting-edge AI language models are either prominent cloud computing providers, such as Google and Microsoft, or closely aligned with them, as OpenAI is with Microsoft. However, purchasers of these companies’ AI tools may not realize that they are inadvertently becoming locked into a heavily subsidized service, which ultimately costs significantly more than their current expenses. Clem Delangue, CEO of Hugging Face, an open-source AI company, has highlighted this concern.

During a recent Senate hearing, OpenAI CEO Sam Altman indirectly acknowledged this issue when Senator Jon Ossoff (D-Ga.) cautioned that if OpenAI were to create ChatGPT in a manner that fosters addiction and negatively affects children, Congress would scrutinize their actions closely. Altman reassured Ossoff that there was no cause for concern, stating that OpenAI strives to design systems that do not prioritize user engagement. In fact, due to limited GPU availability, Altman suggested that minimizing usage of their products is actually preferable.

Developing and training AI language models involves substantial expenses, primarily due to the enormous amount of data and complex software required to identify language patterns. Moreover, AI companies often recruit top-notch researchers whose salaries can rival those of professional athletes. This initial barrier makes it challenging for companies to build their own models, although a few well-funded start-ups, such as Anthropic AI, founded by former OpenAI members with financial support from Google, have managed to succeed.

The Intersection of GPUs, OpenAI’s Evolution, and the Cost of Chatbot Systems

When users interact with chatbots like ChatGPT, Microsoft’s Bing, or Anthropic’s Claude, each query is directed to data centers where powerful supercomputers process the models. These high-performance machines carry out multiple rapid calculations simultaneously. Initially, they interpret the user’s prompt and then work to predict the most plausible response, analyzing each “token” (a four-letter sequence) one at a time.

The computational power required for such tasks heavily relies on graphics processing units (GPUs), originally designed for video games. These GPUs have proven to be the only chips capable of handling the intense computational demands of large language models. Currently, Nvidia is the sole provider of these top-tier GPUs, for which it charges tens of thousands of dollars. The company’s valuation recently skyrocketed to $1 trillion due to the anticipated sales. Additionally, TSMC, the Taiwan-based manufacturer responsible for producing many of these chips, has experienced a similar surge in value.

Elon Musk, the entrepreneur who recently acquired around 10,000 GPUs for his AI start-up, humorously compared the difficulty of obtaining GPUs to that of acquiring drugs. This statement highlights the high demand for GPUs among technology enthusiasts and AI researchers.

The substantial computing requirements in the field of AI also played a significant role in the transformation of OpenAI from a nonprofit organization to a for-profit entity. OpenAI, founded in 2015 with the noble goal of developing AI for the benefit of humanity without being driven by financial motives, had to adapt its approach. In 2019, it transitioned to a for-profit model to attract investment, securing a substantial $1 billion from Microsoft. As a result, Microsoft became OpenAI’s exclusive computing provider and has continued to invest further, contributing an additional $10 billion and integrating OpenAI’s technology into products like Bing, Windows, and others.

The cost of running advanced chatbot systems such as ChatGPT is an ever-evolving figure as companies strive to enhance their efficiency.

The Cost of Generative AI and the Challenge of Wide Availability

In December, shortly after its release, Altman provided an estimate of the cost of ChatGPT, stating that it was likely in the single-digits cents per chat. While this may seem insignificant, when multiplied by the estimated number of over 10 million users per day, the costs become substantial. In February, SemiAnalysis conducted an analysis and determined that OpenAI was spending around $700,000 per day in computing expenses for ChatGPT alone. These costs were based on the processing requirements of GPT-3.5, which was the default model at that time.

Considering the enormous user bases of popular search engines like Microsoft’s Bing, with 100 million daily users, and Google, with over 1 billion reported users, the financial burden of providing access to the most advanced AI models becomes apparent. It is no wonder that tech giants hesitate to make these models widely available to the public.

Sid Sheth, the founder and CEO of d-Matrix, a startup focused on developing more efficient AI chips, expresses concerns about this situation. He believes that the current equation of high computing costs is unsustainable for the democratization and widespread availability of generative AI, as well as for the economy and the environment.

Even a company as financially capable as Google acknowledged the cost implications when introducing its Bard technology in February. Google stated that Bard would initially run on a “lightweight” version of their LaMDA language model because it required significantly less computing power, enabling them to serve a larger user base. In essence, Google recognized that even they could not bear the expenses of providing their most powerful AI technology as a free chatbot.

The need for cost-cutting measures became evident as Bard encountered difficulties during its launch demonstration, leading to a significant drop of $100 billion in Google’s share value. Bing also faced challenges early on, prompting Microsoft to reduce both the personality of its chatbot and the number of questions users could ask in a single conversation.

These errors, often referred to as “hallucinations,” have become a significant concern with AI language models as individuals and businesses increasingly rely on them. Experts explain that these issues are inherent to the models’ fundamental design. They are designed to generate probable sequences of words rather than provide factual statements.

The Shift Towards Affordable and Accessible Generative AI Models

Google’s DeepMind subsidiary developed a chatbot called Sparrow, aimed at combatting misinformation by searching the internet and providing citations. However, Google has not released this chatbot yet.

In the race to make AI language models more affordable, major players in the industry are exploring various approaches. OpenAI’s new GPT-3.5 Turbo model, for example, offers significantly lower query costs compared to its flagship GPT-4 model. Google is also developing its own AI chips, claiming increased efficiency over Nvidia’s offerings, while start-ups like d-Matrix are entering the market with their own alternatives. Additionally, numerous start-ups are leveraging open-source language models like Meta’s LLaMA, enabling them to avoid paying licensing fees to OpenAI or Google. However, these models may not perform as well and may lack necessary safeguards against misuse.

This shift toward smaller and more cost-effective models signifies a sudden reversal in the industry, according to Maryland’s Goldstein. He noted that the focus in recent years had been on creating the largest models possible, primarily for research purposes rather than public use. However, in the past few months, the community has experienced a complete turnaround, with everyone striving to build the most compact models to manage expenses.

Consumers may soon face limitations on unrestricted access to powerful, general-purpose AI models. Microsoft has already begun experimenting with incorporating advertisements into its AI-powered Bing search results. During a Senate hearing, OpenAI’s Altman did not dismiss the possibility of pursuing a similar approach, although he expressed a preference for a paid subscription model.

Both Microsoft and OpenAI are confident that the economics of AI models will eventually make sense. Altman emphasized the tremendous value in these models and expressed certainty in finding effective monetization strategies.

Critics, however, highlight the societal costs associated with generative AI. The substantial computational requirements of these models contribute to greenhouse gas emissions and divert energy resources that could be allocated to other computing tasks, potentially hindering advancements in areas such as healthcare, drug discovery, and cancer detection.

Kasper Groes Albin Ludvigsen, a data scientist, estimated that ChatGPT alone may have consumed as much electricity in January as 175,000 people, equivalent to a midsize city.

Currently, tech giants are willing to incur losses in their pursuit of market dominance with AI chatbots, according to Goldstein. However, if profitability proves elusive, investors will ultimately scrutinize the bottom line, marking the end of the hype curve.

Nevertheless, despite their shortcomings, generative AI tools are expected to be highly attractive to individuals and companies. Goldstein believes that, despite the cost, they offer a significantly more affordable alternative to human labor.