ChatGPT could get more expensive to use in coming years.

The New York Times, citing internal OpenAI docs, reports that OpenAI is planning to raise the price of individual ChatGPT subscriptions from $20 per month to $22 per month by the end of the year. A steeper increase will come over the next five years; by 2029, OpenAI expects it’ll charge $44 per month for ChatGPT Plus.

The aggressive moves reflect pressure on OpenAI from investors to narrow its losses. While the company’s monthly revenue reached $300 million in August, according to the New York Times, OpenAI expects to lose roughly $5 billion this year. Expenditures like staffing, office rent, and AI training infrastructure are to blame. ChatGPT alone was at one point reportedly costing OpenAI $700,000 per day.

OpenAI could face a blowback if it increases prices too quickly. While ChatGPT has roughly 10 million paying users today, surveys suggest that many believe the current $20-per-month price is too high.

OpenAI CEO Sam Altman is leaving the internal commission OpenAI created in May to oversee “critical” safety decisions related to the company’s projects and operations.

In a blog post today, OpenAI said the committee, the Safety and Security Committee, will become an “independent” board oversight group chaired by Carnegie Mellon professor Zico Kolter, and including Quora CEO Adam D’Angelo, retired U.S. Army General Paul Nakasone, and ex-Sony EVP Nicole Seligman. All are existing members of OpenAI’s board of directors.

OpenAI noted in its post that the commission conducted a safety review of o1, OpenAI’s latest AI model, after Altman had stepped down. The group will continue to receive regular briefings from OpenAI safety and security teams, said the company, and retain the power to delay releases until safety concerns are addressed.

“As part of its work, the Safety and Security Committee … will continue to receive regular reports on technical assessments for current and future models, as well as reports of ongoing post-release monitoring,” OpenAI wrote in the post. “[W]e are building upon our model launch processes and practices to establish an integrated safety and security framework with clearly defined success criteria for model launches.”

Altman’s departure from the Safety and Security Committee comes after five U.S. senators raised questions about OpenAI’s policies in a letter addressed to Altman this summer. Nearly half of the OpenAI staff that once focused on AI’s long-term risks have left, and ex-OpenAI researchers have accused Altman of opposing “real” AI regulation in favor of policies that advance OpenAI’s corporate aims.

To their point, OpenAI has dramatically increased its expenditures on federal lobbying, budgeting $800,000 for the first six months of 2024 versus $260,000 for all of last year. Altman also earlier this spring joined the U.S. Department of Homeland Security’s Artificial Intelligence Safety and Security Board, which provides recommendations for the development and deployment of AI throughout U.S. critical infrastructure.

Even with Altman removed, there’s little to suggest the Safety and Security Committee would make difficult decisions that seriously impact OpenAI’s commercial roadmap. Tellingly, OpenAI said in May that it would look to address “valid criticisms” of its work via the commission — “valid criticisms” being in the eye of the beholder, of course.

In an op-ed for The Economist in May, ex-OpenAI board members Helen Toner and Tasha McCauley said that they don’t think OpenAI as it exists today can be trusted to hold itself accountable. “[B]ased on our experience, we believe that self-governance cannot reliably withstand the pressure of profit incentives,” they wrote.

And OpenAI’s profit incentives are growing.

The company is rumored to be in the midst of raising $6.5+ billion in a funding round that’d value OpenAI at over $150 billion. To cinch the deal, OpenAI could reportedly abandon its hybrid nonprofit corporate structure, which sought to cap investors’ returns in part to ensure OpenAI remained aligned with its founding mission: developing artificial general intelligence that “benefits all of humanity.”

How many times does the letter “r” appear in the word “strawberry”? According to formidable AI products like GPT-4o and Claude, the answer is twice.

Large language models (LLMs) can write essays and solve equations in seconds. They can synthesize terabytes of data faster than humans can open up a book. Yet, these seemingly omniscient AIs sometimes fail so spectacularly that the mishap turns into a viral meme, and we all rejoice in relief that maybe there’s still time before we must bow down to our new AI overlords.

The failure of large language models to understand the concepts of letters and syllables is indicative of a larger truth that we often forget: These things don’t have brains. They do not think like we do. They are not human, nor even particularly humanlike.

Most LLMs are built on transformers, a kind of deep learning architecture. Transformer models break text into tokens, which can be full words, syllables, or letters, depending on the model.

“LLMs are based on this transformer architecture, which notably is not actually reading text. What happens when you input a prompt is that it’s translated into an encoding,” Matthew Guzdial, an AI researcher and assistant professor at the University of Alberta, said. “When it sees the word ‘the,’ it has this one encoding of what ‘the’ means, but it does not know about ‘T,’ ‘H,’ ‘E.’”

This is because the transformers are not able to take in or output actual text efficiently. Instead, the text is converted into numerical representations of itself, which is then contextualized to help the AI come up with a logical response. In other words, the AI might know that the tokens “straw” and “berry” make up “strawberry,” but it may not understand that “strawberry” is composed of the letters “s,” “t,” “r,” “a,” “w,” “b,” “e,” “r,” “r,” and “y,” in that specific order. Thus, it cannot tell you how many letters — let alone how many “r”s — appear in the word “strawberry.”

This isn’t an easy issue to fix, since it’s embedded into the very architecture that makes these LLMs work.

Kyle Wiggers dug into this problem last month and spoke to Sheridan Feucht, a PhD student at Northeastern University studying LLM interpretability.

“It’s kind of hard to get around the question of what exactly a ‘word’ should be for a language model, and even if we got human experts to agree on a perfect token vocabulary, models would probably still find it useful to ‘chunk’ things even further,” Feucht said. “My guess would be that there’s no such thing as a perfect tokenizer due to this kind of fuzziness.”

This problem becomes even more complex as an LLM learns more languages. For example, some tokenization methods might assume that a space in a sentence will always precede a new word, but many languages like Chinese, Japanese, Thai, Lao, Korean, Khmer and others do not use spaces to separate words. Google DeepMind AI researcher Yennie Jun found in a 2023 study that some languages need up to 10 times as many tokens as English to communicate the same meaning.

“It’s probably best to let models look at characters directly without imposing tokenization, but right now that’s just computationally infeasible for transformers,” Feucht said.

Image generators like Midjourney and DALL-E don’t use the transformer architecture that lies beneath the hood of text generators like ChatGPT. Instead, image generators usually use diffusion models, which reconstruct an image from noise. Diffusion models are trained on large databases of images, and they’re incentivized to try to re-create something like what they learned from training data.

Asmelash Teka Hadgu, co-founder of Lesan and a fellow at the DAIR Institute, said, “Image generators tend to perform much better on artifacts like cars and people’s faces, and less so on smaller things like fingers and handwriting.”

This could be because these smaller details don’t often appear as prominently in training sets as concepts like how trees usually have green leaves. The problems with diffusion models might be easier to fix than the ones plaguing transformers, though. Some image generators have improved at representing hands, for example, by training on more images of real, human hands.

“Even just last year, all these models were really bad at fingers, and that’s exactly the same problem as text,” Guzdial explained. “They’re getting really good at it locally, so if you look at a hand with six or seven fingers on it, you could say, ‘Oh wow, that looks like a finger.’ Similarly, with the generated text, you could say, that looks like an ‘H,’ and that looks like a ‘P,’ but they’re really bad at structuring these whole things together.”

That’s why, if you ask an AI image generator to create a menu for a Mexican restaurant, you might get normal items like “Tacos,” but you’ll be more likely to find offerings like “Tamilos,” “Enchidaa” and “Burhiltos.”

As these memes about spelling “strawberry” spill across the internet, OpenAI is working on a new AI product code-named Strawberry, which is supposed to be even more adept at reasoning. The growth of LLMs has been limited by the fact that there simply isn’t enough training data in the world to make products like ChatGPT more accurate. But Strawberry can reportedly generate accurate synthetic data to make OpenAI’s LLMs even better. According to The Information, Strawberry can solve the New York Times’ Connections word puzzles, which require creative thinking and pattern recognition to solve and can solve math equations that it hasn’t seen before.

Meanwhile, Google DeepMind recently unveiled AlphaProof and AlphaGeometry 2, AI systems designed for formal math reasoning. Google says these two systems solved four out of six problems from the International Math Olympiad, which would be a good enough performance to earn as silver medal at the prestigious competition.

It’s a bit of a troll that memes about AI being unable to spell “strawberry” are circulating at the same time as reports on OpenAI’s Strawberry. But OpenAI CEO Sam Altman jumped at the opportunity to show us that he’s got a pretty impressive berry yield in his garden.