Imagine asking a sophisticated AI model for tips on making the perfect pizza, only to be told to use glue to help the cheese stick. Or watching it fumble through basic arithmetic problems that a middle schooler could solve with ease. These are the limitations and quirks of generative AI and the large language models (LLMs) that underpin them. They’re happening because AI models are running out of good training data, causing them to plateau. This is a cycle in innovation that repeats throughout history: For a long time, an almost undetectable amount of knowledge and craft builds up around an idea, like an invisible gas. Then, a spark. An explosion of innovation ensues but, of course, eventually stabilizes. This pattern is called an S-Curve. For example:

• TCP/IP: Originating in ideas from the 1960s, TCP/IP saw significant acceleration after its 1974 RFC and stabilized with version 4 in 1981, which still underpins the modern Internet.
• The Browser Wars: During the late 1990s, web browsers rapidly evolved into interactive, programmable platforms. Since then, improvements have largely been incremental.
• Mobile Apps: The iPhone App Store’s launch in 2008 spurred a surge in mobile app innovation. Today, truly novel mobile apps are rare.

The AI revolution is following this curve. In a 1950 paper, Alan Turing was one of the first computer scientists to explore how to build a thinking machine, starting the slow buildup of knowledge. Seventy years later, the spark: A 2017 research paper, Attention Is All You Need, leads to OpenAI’s development of ChatGPT, which convincingly mimics human conversation, unleashing a global shock wave of innovation based upon generative AI technology.

Full opinion : The first wave of AI innovation is over. Here’s what comes next.