A new open-source language model has claimed the throne of the best in the world, according to the latest rankings from Hugging Face, one of the leading platforms for natural language processing (NLP) research and applications. The model, called “Smaug-72B,” was released publicly today by the startup Abacus AI, which helps enterprises solve difficult problems in the artificial intelligence and machine learning space. Smaug-72B is technically a fine-tuned version of “Qwen-72B,” another powerful language model that was released just a few months ago by Qwen, a team of researchers at Alibaba Group. What’s most noteworthy about today’s release is that Smaug-72B outperforms GPT-3.5 and Mistral Medium, two of the most advanced open source large language models developed by OpenAI and Mistral, respectively, in several of the most popular benchmarks. Smaug-72B also surpasses Qwen-72B, the model from which it was derived, by a significant margin in many of these evaluations. According to the Hugging Face Open LLM leaderboard, which measures the performance of open-source language models on a variety of natural language understanding and generation tasks, Smaug-72B is now the first and only open-source model to have an average score more than 80 across all major LLM evaluations. While the model still falls short of the 90-100 point average indicative of human-level performance, its birth signals that open source AI may soon rival Big Tech’s capabilities, which have long been shrouded in secrecy. In short, the release of Smaug-72B could fundamentally reshape how AI progress unfolds, tapping the ingenuity of those beyond just a handful of wealthy companies.

Full story : Open-source AI Smaug-72B outperforms GPT-3.5 and Mistral Medium on Hugging Face benchmarks.