Open-Source Natural Language Processing: EleutherAI’s GPT-J

OpenAI released GPT-3 in June of 2020  – and the results were instantly compelling. By 2021, the MIT Technology Review was proclaiming OpenAI’s GPT-3 “a big step toward AI that can understand and interact with the human world.” It has since commercialized in collaboration with Microsoft.   In response, EleutherAI – a self-described “grassroots collective of researchers working to open-source AI research”  launched GPT-J in July 2020 as a quest to replicate the OpenAI GPT collection of models. The goal is to “break the OpenAI-Microsoft monopoly” through broadening availability and the collective intelligence of open-source development of a competing class of GPT models.