Elon Musk’s xAI released Grok-1’s AI model code and weights, taunting OpenAI in the process.
This release via GitHub and BitTorrent allows researchers and developers worldwide to build and iterate with its 314 billion parameters – some 150 billion more than GPT-3.
xAI aims to democratize access to advanced LLM tech by providing a raw, unrefined version of Grok-1 ready for experimentation in any capacity — even commercially.
░W░E░I░G░H░T░S░I░N░B░I░O░
— Grok (@grok) March 17, 2024
Musk, of course, couldn’t resist a bit of (un)friendly banter over open-sourcing Grok. The ChatGPT X account commented on Grok’s post above, “stole my whole joke,” to which Musk quipped, “Tell us more about the “Open” part of OpenAI …”
Tell us more about the “Open” part of OpenAI …
— Elon Musk (@elonmusk) March 17, 2024
Musk and OpenAI’s founders Sam Altman and Greg Brockman are locked in a legal battle and debate about OpenAI’s dramatic progression from a non-profit open-source research company to a profit-making arm of Microsoft.
Grok is another thorn in the side of OpenAI, which is experiencing pressure from multiple fronts with the recent release of Anthropic’s impressive Claude 3 Opus and Google’s Gemini. Even Apple has joined the LLM fray with its newly released MM1.
However, Grok-1 isn’t immediately ready and accessible for conversational AI applications.
For one, the model hasn’t been fine-tuned with specific instructions or datasets to perform optimally in dialogue systems. This means that additional effort and resources will be necessary to harness Grok-1’s capabilities for such tasks, posing a challenge to those interested in developing conversational AI.
In addition, the sheer size of the model’s weights – a hefty 296GB – means that running the model requires substantial computational resources, including high-end datacenter-class hardware.
However, the AI community is anticipating potential efforts to optimize Grok-1 through quantization, which could reduce the model’s size and computational demands, making it more accessible beyond those with generative AI-friendly rigs.
Grok-1 is truly open-source
One of the most important aspects of Grok-1’s release is xAI’s choice to use the Apache 2.0 license, joining the likes of Mistral’s 7B.
Unlike some licenses that impose more restrictive conditions on the use and distribution of the software, the Apache 2.0 license allows for broad freedom in how the software can be used, modified, and distributed.
Grok weights are out under Apache 2.0: https://t.co/9K4IfarqXK
It’s more open source than other open weights models, which usual come with usage restrictions.
It’s less open source than Pythia, Bloom, and OLMo, which come with training code and reproducible datasets. https://t.co/kxu2anrNiP pic.twitter.com/UeNew30Lzn
— Sebastian Raschka (@rasbt) March 17, 2024
This includes commercial uses, making Grok-1 an attractive foundation for companies and individuals aiming to build upon or integrate the model into their own products and services.
By making Grok-1’s weights and architecture freely available, xAI advances Musk’s vision for open AI and confronts the AI community at large.
Every viable open-source model threatens to erode revenue for closed-source developers like OpenAI and Anthropic.
OpenAI is probably shaken by recent developments from Anthropic, Google, and now xAI.
The community is gearing up for some sort of GPT-5 or Sora release that will see them re-take the upper hand.
