When Meta initially hinted in April that it was developing an open-source model with performance comparable to the top private models from firms like Open AI, it was a first for the AI business.
That model is here today. The largest-ever open-source AI model, Llama 3.1, is being released by Meta. According to the firm, it scores better on several benchmarks than Anthropic’s Claude 3.5 Sonnet and GPT-4o. Along with expanding the Llama-based Meta AI assistant’s language and country of availability, it also adds a capability that allows it to create graphics based on a person’s unique appearance. By the end of this year, CEO Mark Zuckerberg now believes Meta AI to overtake ChatGPT as the most popular assistant.
Compared to the smaller Llama 3 models that were released a few months ago, Llama 3.1 is far more complicated. With 405 billion parameters, the biggest version was trained using more than 16,000 of NVIDIA’s incredibly costly H100 GPUs. Although Meta has not disclosed the exact cost of building Llama 3.1, it is reasonable to assume that it was in the hundreds of millions of dollars, just based on the price of the NVIDIA processors.
So why is Meta still giving Llama free with a license that just needs the consent of businesses with hundreds of millions of users, considering the expense? Similar to how Linux evolved into the open-source operating system that now runs the majority of phones, servers, and devices, Zuckerberg contends in a letter posted on Meta’s corporate blog that open-source AI models will surpass proprietary models and are currently advancing more quickly.
He draws a parallel between Meta’s previous Open Compute Project, which he claims saved the business “billions” by enlisting the assistance of outside firms like HP to enhance and standardize Meta’s data center designs as it was expanding its own capacity, and its current investment in open-source AI. He writes, “I believe the Llama 3.1 release will be an inflection point in the industry where most developers begin to primarily use open source.” He anticipates the similar scenario to play out with AI in the future.
Meta is collaborating with over twenty organizations, including as Microsoft, Amazon, Google, NVIDIA, and Databricks, to assist developers in releasing their own versions of Llama 3.1 into the wild. According to Meta, the manufacturing running costs of Llama 3.1 are around half of those of Open AI’s GPT-4o. The model weights are being made available so businesses may train and fine-tune it using their own data.
It should come as no surprise that Meta is keeping quiet about the data it used to train Llama 3.1. AI industry employees claim that this knowledge is kept confidential as trade secrets, but detractors accuse them of using this as a ploy to postpone the impending wave of copyright litigation.
Meta will state that the 405-billion parameter version of Llama 3.1 outperformed the smaller 70 billion and 8 billion versions through the use of synthetic data, or data produced by a model instead of by people. Llama 3.1 will be well-liked by developers as “a teacher for smaller models that are then deployed” in a “more cost-effective way,” according to Meta’s Vice President of Generative AI, Ahmad Al-Dahle.
Red teaming, also known as adversarial testing, of Llama 3.1 by Meta included searching for possible applications in biochemistry and cyber security for the first time. The “agentic” behaviors that Meta is portraying as arising are another incentive to test the model more rigorously.
Although the most sophisticated 405-billion parameter model of Llama 3.1 is available for free usage in Meta AI, the assistant will automatically switch to the more basic 70-billion model after you exceed a certain number of prompts in a given week. This implies that Meta cannot operate at full capacity due to the 405-billion model’s high cost. Speaking for the firm, Jon Carvill said that after evaluating early usage, it will furnish me with more details on the quick threshold.
For more tech news, don’t forget to check out TechExposed.co
What’s Up with Meta ‘s Llama 3.1?
