Meta, the company behind one of the world’s most prominent open-source large language models, Llama, is preparing for a significant leap in its AI development efforts. In a recent earnings call for Q2 2024, It CEO Mark Zuckerberg disclosed that the next iteration of their AI model, Llama 4, will require nearly ten times the computing power that was needed for its predecessor, Llama 3.

Meta Ramping Up for Future AI Needs

Zuckerberg’s remarks underscore It commitment to staying at the forefront of AI development. “The amount of computing needed to train Llama 4 will likely be almost 10 times more than what we used to train Llama 3, and future models will continue to grow beyond that,” he stated during the call. He emphasized the importance of proactively building infrastructure to meet these demands, despite the uncertainties surrounding future AI trends.

“It’s hard to predict how this will trend multiple generations out into the future. But at this point, I’d rather risk building capacity before it is needed rather than too late, given the long lead times for spinning up new inference projects,” Zuckerberg added, highlighting the strategic foresight necessary to maintain a competitive edge in the rapidly evolving AI landscape.

Meta’s Current AI Landscape

It has been pushing the boundaries of AI with the Llama series. Llama 3, released in April 2024, featured an impressive 80 billion parameters. Just last week, the company introduced Llama 3.1 405B, an upgraded version boasting a staggering 405 billion parameters, making it Meta’s most advanced open-source model to date. These developments reflect Meta’s ambition to create increasingly sophisticated AI models that can handle complex tasks across various domains.

Investment in AI Infrastructure

Susan Li, Meta’s Chief Financial Officer, echoed Zuckerberg’s sentiments, stating that the company is already exploring different data center projects to support the training of future AI models. She acknowledged that these investments would likely drive up capital expenditures in the coming years, with significant increases expected in 2025.

Training large-scale language models is a resource-intensive endeavor. Meta’s capital expenditures soared by nearly 33% to $8.5 billion in Q2 2024, up from $6.4 billion in the same period the previous year. This increase was largely attributed to investments in servers, data centers, and network infrastructure—essential components for sustaining the company’s AI ambitions.

Competitive Landscape and Strategic Flexibility

Meta’s focus on building its own capacity for AI training aligns with broader industry trends. For instance, OpenAI, another major player in the AI space, reportedly spends $3 billion on training models and an additional $4 billion on server rentals, with support from Microsoft. The high costs associated with AI development are a testament to the intense competition and the scale of resources required to stay competitive.

Li also emphasized Meta’s strategic approach to infrastructure development. “As we scale generative AI training capacity to advance our foundation models, we’ll continue to build our infrastructure in a way that provides us with flexibility in how we use it over time. This will allow us to direct training capacity to gen AI inference or to our core ranking and recommendation work, when we expect that doing so would be more valuable,” she explained.

The Future of AI at Meta

While Meta’s AI efforts are gaining traction, the company does not anticipate immediate financial returns from its generative AI products. Despite this, Meta’s consumer-facing Meta AI chatbot has found significant success in markets like India, which has become the largest user base for the service.

Zuckerberg’s vision for Llama 4 and beyond reflects a broader trend in the tech industry: the relentless pursuit of more powerful AI models, even as the costs and complexities of developing such models continue to rise. For Meta, this means investing heavily in infrastructure and maintaining the agility to adapt to the ever-changing demands of AI research and development.

As Meta gears up for the next phase of AI innovation, the industry will be watching closely to see how these investments pay off—and how they might shape the future of artificial intelligence.

ALSO READ THIS BLOG