The first edition of Meta’s LlamaCon has kicked off. Microsoft CEO Satya Nadella was also present and revealed that 30 percent of the company’s code is now written by AI. What are we to make of this?
The question about this percentage was asked by Mark Zuckerberg, CEO of Meta. Although Nadella mentioned “30-40 percent,” he emphasized that this AI share is increasing. Most of the AI-written code is for new projects. This is a remarkable achievement, given that Nadella considers the existing C++ code bases to be of lower quality than the newer Python code.
But what is AI?
Speaking of Zuckerberg, the Meta CEO does not want to mention a percentage himself. According to him, it is difficult to say whether the code in question was completed by AI or written autonomously. That is a huge difference: if autocomplete counts, a huge portion of our current email traffic has been AI-generated for years. Without making that distinction, we do not know exactly what Microsoft means by AI code.
AI programming assistance is also still in a vulnerable stage. In September, we reported on the large amount of unsafe code generated by AI. At the time, commercial models hallucinated 5.2 percent of the code packages used, while open-source LLMs went haywire in 21.7 percent of cases.
In short, even with the improvements made since then, we must remain skeptical about the reliability of AI outputs for coding. However, Zuckerberg notably noted that AI will actually increase the security level of code. It is unclear why he holds this opinion.
Acceleration
Knowing that the margins of error in AI code are decreasing and that IDEs and other development tools are increasingly responding to AI-generated code, the improvement will continue. That is why Nadella and Zuckerberg agree that AI code will account for an increasingly large portion of codebases. Now, that’s also a somewhat tricky way to determine the “success” of AI coding. How many vulnerabilities are there? Did the AI find the right path in just one or a few attempts?
More importantly, was it still a human who made certain software engineering choices? After all, AI tools must provide assistance regardless of the language, packages, or purposes developers have in mind. Zuckerberg also wants to take a different approach, with AI accelerating Llama development. Nadella even expects development tools and compute infrastructures to be redesigned to accommodate AI agents. Clear language, but language that raises more questions than it answers.
Read also: Google is putting AI to work: 25% of new code is AI-generated