Apple recently announced upgrades to the artificial intelligence models that power its Apple Intelligence features across iOS, macOS, and other platforms. However, according to benchmarks published by Apple itself, these updated models still lag behind those developed by leading tech companies like OpenAI, Google, and Meta.
Benchmark Comparisons Reveal Performance Gaps
In a Monday blog post, Apple shared results from human evaluations comparing the quality of text generated by its newest “Apple On-Device” model—which runs offline on devices such as the iPhone—to similarly sized models from Google and Alibaba. Testers rated Apple’s model as comparable but not superior.
The more powerful “Apple Server” model, designed to operate in Apple’s data centers, was rated behind OpenAI’s GPT-4o, now a year old.
Surprisingly, in image analysis tests, human raters preferred Meta’s Llama 4 Scout model over Apple Server. This is notable given that Llama 4 Scout has underperformed compared to Google, Anthropic, and OpenAI’s leading AI systems in other tests.
These benchmark results align with reports suggesting Apple’s AI research has struggled to keep pace in the competitive AI landscape. Recent years have seen underwhelming AI capabilities from Apple, including delays to promised improvements in Siri. Some customers have even filed lawsuits claiming Apple marketed AI features it has yet to deliver.
Model Details and Developer Access
The Apple On-Device model is roughly 3 billion parameters in size and supports functions like text summarization and analysis. (Model size, indicated by the number of parameters, generally correlates with problem-solving ability.) Apple has opened access to this model for third-party developers through its Foundation Models framework.
Apple also noted improvements in both Apple On-Device and Apple Server models related to tool use, efficiency, and multilingual understanding, supporting approximately 15 languages. This advancement partly stems from an expanded training dataset including images, PDFs, documents, infographics, tables, and charts.
Author’s Opinion
Apple’s cautious but incremental AI progress highlights the challenge of competing in a field where leaders like OpenAI and Google aggressively push boundaries. While Apple’s focus on privacy and device-based AI is commendable, it risks falling behind without a breakthrough leap. The company must accelerate innovation and transparency if it wants to regain AI leadership and meet growing user expectations.
Featured image credit: Dries BuyTaert
For more stories like it, click the +Follow button at the top of this page to follow us.