Grok 4.5 gave Elon Musk’s cost-and-performance claim new support after an independent agent benchmark ranked the model first.
Key Points:
- Grok 4.5 scored 51.4% on AutomationBench-AA, ahead of two Claude models.
- The model cost $0.34 per task, far below its closest Anthropic rivals.
- Its higher rule-violation rate remains a risk for enterprise agent deployments.
Grok Benchmark
Artificial Analysis said Grok 4.5 scored 51.4% on AutomationBench-AA, ahead of Claude Fable 5 at 48.6% and Claude Opus 4.8 at 48.5%. The model also cost $0.34 per task, far below Fable 5 at $1.35 and Opus 4.8 at $1.46.
The benchmark adds an outside test to Musk’s claim that Grok 4.5 is an Opus-class model that runs faster and costs less. SpaceXAI took the model public this week, using a 1.5 trillion-parameter V9 foundation and early internal evaluations to frame the launch.
AutomationBench-AA uses 657 tasks across 40 simulated apps, including Gmail, Slack, Salesforce and HubSpot. The test tracks whether an AI agent can complete objectives without breaking guardrails, and Artificial Analysis keeps the task set private to reduce benchmark contamination.
Also Read: Grok 4.5 Challenges OpenAI And Anthropic With Cheaper Agentic AI
Musk's Bold Claim
Artificial Analysis said Grok 4.5 used about 8,000 output tokens per task, roughly a quarter of Opus 4.8’s total. “Its total token usage of 0.44M per task is among the lowest on the leaderboard,” the firm said, adding that low cost came from efficiency and token pricing.
The model completed 79.9% of task objectives and fully passed 21.9% of tasks. In finance, the benchmark’s hardest domain, Grok 4.5 led with 71%, compared with 64% for Fable 5 and 62% for Opus 4.8.
The weakness was compliance. Grok 4.5 logged 0.63 guardrail violations per task, above Opus 4.8 at 0.55 and Google’s Gemini 3.5 Flash at 0.46, a gap that matters for companies using agents near live financial systems.
Musk’s launch pitch focused on a practical trade-off rather than a clean benchmark sweep. Grok 4.5 now has outside evidence for cost and speed, but the higher rule-violation rate shows why enterprise adoption will still depend on reliability.
Read Next: Bitcoin’s Recovery Has A Demand Problem Bulls Cannot Ignore