<\/p>\n
Epoch AI’s latest benchmarking roundup highlights how Chinese and open-source AI models are swiftly narrowing the performance gap with top-tier international models, driven by architectural innovations and increased investment, but still face challenges in mathematical reasoning and deployment stability.<\/p>\n<\/div>\n
The year-end benchmarking roundup released by Epoch AI paints a picture of rapidly advancing but imperfect artificial intelligence: top-tier international models continue to push the frontier in mathematical reasoning, while open-source competitors, especially Chinese models, are narrowing the gap quickly but remain behind on the hardest problems. According to the report by 36Kr, models such as OpenAI\u2019s GPT family and Google\u2019s Gemini performed strongly on the expert-level FrontierMath suite, yet none achieved full marks on the most demanding problems, underlining persistent limits in deep mathematical reasoning. [1]<\/a><\/sup><\/p>\n FrontierMath, the benchmark at the heart of the evaluation, is a deliberately \u201cguessproof\u201d set of 350 expert-crafted problems spanning number theory, real analysis, algebraic geometry and category theory, designed to require hours or days of human-level work on some items. Epoch AI\u2019s public documentation describes the dataset\u2019s four-tier structure, with 300 questions in Levels 1\u20133 and 50 extremely difficult Level 4 problems; only a small subset of questions are public to preserve the test\u2019s integrity. The benchmark requires models to output a Python function that returns an exact answer (typically an integer or a sympy object), and the evaluation enforces hard token and 30-second code-execution limits to ensure reproducibility on commercial hardware. According to Epoch AI, these design choices aim to make scores verifiable and resilient to simple heuristic approaches. [2]<\/a><\/sup>[3]<\/a><\/sup><\/p>\n On Levels 1\u20133 the report finds the highest-performing Chinese open-source models trailing the global frontier by roughly seven months, a gap that Epoch AI and 36Kr characterise as historically small given how recently open-source efforts lagged by years. DeepSeek V3.2 (Thinking) emerged as the only Chinese model to score non-zero on Level 4, correctly solving one of the 48 private Level 4 problems (about 2%), a result that 36Kr highlights as symbolically significant despite its small absolute value. Epoch AI\u2019s LinkedIn post summarises this outcome and underscores that the best open-weight Chinese models are converging rapidly but still face clear challenges on genuinely frontier tasks. [1]<\/a><\/sup>[7]<\/a><\/sup><\/p>\n Technical explanations in the report attribute much of the narrowing gap to architectural and algorithmic advances rather than raw compute: techniques such as multi-head latent attention (MLA), innovations in mixture-of-experts (MoE) designs, and multi-token prediction allowed DeepSeek to reach pre-training parity with larger models like Meta\u2019s Llama 3 while using an order of magnitude less compute, and its released inference model R1 performs comparably to OpenAI\u2019s o1 at far lower development cost. Epoch AI uses third-party APIs for some evaluations and notes that API stability and interface differences can slightly depress public scores, particularly for newly released models, suggesting actual capabilities may sometimes be stronger than the published numbers indicate. [1]<\/a><\/sup>[2]<\/a><\/sup><\/p>\n The report also documents operational constraints on frontier models: GPT-5\u2019s arrival in 2025 produced substantial metric gains over GPT-4 (large percentage lifts across MMLU, MATH, HumanEval and other benchmarks), but expectations were tempered by a faster release cadence and the prior market penetration of intermediate models such as Claude 3.7 and Gemini 2.5. Epoch AI further notes API reliability problems affected certain evaluations, Gemini 3 Pro lost points on multiple Tier 1\u20133 questions due to API errors and xAI\u2019s Grok 4 suffered timeouts on a notable share of Tier 4 items, demonstrating that deployment stability is an increasingly important constraint on realised model performance. The report also points to OpenAI\u2019s disclosed 2024 computing budget, where the majority of spend went to experimental training and research rather than final product runs, as evidence that R&D and discovery carry the lion\u2019s share of frontier costs. [1]<\/a><\/sup><\/p>\n Epoch AI\u2019s capabilities index (ECI) shows an accelerating trajectory for top-tier models: the organisation identifies a breakpoint in April 2024 after which annual capability gains approximately doubled (from about 8.2 points\/year to 15.3 points\/year), a pattern it links to the emergence of efficient inference models and intensified reinforcement learning investment. The acceleration compresses the competitive window between closed-source frontier labs and open-source teams, increasing pressure on community projects to iterate rapidly on algorithms, data and training strategies to remain competitive. Industry-wide trends reported by Epoch AI also include dramatic inference-cost reductions, varying by task complexity from modest to exponential declines, driven by market competition and inference optimisation, though cost benefits are uneven across task types. [1]<\/a><\/sup>[5]<\/a><\/sup>[6]<\/a><\/sup><\/p>\n Taken together, the findings portray an AI landscape where capability growth is fast and broadly accessible, but where the last miles of robust mathematical reasoning and reliable deployment remain concentrated among a handful of well-resourced labs. Open-source projects and consumer-grade hardware are closing in quickly, but maintaining a durable lead will depend on sustained investment in experimentation, algorithmic innovation and production-grade stability. [1]<\/a><\/sup>[2]<\/a><\/sup>[4]<\/a><\/sup><\/p>\n ##Reference Map:<\/p>\n\n