de Curtò J., de Zarzà I., García P., Cabot J., Cano J.C., Calafate C.T.
Information Processing and Management, vol. 63, n° 7, art. no. 104878, 2026
This paper presents a comprehensive cross-platform evaluation of reasoning capabilities in contemporary foundation models, establishing an infrastructure-agnostic benchmark across three computational paradigms: HPC supercomputing (MareNostrum 5), cloud platforms (Nebius AI Studio), and university clusters (a node with eight H200 GPUs). We evaluate 15 foundation models across 79 problems spanning eight academic domains (Physics, Mathematics, Chemistry, Economics, Biology, Statistics, Calculus, and Optimization) through three experimental phases: (1) Baseline establishment: Six models (Mixtral-8x7B, Phi-3, LLaMA 3.1-8B, Gemma-2-9b, Mistral-7B, OLMo-7B) evaluated on 19 problems using MareNostrum 5, establishing methodology and reference performance; (2) Infrastructure validation: The 19-problem benchmark repeated on university cluster (seven models including Falcon-Mamba state-space architecture) and Nebius AI Studio (nine state-of-the-art models: Hermes-4 70B/405B, LLaMA 3.1-405B/3.3-70B, Qwen3 30B/235B, DeepSeek-R1, GPT-OSS 20B/120B) to confirm infrastructure-agnostic reproducibility; (3) Extended evaluation: Full 79-problem assessment on both university cluster and Nebius platforms, probing generalization at scale across architectural diversity. Results challenge prevailing scaling assumptions through a parameter efficiency paradox: Hermes-4-70B (70B parameters) achieves the highest score among extended models (0.598), outperforming both its 405B counterpart (0.573) and Meta’s LLaMA 3.1-405B (0.560). Domain-specific analysis reveals LLaMA 3.1-405B achieves record Calculus performance (0.717), while DeepSeek-R1 sets unprecedented standards for reasoning transparency (0.716 step-accuracy). Qwen3 models demonstrate exceptional consistency, with mean per-evaluation standard deviation of 0.013, the lowest among Nebius models. The findings challenge conventional scaling assumptions, establish training data quality as more critical than model size, and provide actionable guidelines for model selection across educational, production, and research contexts. The tri-infrastructure methodology and 79-problem benchmark enable longitudinal tracking of reasoning capabilities as foundation models evolve.
