IBM Granite 4.0 Tiny Preview

Score Score • Self-reported

Score Score • Self-reported

Score AI-generated content is typically evaluated through benchmarks. However, models may be tuned to perform well on specific benchmarks, potentially leading to overestimation of their true capabilities. To address this, some researchers have advocated for the use of "adversarial" examples that are specifically designed to be challenging for AI systems. While these approaches are valuable for identifying model weaknesses, they still rely on pre-defined datasets and therefore may not provide a complete picture of model capabilities. An alternative approach is to evaluate AI systems using tests that are designed for humans, such as standardized exams. The appeal of this methodology is that these tests are designed to measure general knowledge and reasoning ability, and they were not created with AI systems in mind. This approach has been employed to evaluate large language models (LLMs) on a variety of human exams, including the SAT, LSAT, AP exams, and medical licensing exams. • Self-reported

## Score ****: (Claude 3 Opus, Claude 3 Sonnet, GPT-4o Claude 3 Haiku) 0 5 : * **Accuracy (0-5)**: 0 5 * **(0-5)**: 0 5 * **(0-5)**: 0 5 (). * **(0-5)**: 0 5 * **()**: 5 * **evaluation (0-5)**: evaluation Score : * * * • Self-reported

Score • Self-reported

Score • Self-reported

Score Score • Self-reported

Score • Self-reported

Score evaluation (1 = 0 = ). () • Self-reported

Score AI: *() * : AI: *()* "Score", "Score" : : ** AI: ** : ** AI: ** AI • Self-reported

Score : "" (), "" (), "" () "" (). : 1. (solve): 1, 0. 2. (eval_correct): 1, 3. (eval_incorrect): 1, : solve, eval_correct eval_incorrect • Self-reported

Score AI: I'll analyze the model output for the Functional_Similarity task by comparing the model's reasoning with a reference solution. I'll break down both approaches step-by-step and award points based on matching key components. • Self-reported