Phi 4 Reasoning

Diamond AI: by texts about models artificial intelligence. human. Translate following text descriptions method analysis model AI on Russian language, rules: 1. on language. 2. all and in form (for example: GPT, LLM, API, AIME, GPQA). (for example: "thinking mode" → "mode thinking", "tools" → "tools"). 3. and 4. descriptions. 5. Not information, only then, that maintaining all details. 6. models (for example "GPT-5 nano", "Claude") on 7. benchmarks and on (for example: "AIME", "FrontierMath", "Harvard-MIT Mathematics Tournament"). 8. should be on text, 9. explanations, quotes or — on ONLY translation • Self-reported

Standard evaluation AI: Thinker • Self-reported

Standard evaluation AI • Self-reported

Standard evaluation AI: Standard evaluation • Self-reported

3K-AI: this 3000-() data. In context models machine training and processing language usually matches parts or in 3000 tokens means, that for analysis is used set data specific size • Self-reported

Standard evaluation AI: works better, if at us is about that, how well well work systems, which we For AI this often can be complex task, but this part. We we evaluate its model on various data and benchmarks, but some which usually are used in : * : We accuracy our models on various standard sets data, which evaluate their understanding diverse that, such how and For example, MMLU — this set data from more 14,000 questions with multiple choice, 57 We also over testing for evaluation understanding, especially by more knowledge, for example in exact * and logical thinking: We we evaluate ability models solve different mathematical and logical tasks, from base to mathematics. We we use how standard tests in this field (for example, MATH and GSM8K), so and new, more complex tests, in order to evaluate abilities our models in solving tasks and their reliability in these fields. * Reasoning and : We we measure ability models to and with common (sense) meaning, and also ability interpret such how and For example, HellaSwag verifies ability model choose most that requires from her understanding * : We we evaluate ability our models and code on various languages programming, and also solve different tasks programming, understand and answer on questions about code. We we use data, such how HumanEval for Python, MBPP for various languages programming and Natural2Code for more diverse tasks • Self-reported

Standard evaluation • Self-reported

8/1/24–2/1/25 • Self-reported

Standard evaluation AI: Yes, this is a standard benchmark evaluation. What can I help you with? • Self-reported

Standard evaluation AI: ChatGPT 4o *Process* We've established a standard evaluation protocol for mathematical assessments. 1. The test is presented to the AI one problem at a time. 2. For each problem, the AI is instructed to work step-by-step and provide a final answer. 3. We score responses in a binary fashion - correct or incorrect. 4. No partial credit is awarded. 5. Answers must match the exact form specified in the problem (e.g., reduced fractions, simplified expressions). 6. For multiple-choice questions, only the letter choice is required for scoring. *Implementation notes* - All problems are presented with clear instructions on required answer format. - The AI receives no external feedback during testing. - We test each model variant with the same problems in identical order. - Evaluations are conducted without human intervention or feedback loops. This protocol ensures consistent measurement across different AI systems and provides a clear benchmark for mathematical reasoning capabilities. • Self-reported

2.21 AI: I'll evaluate it on the MMLU benchmark. MMLU stands for Massive Multitask Language Understanding, a benchmark that tests the model on a wide variety of tasks such as elementary mathematics, US history, computer science, law, and more. It measures the model's ability to apply knowledge across these different domains, requiring both factual recall and reasoning. Let me break down the model's performance by category. • Self-reported