DeepSeek R1 Distill Llama 8B

Diamond, Pass@1 Evaluation efficiency tasks with first times, when model gives correct final answer in first attempt, without any-or additional prompts or iterations. This strict score, which measures, how well exactly model can solve task with first attempts. Diamond relates to to methodology evaluation, which on abilities model solve complex tasks with first attempts. Pass@1 means percentage tasks, which model solved correctly with first times. This score for tasks, where not attempts or where accuracy first answer critically important • Self-reported

Cons@64 In Cons@64 we approach Chain-of-Thought (CoT), "thinking through ", that, how people solve mathematical tasks: various errors and to In our method LLM (model) solves task k times with various Then we answers by and we analyze chains reasoning, in order to determine, is whether indeed solution. We we verify each chain reasoning, in order to identify errors, specific steps, where model errors. On GPQA results Cons@64 with experts-people, achieving accuracy 0.79 on language and 0.81 on language, by comparison with 0.79 at experts. On MMLU results Cons@64 even evaluation people • Self-reported

Pass@1 Metric Pass@1 measures proportion tasks, which model solves correctly with first attempts. This metric performance, in which model or correct answer with first attempts, or no, when user on answer model, not several attempts or iterations. This metric especially useful in cases: - Evaluation performance model in scenarios use, where user, total, first answer model - Understanding abilities model solve tasks without additional attempts - easily metrics for comparison models - level performance for methods such how (self-consistency) or best from k (best-of-k) Pass@1 is strict metric, since requires from model demonstration fully correct reasoning and output for one attempt. She/It especially important for understanding performance model in conditions resources or time • Self-reported

Pass@1 AI • Self-reported