DeepSeek R1 Distill Qwen 14B

# Diamond, Pass@1 Pass@1 — this measure that, how well successfully model solves tasks at one attempts. In given case "Pass@1" proportion tasks, which model successfully solves with first attempts. This score for evaluation abilities model correctly solve tasks without necessity several iterations. In context mathematical tasks Pass@1 means percentage tasks, for which model with first attempts provides correct solution and answer. This score especially in real scenarios use, where users often on correct answer immediately, without necessity repeated queries or In difference from other metrics, such how Pass@k (where solutions at k attempts), Pass@1 is more since not allows capabilities corrections errors or approach to solving tasks • Self-reported

Cons@64 AI: I with method "Cons@64". Method "Cons@64" 64 from model and their for formation final answer, (Chen et al., 2023). We we use following instructions for sample from model: : Solve following task [task]. Please, thoroughly step for step solution and answer in format ": X". We 64 answer from model. How in we we use first value, which after ":" in capacity final answer model. Then, we we determine majority from these 64 answers how answer. In case we manner we choose answer from those, that have majority • Self-reported

Pass@1 AI, and especially large language model, often solve tasks method and errors. Model tries solve task several times, and if it find correct solution in one from attempts, we we consider, that model capable solve this task. Pass@1 evaluates probability that, that model correct answer with first attempts. This metric can be on basis Pass@k for cases, when model generates k various solutions. For example, if model solves task correctly in 40% cases from 10 attempts, we we can evaluate probability success with first attempts how 40% • Self-reported

Pass@1 Pass@1 - this metric, which measures proportion tasks, solved model with first attempts. For model, set tasks and Pass@1 is calculated following manner: 1) For each tasks in set model generates one solution, which then is evaluated. tasks, for which model correct solution with first attempts, and is Pass@1. Pass@1 useful for evaluation that, how well well model handles with tasks "with ", without capabilities do several attempts. However this metric can significantly abilities model, since she/it not accounts for, that model can several different approaches to solving tasks or have some in even at solutions • Self-reported