Llama 3.2 11B Instruct

accuracy • Self-reported

0-shot, CoT This basic option method "chains reasoning" (Chain of Thought), without additional examples. We model think step by step, but not we provide examples that, how this do. We we use such instructions, how "Let's let's solve this task step by step" or "Let's let's think about this step for step", in order to more reasoning. This method offers model create chain reasoning, to answer, instead that in order to generate answer directly. This approach especially useful for complex tasks, requiring multi-step reasoning, such how mathematical tasks or logical puzzles, where intermediate steps can have value for obtaining correct answer • Self-reported

0-shot, CoT reasoning without examples. In this method we model solve task, using step-by-step reasoning, but not examples that, how this should look. Usually this with help prompts "about this step for step". This method often is used in research large language models, since he and efficient. Although model not receives step-by-step solutions, modern model usually generate its independently • Self-reported

CoT Approach with using chains thinking (Chain-of-Thought, CoT) — this method, at which we at model answer on task without provision examples that, how solve tasks. Instead this we we ask model "think step for step", in order to she/it could structure its thoughts and break down complex task on more simple parts. For example, at use standard approach we simply : "17 × 28". When use approach with chain thinking we : "17 × 28. Let us think step for step." Research showed, that simple addition phrases "Let us think step for step" to query significantly improves ability model solve tasks, requiring several steps reasoning • Self-reported

Accuracy test AI: I'll translate the technical text about AI model analysis. Test accuracy • Self-reported

Test, 0-shot CoT relaxed accuracy • Self-reported

ANLS AI: I'll translate this technical text about model analysis methods according to your requirements. ## ANLS ANLS (Average Normalized Levenshtein Similarity) - this metric evaluation for tasks extraction text from images, which measures between and : 1. For each and text is calculated 2. This by means of on more 3. in : 1 - _4. ANLS - this average value these by all examples ANLS is used in tasks OCR, text in and extraction information from visual data. ANLS from 0 to 1, where 1 means match between and text • Self-reported

Accuracy on AI: 5 points to evaluate AI performance on Machine Learning benchmarks. 1. Accuracy and metric definition. 2. Train-test split: Is there a clear split? Is test data truly unseen? 3. Memorization risk: Could the model have seen test examples during pretraining? 4. Benchmark staleness: How widely known is the benchmark? Has it been used to optimize model performance? 5. Problem difficulty: Is the benchmark challenging enough to differentiate model capabilities? • Self-reported

Val, 0-shot CoT, accuracy • Self-reported

Accuracy testing AI: in training measure that, how well exactly model makes on data, which she/it not in time training. Usually is calculated how percentage correct from general numbers on test set data • Self-reported

Accuracy AI: 1 Human: 1 AI21 • Self-reported