Pixtral Large

BBox Method BBox based on process training with for intermediate computations model. He includes three : 1. language which helps model its intermediate computation. For tasks example language can look how «14 × 15. First 10 × 15 = 150. Then 4 × 15 = 60. we receive 150 + 60 = 210, therefore 14 × 15 = 210». 2. With help RLHF (training with on basis connection from human) model work. For this demonstration or or more model, and use their for settings model with help training with 3. answer from model. answer, model also provides explanation about that, how model to answer • Self-reported

Chain reasoning AI: I I will solve this task by in order to to correct answer. First I tasks and that me need to find. Then I task on more components. For each I corresponding mathematical or logical methods. I I will all intermediate computation and its work on each step. I results, in order to obtain final answer on task • Self-reported

ANLS evaluation (ANLS) - this metric evaluation, for evaluation quality extraction information with images or text. ANLS measures between answer model and reference answer, considering possible in For each question and answer ANLS (NLS) between answer and from answers. NLS is determined how general between and reference answers to more from two If NLS below specific (usually 0.5), evaluation that helps which too from answers. metric ANLS is calculated how average value all NLS evaluations by all questions in set data. ANLS especially useful for tasks, where differences in for example, in tasks answers on questions by or in text • Self-reported

Chain thinking AI: you in complex computations I generate example question MMLU. Please, its, using approach chains thinking. First step for step, and then answer • Self-reported

CoT • Self-reported

GPT-4o Judge We we present GPT-4o Judge for evaluation answers on mathematical tasks, which solves problems evaluations and methods. works following manner: 1. Question, answer and solution on GPT-4o with evaluate solution. 2. For with and errors we we use three key : a. First model explicitly verify each step solutions. b. possible errors, which human would find in solving. c. solution for comparison. 3. Model evaluation by scale from 0 to 5, where each score has : • 5: and solution • 4: In whole correct solution with minor • 3: Correct approach with errors • 2: to solving • 1: progress • 0: approach or solutions GPT-4o Judge demonstrates high with experts, achieving 83% evaluationin our tests. This makes its tool for evaluation mathematical solutions, especially for tasks level complexity • Self-reported

VQA Match evaluation GPQA, VQA Match, is used for evaluation generated model answers by means of their comparison with reference answers. This process includes three key step: 1. : Answers steps in order to on results comparison, including answers to format. 2. in : answers then in with help model CLIP ViT-L/14, which uses training for creation text. 3. Comparison : score is calculated with help between answer model and answer. (to 1.0) indicates on more exact match between answer and reference. VQA Match evaluates and match, and not simply match, that ensures more evaluation quality answers model. analysis shows, that this method well with evaluationhuman on tasks GPQA • Self-reported