Mistral Small 3.1 24B Base

# A/B-testing for improvements LLM with help thinking ## When large language model (LLM) work with tasks, reasoning, that model show results, when their "think step for step". This to methods prompts, model to more solving tasks. However, how and other prompts, this approach has two key limitations: 1. He is applied to all tasks, although model can more high accuracy at different to tasks. 2. He not always with points view efficiency, since intermediate computation text. In given work we we offer approach, which allows model between various "thinking" in dependency from tasks. We that between: - answer - results ...substantially improves general performance, especially in tasks, requiring mathematical or complex logical conclusions. ## Methodology ### thinking We we determine and we compare main mode thinking, which can models: 1. **mode**: Model answers directly, without intermediate steps reasoning. 2. **mode**: Model complex task on sequence intermediate steps before provision final answer. 3. **Mode **: Model offers answer, then verifies its correctness, errors. 4. **mode**: Model first determines mode for given tasks, and then applies its • Self-reported

accuracy • Self-reported

Accuracy CoT We analysis efficiency model at use method Chain-of-Thought (CoT, chain reasoning) for solutions tasks, requiring reasoning. CoT offers model not simply generate final answer, but perform step-by-step reasoning before answer. Evaluation following manner: 1. model questions, requiring reasoning, and we ask show intermediate steps 2. not only final answer, but and and correctness reasoning 3. accuracy answers, with using CoT, and answers, direct analysis is measurement: - model break down complex tasks on more simple steps - in reasoning model - improvements accuracy reasoning - errors, which can in process reasoning • Self-reported

0-shot CoT Method thinking by chain without examples (0-shot Chain-of-Thought, 0-shot CoT) encourages model solve tasks step by step, without provision specific examples such process reasoning. This approach uses general prompts, such how "Let's let's solve this task step for step" or "Let's ", in order to stimulate model to process reasoning before final answer. Method 0-shot CoT especially useful, when no capabilities or provide samples reasoning for specific tasks. He usually outperforms simple prompts without reasoning, so how allows model break down complex task on more managed subtasks, that probability errors at solving multi-step tasks. not less, 0-shot CoT usually other CoT, in which are used examples, since model not receives specific about that, how manner structure its thinking for specific tasks. for examples quality reasoning depends from abilities model independently way analysis tasks • Self-reported

5-shot • Self-reported