Claude 3.5 Haiku

Verification without examples • Self-reported

standard • Self-reported

0-shot CoT Zero-shot Chain-of-Thought (CoT) — this method improvements reasoning LLM through instructions "step for step", without necessity demonstration examples. This approach, in work Kojima et al. (2022), offers simple way stimulate step-by-step reasoning in models, that often leads to performance in tasks, requiring complex reasoning. When use 0-shot CoT instruction model on intermediate steps reasoning, before than to answer. that such approach especially efficient for large language models. 0-shot CoT : sufficiently phrase "let's let's think step for step" (or ) to query. This encourages model its thoughts sequentially, that helps in solving complex tasks, including and logical tasks. In difference from few-shot CoT, which requires provision examples with reasoning, 0-shot CoT not requires examples and can without additional training model • Self-reported

0-shot CoT Method "0-shot CoT" (thinking with example) — this approach to solving tasks models artificial intelligence, at which model provides step-by-step reasoning without preliminary examples solutions similar tasks. In difference from "few-shot CoT", where model show several examples with solutions, in 0-shot approach model should independently strategy reasoning. Usually this simple prompts "Let's let's solve this step for step" (or "Let's think step by step" in ) after assignments. This method especially efficient for modern LLM, so how their ability to without necessity in additional examples. Research showed, that such simple significantly improves performance models at solving complex tasks, requiring logical thinking, computations or multi-step reasoning. advantages 0-shot CoT: • Not requires examples • in context query • Allows model apply own reasoning • on tasks This approach at work with models GPT, Claude and other LLM for improvements quality their answers at solving complex tasks • Self-reported

3-shot F1-measure F1-measure usually is used for measurement accuracy binary classification. She/It is calculated how harmonic average accuracy (proportion correct positive predictions among all positive predictions) and completeness (proportion correct positive predictions among all actual positive instances). In context LLM we this metric for tasks with answer, where system should generate actual information. We F1-between that, that should was be (), and that, that actually was (). Metric "3-shot F1-measure" means, that model show 3 example question/answer before in order to she/it could format execution tasks • Self-reported

0-shot CoT Chain-of-thought — this method prompting language models to "aloud" at solving complex tasks, which was first presented in work Wei et al. (2022). In standard approach 0-shot CoT model explicitly indicates solve task step by step, often with using phrases "Let's let's solve this step for step". By comparison with direct query, 0-shot CoT gives models for execution intermediate computations and reasoning before provision final answer. This especially useful for mathematical tasks, tasks common (sense) meaning and logical puzzles, where problems on steps can probability errors. Examples 0-shot CoT: • "Let's let's solve this step for step." • "this problem step by step." • "For solutions this tasks I I will think sequentially." 0-shot CoT requires on since not requires provision examples, and often significantly improves performance by comparison with for majority models at solving tasks, requiring reasoning • Self-reported

# 0-shot CoT Method 0-shot CoT ("chain thinking" without examples) — this way encourage model reason step by step, not showing it examples that, how should look such reasoning. He was presented in Kojima et al. (2022) and based on more research Chain-of-Thought (Wei et al., 2022). ## Method In order to use 0-shot CoT, simply phrases, to such how "Let us reason step for step" in end query. This encourages model execute step-by-step reasoning, before than give final answer. ## How this works Approach 0-shot CoT allows models: 1. complex tasks on more simple subtasks 2. intermediate results 3. and correct errors in reasoning 4. solution ## Example use For tasks: "If at was 5 he 2, and then still 3, how many at him ?" **Query with 0-shot CoT**: "If at was 5 he 2, and then still 3, how many at him ? Let us reason step for step." **Answer model**: "with that, that at 5 2 apples, therefore at him 5 - 2 = 3 apples. Then still 3 apples, therefore at him 3 + 3 = 6 Answer: at 6 " ## When this useful 0-shot CoT especially useful for: - tasks - puzzles - on reasoning - problems ## Limitations - Not always generates chain reasoning - Quality reasoning depends from basic abilities model - Can generate but incorrect steps reasoning • Self-reported

Standard AI: I that exists simple solution: 6 in time means, that when in 12:00, in 6:00. if in 9:00 in this will 15:00 in in 10:30 in that matches 16:30 in • Self-reported

Standard • Self-reported