Mistral Large 2

Accuracy In context evaluation model LLM accuracy relates to to abilities model correctly answer on questions or solve tasks. Accuracy can measure, proportion correct answers from general numbers questions. In capacity accuracy often is used evaluation, verification or comparison with reference answers. Accuracy is important evaluation, since gives representation about that, how well model at answer on questions in specific domain field or in various contexts. She/It helps identify strong side and limitations model, and also determine field, requiring further • Self-reported

Pass@1 Method evaluation Pass@1 measures percentage tasks, which model solves successfully with first attempts. This direct score quality work model at without capabilities corrections or attempts. metric especially important for scenarios, where users and exact answer, or in cases, when attempts from-for limitations by time, or systems. When Pass@1: • Model receives task one times • one solution • Solution is evaluated how correct or • Final score — percentage correct answers from all tasks High score Pass@1 indicates on reliability model and her/its ability find solutions without additional iterations. This metric often is used in analysis various models for determination their base performance • Self-reported

Accuracy AI-generated answers can sound plausible but still be incorrect. To measure how well LLMs can produce factually accurate answers, we can present them with test questions with known answers, and assess the percentage of their responses that are correct. Benchmarks: - General Knowledge: MMLU, a comprehensive test covering 57 subjects from STEM to humanities. - Advanced Knowledge: GPQA (Graduate-level Professional Questions & Answers), which tests expert-level knowledge. - Mathematical Reasoning: GSM8K for grade school math problems, and MATH for competition-level problems. This metric helps evaluate if models can reliably generate factual information rather than just producing coherent-sounding text. High accuracy suggests an LLM can be trusted to provide correct information within its training domain. • Self-reported

Accuracy AI: The model gave the correct answer. Human: Model correct answer • Self-reported

Score Evaluation • Self-reported