Kimi K2 Instruct

Standard evaluation • Self-reported

Pass@1 Pass@1 Pass@k, k Pass@1 Pass@1 Pass@1 Pass@1 • Self-reported

Accuracy AI: I'll compute the accuracy of the model's responses by comparing them to the ground truth answers. For multiple-choice questions, I'll check if the model selected the correct option (A, B, C, or D). For open-ended questions that require numerical answers, I'll check if the model's final answer matches the correct value. I'll be lenient with formatting differences (e.g., "5" vs "5.0" vs "five"). For open-ended questions that require textual answers, I'll assess whether the model's response contains the key elements of the correct answer. I'll report the overall accuracy as the percentage of questions answered correctly, and break down performance by question type and difficulty level. • Self-reported

Diamond Avg@8 Diamond Avg@8 — 8 Diamond Avg@8 : 1. 2. 3. 8 4. 8 Diamond Avg@8 • Self-reported

Accuracy AI: 0 • Self-reported

Accuracy AI: 0.0/1.0 • Self-reported

64 • Self-reported

Avg@64 • Self-reported

Accuracy AI • Self-reported

Accuracy AI: "Accuracy" refers to how often a model makes correct predictions or provides correct answers. For simple tasks like "Is this image a cat or a dog?", accuracy is straightforward - the percentage of correct classifications. For complex tasks like answering multi-step math problems or open-ended questions, accuracy becomes more nuanced: 1. Partial correctness may apply (getting part of a multi-step solution right) 2. Multiple valid answers may exist 3. Context and interpretation matter When evaluating large language models, accuracy can be measured through: - Benchmark performance (scores on standardized tests) - Human evaluation (experts judging correctness) - Comparison to reference answers - Self-consistency (agreement across multiple attempts) Improving accuracy typically involves: - More/better training data - Enhanced model architectures - Better fine-tuning techniques - Improved prompting methods High accuracy is critical for high-stakes applications but must be balanced with other considerations like speed, transparency, and resource efficiency. • Self-reported

16 • Self-reported

Correct • Self-reported

Avg@32 AI: * • Self-reported

Pass@1 Pass@1 n=k (20 ), evaluation Pass@1, Pass@k k • Self-reported

Accuracy () • Self-reported

"", LLM : - "X" - "" - "" • Self-reported

Pass@1 — Pass@1 Pass@1 (n=100) • Self-reported

Pass@1 Pass@1 Pass@1 : 1. 2. 3. () 4. : Pass@1 = () / () Pass@1 Pass@k (), Pass@1 • Self-reported

Accuracy AI Accuracy : accuracy - GPT-4o, accuracy, accuracy accuracy GPT-4o • Self-reported

EM • Self-reported

EM • Self-reported

Accuracy AI: [model] is a powerful artificial intelligence language model developed by OpenAI. In this test, we assess its accuracy in answering questions correctly. Accuracy refers to the model's ability to provide factually correct responses without making errors or generating false information. To evaluate accuracy, we present the model with questions that have verifiable answers across different domains including science, history, mathematics, and general knowledge. We then compare the model's answers against established facts from reliable sources. Factors affecting accuracy include: 1. Knowledge cutoff limitations 2. Training data quality and comprehensiveness 3. Inherent limitations in pattern recognition 4. Prompt specificity and clarity The accuracy assessment gives us insight into how reliable [model] is as an information source and helps identify areas where additional training or improvements may be needed. • Self-reported

# Pass@1 Pass@1 - : 1. 2. () 3. Pass@1 - Pass@1 • Self-reported

Pass@1 AI: Pass@1 Pass@1, 1000 (GSM8K) • Self-reported

Pass@1 Pass@1 Pass@1 Pass@1 Pass@k (k > 1) • Self-reported

4 • Self-reported

Standard evaluation • Self-reported

Accuracy AI: • Self-reported

Standard evaluation • Self-reported

Standard evaluation • Self-reported

thinking, : - : : : - : : • Self-reported

temperature top_p, seed. : 1. : 2. Score : 3. : 4. evaluation: • Self-reported

Avg@4 Avg@4 4 4 • Self-reported

4 • Self-reported

Avg@4 4 4 • Self-reported

AI: evaluation LLM • • — • evaluation — evaluation • • • : • • • • accuracy • Self-reported

Accuracy AI: The ability to correctly predict outputs compared to ground truth. • Self-reported

Accuracy ChatGPT AI: I'm going to solve this step-by-step. To find the smallest positive integer k such that a^k ≡ 1 (mod n), I need to determine the order of a modulo n. Given: - n = 15 - a = 4 First, I'll check if a is relatively prime to n by computing gcd(a,n) = gcd(4,15). 15 = 4*3 + 3 4 = 3*1 + 1 3 = 1*3 + 0 So gcd(4,15) = 1, which means a and n are relatively prime. Now I'll compute powers of a modulo n: 4^1 ≡ 4 (mod 15) 4^2 ≡ 16 ≡ 1 (mod 15) So 4^2 ≡ 1 (mod 15), which means the smallest positive integer k such that a^k ≡ 1 (mod n) is k = 2. Therefore, k = 2 is the answer. • Self-reported