Granite 3.3 8B Base

Score : Measurement Approach Measurements need to assess the behavior of AI systems to provide confidence to a variety of stakeholders that the system meets societal expectations for its behavior. These behaviors and expectations include: - The system's adherence to established procedures to achieve intended goals - The system's potential for manipulation of users - The system's potential for harmful outputs - The system's competence in specific domains One method for measurement is self-evaluation, which evaluates a model on relevant benchmarks or metrics, or evaluates a model's adherence to responsible use strategies. Self-evaluation can include the use of synthetic evaluators, and both human and automatic evaluation. Frontier developers and governments should create measurement approaches that are strong enough to catch novel, unanticipated risks from frontier AI. However, it is difficult to measure risk—risks may be highly context-dependent or imprecisely defined. Quantitative measures should be complemented with regular qualitative assessments by independent third-party evaluators. Self-evaluation should also involve multiple methodologies and consider difficult-to-detect behaviors, such as deception. Frontier developers could publish results of self-evaluation across the development cycle to understand how risks emerge as systems improve, and to allow the broader community to give feedback on their risk assessment approaches • Self-reported

## Score 0 5: - 0: 1: 2: 3: 4: 5: : evaluation 5 5 • Self-reported

Score • Self-reported

Score AI: : 1. 2. () 3. 4. 5. evaluation • Self-reported

# OLMES OLMES (Open-Language Model External Stimulus) - : 1. ****: 2. ****: 3. ****: OLMES : - OLMES : - OLMES • Self-reported

(0 1 ) (0 10) • Self-reported

OLMES () • Self-reported

Score : 1. 2. 2. : - : : 3. : - : : 4. 0 5: 5: 4: Correct 3: Correct 2: 1: 0: 5 • Self-reported

# Score (explanation-first), (answer-first), (validity), (factual correctness) • Self-reported

Not specified • Self-reported

## Score "", "" "". evaluation0 3 • Self-reported

Score Score • Self-reported

Arena Hard Claude RLHF Arena: Hard. Arena Hard RLHF (). Claude: RLHF Arena: Hard — : - : Claude : : : Anthropic Claude, Anthropic • Self-reported

Not specified (OLMES) • Self-reported

# OLMES OLMES () — ## OLMES ## OLMES : 1. 2. 3. ## OLMES • Self-reported

OLMES AI: OLMES (Online Large Model Evaluation System) - nolmo.ai, LLM, OLMES : • API. • • OLMES : • • • • • Self-reported

Not specified • Self-reported

Score • Self-reported

Score • Self-reported

Score • Self-reported