Stop Guessing, Start Measuring: Evaluations with Pydantic AI

Event

MLcon San Diego 2026

Date

2–3 June 2026

Location

San Diego, California

Venue

Hyatt Regency La Jolla at Aventine

I spoke at MLcon San Diego about evaluating AI agents with Pydantic AI.

The session focused on a practical question: how do you know whether an AI agent is working well? Instead of relying on a few convincing outputs, evaluations provide a repeatable way to assess behaviour and measure whether changes improve the system.

Key takeaways

• Define measurable criteria for successful agent behaviour.
• Use evaluations to assess AI agents with Pydantic AI.
• Compare results across changes instead of relying on intuition.