About

Meridian Labs is a 501(c)(3) non-profit building open source tools for testing, evaluating, and researching frontier AI models. Our goal is to support work on AI safety, alignment and security by creating a common platform shared by governments, non-profits, academia, and model developers. Our tools enable rigorous model testing and in-depth analysis of agent behavior and alignment.

The core of our open-source offerings is Inspect AI, an evaluation framework developed at the UK AISI in collaboration with Meridian’s founding team. Inspect AI is now the standard evaluation framework for many government organizations including UK AISI, US CAISI, EU AI Office, Japan AISI, Korea AISI, and others. It is also the standard framework for a wide variety of NGOs including METR, Apollo, Epoch, HAL, SecureBio, Redwood, and RAND.

Our Work

Meridian works on a wide variety of open source projects, including:

  1. Inspect AI frontier AI evaluation framework which provides researchers, developers, and policy makers with the tools to conduct rigorous, repeatable assessments of AI capabilities and behaviors.

  2. Inspect Scout, a framework for analysis and monitoring of AI agents (both in the lab and in production). Scout is in wide use at UK AISI, US CAISI, METR, and Apollo, and expect it to be an important technical foundation for AI alignment and monitoring.

  3. Insect Petri, an automated alignment auditing tool that orchestrates multi-turn interactions between auditor and target models (note this was originally developed at Anthropic and is being transitioned to Meridian).

  4. Inspect Flow, a configuration and workflow management tool for AI evaluations that enables systematic experimentation and running large scale evaluation sets (e.g. for auditing / pre-deployment testing). Currently in use at UK AISI and US CAISI.

Many of our current users also do work in the adjacent fields of alignment, control, and mechanistic interpretability. Inspect AI is often used as part of these workflows (for example, ControlArena from UK AISI, LinuxArena from Redwood, and the TransformerLens and nnterp Inspect model providers for interpretability).

Goals and Plan

Our goal is to create open source software that makes leading edge AI testing and evaluation tools broadly available. Future plans include:

  • Evolving our current projects in response to user needs and the evolving ecosystem, maintaining a strong and up to date foundation for evaluating and understanding models.

  • Providing an avenue for AI Safety Institutes, research organizations, and safety teams at leading labs to transfer selected pieces of their work into more broadly available open source projects.

Projects in this vein are likely to include automated alignment auditing, control frameworks, interpretability tools, and means of facilitating privacy-preserving analysis of language model interactions.

We also believe that automated workflows for evaluations, monitoring, and alignment research will become increasingly important, and our tools will be designed and built for use by humans and models alike.