Defense · AI · Evaluation

Scott
Mastin

Defense AI Practitioner

I evaluate AI systems for the Department of Defense — designing rigorous test and evaluation frameworks that determine whether AI is ready for the mission.

AI Eval Corp Read My Writing
Scott Mastin — Defense AI Practitioner, close-up portrait
Background

Seventeen years at the
frontier of AI
and national security

I spent my career at the intersection of artificial intelligence and national security. From the Army Research Laboratory to Project Maven to the Chief Digital and Artificial Intelligence Office (CDAO), I've been in the room where decisions are made about whether AI systems are reliable enough to deploy.

Today I run AI Eval Corporation, a boutique defense consultancy focused exclusively on AI test and evaluation. We build vendor-agnostic evaluation frameworks, design operational test plans, and ensure the Department of Defense has honest, defensible assessments of the AI tools it fields.

I'm a Service-Disabled Veteran-Owned Small Business (SDVOSB) certified and Tradewinds approved — working with organizations like JHU/APL and CDAO on the most consequential AI evaluation challenges in government.

01

Army Research Lab

AI research and development for Army modernization programs — laying the technical groundwork for DoD's AI enterprise.

02

Project Maven

DoD's flagship AI initiative for combat operations. Supporting evaluation of AI capabilities at the mission edge.

03

CDAO T&E

Test and evaluation services for the Chief Digital and AI Office — the principal advisor to the Secretary of Defense on AI.

04

SDVOSB Certified

Service-Disabled Veteran-Owned Small Business. Tradewinds approved. Supporting CDAO through JHU/APL.

Writing & Insights

Long-form analysis on
the hardest problems
in defense AI

I write on Substack about AI test and evaluation, defense policy, and what evaluation frameworks actually need to measure — based on direct experience building them.

Substack

Long-form analysis on AI T&E, defense policy, and what evaluation frameworks actually need to measure.

Read on Substack

LinkedIn

Short takes on real problems in AI T&E — practical observations from ongoing work in the field.

Connect on LinkedIn
  • AI Evaluation Frameworks
  • Defense AI Policy
  • Operational Testing
  • Model Assessment
  • Benchmark Design
  • Vendor Accountability
AI Eval Corporation

Independent
AI evaluation
for the DoD

AI Eval Corporation provides independent, rigorous AI evaluation services to the Department of Defense and intelligence community. We specialize in building evaluation infrastructure that works in operational environments — including air-gapped networks like SIPR and JWICS.

SDVOSB Tradewinds Approved Founded 2023 Washington D.C.
Visit aieval.org

  • Evaluation Frameworks

    Vendor-agnostic harnesses for model, system, and operational-level AI evaluation. Built to work in classified and air-gapped environments.

  • Operational T&E Plans

    Test plans aligned with the Alternative T&E Pathway and Software Acquisition Pathway — grounded in how AI actually performs in the field.

  • Accreditation Support

    Evaluation artifacts for ATO packages, requirements traceability, and compliance documentation. Making evaluation defensible to the people who approve deployment.

Contact

Let's talk
about
AI evaluation

Whether you're fielding AI systems in a defense environment or navigating the evaluation landscape, I'm happy to connect.

Email

leon@aieval.org

LinkedIn

linkedin.com/in/scottmastin

Location

Washington, D.C. Metro Area