Clear calls on buys and launches.
An independent read on what a system actually does on your work, what it will cost to run, and whether to buy, ship, wait, or walk.
AI Eval Corp SDVOSB Tradewinds
Senior AI counsel for the leaders who own the decision.
I advise commercial leaders putting AI into business workflows, and defense executives with the same decisions: what to buy, what to ship, how to govern it, and how to know it works.
30 minutes with me, not a sales team. No preparation needed.
Prefer email? scott@aieval.org
Track record
Project MavenAI T&E program manager, AWCFT, 2019 to 2023
Maven under CDAOMaven T&E lead, 2023 to present
CDAO Rapid Test and Evaluation CellJohns Hopkins APL subcontract
Army Research LaboratorySensor and imagery AI testing, 2007 to 2019
Advisory
Better AI decisions, made faster and defended with evidence.
A risk picture leadership can act on.
Failure modes, ownership, and accountability laid out plainly for you, your board, or your chain of command.
Evidence standards that hold up.
What to measure, what claims to lock in with vendors and teams, and how much evidence is enough to decide.
A steady independent voice.
Someone beside you and your team through vendor pitches, program reviews, and internal debates, with no stake in which product you pick.
Working together
What month-to-month advisory looks like.
You get direct access to me and a steady rhythm of working sessions, built around the decisions on your calendar.
A typical month can include
- Regular working sessions with you, or with you and your key people
- Direct email and phone access between sessions for time-sensitive questions
- Review of the documents in front of you: vendor proposals, evaluation results, program briefs, contracts, or policy drafts
- A seat in the meeting that matters, such as a vendor review, board prep, or program review, when you need it
- Short written notes on decisions made, open risks, and what to watch next
Decisions I help with
- Buy, build, or wait on an AI system
- Whether a system is ready to ship or field
- What evidence to require from vendors and internal teams
- How to set up testing and evaluation so claims hold up
- Governance: who owns the model, the data, and the failure
- How to brief a board, leadership, or oversight body on AI risk
- Acquisition readiness for defense programs
Built for leaders who carry the AI decision.
- Executives at companies making their first major AI commitments, or cleaning up ones already made
- CTOs, CIOs, and chief data and AI officers who want an independent voice beside their team and vendors
- Defense and national security leaders, program executives, and program managers heading toward a buy, fielding, or evaluation decision
How it works
Three steps to your first working session.
01
Intro call.
30 minutes. You tell me the decision in front of you, the system or program, and what is driving the timing.
02
Set the scope.
I send a one-page scope with cadence, access, and a monthly fee.
03
Get to work.
We schedule the first working session as soon as the scope is signed. Contracting runs through AI Eval Corporation.
Experience
Nearly two decades of testing AI where the stakes are high.
Scott Mastin
President, AI Eval Corporation
I have spent my career answering one question for senior decision makers: does this AI system work well enough to trust with the mission? The same question applies in a boardroom.
- Project Maven. AI test and evaluation program manager for the Algorithmic Warfare Cross-Functional Team (AWCFT), across all lines of effort, 2019 to 2023.
- Maven under CDAO. Maven test and evaluation lead under the Chief Digital and Artificial Intelligence Office (CDAO), 2023 to present, following Maven's move to CDAO and NGA.
- CDAO Rapid Test and Evaluation Cell. Test and evaluation work under a Johns Hopkins APL subcontract.
- Army Research Laboratory. AI testing for sensor and imagery systems, 2007 to 2019.
You get advice shaped by years of telling senior leaders plainly whether a system is ready, and what it will take if it is not.
- AI Eval Corp
- Maven / CDAO
- ARL
- SDVOSB
- Tradewinds
Engagements are contracted through AI Eval Corporation, an SDVOSB that builds AI tools with test and evaluation built in. Learn more at aieval.org.
Common questions
What happens on the intro call?
You tell me the decision in front of you and what is driving the timing. I tell you plainly whether I can help and what ongoing advisory would cover. If it is a fit, I send a one-page scope afterward. If it is not, you still leave with a clearer next step.
What does ongoing advisory cost?
Pricing depends on scope and cadence, so I set it per engagement after the intro call. It is a flat monthly fee, and you can adjust month to month.
What happens in a typical month?
It depends on what you need. A typical month can include regular working sessions, direct access between them, review of the documents behind your decisions, and short written notes on decisions and open risks. See "What month-to-month advisory looks like" above.
How is this different from hiring a consulting firm?
You work with me directly from the first call. Advice starts in the first session, and the engagement flexes month to month with your decisions.
Is your advice independent?
Yes. I do not resell third-party platforms or take referral fees from vendors, so my recommendation is not tied to what you buy.
Do you work with commercial companies or only defense?
Both. The core questions are the same in a company and a program office: does it work on your task, what breaks, who owns it, and what evidence is enough to decide.
Who do I contract with?
AI Eval Corporation, the company I lead. It handles contracting and invoicing for all engagements. aieval.org
Start with a short conversation.
Tell me the decision in front of you and what is driving it. In 30 minutes we will know whether ongoing advisory is the right fit and what it should cover.
Book a 30-minute intro