The problem
SR 11-7 is clear about one thing above all, the party that validates a model cannot be the party that built it. That is the whole point of independent validation, and it is exactly the part that does not scale. Your model inventory grows every quarter (every new scorecard, every ML re-fit, every vendor model you inherit), and your second-line validation team does not grow with it. So the backlog builds, the revalidations slip, and eventually an examiner writes it up.
The tools that promise to help usually do one narrow thing (backtesting alone, or drift alone), or they are a person-heavy audit that costs six figures per model, or they are the new flavor of the month, an AI grading your AI with a confidence score. None of those are what SR 11-7 asks for. SR 11-7 asks for an independent party to re-derive the answer and show its work.
How it works
AYA Model-Risk Validation is one deterministic workflow with four legs, and every number it produces leaves an auditable receipt.
- Bench. It scores the model quantitatively, discrimination (AUC and Gini), backtesting (observed exceedances against your confidence level, placed in a Basel-style green, amber, or red zone), population drift (a stability index between the base and current distributions), and champion-versus-challenger (the delta and the winner). These are closed-form statistics computed over the arrays you supply. No model of ours is grading your model.
- Coverage. It checks the validation against a declared SR 11-7 expected-set (conceptual soundness, ongoing monitoring, outcomes analysis, discrimination, stability, benchmarking, data quality, governance, documentation) and it is fail-closed, the review is complete only when zero required elements are missing. Ninety-nine percent with one required gap stays incomplete, never rounded up.
- Verify. It re-walks the governance and applicability determinations deterministically, and grades them, either it agrees, or it names the business failure (a wrong determination, an invented citation, a gap guessed through) and hands back the correct answer.
- Receipts. Every leg emits an auditable receipt, the bench numbers, the coverage verdict with a ranked worklist of what is missing, and the disagreement report with the correct answer attached. A scorer error is never scored as a pass. The whole thing fails closed.
What you get
A validation package you can put in front of an examiner. Bench receipts with the traffic-light zones, a fail-closed coverage verdict that tells you exactly which required SR 11-7 elements are covered and which are not, and a disagreement report that re-derives each governance determination and shows where (if anywhere) the model's paperwork was wrong.
Who it is for
Model Risk Management and validation functions at banks, credit unions, and fintech lenders under SR 11-7 or OCC 2011-12, second-line risk teams watching the model inventory outrun their headcount, and RegTech or insurance platforms whose own customers have to show independent validation. It is not for low-stakes, non-regulated ML where a wrong answer costs nothing, and it is not a chatbot.
Pricing
The prices below are hypotheses we are validating with early partners, not a fixed rate card. We price the ladder, not a number.
- Pilot, a fixed-price independent validation of one model, you see the whole package (bench receipts, coverage verdict, disagreement report) before you commit to anything.
- Per model, annual, independent validation and revalidation for each model in the inventory, once the pilot has proven the assurance is real.
- Platform, the deterministic workflow licensed into your MRM stack, continuous monitoring, drift alerts, and coverage completeness across the whole inventory. The IP is licensed, never assigned.
Any step that moves money is fail-closed and pauses for your explicit confirmation.
Proof (and how we dogfood it)
We run this product's own discipline on ourselves. Every capability AYA ships terminates as a verification receipt, the same shape of proof this product sells, and the bench kernel is producer-agnostic, so it scores AYA's own model outputs (including our eval-plane benchmark numbers) exactly the way it would score a customer's. We eat the validator we sell.
Concretely, for this build, the deliverable workflow was quality-scored and its composed pieces were confirmed to resolve on disk, and the new bench kernel was unit-checked, it produced the correct AUC, the correct backtesting zone, the correct stability index, and the correct champion-challenger call, and it refused (returned a failure, not a fabricated pass) when handed malformed input.
Honest note
This is a structural build, not a live deployment. The bench kernel computes correctly in isolation and fails closed on bad input, but it has not yet been run through a booted mesh, and nothing here has been run against a real customer model, a live model inventory, or a real batch of determinations. There is no live spend and no examiner-facing package produced yet. The first real test is a runtime witness of the new bench path plus read-only access to an actual model, and until that happens we will not show a bench number we cannot reproduce on demand. The bench leg is the one net-new piece of code in this product, and it is marked for review.
*This page is a specification. The capability it describes is not built yet, and nothing here is a claim that it runs today.*