Validation Center
Phase 7Recommendations onlyObservation is complete. Every AI recommendation is now compared against approved coordinator behavior before TC Pros earns additional autonomy. Validation records are append-only, and no workflow gains autonomy without human approval.
Shadow accuracy
79%
AI prediction matched the coordinator's eventual action
n=412 · coverage 68% · conf 0.86
Expert agreement
72%
Reviewer confirmed recommendation as correct or near-correct
n=288 · coverage 61% · conf 0.81
Prediction accuracy
74%
Predicted next action within the observed window
n=355 · coverage 63% · conf 0.80
Playbook validation
55%
Playbooks meeting stable-evidence thresholds
n=10 · coverage 60% · conf 0.64
Exception validation
48%
Exception scenarios with approved, reusable resolutions
n=27 · coverage 52% · conf 0.59
Knowledge approval
41%
Candidates approved out of those submitted for review
n=22 · coverage 58% · conf 0.66
Readiness score
62%
Weighted composite across coverage, accuracy, agreement, stability
coverage 61% · conf 0.71
Autonomy readiness
24%
Share of workflow steps rated AUTONOMY_CANDIDATE
coverage 61% · conf 0.70
Expert validation · append-only
| id | step | ai recommendation | expert action | result | conf | reviewer |
|---|---|---|---|---|---|---|
| VAL-7101TX-2041 | Title commitment review | Flag Schedule B-II easement, request survey exception deletion | Requested survey exception deletion + T-19 endorsementAI omitted endorsement typically requested by this title officeevidence · commitment_v2.pdf p.4 · email 08:39 | Partially Matched | 0.77 | M. Alvarez |
| VAL-7102TX-2039 | Payoff request | Request payoff good-through close + 4 days | Requested payoff good-through close + 4 daysIdentical timing and bufferevidence · payoff_request.eml · EXC-51 pattern | Matched | 0.93 | D. Okafor |
| VAL-7103TX-2033 | Closing scheduling | Schedule signing 2 days before funding deadline | Scheduled signing 4 days before, mobile notaryOut-of-state seller required mobile notary lead time — context not in AI inputsevidence · seller_profile · calendar event | Different | 0.71 | R. Chen |
| VAL-7104TX-2028 | Earnest money verification | Accept wire confirmation screenshot as proof | Required escrow-of-record receiptScreenshot is not an acceptable evidence class under wire-fraud policyevidence · CONSTITUTION §Security · escrow receipt | Rejected | 0.88 | S. Patel |
| VAL-7105TX-2012 | HOA estoppel ordering | Order estoppel at day 5 | Ordered at day 5Timing and vendor identicalevidence · estoppel_order.pdf | Matched | 0.90 | M. Alvarez |
| VAL-7106TX-1994 | Novation lender notice | No action recorded | Sent lender notice with recorded assignmentAI had no observation coverage for novation lender noticesevidence · coverage gap: novation | Unknown | 0.34 | D. Okafor |
Agreement mix
288
validated recommendations
Historical validation is never overwritten. Re-review appends a new record with its own reviewer, reason and evidence.
Playbook validation
| playbook | obs | approvals | exc/file | pred acc | completion | revisions | conf | recommendation |
|---|---|---|---|---|---|---|---|---|
| Traditional purchase | 148 | 121 | 0.4 | 86% | 94% | 9 | 0.89 | Stable |
| Wholesale assignment | 96 | 78 | 0.6 | 81% | 90% | 12 | 0.83 | Stable |
| Double close | 61 | 39 | 1.3 | 68% | 82% | 21 | 0.70 | Needs Review |
| Subject-to | 54 | 31 | 1.6 | 63% | 79% | 26 | 0.66 | Needs Review |
| Seller financing | 44 | 33 | 0.9 | 74% | 88% | 11 | 0.72 | Needs Review |
| Novation | 19 | 11 | 1.8 | 58% | 74% | 14 | 0.44 | Insufficient Evidence |
| Lease option | 7 | 3 | 1.4 | 50% | 71% | 5 | 0.27 | Insufficient Evidence |
| Land purchase | 6 | 4 | 0.8 | 55% | 83% | 3 | 0.25 | Insufficient Evidence |
| Commercial purchase | 3 | 1 | 2.0 | 40% | 67% | 2 | 0.18 | Insufficient Evidence |
| Hybrid deal | 2 | 0 | 2.5 | 35% | 50% | 2 | 0.12 | Insufficient Evidence |
Decision validation
Observed expert behavior is evidence, not ground truth. Alternative approved paths are retained; a majority pattern never invalidates a documented minority path.
When to order title
conf 0.87- predicted timing
- Day 1 after acceptance
- observed timing
- Day 1 (median), Day 2 (p75)
- predicted action
- Open escrow + order commitment together
- observed action
- Open escrow, order commitment same call
outcome · Commitment 1.8 days faster than cohort
evidence · 48 files · escrow open timestamps
Alternative approved paths: Order commitment after EM clears (2 coordinators, similar outcome)
Curative escalation point
conf 0.69- predicted timing
- Day 4 of open defect
- observed timing
- Day 6 (median)
- predicted action
- Escalate to title manager
- observed action
- Second underwriter email, then escalate
outcome · Escalation at day 4 shortened cure by 1.2 days where used
evidence · 31 curative episodes
Alternative approved paths: Escalate immediately for lien defects (small sample, n=6)
Signing scheduling buffer
conf 0.78- predicted timing
- Close − 2 days
- observed timing
- Close − 3 to − 4 days
- predicted action
- Book signing room
- observed action
- Book signing room or mobile notary by party location
outcome · Larger buffer reduced reschedules by 34%
evidence · 62 signings
Alternative approved paths: Same-day signing for local cash deals (n=11, no failures)
Payoff refresh
conf 0.93- predicted timing
- Close − 5 days
- observed timing
- Close − 5 days
- predicted action
- Request good-through + 4
- observed action
- Request good-through + 4
outcome · Expired-payoff incidents down 88%
evidence · 29 payoffs
Exception library
Payoff expires before scheduled closing
EXL-01 · observed 29× · conf 0.93
Re-request payoff good-through close + 4 business days
Outcome · 88% of expiry incidents eliminated
docs · Payoff statement, Authorization to release
comms · Lender payoff dept, Title processor
approvals · Coordinator
Seller deceased — probate required
EXL-02 · observed 11× · conf 0.74
Confirm letters testamentary, add estate signer block, title underwriter sign-off
Outcome · Closed with average +14 days
docs · Letters testamentary, Death certificate, Estate deed
comms · Attorney, Underwriter, Buyer agent
approvals · Coordinator, Title underwriter
Undisclosed judgment lien discovered in curative
EXL-03 · observed 18× · conf 0.81
Order payoff demand, escrow holdback if unresolvable pre-close
Outcome · 72% cured pre-close, 28% holdback
docs · Lien release, Holdback agreement
comms · Lien holder, Escrow officer
approvals · Coordinator, Both parties
HOA estoppel returned after closing date
EXL-04 · observed 14× · conf 0.68
Rush fee authorization + interim ledger from management company
Outcome · Median delay reduced 3.1 days
docs · Estoppel, Rush authorization
comms · HOA management, Buyer
approvals · Coordinator, Buyer
Subject-to lender due-on-sale inquiry
EXL-05 · observed 7× · conf 0.41
Attorney-reviewed response; no representation made by TC
Outcome · No accelerations observed (small sample)
docs · Attorney memo
comms · Attorney
approvals · Attorney, Coordinator
Trust-held property with successor trustee
EXL-06 · observed 9× · conf 0.66
Certificate of trust + trustee authority verification before signing packet
Outcome · Signing rejections eliminated in observed set
docs · Certificate of trust, Trust excerpt
comms · Trustee, Title underwriter
approvals · Title underwriter
Only approved exceptions become reusable. Pending and insufficient-evidence entries are documentation, not behavior.
AI readiness score
| scope | subject | coverage | accuracy | agreement | exc stability | calibration | override | score | verdict |
|---|---|---|---|---|---|---|---|---|---|
| Transaction type | Traditional purchase | 88% | 86% | 84% | 82% | 79% | 11% | 0.84 | AUTONOMY_CANDIDATE |
| Transaction type | Wholesale assignment | 79% | 81% | 77% | 74% | 72% | 18% | 0.76 | READY_FOR_APPROVAL |
| Transaction type | Subject-to | 52% | 63% | 58% | 44% | 60% | 37% | 0.54 | LIMITED_READY |
| Transaction type | Commercial purchase | 12% | 40% | 33% | 20% | 31% | 62% | 0.26 | NOT_READY |
| Playbook | Traditional purchase playbook | 90% | 86% | 85% | 83% | 80% | 10% | 0.85 | AUTONOMY_CANDIDATE |
| Playbook | Double close playbook | 61% | 68% | 62% | 49% | 63% | 31% | 0.59 | LIMITED_READY |
| Workflow step | Order title & open escrow | 94% | 92% | 90% | 90% | 85% | 6% | 0.91 | AUTONOMY_CANDIDATE |
| Workflow step | Payoff request & refresh | 86% | 90% | 88% | 87% | 82% | 8% | 0.87 | AUTONOMY_CANDIDATE |
| Workflow step | Curative escalation | 63% | 69% | 64% | 55% | 66% | 29% | 0.62 | READY_FOR_APPROVAL |
| Workflow step | Earnest money release | 81% | 84% | 80% | 78% | 77% | 14% | 0.80 | NOT_READY |
| Bot | Title Bot | 85% | 87% | 83% | 80% | 78% | 12% | 0.83 | AUTONOMY_CANDIDATE |
| Bot | Curative Bot | 60% | 67% | 61% | 52% | 64% | 30% | 0.60 | LIMITED_READY |
| Bot | Communication Bot | 77% | 80% | 75% | 72% | 74% | 19% | 0.76 | READY_FOR_APPROVAL |
| Skill | Deadline extraction from contract | 92% | 94% | 91% | 89% | 87% | 5% | 0.92 | AUTONOMY_CANDIDATE |
| Skill | Commitment exception summarization | 70% | 73% | 69% | 65% | 68% | 24% | 0.69 | READY_FOR_APPROVAL |
| Exception type | Payoff expiry | 88% | 91% | 90% | 88% | 84% | 7% | 0.89 | AUTONOMY_CANDIDATE |
| Exception type | Probate estate | 48% | 66% | 62% | 50% | 58% | 33% | 0.55 | LIMITED_READY |
| Exception type | Due-on-sale inquiry | 17% | 45% | 40% | 22% | 35% | 58% | 0.30 | NOT_READY |
Earnest money release scores well on accuracy but is a permanent human-only gate — funds movement and term changes can never become autonomy candidates regardless of score.
Human review analytics
median review 3h 12m · p90 19h 40m · n=288
Most edited steps
- Title commitment exception summary41
Missing office-specific endorsements
- Buyer status update email33
Tone and detail level adjusted
- Curative escalation message24
Escalation target changed
- Closing scheduling proposal19
Party location / notary constraints
Most rejected suggestions
- Accept wire screenshot as EM proof14
Evidence class not permitted
- Auto-send lender payoff authorization9
Requires signed authorization
- Propose closing date change7
Term change — human only
Most trusted suggestions
- Deadline extraction from executed contract132
edit rate 4%
- Payoff good-through + 4 request71
edit rate 5%
- Estoppel ordering reminder58
edit rate 7%
Knowledge promotion gates
Payoff good-through close + 4 business days
KP-701 · All playbooks with existing lien payoff · conf 0.93
- ✓ Minimum observations29 / ≥ 20
- ✓ Minimum approvals24 / ≥ 10
- ✓ Minimum confidence0.93 / ≥ 0.80
- ✓ Minimum coverage0.88 / ≥ 0.70
- ✓ No governance conflict0 / 0 open
Escrow-of-record receipt required for EM verification
KP-702 · All playbooks · conf 0.90
- ✓ Minimum observations22 / ≥ 20
- ✓ Minimum approvals22 / ≥ 10
- ✓ Minimum confidence0.90 / ≥ 0.80
- ✓ Minimum coverage0.92 / ≥ 0.70
- ✓ No governance conflict0 / 0 open
Escalate curative at day 4 of open defect
KP-703 · Traditional purchase, Wholesale assignment · conf 0.69
- ✓ Minimum observations31 / ≥ 20
- ✕ Minimum approvals8 / ≥ 10
- ✕ Minimum confidence0.69 / ≥ 0.80
- ✕ Minimum coverage0.63 / ≥ 0.70
- ✓ No governance conflict0 / 0 open
Subject-to due-on-sale response template
KP-704 · Subject-to · conf 0.41
- ✕ Minimum observations7 / ≥ 20
- ✕ Minimum approvals3 / ≥ 10
- ✕ Minimum confidence0.41 / ≥ 0.80
- ✕ Minimum coverage0.17 / ≥ 0.70
- ✕ No governance conflict1 — legal advice boundary / 0 open
Promotion produces a proposal only. TC Pros never publishes learning; Assistant controls publication.
Cross-team comparison · anonymized
| org | workflow similarity | playbook variations | exception rate | median stage days | median response | files |
|---|---|---|---|---|---|---|
| Org A | 91% | 2 | 0.42 | 3.1 | 2.4h | 148 |
| Org B | 84% | 4 | 0.58 | 3.9 | 4.1h | 96 |
| Org C | 72% | 7 | 0.81 | 5.2 | 7.8h | 61 |
| Org D | 66% | 9 | 0.94 | 6.0 | 9.3h | 24 |
Organizations are pseudonymous and aggregate-only. No customer, property, party or coordinator identity crosses an org boundary.
Validated workflows
14/38
conf 0.82
17
pending validation
Knowledge gaps
- Commercial purchasecov 12% · conf 0.18
- Lease optioncov 21% · conf 0.27
- Hybrid dealscov 9% · conf 0.12
- Land purchasecov 24% · conf 0.25
High-risk areas
- Subject-to due-on-sale handling0.41
Legal boundary + tiny sample
- Double close funding sequencing0.59
Override rate 31%, unstable exceptions
- Probate estate signing authority0.55
Underwriter-dependent variance
Top expert patterns
- Payoff good-through + 4n=29 · 0.93
- EM verified at escrow of recordn=22 · 0.90
- Estoppel ordered day 5n=26 · 0.84
- Signing booked close − 3n=62 · 0.78
Replay validation
| file | step | expert action | ai prediction | observed outcome | result | learning candidate |
|---|---|---|---|---|---|---|
| TX-2012 | Payoff request | Requested day 12 with +4 buffer | Request day 12 with +4 buffer | Closed on schedulealt · Request at commitment receipt (Org B norm) | Matched | Reinforce payoff buffer rule confidence |
| TX-1994 | Novation lender notice | Sent notice with recorded assignment | No prediction — no coverage | Closed +6 days | Unknown | Open coverage gap: novation lender notice sequence |
| TX-1961 | Curative escalation | Escalated day 7 | Escalate day 4 | Cure took 9 daysalt · Day 4 escalation cured in 6 days on 11 comparable files | Different | Propose day-4 escalation trial for review |
| TX-1948 | Signing scheduling | Booked close − 4 | Book close − 2 | No reschedulealt · Close − 2 succeeded on 19 local cash files | Partially Matched | Condition buffer on party location |
Replay validation generates learning candidates only. No rule, playbook or automation changes as a result of a replay run.
Learning submissions · via Assistant
- Playbook candidateconf 0.69
Day-4 curative escalation trial for traditional purchase
evidence · 31 curative episodes · routed to Assistant
- Skill improvementconf 0.77
Commitment summarizer to include office-specific endorsements
evidence · 41 edits on same step · routed to Assistant
- Workflow improvementconf 0.78
Condition signing buffer on party location
evidence · 62 signings · routed to Assistant
- Exception improvementconf 0.68
Promote HOA estoppel rush path to reusable
evidence · 14 occurrences · routed to Assistant
- Confidence calibrationconf 0.66
Lower subject-to prior; model overconfident by 0.14
evidence · 54 observations · routed to Assistant
TC Pros never publishes learning directly. Every submission is a proposal for Assistant.
Governance
TC Pros owns
does not own
invariants
- · Validation records are append-only — history is never overwritten
- · Every readiness verdict is a recommendation, never an activation
- · Funds movement and term changes stay human-only regardless of score
- · Unknown remains unknown — no verdict is issued below evidence thresholds
- · Assistant is the sole orchestration and publication layer