Data, Evaluation & Trustworthy AI¶
Evaluation runs through every method: data suitability, task-aligned metrics and reliability under shift. Use this page as an experimental checklist, not another long course.
Map and priority¶
| Depth | Topic | Understand |
|---|---|---|
| P0 | Data and design | Provenance, licensing, cleaning, duplicates, missingness, leakage, splits and held-out evaluation |
| P0 | Task metrics | Different metrics for classification, regression, retrieval, generation and control; variance and failures |
| P1 | Uncertainty and robustness | Calibration, confidence, OOD, noise, shift, anomalies and abstention |
| P1 | Interpretation and traceability | Attribution, visualization, ablation and errors; explanation is not causal proof |
| P2 | Frontier reliability | Contamination, hallucination, tool traces, reward gaming, safety/privacy and bias; test the scenario |
Core mechanistic intuition · Reliability & evaluation¶
- Confidence vs. correctness: Modern deep neural networks with Softmax output are routinely overconfident. Calibration (e.g., Temperature Scaling) measures whether predicted probabilities align with observed empirical frequencies.
- Out-of-Distribution (OOD) & domain shift: Identical distribution (i.i.d.) is the fundamental premise of empirical risk minimization. When sensor drift or environmental conditions change, performance can degrade abruptly; density or reconstruction-based rejection filters are essential.
- Hidden leakage pathways: Leakage extends far beyond identical sample duplication. Preprocessing statistics computed across the entire corpus, random splits on autocorrelated time series, and cross-split sensor contamination yield illusory benchmark results.
Selected resources¶
-
:material-table-split:{ .lg .middle } Validation & Splits · scikit-learn CV
Understand temporal splits (TimeSeriesSplit), grouping (GroupKFold) and stratification to avoid leakage.
-
Calibration · scikit-learn Probability Calibration
Diagnose deep neural network overconfidence; evaluate reliability diagrams and temperature scaling.
Guided experiment · AI-7 · Reliability checks on an existing model¶
- Select an existing project; specify success, splits and metrics without adding models.
- Define representative noise/missingness/condition changes and distinguish their origins.
- Report errors by subgroup/condition; check calibration or generation support as appropriate.
- Make one improvement and check gains/regressions on reserved examples.
Acceptance: Report reproducible conditions, evidence, failures and limits; historical completion checkmarks are not independent project evidence.
Check yourself · Critical thinking¶
- Why is Expected Calibration Error (ECE) frequently more informative than raw accuracy in mission-critical safety or damage detection?
- How can a lightweight selective classification or uncertainty-based abstention mechanism be deployed to prevent confident failures on anomalous inputs?
Reuse these checks throughout the roadmap; LLM and robot-specific details stay in their sections.