A list of papers that studies out-of-distribution (OOD) detection and misclassification detection (MisD)

PyTorch implementation of our ECCV 2022 paper "Rethinking Confidence Calibration for Failure Prediction"

A decision-safety lab for loan approval: trains a baseline classifier, calibrates probabilities (ECE/Brier), sweeps confidence thresholds to build a coverage, quality frontier and outputs a defensible abstention policy (auto-decide vs review). Includes a Streamlit dashboard for report cards, triage UI, and data quality checks.

Longform article reframing abstention (reject option / selective prediction) as product design, not model weakness. Covers coverage as a KPI, calibration as a prerequisite, threshold selection under review capacity and risk, queue/UX design for human-in-the-loop workflows, and anti-patterns that break safety in production.

Decision-safe evaluation + Streamlit dashboard for AI vs Human vs Post-Edited AI text detection. Generates a reliability report card (Accuracy, Macro F1, ECE, Brier), calibration plots, confidence histograms, and a coverage-vs-performance abstention curve. Recommends an operating threshold for human-review routing.

Promises and Pitfalls of Threshold-based Auto-labeling (NeurIPS 2023, Spotlight)

Pearls from Pebbles: Improved Confidence Functions for Auto-labeling (NeurIPS 2024)

Code for "Budgeted Classification with Rejection: An Evolutionary Method with Multiple Objectives", at IEEE CEC 2022

Robust Selective Classification of Skin Lesions with Asymmetric Costs

More human oversight can make an AI agent less safe. Headroom is a human-in-the-loop firewall for coding agents that measures when to trust the human: oversight as resource allocation, not just classification.

Reliable and Efficient Selective Machine Learning

Event-level risk-controlled triage for conjunction data message streams

Official implementation of the ICLR paper "Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix It"

Safety-centered uncertainty evaluation for RETFound retinal diabetic retinopathy screening under dataset shift.

Code for our paper analyzing the looseness of the upper bound on selective classification performance.

Missingness-aware abstention for selective classification under MCAR/MAR/MNAR label missingness.

New Zealand-localized phishing and smishing detection with selective abstention. Combines a localized feature set (IRD, NZ Post, banks, te reo cues, .govt.nz lookalikes) with classical baselines, fine-tuned transformers, and LLM-as-classifier (Gemini), under a class-conditional deferral layer for human review. Independent research.

Risk-controlled MIP morphology classification with a reproducible local HistoNexa-MIP reviewer application.

Enable reliable loan decisions by calibrating model scores into defendable actions: auto-approve, auto-reject, or manual review policies.