classification (forecasting confusion matrix)

Forecasting-regime confusion-matrix primitives (v0.2.0). These turn the alarm-vs-seizure match counts into the standard binary-classifier scores (specificity, PPV, NPV, F1) and the observed lead time, under the SOP-length-opportunity true-negative convention documented in The Alarm Confusion Matrix: Definitions and the SPH/SOP Framework and ADR-0001.

This is an internal module (_classification); its results are surfaced to users through forecasting (evaluate / evaluate_stream) and the report (MetricsReport) fields specificity / ppv / npv / forecasting_f1 / n_tn / n_opportunities / lead_time_mean / lead_time_median. It is documented here so the exact TN convention, the NaN-on-empty (fail-loud) behaviour, and the lead-time definition are discoverable in the API reference.

Forecasting-regime classification primitives.

The forecasting (alarm/seizure-prediction) regime already counts true positives (caught seizures) and false positives (alarms that catch nothing) in forecasting.evaluate via _alarm.alarm_match. This module turns those raw counts into the standard binary-classifier scores (specificity, PPV, NPV, F1) and the observed lead/warning time, under one explicit, documented convention.

Confusion-matrix convention (alarm / prediction-opportunity basis)

Counts are defined on a mixed basis that mirrors how the seizure- prediction literature (Snyder 2008; Schelter/Winterhalder 2006; Mormann 2007) scores an alarm system:

  • TP — number of seizures caught: a seizure at t_s is a TP iff some alarm fires at t_a with t_a + sph <= t_s <= t_a + sph + sop (this is _alarm.alarm_match seizure_caught.sum()).

  • FN — seizures not caught = n_seizures - TP.

  • FP — alarms that catch no seizure = alarm_useful False count.

  • TN — interictal “prediction opportunities” in which the system correctly stayed silent. The interictal time (the FP/hr denominator, with each seizure’s [t_s - sop - sph, t_s + sop] window removed) is partitioned into non-overlapping SOP-length opportunities: n_opportunities = floor(interictal_seconds / sop). Each opportunity that did not contain a false alarm is a TN, so TN = max(0, n_opportunities - FP).

From these:

  • sensitivity (recall) = TP / (TP + FN) [per-seizure]

  • specificity = TN / (TN + FP) [per-opportunity]

  • ppv (alarm precision) = TP / (TP + FP)

  • npv = TN / (TN + FN)

  • f1 = 2·TP / (2·TP + FP + FN)

Caveats a reviewer should confirm

“True negative” is genuinely convention-dependent for an alarm system — there is no canonical unit of “a correctly-quiet interictal moment”. The SOP-length-opportunity convention is the most defensible packaged choice (it matches the time unit the alarm validity window uses and the interictal FP denominator), but specificity/NPV scale with the chosen opportunity length. They are therefore reported alongside the raw n_opportunities (in extras) so the denominator is always visible. ppv and f1 do not depend on the TN convention.

Observed lead time

Distinct from the SPH constraint: for each caught seizure, the observed lead time is t_s - t_a_earliest, where t_a_earliest is the earliest alarm whose validity window covers that seizure. By construction it is >= sph. The per-seizure array plus mean/median summaries describe how much warning the system actually delivered.

class scitex_seizure_metrics._classification.AlarmClassification(tp, fp, fn, tn, n_opportunities, sensitivity, specificity, ppv, npv, f1)[source]

Bases: object

Forecasting-regime confusion matrix and derived classifier scores.

All fields use the convention documented at module level. tn and the metrics that depend on it (specificity, npv) are computed from interictal SOP-length opportunities; n_opportunities exposes that denominator.

Parameters:
f1: float
fn: int
fp: int
n_opportunities: int
npv: float
ppv: float
sensitivity: float
specificity: float
tn: int
tp: int
scitex_seizure_metrics._classification.alarm_classification(n_tp, n_fp, n_seizures, interictal_seconds, sop_seconds)[source]

Build the alarm-regime confusion matrix and classifier scores.

Parameters:

  • n_tp (int) – number of caught seizures (alarm_match seizure_caught sum).

  • n_fp (int) – number of alarms that caught no seizure.

  • n_seizures (int) – total reference seizures.

  • interictal_seconds (float) – interictal duration (FP/hr denominator).

  • sop_seconds (float) – Seizure Occurrence Period — the opportunity length.

Returns:

AlarmClassification with TP/FP/FN/TN, n_opportunities, and the sensitivity/specificity/PPV/NPV/F1 scores.

Return type:

AlarmClassification

scitex_seizure_metrics._classification.lead_time_summary(lead_times)[source]

Mean/median/min/max summary of an observed-lead-time array.

Returns NaN summaries (not silent zeros) when no seizure is caught, so an empty result is never mistaken for “0 s lead”.

Parameters:

lead_times (ndarray)

Return type:

dict

scitex_seizure_metrics._classification.observed_lead_times(alarms, seizures, sph, sop)[source]

Observed lead time per caught seizure.

For each seizure that is caught, the observed lead time is t_s - t_a_earliest, where t_a_earliest is the earliest alarm whose validity window [t_a + sph, t_a + sph + sop] covers the seizure. Uncaught seizures contribute no entry.

By construction every returned value is >= sph and <= sph + sop.

Parameters:

  • alarms (ndarray) – alarm onset times (seconds).

  • seizures (ndarray) – seizure onset times (seconds).

  • sph (float) – Seizure Prediction Horizon (seconds).

  • sop (float) – Seizure Occurrence Period (seconds).

Returns:

1-D float array of observed lead times, one per caught seizure (empty array if no seizure is caught).

Return type:

ndarray