sensitivity_tiw

Empirical sensitivity vs time-in-warning trade-off — the field-standard forecasting view (Karoly 2017 Brain 140:2169 Fig 6). Sweep the decision threshold and trace seizure-level sensitivity against time-in-warning; chance is the diagonal (sensitivity == time-in-warning). The empirical complement to the analytic bridge. See Sensitivity vs Time-in-Warning: Mathematical Derivation for the chance-diagonal derivation and a worked example.

Empirical sensitivity vs time-in-warning (TiW) trade-off.

The field-standard forecasting view (Karoly et al. 2017, Brain 140:2169, Fig 6; Karoly et al. 2019): sweep the decision threshold and trace seizure-level sensitivity against time-in-warning (the fraction of recorded time the alarm is ON). The empirical complement to scitex_seizure_metrics.bridge.sample_to_alarm() (analytic sample-to-alarm bounds).

Public API:

The plotter lives in scitex_seizure_metrics.plots.sensitivity_tiw() (package convention: all plots in plots.py).

References

  • Karoly PJ et al., Brain 2017; 140: 2169-2182. doi:10.1093/brain/awx173

  • Karoly PJ et al., Lancet Neurology 2019.

  • Mormann F et al., Brain 2007; 130: 314-333.

  • docs/math/sensitivity_tiw.md.

class scitex_seizure_metrics.sensitivity_tiw.SensitivityTiWCurve(thresholds, tiw, sensitivity, n_seizures, improvement_over_chance, sensitivity_at_target_tiw, tiw_at_target_sensitivity, target_tiw=0.2, target_sensitivity=0.75, name='', notes=())[source]

Bases: object

Empirical sensitivity-vs-time-in-warning operating curve.

Arrays are ordered by ascending time-in-warning so the curve plots / integrates left-to-right against the chance diagonal.

Attrs:

thresholds: decision thresholds, aligned with the other arrays. tiw: time-in-warning fraction in [0, 1] at each threshold. sensitivity: seizure-level sensitivity in [0, 1] at each

threshold.

n_seizures: number of seizures (sensitivity denominator). improvement_over_chance: signed trapezoidal area between the

curve and the chance diagonal, integrated over TiW. Positive means the forecaster is, on net, above a time-matched coin.

sensitivity_at_target_tiw: best sensitivity achievable at

TiW <= target_tiw (NaN if no operating point fits).

tiw_at_target_sensitivity: smallest TiW at which the curve first

reaches target_sensitivity (NaN if never reached).

target_tiw: TiW operating point the scalar was read at. target_sensitivity: sensitivity operating point the scalar was

read at.

name: identifier carried from the caller. notes: free-form caveats.

Parameters:
improvement_over_chance: float
n_seizures: int
name: str = ''
notes: tuple[str, ...] = ()
sensitivity: ndarray
sensitivity_at_target_tiw: float
summary()[source]

Flat dict of the summary scalars (no per-threshold arrays).

Return type:

dict

target_sensitivity: float = 0.75
target_tiw: float = 0.2
thresholds: ndarray
tiw: ndarray
tiw_at_target_sensitivity: float
to_frame()[source]

One row per threshold as a pandas DataFrame (curve only).

class scitex_seizure_metrics.sensitivity_tiw.TiWSignificance(tiw, sensitivity, chance_sensitivity, n_seizures, n_caught, p_value, ci_low, ci_high, method='binomial', n_surrogate=0)[source]

Bases: object

Result of an above-chance test at one operating point.

Attrs:

tiw: time-in-warning fraction of the tested operating point. sensitivity: observed sensitivity at that point. chance_sensitivity: the diagonal value (== tiw). n_seizures: number of seizures (binomial n). n_caught: number of seizures caught (binomial successes). p_value: one-sided p-value for H1: sensitivity > chance. ci_low / ci_high: Wilson interval on the true catch rate. method: “binomial” (exact) or “surrogate” (permutation). n_surrogate: number of surrogate draws (surrogate method only).

Parameters:
chance_sensitivity: float
ci_high: float
ci_low: float
method: str = 'binomial'
n_caught: int
n_seizures: int
n_surrogate: int = 0
p_value: float
sensitivity: float
tiw: float
scitex_seizure_metrics.sensitivity_tiw.binomial_above_chance(*, n_caught, n_seizures, tiw, ci=0.95)[source]

Exact binomial test that observed sensitivity beats chance.

Under the null (a time-matched random alarm) each of n_seizures seizures is caught independently with probability \(p_0 = \text{tiw}\). The one-sided p-value is \(\Pr(X \ge n_{\text{caught}})\), \(X \sim \text{Binom}(n, p_0)\).

Parameters:

  • n_caught (int) – seizures caught at the operating point.

  • n_seizures (int) – total seizures (binomial n).

  • tiw (float) – time-in-warning fraction (the null catch probability).

  • ci (float) – confidence level for the Wilson interval on the catch rate.

Returns:

TiWSignificance with method == "binomial".

Raises:

ValueError – if n_seizures <= 0 or tiw not in [0, 1].

Return type:

TiWSignificance

scitex_seizure_metrics.sensitivity_tiw.chance_sensitivity(tiw)[source]

Analytic chance sensitivity for a time-matched random alarm.

A random warning ON for a fraction tiw of the recording overlaps each seizure’s pre-ictal window with probability tiw in expectation, so \(\text{sens}_{\text{chance}} = \text{tiw}\) (the diagonal). See docs/math/sensitivity_tiw.md.

Parameters:

tiw (float)

Return type:

float

scitex_seizure_metrics.sensitivity_tiw.monotone_upper_envelope(tiw, sens)[source]

The achievable (TiW, sensitivity) frontier as a non-decreasing step.

A forecaster can always discard signal to move down the curve, so the meaningful operating frontier at any time-in-warning budget is the running maximum of sensitivity over all operating points whose TiW does not exceed that budget. This collapses duplicate-TiW points to their best sensitivity and then takes the cumulative max, giving the monotone non-decreasing envelope that sensitivity_at_tiw() reads off (so a marker placed at (target_tiw, sensitivity_at_tiw(target_tiw)) lands exactly on this envelope, never floating above or below the drawn line).

Returns:

(env_tiw, env_sens) sorted by ascending TiW. Empty input returns two empty arrays.

Parameters:
Return type:

tuple[ndarray, ndarray]

scitex_seizure_metrics.sensitivity_tiw.seizures_from_labels(labels, times)[source]

Derive one onset time per contiguous pre-ictal (label==1) run.

The onset is placed at the END of the pre-ictal run (the first timestamp after the run) because pre-ictal windows precede the seizure. If a run reaches the end of the recording, the onset is placed one cadence after the last labelled window.

Parameters:

  • labels – per-window binary pre-ictal labels.

  • times – matching per-window timestamps (seconds).

Returns:

np.ndarray of seizure onset times (seconds), sorted ascending.

Return type:

ndarray

scitex_seizure_metrics.sensitivity_tiw.sensitivity_tiw_curve(scores, policy, *, labels=None, seizure_times=None, times=None, thresholds=None, n_thresholds=41, target_tiw=0.2, target_sensitivity=0.75, name='')[source]

Empirical sensitivity-vs-time-in-warning trade-off curve.

Sweep the decision threshold over the score range; at each threshold convert the score stream into alarms (via the policy’s refractory/merge rules) and measure time-in-warning and seizure-level sensitivity. The empirical complement to bridge.sample_to_alarm() and the curve behind Karoly 2017 Fig 6.

Parameters:

  • scores – 1-D per-window predicted scores / probabilities.

  • policy (AlarmPolicy) – AlarmPolicy (SPH, SOP, refractory, merge rule). The policy’s alarm_threshold is ignored — the threshold is what gets swept.

  • labels – optional per-window binary pre-ictal labels (mode A).

  • seizure_times – optional seizure onset timestamps (mode B). One of labels / seizure_times is required.

  • times – optional per-window timestamps (seconds). Defaults to a unit-cadence index [0, 1, 2, ...] if omitted.

  • thresholds (Iterable[float] | None) – explicit thresholds to sweep. If None, n_thresholds values spanning the observed score range are used.

  • n_thresholds (int) – number of thresholds when thresholds is None.

  • target_tiw (float) – TiW budget for sensitivity_at_target_tiw (Karoly’s common 0.20 / 20 %).

  • target_sensitivity (float) – target for tiw_at_target_sensitivity.

  • name (str) – identifier carried into the result.

Returns:

SensitivityTiWCurve.

Raises:

ValueError – neither labels nor seizure_times given, or shape mismatch.

Return type:

SensitivityTiWCurve

scitex_seizure_metrics.sensitivity_tiw.surrogate_above_chance(scores, policy, *, threshold, labels=None, seizure_times=None, times=None, n_surrogate=1000, rng_seed=0, ci=0.95)[source]

Permutation test: observed sensitivity vs TiW-matched surrogates.

Karoly-style surrogate analysis using circular time-shift surrogates of the score stream. We measure the observed operating point at threshold (its TiW and sensitivity), then n_surrogate times roll the score stream by a random offset and recompute sensitivity at the same threshold. A circular shift preserves the score multiset and the warning-block geometry exactly — hence the time-in-warning is unchanged — while destroying any phase-locking of the scores to the seizures. The one-sided p-value is

\[p = \frac{1 + \#\{\text{surrogate sens} \ge \text{observed sens}\}} {1 + n_{\text{surrogate}}} .\]

Complements binomial_above_chance(): the binomial test assumes independent per-seizure catches at rate tiw; the surrogate test makes no independence assumption and honours the real warning autocorrelation (run lengths, SOP, refractory). Because TiW is held fixed, a forecaster that merely warns a lot — without locking to seizures — is correctly judged non-significant.

Parameters:

  • scores – per-window scores (same as sensitivity_tiw_curve).

  • policy (AlarmPolicy) – AlarmPolicy.

  • threshold (float) – operating-point threshold to test.

  • times (labels / seizure_times /) – input modes.

  • n_surrogate (int) – number of circular-shift surrogates.

  • rng_seed (int) – RNG seed for reproducibility.

  • ci (float) – confidence level for the Wilson interval.

Returns:

TiWSignificance with method == "surrogate".

Raises:

ValueError – if there are no seizures.

Return type:

TiWSignificance