Evaluate Shift Recovery Against Simulated Ground Truth

Compare inferred shift locations against known simulated shift locations across multiple datasets using both strict node matching and fuzzy matching within a configurable node distance. This is a package-clean export of the manuscript's evaluate_shift_recovery() function.

Usage

evaluateShiftRecovery(
  simdata,
  simresults,
  fuzzy_distance = 2,
  weighted = TRUE,
  verbose = TRUE
)

Arguments

simdata

A list of simulated shifted datasets, typically the simdata component returned by runShiftRecoverySimulationStudy(). Each element must contain at least paintedTree and shiftNodes.

simresults

A list of search results corresponding to simdata, usually the results component returned by runShiftRecoverySimulationStudy(). Each element should contain shift_nodes_no_uncertainty and num_candidates. Results from searchOptimalConfiguration() also carry candidate_nodes, which allows candidate-aware specificity calculations; legacy result objects without that component retain the historical count-only calculation. If weighted = TRUE, ic_weights is also used when available.

fuzzy_distance

Integer node distance threshold used for fuzzy matching.

weighted

Logical; if TRUE, compute weighted precision/recall/F1 using the inferred-node IC weights from each search result when available.

verbose

Logical; if TRUE, print a compact summary of the strict, fuzzy, and weighted metrics.

Value

A list of class bifrost_shift_recovery_evaluation with five components:

strict

Strict precision, recall, F1, specificity, false-positive rate, and balanced accuracy.

fuzzy

The same metrics under fuzzy matching.

weighted

Weighted strict and fuzzy precision/recall/F1 summaries when weighted = TRUE; otherwise NULL.

counts

Aggregated contingency-table counts for the strict and fuzzy matching schemes.

n_evaluable_replicates

Number of replicate pairs included in the aggregated counts after failed or structurally incomplete records are excluded.

Details

The evaluation uses three complementary summaries:

• strict metrics count only exact node matches,

• fuzzy metrics allow inferred shifts within fuzzy_distance nodes of a true shift, using a greedy one-to-one assignment, and

• weighted metrics apply IC weights only to inferred nodes, following the simulation-study implementation.

Search results carrying a non-empty error field are retained by the study wrappers for diagnosis but excluded from recovery counts and metrics.

For current search results, true-negative counts are calculated over the explicit candidate_nodes universe. True shifts excluded by the search's candidate filter still contribute false negatives, but are not subtracted from the number of candidate negatives. This prevents specificity and balanced accuracy from being distorted when a simulated shift is not an eligible search candidate. Supplied candidate-node vectors must contain unique, valid whole-number node IDs and agree with num_candidates. For compatibility, result objects that provide only num_candidates use the historical count-only formula.

When a replicate has no evaluable candidate shifts (num_candidates == 0), recall-style quantities can still be computed from the true and inferred shifts, but specificity, false-positive rate, and balanced accuracy are returned as NA because there are no evaluable negatives.

See also

runShiftRecoverySimulationStudy(), simulateShiftedDataset()

Examples

if (FALSE) { # \dontrun{
# Usually called on the output of runShiftRecoverySimulationStudy():
# evaluateShiftRecovery(study$simdata, study$results, fuzzy_distance = 2)
} # }