Select Tuned Search Parameters from a Fixed-IC Grid

Choose a recommended shift_acceptance_threshold and min_descendant_tips combination from a bifrost_search_tuning_grid produced by runSearchTuningGrid(). Selection happens within a single IC family, making it easy to report one tuned workflow for GIC and another for BIC.

Usage

selectTunedSearchParameters(
  tuning_grid,
  max_false_positive_rate = 0.05,
  max_any_false_positive = 0.2,
  min_evaluable_fraction = 0.9,
  primary_metric = c("fuzzy_balanced_accuracy", "fuzzy_f1", "fuzzy_recall", "strict_f1",
    "weighted_fuzzy_f1"),
  scenario_weights = c(proportional = 0.5, correlation = 0.5),
  tie_break = c("conservative", "liberal"),
  allow_infeasible = FALSE
)

Arguments

tuning_grid

A bifrost_search_tuning_grid returned by runSearchTuningGrid().

max_false_positive_rate

Maximum acceptable mean false-positive rate under the null study, as a finite number between zero and one.

max_any_false_positive

Maximum acceptable fraction of null replicates that infer at least one shift.

min_evaluable_fraction

Minimum acceptable fraction of replicates with at least one candidate shift. This filter is applied to the null, proportional, and integration-rate summaries.

primary_metric

Character scalar indicating the recovery metric used to rank feasible settings. Defaults to "fuzzy_balanced_accuracy". Supported legacy values are "fuzzy_f1", "fuzzy_recall", "strict_f1", and "weighted_fuzzy_f1".

scenario_weights

Numeric length-2 vector giving the weights applied to the proportional and integration-rate scenarios when forming the ranking score. The internal weight name correlation is retained for compatibility. If unnamed, the first value is used for proportional and the second for the integration-rate scenario.

tie_break

Character scalar. "conservative" prefers larger thresholds and larger min_descendant_tips when the primary ranking score is tied; "liberal" prefers the smaller values.

allow_infeasible

Logical. If FALSE (the default), stop without a recommendation when no rankable setting satisfies every constraint. Set to TRUE to retain the historical diagnostic fallback that ranks the full rankable grid and marks used_all_settings = TRUE.

Value

A list of class bifrost_search_tuning_selection with components:

selected_row

The chosen row from tuning_grid$summary_table, augmented with a ranking score.

recommended_search_options

A search-options list containing the tuned controls from the simulation grid. For response-only empirical searches, the inherited intercept-only formula can usually be used directly. For empirical analyses with covariates, carry over the tuned controls but set formula to the empirical analysis formula.

feasible_table

The filtered and ranked table used for selection.

n_feasible_settings

The number of settings that passed the supplied constraints.

used_all_settings

Logical indicating whether the selector had to fall back to ranking the full grid because no feasible settings remained.

Details

The selector uses a two-step decision rule. First, it filters out settings that violate the null false-positive or evaluability constraints. Second, it ranks the remaining settings using a weighted average of the chosen recovery metric across the proportional and integration-rate scenarios. By default, this ranking metric is fuzzy balanced accuracy.

Only settings with finite recovery metrics for every scenario having positive weight are rankable. If no rankable settings pass the filters, the function stops by default rather than returning an unsupported recommendation. With allow_infeasible = TRUE, it warns, falls back to the full rankable grid, and records that no feasible settings were available under the supplied constraints. If the grid contains no rankable settings, the function always stops.

See also

runSearchTuningGrid(), searchOptimalConfiguration()

Examples

if (FALSE) { # \dontrun{
# Usually called after runSearchTuningGrid():
# tuned <- selectTunedSearchParameters(gic_grid)
# tuned$recommended_search_options
} # }