Resampling Strategies

Resampling Strategies

Overview

Comprehensive guide to resampling methods for model validation using the rsample package. Covers cross-validation, bootstrapping, and specialized resampling for time series and grouped data.

Data Splitting

Basic Train/Test Split

library(rsample)

set.seed(123)

# Simple split (75% training)
split 
  mutate(
    model = map(splits, ~ lm(y ~ x, data = analysis(.x))),
    coefs = map(model, tidy)
  ) |>
  unnest(coefs)

# Calculate confidence intervals
boot_ci 
  tune_grid(
    resamples = outer_folds,  # outer loop
    grid = 20
  )

Time Series Resampling

Rolling Origin

# Time series cross-validation
ts_folds 
  fit_resamples(
    resamples = folds,
    metrics = metric_set(rmse, rsq, mae),
    control = control_resamples(save_pred = TRUE)
  )

# Collect metrics
collect_metrics(results)

# Collect predictions
collect_predictions(results)

Custom Resampling

# Create custom resample object
custom_splits  10000)
folds 
  tune_grid(
    resamples = vfold_cv(train, v = 10, strata = outcome),
    grid = 20,
    metrics = metric_set(roc_auc, accuracy)
  )

# Tune with bootstrap
tune_results 
  tune_bayes(
    resamples = bootstraps(train, times = 25),
    iter = 50
  )

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: choxos
• Source: choxos/BiostatAgent
• License: MIT

Install and usage instructions live in the source repository linked above.