“The future enters into us, in order to transform itself in us, long before it happens.” - Rainer Maria Rilke
“Time discovers truth.” - Seneca
“All models are wrong, but some are useful.” - George E. P. Box
CRAFT stands for Conditional Regime Analog Forecasting with Trajectories. It asks a simple, practical question: when the latest trajectory looks like this, which historical future trajectories tended to follow?
No prophecy, no fog machine, no heroic point forecast wearing a cape. CRAFT keeps the workflow probabilistic: it turns recent movement profiles into regime labels, estimates what those labels usually lead to, samples matching historical future trajectories, and then fits smooth forecast distributions. The point is not to declare that tomorrow has been solved. The point is to keep uncertainty organized enough that it can sit at the table and use cutlery.
Trajectory preparation.
trajectory_embedding() converts each numeric series into
past and future cumulative-change windows. Past windows describe what
was known at a time point. Future windows describe what happened
next.
Regime detection.
svd_changepoint_regimes() compresses trajectory matrices
with singular value decomposition and detects changepoint-based regimes
on the resulting factors. This keeps the regime map compact even when
several assets and horizons are involved.
Conditional transitions.
craft_fit() learns how past-regime labels map into
future-regime labels. The transition model is intentionally modest: it
estimates conditional probabilities and leaves the drama to the
data.
Analog sampling. Historical future trajectories are resampled according to the latest regime probabilities. Optional tail penalties can make extreme analogues less eager to jump into the forecast parade.
Forecast distributions.
trajectory_forecast() fits smooth empirical distributions
to sampled trajectory draws, optionally reconstructing them into level
forecasts from the latest observed values.
We start with three synthetic level series. They share a little common movement, but not enough to become a committee.
library(CRAFT)
set.seed(42)
n <- 120
market_pulse <- cumsum(rnorm(n, mean = 0, sd = 0.004))
rate_pulse <- cumsum(rnorm(n, mean = 0, sd = 0.003))
series <- data.frame(
asset_a = cumprod(1 + 0.0010 + market_pulse + rnorm(n, 0, 0.008)),
asset_b = cumprod(1 + 0.0005 + 0.6 * market_pulse - 0.2 * rate_pulse +
rnorm(n, 0, 0.010)),
asset_c = cumprod(1 + 0.0008 - 0.3 * market_pulse + 0.5 * rate_pulse +
rnorm(n, 0, 0.007))
)
tail(series, 3)
#> asset_a asset_b asset_c
#> 118 3.865509 4.287691 0.1353390
#> 119 3.913264 4.369108 0.1304232
#> 120 3.923525 4.459543 0.1283990A trajectory window is a compact profile of cumulative changes. Backward-looking windows feed the regime detector; forward-looking windows become the pool of historical analogues.
trajectories <- trajectory_embedding(series, trajectory_window = 5)
names(trajectories)
#> [1] "past_trajectories" "future_trajectories" "latest_trajectory"
#> [4] "time_index"
dim(trajectories$past_trajectories)
#> [1] 110 15
dim(trajectories$future_trajectories)
#> [1] 110 15
head(round(trajectories$past_trajectories[, 1:5], 4), 3)
#> asset_a_cum_lag_1 asset_a_cum_lag_2 asset_a_cum_lag_3 asset_a_cum_lag_4
#> [1,] 0.0258 0.0246 0.0432 0.0596
#> [2,] 0.0236 0.0500 0.0487 0.0678
#> [3,] 0.0149 0.0388 0.0656 0.0643
#> asset_a_cum_lag_5
#> [1,] 0.0726
#> [2,] 0.0846
#> [3,] 0.0837Before fitting the full model, we can use
trajectory_forecast() directly. Here we take the realized
future trajectories for one asset and estimate a predictive interface
with density, distribution, quantile, and random-generation functions.
Very civilized. Almost suspiciously civilized.
asset_a_cols <- grep(
"^asset_a_cum_lead_",
colnames(trajectories$future_trajectories),
value = TRUE
)
asset_a_draws <- as.data.frame(trajectories$future_trajectories[, asset_a_cols, drop = FALSE])
asset_a_fc <- trajectory_forecast(
asset_a_draws,
probs = seq(0.05, 0.95, length.out = 40),
min_unique = 5,
verbose = FALSE
)
names(asset_a_fc)
#> [1] "asset_a_cum_lead_1" "asset_a_cum_lead_2" "asset_a_cum_lead_3"
#> [4] "asset_a_cum_lead_4" "asset_a_cum_lead_5"
asset_a_fc[["asset_a_cum_lead_5"]]$qfun(c(0.05, 0.50, 0.95))
#> [1] -0.02352891 0.06644611 0.12671289Now we let craft_fit() run the whole sequence:
trajectory embedding, regime labelling, conditional transition
probabilities, future-trajectory sampling, and forecast distribution
fitting. The settings below are deliberately small so the vignette does
not ask your laptop fan to write a resignation letter.
fit <- craft_fit(
series,
window = 5,
n_draws = 80,
n_factors = 1,
min_segment = 5,
max_regimes_per_factor = 3,
n_testing = 0,
return_train_probs = FALSE,
verbose = FALSE,
seed = 123
)
class(fit)
#> [1] "craft_fit" "regime_forecast_v5"
#> [3] "regime_forecast_transition" "regime_forecast"
fit$valid_joint_acc
#> [1] 1
names(fit$return_dists)
#> [1] "asset_a" "asset_b" "asset_c"The fitted object keeps both the machinery and the useful bits: trajectory matrices, regime models, latest labels, transition probabilities, sampled future trajectories, fitted return distributions, fitted level distributions, and diagnostics.
dim(fit$trajectory_draws)
#> [1] 80 15
names(fit$diagnostics)
#> [1] "latest_unseen_counts" "latest_unseen_total"
#> [3] "posterior_entropy" "sampler_weight_effective_n"
#> [5] "trajectory_columns_used_names" "expected_trajectory_columns"
#> [7] "class_balance" "transition"
#> [9] "valid_split" "temperature"
#> [11] "factor_selection" "min_segment"
#> [13] "tail_penalty"
round(fit$sampler_weights, 3)
#> regime_dim_1
#> 1
round(fit$diagnostics$posterior_entropy, 3)
#> regime_dim_1
#> 0.093craft_predict() reuses the fitted model. With no
newdata, it resamples from the latest trajectory already
stored in the fit. With newdata, it transforms a fresh
history and runs the same conditional analogue logic again.
pred <- craft_predict(fit, n_draws = 50, seed = 7)
class(pred)
#> [1] "craft_prediction" "regime_forecast_prediction"
dim(pred$trajectory_draws)
#> [1] 50 15
horizon_name <- tail(names(pred$return_dists$asset_a), 1)
pred$return_dists$asset_a[[horizon_name]]$qfun(c(0.05, 0.50, 0.95))
#> [1] -0.0425328 0.1014361 0.1314558You can also request only the part you need. This is helpful when a dashboard wants sampled trajectories while another report wants regime probabilities. Different rooms, same house.
labels <- craft_predict(fit, type = "labels")
probs <- craft_predict(fit, type = "probabilities")
draws <- craft_predict(fit, type = "trajectory_draws", n_draws = 10, seed = 9)
labels
#> regime_dim_1
#> 1 3
lapply(probs, function(x) round(x[1, ], 3))
#> $regime_dim_1
#> 1 2
#> 1 0.012 0.988
dim(draws)
#> [1] 10 15A few practical habits help:
fit$valid_joint_acc and the transition
diagnostics. They tell you whether the regime transition model is
learning anything more useful than a shrug.fit$diagnostics$posterior_entropy. High entropy
means the latest regime probabilities are diffuse. That is not
necessarily bad; it may be the model being appropriately humble.fit$trajectory_draws. These are the sampled
analogue futures. If they look absurd, the distributions fitted on top
of them will be beautifully absurd, which is still absurd.fit$level_dists when you want forecast levels and
fit$return_dists when you want cumulative-change
forecasts.CRAFT is a memory-based probabilistic forecasting workflow: profile the recent past, find comparable regime structure, sample what historically came next, and wrap the result in forecast distributions that can answer quantile, density, probability, and simulation questions.
It will not make time behave. Time has never accepted calendar invites from models. But CRAFT gives the next trajectory a disciplined set of historical analogues, and that is often a much better conversation than a lonely point forecast pretending to be certain.
Enzoi.