Introduction
The curse of dimensionality is usually stated as a warning: every parameter you add multiplies the size of the search grid, so a high-dimensional strategy is hopeless to sweep [1]. That framing prices every dimension identically. In a backtest, they are not identical — they differ in unit cost by the ratio of an indicator rebuild to a decision-rule pass, and once that is named the right question is not how many parameters but how many expensive ones.
Split a strategy's parameters by what re-evaluating them forces you to recompute. Some parameters change the indicators — the feature arrays a decision is later made against. A moving-average length is one: change it and the indicator must be rebuilt across the entire price series, an O(np) windowed pass for a window of length p over n bars. Other parameters change only the decision rule applied to fixed indicators. A crossover margin is one: change it and no indicator moves; you re-run a single O(n) pass over the already-computed signal arrays, applying the gate and booking fills. We call the first the expensive axis and the second the cheap axis.
The naive way to sweep a two-axis grid is a single loop: pick an (\text{indicator},\text{threshold}) pair, build the indicators from scratch, simulate, score, repeat. For the overwhelming majority of adjacent trials, that loop recomputes indicators that did not change — nudge the margin threshold and the moving average over tens of thousands of bars is bit-for-bit identical to the previous trial, because the threshold appears nowhere in the indicator's definition. The naive loop rebuilds it anyway. The entire optimization studied here is the refusal to do that: build each distinct indicator config once, cache its signal arrays, and sweep every threshold against the cache. This is memoization [3] keyed on the expensive axis, and it is correct precisely because the indicator is a pure function of the indicator parameters alone.
Two properties make this worth a controlled study rather than a footnote. First, the speedup is not free of a ceiling: it is bounded by how much more one indicator build costs than one threshold pass, a workload-dependent ratio we measure rather than assume. Second, reordering a computation invites bugs, so the optimization is only trustworthy under a gate: the cached scheme must return the identical result — the same trades, the same PnL, on every pair — as the naive grid it replaces. This paper quantifies, from one seeded run, the unit cost of each axis and their ratio; the equivalence gate over every pair; the measured speedup against the analytic cost-model prediction and the ideal ceiling T; and how the speedup scales as the threshold cardinality grows.
Scope and honesty constraints.
All numbers come from one seeded run of one public harness
(scripts/run_all.py, methods in scripts/axes.py), with
NumPy [2] as the only dependency. Wall-clock times are
pure-NumPy CPU measurements and will vary by machine; what does not vary is
the equivalence gate (the two schemes are bit-identical by construction) and the
T\times reduction in indicator builds (a pure
count). The realizable wall-clock speedup on this workload is capped by the modest
build/threshold cost ratio (3.92\times here), well
short of the T-fold ideal; a heavier indicator (a
higher-timeframe resample, a deeper composition) would push the ratio up and the
realized speedup toward T, but those magnitudes are
not what this workload demonstrates and we do not claim them.
The cost model
Factorization.
Write the evaluation of one full parameter vector
\theta=(\alpha,\tau), where
\alpha are the indicator parameters and
\tau a threshold, as two stages:
\begin{equation}
\label{eq:factor}
\mathrm{signals} = \mathrm{build}(\alpha), \qquad
\mathrm{score} = \mathrm{simulate}(\mathrm{signals},\,\tau).
\end{equation} The first stage does not read \tau:
the signal arrays are a function of the indicator parameters
\alpha alone. This is not a lucky
implementation detail but the load-bearing fact — it is what licenses holding
\alpha fixed, varying \tau
freely, and treating \mathrm{signals} as a constant
computed once. In the harness, build_signals is the expensive stage (it
bumps an indicator-build counter) and simulate is the cheap stage (it
reads separation straight from the cache).
Two schemes over an I\times T grid.
Let I be the number of distinct indicator configs, T the number of thresholds, c_{\mathrm{ind}} the wall-time of one indicator build and c_{\mathrm{thr}} that of one threshold pass. Both schemes perform I\cdot T threshold passes; they differ only in how many indicator builds they perform. The naive full grid rebuilds inside the inner loop, doing I\cdot T builds; the cached scheme builds each config once, doing I: \begin{equation} \label{eq:costs} \mathrm{cost}_{\mathrm{naive}} = I\,T\,c_{\mathrm{ind}} + I\,T\,c_{\mathrm{thr}}, \qquad \mathrm{cost}_{\mathrm{cached}} = I\,c_{\mathrm{ind}} + I\,T\,c_{\mathrm{thr}}. \end{equation}
The speedup identity.
Their ratio is independent of I: \begin{equation} \label{eq:speedup} \mathrm{speedup} = \frac{\mathrm{cost}_{\mathrm{naive}}}{\mathrm{cost}_{\mathrm{cached}}} = \frac{T\,c_{\mathrm{ind}} + T\,c_{\mathrm{thr}}}{c_{\mathrm{ind}} + T\,c_{\mathrm{thr}}} = T\cdot\frac{c_{\mathrm{ind}}+c_{\mathrm{thr}}}{c_{\mathrm{ind}}+T\,c_{\mathrm{thr}}}. \end{equation} Two limits read the formula. As c_{\mathrm{ind}}/c_{\mathrm{thr}}\to\infty (the indicator dominates, the whole premise of the optimization), the T c_{\mathrm{thr}} term is negligible against c_{\mathrm{ind}} and the speedup \to T, the threshold cardinality: every threshold pass becomes nearly free and you pay only for the I builds you could not avoid. Conversely, when c_{\mathrm{ind}}=c_{\mathrm{thr}} the speedup is 2T/(1+T)<2: if a build costs the same as a pass there is almost nothing to cache. So T is an ideal ceiling, approached only in proportion to how expensive the indicator axis is relative to the threshold axis. The realized speedup on a given workload is Eq. (3) evaluated at that workload's measured c_{\mathrm{ind}},c_{\mathrm{thr}}; the gap between it and T is the honest cost of the threshold passes you still run.
The equivalence gate
Reordering a computation is only sound if it changes nothing observable. The
cached scheme computes the same set of (\alpha,\tau)
results as the naive grid, in a different order (all thresholds for one
\alpha before moving on, rather than interleaved),
and reusing an indicator array across thresholds rather than rebuilding it. Because
both schemes call the identical build_signals and simulate
on the identical inputs, every result should be bit-identical, not merely
close: any difference is a caching bug, not floating-point rounding.
The gate makes that a hard check. For every (\alpha,\tau)
key it compares the two schemes' simulate outputs field by field —
trade count, entry and exit bar indices, entry and exit prices, and PnL —
under exact equality, and requires the key sets to match. It passes iff there are
zero mismatches over all I\cdot T pairs. The test
suite additionally shows the gate is not vacuous: a single perturbed cached PnL is
detected. This is the discrete, downstream invariant that certifies the speedup is a
pure performance change: a correct cache moves no trade, so a passing gate means the
numbers in Section 4 were bought with ordering alone.
Results
All results are one seeded run (seed 0, Python
3.12.3, NumPy 2.5.1) of scripts/run_all.py on a geometric-random-walk
close series (n=40{,}000, initial price
30{,}000, per-bar log-return standard deviation
6\times10^{-4}, zero drift). The strategy is an
MA-cross long/flat rule: an HMA-like fast moving average against a slower one,
entered when their signed separation (in percent of price) exceeds a margin
threshold and exited past the symmetric lower band, executed one bar delayed at the
close (leak-free). The indicator grid is the I=12
configs (\text{fast},\text{slow}) with fast
\in\{8,16,24\} and slow
\in\{48,72,96,120\}; the threshold grid is
T margins evenly spaced in
[0.02,0.30] percent.
Per-axis unit cost.
Table 1 times one indicator build and one threshold pass in isolation (warm-up discarded, best-of-repeat). The build costs 2.361~ms and the pass 0.602~ms, a ratio of 3.92: one indicator rebuild is worth just under four threshold passes on this workload. This modest ratio is the whole story of why the realized speedup lands where it does — it is the c_{\mathrm{ind}}/c_{\mathrm{thr}} that Eq. (3) evaluates. It is also honest about the workload: a single-timeframe NumPy moving average is cheap; a higher-timeframe resample or a deeper indicator composition would raise this ratio and push the realizable speedup toward T.
| Quantity | Value |
|---|---|
| cost of one indicator build c_{\mathrm{ind}} | 2.361~ms |
| cost of one threshold pass c_{\mathrm{thr}} | 0.602~ms |
| cost ratio c_{\mathrm{ind}}/c_{\mathrm{thr}} | 3.92 |
The equivalence gate and the build count.
Over the full I\times T = 12\times 64 = 768 grid, the gate confirms equivalence on all 768 pairs with 0 mismatches: the cached scheme reproduces the naive grid exactly (a combined 334{,}996 trades and 5894.88 total PnL percentage points, identical under both). Table 2 is the honest cost account behind the speedup. Both schemes run 768 threshold passes, but the naive grid performs 768 indicator builds where the cached scheme performs 12 — a reduction of exactly 768/12 = 64 = T. The build count is a pure integer tally, independent of any machine: the cached scheme provably does T\times fewer of the expensive operation, which is the mechanism Eq. (3) prices.
| Quantity | Naive grid | Cached scheme |
|---|---|---|
| Indicator builds | 768 | 12 |
| Threshold passes | 768 | 768 |
| Indicator-build ratio (naive/cached) | 64.0 | |
| Pairs compared | 768 | |
| Mismatches (bit-exact) | 0 | |
| Equivalence confirmed | yes | |
Measured versus analytic speedup.
Table 3 places the measured wall-clock speedup next to the cost-model prediction and the ideal ceiling on the same 768-pair grid. The naive grid runs in 3.118~s, the cached scheme in 0.481~s, a measured speedup of 6.48\times. The cost model Eq. (3) evaluated at the measured c_{\mathrm{ind}},c_{\mathrm{thr}} predicts 4.64\times; the ideal ceiling is T=64\times. The measured speedup exceeds the cost-model prediction by a factor 1.40 — the per-op costs of Table 1, timed on single isolated calls, slightly understate the amortization the batched cached loop achieves (repeated builds share warm caches and code paths), so the realized speedup runs a little ahead of the naive-cost prediction while remaining an order of magnitude below the ideal T (the measured/T ratio is 0.101). The realizable speedup is capped, exactly as Section 2 predicts, by the 3.92\times cost ratio: you cannot save more than the redundant builds were worth.
| Quantity | Value |
|---|---|
| naive grid wall time | 3.118~s |
| cached scheme wall time | 0.481~s |
| measured speedup | 6.48\times |
| analytic speedup from costs (Eq. (3)) | 4.64\times |
| ideal ceiling T | 64\times |
| measured / analytic-from-costs | 1.40 |
| measured / ideal T | 0.101 |
Scaling with the threshold cardinality.
The mechanism predicts that the speedup grows with
T: the more thresholds share one cached indicator,
the larger the fraction of builds that were redundant. Table 4 sweeps
T\in\{4,8,16,32,64,128\} with the indicator grid
held fixed. The measured speedup rises from 2.82\times
at T=4 to 6.45\times
at T=128, and the cost-model prediction rises
alongside it from 2.49\times to
4.78\times, both climbing toward the ideal
T they can never reach at this cost ratio. The
empirical slope of measured speedup against T is
positive (0.021 per unit
T) with correlation
0.78; the growth is monotone in the endpoints and
the harness records speedup_grows_with_T as true. (Individual timed
rows carry scheduler noise — the T=64 row here
reads 5.30\times, a separately timed run from the
headline 6.48\times of Table 3 — which is
exactly why we report the sweep and its trend rather than lean on any single
wall-clock number.)
| T | naive / cached wall (s) | measured speedup | analytic from costs |
|---|---|---|---|
| 4 | 0.183\ /\ 0.065 | 2.82\times | 2.49\times |
| 8 | 0.415\ /\ 0.107 | 3.87\times | 3.30\times |
| 16 | 0.777\ /\ 0.151 | 5.15\times | 3.95\times |
| 32 | 1.505\ /\ 0.265 | 5.69\times | 4.39\times |
| 64 | 2.986\ /\ 0.564 | 5.30\times | 4.64\times |
| 128 | 6.181\ /\ 0.959 | 6.45\times | 4.78\times |
Discussion
The curse of dimensionality is a curse of expensive dimensionality.
The size of the grid grows in the total parameter count [1], but the cost of covering it grows only in the count of expensive dimensions. Once the axes are separated, thresholds expand the grid without meaningfully expanding the compute bill: each additional threshold is one cheap pass over arrays that never move. The honest way to reason about search cost is not “how many parameters” but “how many expensive parameters, and how coarse can their grid be” — keep the indicator grid small and deliberate, be lavish on the threshold axis.
Memoize on the invariant, and only on the invariant.
The cache is keyed on the expensive axis alone because the indicator is a pure function of the indicator parameters [3]. The correctness of that key is not a convenience; it is the premise the equivalence gate verifies. Cache on something the threshold secretly touches — a look-ahead leak that peeks at the decision, say — and the gate would fire. That the gate passes bit-exactly on all 768 pairs is the evidence that the factorization Eq. (1) holds for this strategy.
Know your cost ratio before you quote a speedup.
The ideal ceiling T is seductive and almost never realized. What you actually get is Eq. (3) at your workload's c_{\mathrm{ind}}/c_{\mathrm{thr}}. On this deliberately light single-timeframe workload that ratio is 3.92, so a 64-threshold sweep buys 6.5\times, not 64\times. A heavier indicator would move the ratio and the realized speedup up together; the scaling table shows the shape of that climb. The number to report is the measured one at your ratio, with the ideal T named as the ceiling it is, never as the result.
Limitations
Machine-dependent wall-clock. The speedup, cost-ratio, and timing numbers are pure-NumPy CPU measurements on one machine and will differ elsewhere. What is machine-invariant is the equivalence gate (bit-identical by construction), the indicator-build counts (768 vs 12, a pure tally), and the \sim\!T scaling of the speedup. We report timings for concreteness, not as portable constants.
Realizable speedup capped by the cost ratio. The measured 6.48\times is far below the ideal T=64 because one indicator build costs only 3.92\times one threshold pass on this workload. This is a property of the (deliberately light) indicator, not a limit of the method: as c_{\mathrm{ind}}/c_{\mathrm{thr}} grows the realized speedup approaches T. The accompanying blog post's \sim\!1{,}600\times cost ratio is a different, heavier workload (a full-year multi-timeframe engine); that magnitude is not reproduced here and is not claimed — this workload's ratio is \sim\!4\times, which is precisely why its realized speedup is single-digit.
One synthetic DGP and one strategy family. Numbers are specific to a single seeded geometric-random-walk series and a single MA-cross long/flat strategy. The factorization and the gate generalize to any strategy whose features are threshold-invariant, but the specific costs and speedups are tied to these choices.
Performance, not statistics. Making the threshold axis cheap makes it easy to run many trials, which is a multiple-testing hazard the speedup does nothing to address. This paper is strictly about the compute cost and its correctness gate, not about how to discount a densely searched threshold grid.
Conclusion
A strategy's parameters split into an expensive indicator axis, whose values must be recomputed across the whole series, and a cheap threshold axis, a single pass over precomputed signals. Because the indicator builder never reads the threshold, the naive full grid recomputes a threshold-invariant quantity T times over; caching each indicator config once and sweeping all thresholds against the cache removes exactly those redundant builds, at a speedup T(c_{\mathrm{ind}}+c_{\mathrm{thr}})/(c_{\mathrm{ind}}+T c_{\mathrm{thr}}) that tends to the threshold cardinality T as the indicator cost dominates. On a seeded 768-pair grid the equivalence gate confirms the cached scheme is bit-identical to the naive grid on every pair, the indicator-build count falls from 768 to 12 (exactly T=64\times), and the measured speedup is 6.48\times — bracketing the cost-model prediction of 4.64\times and far below the ideal T=64, because on this light workload one build costs only 3.92\times one pass. The speedup grows with T as the mechanism predicts. The lesson is to reprice the search: separate the axes, memoize on the invariant one, verify with an equivalence gate that the cache moved no trade, and quote the speedup your cost ratio actually delivers rather than the ceiling T it can only approach.
Reproducibility.
All numbers derive from one seeded run. scripts/run_all.py
regenerates results/results.json from seed 0
(Python 3.12.3, NumPy 2.5.1); the indicator/threshold axes, the two search schemes,
the equivalence gate, and the cost model are in scripts/axes.py. The
synthetic series is a geometric random walk from
numpy.random.default_rng(0) (initial price
30{,}000, per-bar log-return standard deviation
6\times10^{-4}, zero drift,
n=40{,}000). tests/ contains
deterministic invariant tests for the equivalence gate, the build-count account, and
the T-scaling of the speedup. Wall-clock times will
vary by machine; the gate and the build counts are exact and reproduce
bit-for-bit.