Abstract

We report two negative results on the strategy-selection stage of a production automated-trading pipeline. In the first study, 1,971 blind out-of-sample tests — fourteen strategy families at three bar sizes across 59 instruments — were run with selection pinned to a cutoff twelve months in the past and each family’s champion frozen and replayed over the unseen year. Over that year crypto fell 63.1% and the strategies returned −28.2%; US equities rose 22.1% and the strategies returned −22.1%. The near-identical loss in opposite regimes identifies the apparent defensiveness of these systems as low market exposure rather than skill. In-sample metrics barely rank the future: Spearman ρ between realised out-of-sample return and the training return is 0.312 (crypto), above the 0.273 of the embargoed holdout that exists to correct it. A profitable configuration existed for 58 of 59 instruments; the protocol almost never chose it.

The second study tests the obvious replacement. If fitted metrics do not predict, ranking candidates by realised out-of-sample return might. Over 52 weekly anchors (2024-07-01 to 2025-06-23), 8,046 picks across 53 instruments were ranked on a 365-day out-of-sample window and replayed on the week that followed. The rank correlation between the two is −0.059: the ranking does not persist even one week. Mean return by rank is not monotone (−0.056%, −0.265%, −0.129% per week for ranks 1–3), the top-ranked configuration changes in 27.3% of weeks, and an equal-weight portfolio of every pick compounds to −7.69% over the year. We conclude that neither the fitted signal nor its realised-return replacement carries usable predictive information at this horizon, and that the bottleneck is selection rather than the strategy library. No result here supports allocating capital to these strategies.

Keywords: out-of-sample validation · overfitting · walk-forward analysis · strategy selection · cryptocurrency · US equities · negative results

1Introduction

A parameter sweep that searches thousands of configurations and keeps the best one will always report a good backtest. The question that matters is whether the configuration it kept was kept for a reason that survives contact with unseen data. This paper answers that question twice, for the same production pipeline, and gets the same answer both times: no.

The system under study searches fourteen strategy families over a grid of parameters, at three bar sizes, for each instrument in a 59-instrument universe. Candidates pass through a fixed protocol — a training window, a 35-day embargo, an out-of-sample holdout, a set of viability tiers, and a buy-and-hold comparison — and the survivor is promoted with a quality rating from one to five stars. That protocol is conventional. It is also, we find, close to uninformative about what happens next.

Two studies are reported. Study I asks whether anything observable at selection time predicts forward return: it freezes the entire protocol at a cutoff twelve months in the past, replays every family’s champion over the unseen year, and correlates each selection-time metric against what actually happened. Study II asks the follow-up question that Study I forces. If fitted metrics do not predict, the natural replacement is to rank candidates by their realised out-of-sample return — to promote what has actually been working. That substitution is only sound if such a ranking persists, and Study II measures the persistence directly over 52 consecutive weekly anchors.

Study I removes the fitted signal. Study II removes the obvious thing you would replace it with. What remains is a selection problem with no known solution, sitting on top of a library that demonstrably contains profitable configurations.

Both results are negative and are reported as such. We have not found a selection rule that works; we have measured, with some precision, how badly two candidate rules fail. The practical consequences for the pipeline are set out in §5, and the reasons not to over-read any of this are in §6.

2Data

2.1Instruments and venues

The universe is the pipeline’s own tradeable set: 31 cryptocurrency perpetual-swap symbols priced from Binance spot/perp OHLCV, and 30 US equities and ETFs (9 ETFs, 21 large caps) from Alpaca IEX bars. Equity series are regular-trading-hours only — there is no feed between 16:00 and 09:30 ET — and the NYSE calendar including half-days gates every session. Crypto trades continuously.

Bars are stored at 15-minute, 1-hour and 4-hour resolution. Equities are validated on native 15m and 1h only; their 4h series is derived from 1h and appears in this paper as research material, never as a tradeable interval.

2.2Coverage, and why the panel is ragged

Instrument histories differ by years, and the protocol has a hard minimum: 180 days of training, a 35-day embargo and at least 90 days of holdout must fit before any selection cutoff. Symbols that cannot meet it fail closed and are reported as untestable rather than being quietly dropped or padded. In Study I that removed two crypto symbols (HYPE, ONDO), leaving 59 instruments. In Study II, where the requirement compounds with a 365-day ranking window to 670 days of prior bars per anchor, coverage is reported per anchor. Table 1 gives the breakdown for both studies.

Table 1. Instrument coverage in both studies. Symbols that cannot support the protocol’s window minimums fail closed and are reported here rather than dropped or padded.

Population	Tickers	Detail
Study I — submitted	61	31 crypto + 30 US equities
testable at the cutoff	59	29 crypto, 30 equities
untestable	2	HYPE, ONDO — listed too recently for 180d training + 35d embargo + 90d holdout
Study II — submitted	61	the same universe
reached the analyser	60	HYPE had no durable bars at any anchor
scored ≥ 1 anchor	53	23 crypto, 30 equities
scored all 52 anchors	50	three more scored 16, 22, 44
scored 0 anchors	7	ENA, JUP, ONDO, PENGU, POL, TAO, TON — all listed too recently

Equity history is itself unequal. Only SPY, QQQ, AAPL and NVDA are backfilled to 2023 and were selected on the full 790-day lookback; the other 26 tickers begin 2024-06-06 and were selected on a 428-day lookback of which roughly 303 days are training. All 30 cleared the protocol’s minimums, but they did not all get the same amount of evidence.

2.3A known hole in the tape

Binance served nothing for five consecutive 15-minute bars on 2023-03-24 (12:45–13:45 UTC). This is a venue outage, not a backfill failure: the exchange still returns nothing for that window and no refetch can close it. Study II does not clamp its windows around the gap — the five bars are simply absent and the surrounding bars are treated as adjacent. A 75-minute hole once in a four-year 15-minute series moves no indicator materially, whereas clamping every training window to start after it would delete the study’s entire first year of anchors. The trade is deliberate and is recorded here so it can be argued with.

2.4Cost model

Every simulated fill in both studies pays 4 basis points taker, 2 basis points maker and 5 basis points slippage; equities are levered 2×. This is the same cost model the production sweep validates against, so the numbers here are directly comparable to the ones the pipeline reports about itself. Equity fills additionally respect session semantics: a gap through a stop fills at the next session’s open, never at the stop price.

3Method

3.1The selection protocol

Both studies re-run the production protocol, disjoint_holdout_v1, unmodified. For a cutoff C it builds three strictly ordered, disjoint windows ending at or before C: a training window of at least 180 days, a 35-day embargo, and a holdout of at least 90 days or 20% of the lookback. A grid search fits parameters on training only; candidates are filtered through six descending viability tiers and a training-only buy-and-hold comparison at 6, 12 and 24 months; the survivor is scored on the holdout and assigned a 1–5 star rating. Nothing that closes after C is visible to any part of this.

3.2Study I — the blind matrix

The cutoff was pinned to 2025-08-09, twelve months before the run. For every instrument, the protocol was executed to completion and the winning configuration of every one of the fourteen families was retained — not only the family the pipeline would have promoted. Each was then frozen and replayed exactly once over the following year, 1,971 tests in total.

Retaining all fourteen champions is what separates two failure modes the pipeline could not previously tell apart: the library has no edge versus the library has edge and the picker misses it. With every family measured on the same frozen year, the best available configuration per instrument is observable after the fact, and the distance between it and the protocol’s actual pick is selection error by construction.

3.3Study II — 52 weekly anchors

Study II replaces the single cutoff with a rolling one. At each Monday anchor A from 2024-07-01 to 2025-06-23, three nested windows are built:

Selection, ending at the cutoff C = A − 365 days. Training, embargo and holdout all close at or before C, so every candidate’s parameters are fitted on bars older than C.
Ranking, the window [C, A). Each family champion is replayed here and ranked by realised return. This window is blind with respect to selection.
Test, the week [A, A+7d). The top 3 ranked configurations are replayed once more. This window is blind with respect to both selection and ranking, and it is the only number that scores the rule.

Rolling A forward one week at a time converts “does the out-of-sample ranking persist?” into 52 out-of-sample observations per instrument. The full selection sweep is re-run at every (instrument, anchor) pair, because the training window moves with C; 8,046 picks were scored in total.

3.4Controls

No cached features. Indicators are recomputed in memory from bar zero. The persistent feature cache is neither read nor written, so no stale or version-skewed cache row can influence a number.
No look-ahead. Every series is truncated at the cutoff before selection, and the harness asserts that the last selection-visible bar closes at or before it. Window ordering was verified across all artifacts: training → embargo → holdout are strictly ordered and end at or before the cutoff, and every out-of-sample bar starts at or after it.
No writes. Both studies are read-only against the production database. The live strategy table was read once, for the comparison in §4.6, and never modified. Nothing measured here was promoted.
Failures are reported, not filled. Instruments that cannot support the protocol at a given cutoff are recorded as untestable or insufficient-history for that anchor. The ragged panel that results is part of the result.

4Results

4.1Two opposite markets, the same loss

Table 2 is the headline. Over the frozen year the crypto tape fell 63.1% and the equity tape rose 22.1%. The median strategy returned −28.2% and −22.1% respectively — the same loss, in opposite regimes.

Table 2. Study I headline. One frozen year, two opposite tapes, near-identical losses. “Strategies” is the median across every family×interval champion; “protocol pick” is the single configuration the live selection rule would have promoted.

Cohort	Tickers	Tests	Buy and hold	Strategies (median)	Beat B&H	Protocol pick
Crypto	29	1132	−63.1%	−28.2%	79.3%	−17.5%
US equities	30	839	+22.1%	−22.1%	16.7%	−21.5%

The symmetry is the finding. Against the crypto crash, a system that is mostly out of the market reads as protection: 79.3% of picks beat buy-and-hold. Against the equity rally the identical behaviour is simply expensive, and the same comparison collapses to 16.7%. “Beats buy-and-hold” is the metric the sweep optimises toward, and it flatters low exposure. Absolute return is negative in both regimes.

4.2The edge exists; the picker misses it

58 of the 59 testable instruments had at least one strategy×interval configuration that made money out of sample (29/29 crypto, 29/30 equities). Table 3 sets the protocol’s actual pick against three alternatives, including that hindsight ceiling.

Table 3. Study I selection rules, one pick per ticker. Every rule loses money out of sample; picking by the embargoed holdout — the stage the protocol adds specifically to correct overfitting — is the worst rule in both cohorts.

Rule applied at the cutoff	Crypto median	Profitable	Equity median	Profitable
Best training return	−26.2%	20.7%	−11.5%	36.7%
Best holdout return	−33.3%	24.1%	−25.1%	36.7%
What the protocol promotes	−17.5%	27.6%	−21.5%	20.0%
Best available in hindsight	+50.9%	100.0%	+31.1%	96.7%

The hindsight row is an upper bound measured after the fact, not an achievable rule. It is reported to size the selection problem: it ranks configurations the protocol had already generated and then placed below its actual pick.

Two things stand out. The gap between the protocol’s −17.5% / −21.5% and the ceiling’s +50.9% / +31.1% is selection error, not a missing strategy — the ceiling ranks configurations the protocol itself generated and then placed below its choice. And picking by the embargoed holdout is the worst rule tested in both cohorts, worse than picking by the training return the holdout exists to correct.

4.3Decay, and what the metrics predict

Figure 1 tracks the same configurations through the three stages at which they are measured. Median training return (+11.1% crypto, +2.4% equity) is fitted by construction. The embargoed holdout — the number the star rating is minted from — already gives most of it back. The frozen year takes the rest.

Figure 1. Study I metric decay. Median return of the same family×interval champions at each of the three stages at which they are measured, with the buy-and-hold return of the frozen year marked as a dashed reference. Fitting is visible as the drop from training to holdout; the drop from holdout to out of sample is the part the protocol does not catch.

Table 4. Study I rank correlations against realised out-of-sample return (Spearman ρ, all family×interval champions). The training return predicts marginally better than the embargoed holdout that is meant to correct it.

Predictor available at the cutoff	Crypto	US equities
Training return	0.312	0.290
Embargoed holdout return	0.273	0.266
Star rating (1–5)	0.237	0.213

Table 4 puts the three selection-time metrics side by side. In both cohorts the training return ranks the future marginally better than the holdout does. That is the signature of a validation stage that is not adding information: it measures the same fitted noise one window later. The star rating, which is derived from the holdout, inherits the problem, and its ordering is not even monotone (Table 5).

Table 5. Study I star-rating calibration. The rating is minted from the embargoed holdout at the cutoff; the ordering is not monotone in either cohort.

Rating at cutoff	Crypto n	Crypto median OOS	Equity n	Equity median OOS
5★	106	−20.3%	54	−20.9%
4★	101	−31.9%	51	−19.1%
3★	153	−24.6%	139	−15.4%
2★	350	−13.8%	259	−17.2%
1★	422	−46.3%	336	−32.2%

1★ is the one rating that carries information, and only as an exclusion: it marks configurations that were never viable in sample at all.

One family illustrates the whole mechanism. PMax leads the crypto training column at +75.3%, prints −8.4% on the holdout, and returns −20.8% over the frozen year. A configuration can look like the best thing in the sweep and be worthless forward.

4.4Bar size is the largest effect

The strongest and cleanest gradient in Study I is not a strategy property at all. Holding family, instrument and year fixed and varying only the bar size moves the crypto median from −48.8% at 15 minutes to −9.9% at four hours, monotonically (Table 6, Figure 2). Crypto 4h champions were profitable 4.6× more often than 15m champions.

Figure 2. Study I median out-of-sample return by bar size. Same families, same instruments, same frozen year — only the bar size differs. Monotone in both cohorts and in every family taken separately.

Table 6. Study I by bar size — the largest single effect measured, monotone in both cohorts across all fourteen families.

Bar	Crypto n	Crypto median OOS	Profitable	Equity n	Equity median OOS	Profitable
15m	382	−48.8%	7.3%	420	−28.4%	15.5%
1h	378	−24.8%	20.9%	419	−15.7%	22.4%
4h	372	−9.9%	33.6%	396	−10.7%	28.0%

Equity 4h is research-only: production validates US equities on native 15m and 1h bars, and the 4h equity series is derived from 1h. The gradient is the finding, and it is present inside the traded intervals too.

Unlike the correlation results this is not a subtle statistical claim: the 15-minute tape is where fee drag and noise consume the edge. Of all the findings in Study I this is the one that translated directly into a pipeline change (§5).

4.5Which families held up

Table 7. Study I family league, ranked by median crypto out-of-sample return. Not one family has a positive median in either cohort.

Strategy family	Crypto median OOS	Profitable	Equity median OOS	Profitable
DoubleTap	−4.7%	37.9%	−0.3%	45.0%
MultiTFConfluence	−5.8%	29.9%	−7.6%	13.3%
VWAPMeanReversion	−10.8%	32.2%	−12.4%	16.7%
PMax	−20.8%	28.7%	−8.3%	40.0%
VolatilityRegime	−23.6%	19.5%	−17.9%	20.3%
TrendFollowing	−25.2%	12.6%	−27.7%	16.7%
MomentumRSI	−25.4%	19.5%	−28.9%	11.7%
ZScoreComposite	−31.0%	21.8%	−20.7%	15.0%
SSLQQE	−36.7%	11.5%	−27.8%	20.0%
BreakoutTrailingStop	−37.1%	26.4%	−23.5%	23.3%
Breakout	−43.2%	17.2%	−38.8%	11.7%
MeanReversion	−57.1%	8.0%	−45.5%	13.3%
OpeningRangeBreakout	−65.2%	2.3%	−53.5%	0.0%

GapFill is omitted: it produced only 6 crypto champions because a 24/7 tape has no session gaps to fade. On equities, where it does fire, it finished mid-table at −21.9%. OpeningRangeBreakout (last row) was profitable on 0 of 60 equity tests — the asset class it was designed for — and has since been removed from the search.

Table 7 ranks the families. Not one has a positive median in either cohort, and family ranking by training return and by out-of-sample return are close to unrelated. VWAPMeanReversion has a negative median crypto training return (−9.6%) and finishes third out of sample; PMax leads training and finishes fifth. Ranking families by backtest return is not merely noisy, it is misleading.

4.6The live rows look good for the wrong reason

Replaying the strategy rows that were actually approved and live in production over the same year gives a median of +10.6% on crypto — apparently far better than the blind picks’ −17.5%. It is not evidence that production is well tuned. Those rows carry validation dates inside the test window, so their parameters were fitted on windows overlapping the very year they are scored on. They are contaminated by construction, and the gap between the two numbers is a direct measure of how much of production’s apparent quality is in-sample fitting. The equity rows, with the same advantage and a rising tape, still beat buy-and-hold only 25.3% of the time and returned a median −3.8%.

4.7Study II: the ranking does not persist

Study I leaves one obvious move: stop ranking by fitted metrics and rank by realised out-of-sample return instead. Table 8 tests it directly.

Table 8. Study II persistence. Spearman ρ between a pick’s realised return over the 365-day ranking window and its return in the week that follows. Zero, or very slightly negative, everywhere.

Population	Picks	ρ (rank window → test week)
All scored picks	8046	−0.059
Crypto only	3366	−0.028
US equities only	4680	−0.022
Rank-1 picks only	2682	−0.059

The correlation between a configuration’s realised return over a 365-day out-of-sample window and its return in the very next week is −0.059. Restricting to the strongest picks does not help (−0.059 for rank-1 only), and neither asset class differs materially. A ranking that does not survive one week cannot be the basis of a weekly promotion rule.

Table 9. Study II by rank. If the ranking carried information, rank 1 would beat rank 2 would beat rank 3. It does not; and around half of the picks never open a position at all.

Rank	Picks	Mean return	Median	Positive	Never traded	Trades / week	Beat B&H
Rank 1	2682	−0.056%	0.00%	22.1%	51.0%	1.05	43.7%
Rank 2	2682	−0.265%	0.00%	20.0%	54.1%	1.10	43.7%
Rank 3	2682	−0.129%	0.00%	16.6%	61.0%	0.92	44.8%

Returns are per test week, in percent of the notional the simulator allocates to that pick. The median is 0.00% in every bucket because the modal week has no trade.

Table 9 shows the same thing from the other side. Mean weekly return is not monotone in rank: rank 2 (−0.265%) is worse than both rank 1 (−0.056%) and rank 3 (−0.129%). The top-ranked configuration is also unstable, changing in 27.3% of consecutive weeks — so a rule that followed it would be re-selecting a different strategy roughly every fourth week, paying costs for a signal that carries no information.

An inactivity caveat that cuts both ways. 51.0% of rank-1 picks never open a position during their test week, rising to 61.0% at rank 3. Those weeks contribute an exact 0.00%, which is why every median in Table 9 is zero. This compresses the measured spread and makes the near-zero correlation partly a statement about sparsity rather than about signal. It is also the honest description of the rule’s behaviour: much of the time it does nothing, and when it does something the outcome is uncorrelated with why it was chosen.

4.8Study II: what the panel returned

An equal-weight portfolio of every scored pick compounds to −7.69% over the 52 weeks, with 30.8% of weeks positive and a mean of −0.152% per week (Figure 3). The drawdown is not dramatic; it is a slow, consistent bleed, which is what a zero-information rule paying transaction costs should look like.

Figure 3. Study II equal-weight portfolio across 52 weekly anchors. Bars give the mean test-week return of every scored pick at that anchor (left axis); the line is the compounded result (right axis). No single week does the damage.

Table 10. Study II outcomes over the whole 52-week panel, per ticker. “Profitable” and “beat buy-and-hold” are largely different tickers — the second is mostly a list of instruments whose tape fell.

Outcome	Tickers	Of	Which
Compounded return > 0	10	53	AAPL, ADA, ARB, ATOM, DOT, ETH, SOL, SUI, TRX, TSLA
Beat buy and hold	10	53	AMD, APT, ARB, ATOM, AVAX, DOT, ETH, NEAR, OP, UNH
Both at once	4	53	ARB, ATOM, DOT, ETH

Table 10 makes a distinction that the pipeline’s own reporting historically blurred. 10 of 53 instruments finished the panel profitable, and 10 beat buy-and-hold — but only 4 did both. The two lists barely overlap because they measure different things: beating buy-and-hold on this panel mostly means the instrument fell and the rule was not in it.

Figure 4. Study II per-instrument outcome against buy-and-hold over the same weeks. The horizontal axis is on a signed log scale to fit a +469% tape alongside a −46% one; the dashed diagonal is parity with holding the asset. Vertical spread is narrow, horizontal spread covers two orders of magnitude.

All 53 scored instruments are plotted and labelled, none clipped; a hairline ties a label to its point where crowding moved it. Hover any point for both returns.

Figure 4 plots every instrument’s walk-forward return against its own buy-and-hold over the same weeks. Points below the dashed parity line lost to simply holding the asset; the great majority are, and the vertical spread is narrow while the horizontal spread covers two orders of magnitude. The rule neither tracked the winners nor avoided the losers.

Table 11. Study II worst rally misses — the six tickers with the largest losses while their own tape rose. Ordered by walk-forward return.

Ticker	Class	Anchors	Walk-forward	Buy and hold	Shortfall
HBAR	Crypto	52	−51.4%	+121.6%	−172.9%
AAVE	Crypto	52	−42.0%	+469.4%	−511.4%
LTC	Crypto	52	−38.6%	+95.4%	−134.0%
AVGO	Equity	52	−34.9%	+85.0%	−119.9%
DOGE	Crypto	52	−32.7%	+48.7%	−81.4%
NVDA	Equity	52	−32.6%	+63.2%	−95.9%

Ranked instead by shortfall, the order changes: AAVE (−511%), XRP (−482%), PLTR (−416%), HBAR (−173%). Either ordering makes the same point.

The extreme cases in Table 11 are instructive. AAVE rose +469% over the panel while the rule returned −42.0%. These are not tail events in a working system; they are the same behaviour Study I identified — low exposure — priced by a rising tape.

For completeness: choosing, in hindsight, the single best-performing configuration per instrument across the whole panel yields a median +12.01% compounded, positive for 41 of 53 instruments. As in Study I this is an upper bound on what better selection could have captured, not a strategy — it requires knowing the answer in advance.

5Discussion

The two studies compose into a single, uncomfortable statement. The metrics the pipeline selects on do not rank forward return (Study I), and the most natural non-fitted replacement — realised out-of-sample performance — does not rank it either (Study II). Between them they eliminate both the incumbent signal and its obvious successor.

What survives scrutiny is narrower and worth stating precisely:

Absolute return is the only defensible yardstick. “Beats buy-and-hold” is regime-dependent and rewards being out of the market. The same behaviour scored 79.3% against a crash and 16.7% against a rally while losing money in both.
The star rating is not a forecast. Rank correlation with realised return is 0.237 / 0.213 and the ordering is not monotone. Its only defensible use is ordering eligibility; 1★ retains value as an exclusion.
Bar size dominates strategy choice. A 38.9-point monotone gradient across every family is larger than any difference between families, and unlike the correlation results it is robust.
The library is not the bottleneck. A profitable configuration existed for 58 of 59 instruments. Generating candidates is solved; choosing among them is not.

Three changes followed from Study I and are already in the production sweep: candidate ranking now prefers 4-hour bars ahead of the in-sample viability tier, which moves the median crypto selection from −17.5% to −5.9% when this study’s own artifacts are re-ranked through it; OpeningRangeBreakout was removed from the search after finishing last of fourteen in both cohorts, independently reproducing an earlier benchmark on different windows [2]; and a daily job now re-runs the whole protocol and records the realised forward ranking as evidence.

Study II is the reason a fourth change did not follow. Rewiring execution onto that forward ranking was the obvious next step, and the persistence measurement says it would have been a mistake: a −0.059 correlation and a 27.3% weekly turnover describe a rule that would churn positions and pay costs for nothing. The daily job continues to accumulate evidence; nothing was promoted on the strength of it.

The useful reading of these results is not “these strategies are bad.” It is that a validation protocol built from a training window, an embargo and a holdout — the standard construction — produced a selection signal with a rank correlation of roughly 0.3 against the future, and a ranking built from realised out-of-sample returns produced one indistinguishable from zero. Both are measurements of the same missing quantity.

6Limitations

These are the reasons not to over-read anything above. They are ordered by how much they should change your confidence.

Two regimes, one of each. Study I covers twelve months of a crypto crash and twelve of an equity rally; Study II covers a single overlapping year. That is enough to demonstrate the exposure asymmetry and to reject the persistence hypothesis at this horizon. It is not enough to characterise behaviour across a full cycle, and a different pair of years could move every point estimate here.
Hindsight ceilings are upper bounds, not strategies. The +50.9% / +31.1% of Study I and the +12.01% median of Study II are computed by choosing, after the fact, the configuration that turned out best. They size the prize available to better selection. No rule can capture them and none is proposed.
Sparsity inflates the persistence result. Roughly half of Study II’s picks never trade in their test week (Table 9). A correlation computed over a sample where the modal observation is exactly zero is a weaker instrument than the sample size suggests. The direction of the finding is unaffected — there is no positive correlation hiding in the traded subset — but the precision is overstated by n.
The panel is ragged. 50 of 53 instruments contributed all 52 anchors; three contributed 16, 22, 44 and 7 contributed none, all because they listed too recently to satisfy the 670-day history requirement. Aggregate statistics therefore weight long-listed instruments more heavily, and the crypto cohort in particular is 23 names rather than 31.
A known gap in the tape. The 2023-03-24 Binance outage (§2.3) sits inside the crypto 15-minute series and is not clamped around. We judge a 75-minute hole immaterial to any indicator used here; that judgement is not free of assumption.
Current code, historical data. These are the selections today’s protocol would have made, not the ones that ran at the time. Study I also had to be re-run after a calendar defect was found mid-study: an exchange calendar that began in 2024 made 2023 holidays look like missing bars and silently clamped four deep-history equity tickers’ training windows forward. Those four were re-run on the corrected calendar with zero gap warnings.
Unequal equity history. 26 of 30 equity tickers were selected on a 428-day lookback against 790 days for the other four (§2.2). All cleared the protocol minimums; they did not all get the same evidence.
Single pipeline, single cost model. Everything here describes one implementation of one protocol at one cost assumption. The negative results are about this construction, not a general claim about technical trading.

7Conclusion

Across 1,971 blind out-of-sample tests and 8,046 walk-forward observations, we could not find a selection signal that predicts forward return. In-sample metrics rank it at ρ ≈ 0.312 at best, with the embargoed holdout ranking it worse than the training return it corrects; realised out-of-sample performance ranks it at −0.059, which is to say not at all. In two opposite market regimes the pipeline lost approximately the same amount, which identifies its apparent defensiveness as low exposure rather than skill.

The one durable, actionable effect we did find is structural rather than strategic: longer bars are monotonically better, by a margin larger than any difference between strategy families. That result changed the production sweep. Nothing else here did, and nothing measured in either study was promoted to trade.

We publish both results because negative results about selection are scarce and expensive to produce, and because the alternative — reporting the backtest and omitting the year that followed — is how this class of system usually gets described. A profitable configuration existed for 58 of 59 instruments. Finding it in advance remains an unsolved problem, and until it is solved these strategies are not a reason to allocate capital.

References

Zarattini, C. and Aziz, A. (2023). Can Day Trading Really Be Profitable? Evidence of Sustainable Long-Term Profits from Opening Range Breakout (ORB) Day Trading Strategy vs. Benchmark in the US Stock Market. SSRN Working Paper. — the source for the OpeningRangeBreakout family evaluated in §4.5, which finished last of fourteen in both cohorts of Study I.
Meet Zoro Research (2026). TradingView strategy benchmark. Internal report, 7 August 2026. — an independent evaluation on different windows that reached the same conclusion about OpeningRangeBreakout.
Meet Zoro Research (2026). Classic out-of-sample test — every strategy, every ticker. Internal report, 9 August 2026. — the full Study I writeup, including per-instrument tables omitted here.

Reproducibility. Study I harness: scripts/research/oos-full-matrix-study.ts. Study II harness: scripts/research/oos-weekly-walkforward.ts, aggregated by scripts/research/oos-weekly-walkforward-analyze.py. This document is generated by scripts/research/oos-paper.py from the distilled dataset embedded below, which is itself checked in alongside this file. Both studies are read-only against the production database.

Not investment advice. This is a research report about a measurement failure. It describes strategies that lost money out of sample in both regimes tested and it recommends none of them. Nothing here is a recommendation to buy, sell, or hold any instrument.

Meet Zoro Research · 10 August 2026 · generated 2026-08-10T18:49:11+00:00