A signal can appear accurate in a historical test because the test quietly includes information unavailable when the decision would have been made. Reconstructing the information timeline is a stronger check than counting winners alone.
The documentation identifies two evaluation hazards
scikit-learn describes data leakage as using information unavailable at prediction time when building a model, producing overly optimistic evaluation. Its cross-validation documentation separately warns that time-dependent observations require care and describes evaluation on later observations using time-series splits. These are methodological principles, not evidence that any particular trading signal works. scikit-learn documentation — Common pitfalls: data leakage scikit-learn documentation — Cross-validation: time series
A candle can contain the outcome it is supposed to predict
Imagine a signal generated at 10:00:00 for a contract expiring at 10:01:00. A historical dataset supplies the high and low of the entire 10:00–10:01 candle as inputs. Those values are not fully known at 10:00:00; they include prices observed during the prediction interval.
The test may therefore let the signal inspect part of the future it claims to predict. Shifting a label column is not enough to fix that problem if the input features still carry later information. Every field needs an availability timestamp as well as a timestamp describing its reference period.
Economic data can create the same problem. A revised number available today might differ from the release that traders actually saw. A backtest using the revised series should not describe itself as a reconstruction of historical decision information unless that difference is addressed.
Model selection can contaminate evaluation even without a coding error. Suppose many indicator settings are tried and the highest-scoring one is selected using the final test period. That period has influenced the choice; it no longer serves as an untouched evaluation of the chosen specification.
A chronological split is useful but may still need separation around overlapping outcome windows. If neighbouring observations share much of the same future price path, treating them as independent evidence can exaggerate the effective sample size. The split design should reflect the prediction horizon.
Audit availability before reviewing accuracy
List each input’s earliest availability, preserve release versions where relevant, and separate development choices from final evaluation. Check preprocessing and feature selection as well as the trading rule itself.
Finally, assess the resulting decisions against contract prices, costs and settlement definitions. A cleaner accuracy estimate is still not a profitability guarantee. The first requirement is simpler: the test must not use tomorrow’s information to describe yesterday’s decision.