reference · Performance & testing
Out-of-Sample Testing: Freeze Rules before Evaluating Them
An out-of-sample evaluation applies a previously chosen procedure to data that did not guide that choice. Its credibility comes from the chronology and preserved rules, not from the label on a chart.
Key points
- Freeze the complete decision procedure, including preprocessing and costs.
- Keep model development separate from the final evaluation decision.
- A later successful sample is evidence with limitations, not a performance guarantee.
Define what is frozen
The procedure includes more than entry thresholds. It also includes the instrument universe, feature construction, sizing rule, costs, execution model, missing-data handling and evaluation criterion. If any of these are adjusted after reading evaluation outcomes, the tested procedure has changed.
MetaTrader documents forward testing during optimization. A software split can help organize a workflow, but it cannot prevent a researcher from repeatedly selecting based on the forward results. Preserve the decision timeline outside the final performance chart.
Use a chronological worked design
Imagine a historical dataset spanning 2019 through 2025. A hypothetical research plan uses 2019–2022 for development, 2023 for choosing among a small predeclared set of procedures, and 2024–2025 for a final evaluation. The precise dates are illustrative and do not claim that this split is sufficient for every strategy.
Before examining 2024–2025, archive the selected code, settings and criteria. Suppose development produced 240 trades and the final interval produces only 12. A positive result in the final interval does not supply the same evidence as hundreds of independent observations. The evaluation must report the actual count, duration and regimes encountered.
Prevent boundary leakage
Training labels that extend into the evaluation period can carry future outcomes across the split. For example, a five-day outcome assigned to the last training day may depend on four evaluation days. Review the exact label horizon and use an appropriate exclusion or separation rule for the research design.
Preprocessing must also respect the boundary. A scaler, feature selector or universe chosen using all dates is not made out-of-sample by splitting only the final trading calculation. Data revisions and same-bar execution assumptions need their own availability checks.
Understand walk-forward evaluation
A predeclared rolling process may fit on an earlier window, evaluate on the next interval, then advance. That can test an adaptive procedure, provided each step uses only earlier information and the rule for advancing is fixed. Combining these forward segments differs from repeatedly choosing whichever segment looks best.
Every manual revision after seeing outcomes should be logged. Previously inspected data can still be useful for development, but it should not be described as untouched evidence. An additional fresh evaluation is a new experiment with its own sample limitations.
Interpret a failure constructively
If evaluation performance degrades, investigate documented hypotheses such as costs, regime dependence or model error rather than silently changing the test dates. Preserve negative results and explain what changed. For copied accounts, evaluate the actual follower implementation where possible: source fills alone omit follower timing, rounding and account constraints. No backtest split proves live execution quality or guarantees future returns, but a transparent chronology makes the evidence meaningfully reviewable.
Questions and answers
Can I tune settings after seeing the out-of-sample result?
You can continue development, but that result has then influenced the process. Preserve the change and avoid presenting the same period as untouched validation of the revised settings.
Is a random train/test split always suitable for trading data?
No. Time dependence, overlapping labels and availability rules can make a random split misleading. The split must fit the chronological question being tested.
Sources and further checks
Use the current source for your exact instrument, account and platform. Referencing a general specification does not establish support for every TradeCopier workflow.
- MetaTrader 5: optimization and forward testing · Checked September 19, 2026
- Bailey and coauthors: backtest selection and evaluation · Checked September 19, 2026
Found an error? Send a correction with this page's address and a primary source. See our editorial standards for how we handle examples, claims and revisions.
