1. Start with a falsifiable research hypothesis
Backtests should evaluate a clearly stated hypothesis, not search for profitable parameter combinations. Define the economic rationale, expected behavior, and failure conditions before running simulations.
2. Use walk-forward validation as default
A single train/test split is fragile. Walk-forward evaluation repeatedly re-estimates models on rolling windows and tests on unseen future slices, providing better evidence across shifting market regimes.
3. Include implementation frictions
Backtests should include realistic assumptions for every rebalance decision:
- Transaction costs, bid-ask spreads, and market impact proxies.
- Execution delay, slippage, and partial fill assumptions.
- Liquidity and volume participation limits for position sizing.
4. Eliminate data leakage and survivorship bias
Each rebalance must use only information available at that timestamp. Universe construction should include delisted names where possible to avoid survivorship inflation. Bias control is mandatory for credibility.
5. Evaluate distributions, not only averages
Mean returns can hide severe tail behavior. Evaluate drawdown depth and duration, left-tail outcomes, volatility clustering, and dependency on specific market windows.
6. Apply robustness and perturbation tests
Perturb key parameters, shift rebalance dates, and test alternative cost assumptions. A robust strategy should degrade gracefully, not collapse under minor configuration changes.
7. Separate model selection from final evaluation
Use a dedicated validation stage for model selection and preserve a final untouched test period for performance estimation. Re-using evaluation windows for tuning materially overstates expected live results.
8. Maintain an auditable experiment log
Every experiment should record assumptions, input versions, parameter sets, and outcomes. This supports review, reproducibility, and post-deployment learning when live behavior diverges from simulation.
9. Pre-deployment release checklist
- Hypothesis and economic rationale documented.
- Walk-forward and out-of-sample results meet policy thresholds.
- Cost and slippage assumptions stress-tested.
- Bias checks completed and signed off.
- Monitoring thresholds defined for live performance drift.
10. Governance note
Backtesting is a risk-management tool, not a marketing artifact. Decision teams should treat it as one part of a broader investment governance process that includes policy, oversight, and periodic review.