Backtesting

Walk-forward analysis vs a single out-of-sample check

What walk-forward analysis adds over one held-back check period, what Veloce's parameter sweeps do and don't do, and how to run a walk-forward test by hand.

By
Veloce team
Published
Reading time
9 min read
On this page
  1. Why a check period at all
  2. How big the luck can be
  3. What a Veloce sweep does
  4. What a sweep does not do
  5. Walk-forward analysis
  6. Rolling and anchored windows
  7. Walk-forward efficiency
  8. Running a walk-forward test with Veloce sweeps
  9. Structural breaks in Indian F&O data
  10. Common questions
  11. A checklist
  12. Try it in Veloce

A single out-of-sample check tunes a strategy on one stretch of history and tests the chosen settings once on the stretch that follows. Walk-forward analysis repeats that experiment many times: tune on a window, check on the next one, move both forward, and judge the strategy on all the check periods joined together. One check gives one verdict, and one verdict can be lucky. Several consecutive verdicts show whether the way you choose settings keeps working. Veloce’s sweeps do the first kind of test, not the second, and this guide explains exactly what they do, where that leaves gaps, and how to run a walk-forward test by hand with them.

Why a check period at all

Every backtest that tries more than one setting has a selection problem. If you try twelve combinations of a moving average crossover, the best of the twelve did well partly because it suits the market and partly because it happened to catch the right days. The second part does not repeat. The overfitting guide covers the mechanics.

A check period measures how much of the tuning result survives on days the choice never saw. The difference between the tuning result and the check result is a rough estimate of how much was luck.

How big the luck can be

The following illustrative simulation, written in plain Python and not drawn from market data, shows the effect. Twelve combinations each trade 80 times in a tuning period and 80 times in a check period. Each trade’s P&L is random with a standard deviation of ₹1,000. The best combination on the tuning period is picked and its check result recorded, and the whole experiment is repeated 2,000 times.

True edge per trade Average tuning P&L of the best combination Its average check P&L Check periods with a profit
None (₹0) +₹14,547 −₹305 49%
Small (₹50) +₹18,676 +₹3,984 67%

With no edge at all, the best of twelve still made about ₹14,500 on the tuning period. On the check period it made nothing on average, and was profitable about half the time, like a coin. With a small real edge of ₹50 a trade, the check result averaged about ₹4,000, close to the true 80 × ₹50, while the tuning result overstated it more than four times.

Two lessons follow. First, the tuning number of the winning combination is not an estimate of anything; it is inflated by the selection. Second, even with a real edge, a third of single check periods showed a loss. One check is weak evidence in either direction.

What a Veloce sweep does

A sweep in Veloce takes a strategy, a tuning period, a list of settings to vary, and optionally a check end date. Here is what happens, step by step, as implemented in the sweep code:

  1. Every combination. The values you list for each setting are combined in full. Three fast lengths and two slow lengths make six runs; add two candle sizes and it is twelve.
  2. Tune on one period. Each combination is backtested on the tuning period, from your start date to your end date, with the same capital, lot sizes, slippage and charges.
  3. Skip what cannot run. A combination the strategy rejects (a fast average longer than the slow one, a candle that does not fit an opening range, a spec field out of bounds) is listed as skipped with the reason, in its place, rather than stopping the sweep.
  4. Rank by net P&L. The runs are sorted by net profit after charges on the tuning period, best first.
  5. Check the best few. If you set a check end date, the top combinations (three by default) are backtested again from the day after the tuning period ends until the check end date. The check end must be after the tuning end.
  6. Report both. For each run you see net P&L, maximum drawdown, number of trades, win rate, profit factor, fees and peak capital at risk, for the tuning period and, for the checked ones, the check period too.

For example, sweep the SMA buyer’s fast average over 3 and 5 and its slow average over 8 and 13, tuned on 1 August to 30 September 2026 and checked until 19 October. The title of the result reads “tuned from 2026-08-01 to 2026-09-30, checked from 2026-10-01 to 2026-10-19”. Each ranked line shows the settings, the tuning measures, and for the top three a second line starting with “check”. Sweeps run from the app with a check-until date, and strategy specs can be swept by field path; see the strategy spec guide.

tuning period: every combination check period: top 3 ranked by tuning net P&L; the check does not re-rank
One split: every combination tunes on the same period, and only the best few are checked, once.

What a sweep does not do

Knowing the limits matters as much as knowing the method.

  • It uses one split. There is one tuning period and one check period. It does not roll windows forward or repeat the experiment.
  • It does not re-rank on the check. The order is set by the tuning result. The check is shown beside it, so you can see decay, but the sweep does not pick a winner from the check period, which would turn the check into tuning.
  • It ranks only by net P&L. A combination with a higher net profit and a much deeper drawdown ranks above a smoother one. Read the drawdown, profit factor and peak capital at risk columns yourself before preferring a row.
  • It does not correct for the number of combinations. Sixteen combinations give more chances for a lucky winner than four. Nothing in the ranking adjusts for that.
  • It does not stitch check periods. There is no combined out-of-sample equity curve and no walk-forward efficiency figure.
  • It cannot stop you reusing the check. If you change the strategy after seeing a poor check and run the sweep again on the same dates, the check period has become part of your tuning.

None of these is hidden; they follow from what the sweep is: a grid search with one honest look at later data.

Walk-forward analysis

Walk-forward analysis repeats the tune-then-check step through time:

  1. Tune on window 1, pick the best settings, and record how they do on the period right after.
  2. Move both windows forward by the length of the check period.
  3. Repeat until the data runs out.
  4. Join the check periods. That joined record is the evidence; the tuning results are not.

Rolling and anchored windows

Kind Tuning window Adapts to change Tunes on
Rolling Fixed length, drops old data Faster Less data per step
Anchored Fixed start, grows Slower More data per step

Walk-forward efficiency

A common summary compares out-of-sample performance with in-sample performance over the same length of time. If the best settings made ₹20,000 a month in tuning and ₹8,000 a month in the following checks on average, the efficiency is 40%. A figure near zero or negative says the selection is mostly fitting noise. There is no universal threshold, and the figure is only as reliable as the number of trades behind it.

Running a walk-forward test with Veloce sweeps

You can build a rolling walk-forward test from a series of sweeps, each with the same settings grid and a check period right after its tuning window. With four-month tuning windows and one-month checks:

Step Tuning period Check period
1 1 January to 30 April 2026 1 to 31 May 2026
2 1 February to 31 May 2026 1 to 30 June 2026
3 1 March to 30 June 2026 1 to 31 July 2026
4 1 April to 31 July 2026 1 to 31 August 2026
5 1 May to 31 August 2026 1 to 30 September 2026
6 1 June to 30 September 2026 1 to 31 October 2026

For each step, run the sweep, keep only the top-ranked combination, and record its check-period net P&L, drawdown and trade count. Six check months joined together are the out-of-sample record. Sweeps are stored in your run history with their full request and result, so the six steps stay together for comparison.

Then read:

  • The joined check P&L. Is it positive across the six months, and how much of it comes from one month?
  • Which settings won each step. If the winner changes every month, the strategy has no stable best setting, and the edge, if any, is in the rule rather than the numbers.
  • Efficiency. Compare the average monthly tuning P&L of the winners with the average check P&L.

To keep it honest, decide the grid, the windows and the rule for picking (“top net P&L”) before step 1, and do not change them after seeing a result.

Structural breaks in Indian F&O data

Walk-forward windows assume that the past resembles the near future. Indian index options have changed under traders’ feet several times recently, and a window that straddles a change mixes two markets:

Date Change Effect on a backtest
1 September 2025 NIFTY weekly expiry moved from Thursday to Tuesday Day-of-week rules shift
January 2026 expiries NIFTY lot from 75 to 65 units Rupee P&L per lot changes
1 April 2026 STT on option sells from 0.1% to 0.15% of premium Costs rise for every sell
3 August 2026 F&O close extended to 15:40, cash closing auction from 15:15 Late-session candles change meaning
7 December 2026 F&O close 15:45, expiring index contracts stop at 15:30 See the December session guide

Veloce applies the lot size, charges and expiry day in force on each date, so a backtest across a change is priced correctly. What it cannot do is make the market behave the same before and after. If a walk-forward step’s tuning window contains a change and its check window does not, read that step with care.

Common questions

How long should the check period be? Long enough for dozens of trades. An intraday strategy that trades twice a day gets about 40 trades in a month; a weekly iron condor gets four or five, so it needs a much longer check.

Should I tune on more combinations to find a better one? More combinations make the best tuning result look better and the gap to the check larger. Prefer a small grid of sensible values and look for a plateau, a region of settings that all work.

Is walk-forward always better than one check? It gives more verdicts from the same data, but each tuning window is shorter. For strategies with few trades, one long tuning period and one long check can be the more reliable test.

What if the check result is better than the tuning result? It happens, usually by chance. It is not evidence that the strategy improves over time.

A checklist

Check Why
Fix the grid and the picking rule before looking at data Choices made after looking are tuning
Use the check period once per idea Reuse turns it into tuning data
Read drawdown and profit factor, not just rank Sweeps rank by net P&L only
Prefer plateaus to peaks A lone peak is often luck
Count trades in each check period Few trades give a weak verdict
Note structural breaks inside windows Expiry days, lots, costs and hours changed
Treat the winner’s tuning P&L as inflated It was selected for being high

Try it in Veloce

Veloce runs parameter sweeps on NIFTY and BANK NIFTY options, SENSEX options and MCX gold futures on one-minute data, with every combination priced with next-open fills, slippage and itemised charges, and the best few checked on a later period. The parameter sweeps page describes the options, and reports explains each measure in the result.

Create a free Veloce account and run your next sweep with a check-until date, or as six steps of a walk-forward test. To judge how much of a single result could be luck, see the Monte Carlo guide.

Frequently asked questions

What is the difference between walk-forward analysis and out-of-sample testing?
An out-of-sample test tunes a strategy on one period and checks the chosen settings once on a later period. Walk-forward analysis repeats that split many times, rolling the tuning and checking windows forward through the data and judging the strategy on all the check periods joined together.
Does Veloce run walk-forward analysis?
Not automatically. A Veloce sweep tunes every combination of settings on one period, ranks them by net P&L and reruns the best few on one later check period. You can run a walk-forward test by hand as a series of sweeps over consecutive windows and compare their check results in your run history.
What ratio of in-sample to out-of-sample data should I use?
Ratios of three or four to one are common conventions, but the binding constraint is the number of trades, not calendar time. A check period with ten trades says little whatever its length. Choose windows long enough that each check period has dozens of trades.
What is the difference between anchored and rolling walk-forward?
Anchored walk-forward keeps the tuning window's start fixed, so each step tunes on more history. Rolling walk-forward keeps the tuning window the same length and drops the oldest data, which adapts faster to changes but tunes on less.
Can I look at the check period more than once?
Each time you change a strategy after seeing its check result, the check period becomes part of the tuning. Use it once per idea. If you need another look, hold back a further period you have not seen.
Does a good walk-forward result prove a strategy will work live?
No. It shows that the way you choose settings held up on data the choice did not see, over the periods tested. Markets change, and costs, slippage and lot sizes in future may differ from the past.

Sources

  1. Wikipedia: Walk forward optimizationen.wikipedia.org, accessed
  2. MQL5 blog: The robustness tests most strategies fail, walk-forward and out-of-sample explainedmql5.com, accessed
  3. NSE circular NSE/FAOP/70616: revision in market lot of derivative contracts on indices (October 3, 2025)nsearchives.nseindia.com, accessed
  4. Union Budget 2026-27, Memorandum explaining the provisions of the Finance Bill (STT on options and futures)indiabudget.gov.in, accessed
esc
↑↓Move↵Open