Workflow
The Tutorial teaches the mechanics. This page teaches the judgement: what you do at each stage, what answer counts as “pass”, and what to do when it does not.
It is a single worked example, start to finish, on a real script from the repository. It ends in a negative result — on purpose, because a workflow that only ever produces wins is not teaching you anything.
The loop at a glance
| # | Stage | The question | The gate — if this fails |
|---|---|---|---|
| 1 | Frame | What exactly am I trying to find out? | Unclear question → do not start |
| 2 | Baseline | Does the script do anything at all, untuned? | Zero or near-zero trades → fix the script, not the parameters |
| 3 | Sanity | Is the baseline’s profit an artefact of costs or of the fill assumption? | Unrealistic result → distrust everything downstream |
| 4 | Search | Is there a region of parameter space that is reliably better? | Run the optimizer |
| 5 | Triage | Is the winner a broad region or a single spike? | Single spike → not a finding |
| 6 | Stress | Does the result survive a different seed and a different period? | Dies under either → not a finding |
| 7 | Decide | Real edge, or not? | — |
| 8 | Record | What would I have to see to change my mind? | — |
Every stage can send you back. That is the normal path, not a failure.
1 — Frame the question
Bad: “Find the best EMA cross parameters.”
That is not a question, because there is no way to be wrong. Any maximum will do.
Better: “On BTCUSDT 1h over 2023–2024, is there a region of EMA-cross parameters that produces a positive, stable result after a 0.1 % round-trip cost?”
Write the question down. It determines your market, your timeframe, your period, your cost assumption — and crucially, your metric. Decide what “good” means before you look at any number, or you will choose a metric that flatters the result you already have.
Choosing an objective before you start
The optimiser maximises whatever metric you hand it. That makes the choice load-bearing and easy to get wrong:
- Profit factor — gross profit over gross loss. Sensitive to a single outlier win.
- Net profit — the most intuitive, and the most easily gamed by a long lucky streak.
- Max drawdown — not an objective on its own; it is a constraint.
- Sharpe — return per unit of volatility. Useful, but assumes a roughly normal distribution, which trade P&L is not.
A common and defensible choice is profit factor, with a minimum trade count and a maximum-drawdown ceiling applied as filters. That is deliberately boring: it is hard to fool.
2 — Baseline
Run the script once, with its shipped defaults. No optimisation.
For this example: ema_cross.pine on BTCUSDT, 1h, defaults fast=9, slow=21,
rsi=14, atr=14.
Gate: does the script fire often enough to measure?
| Trade count | Verdict |
|---|---|
| 0–3 | Useless. Loosen the entry condition before doing anything else. |
| 4–15 | Marginal — expect a wide spread in the results. |
| 20+ | Good enough to optimise. |
Why trade count gates everything downstream
Every metric here is an average over trades. With 5 trades, a profit factor of 3 means one
good trade out of five — which is a coin flip, not a strategy. Statistical noise scales as
roughly 1 / √n, so to halve the noise you need to quadruple the sample.
This is also why the sweep tooling enforces a minimum trade count in its ranking: ranking a thousand cells by profit factor with no trade-count filter is ranking by luck.
3 — Sanity
Before trusting the baseline, check that the number is not an artefact.
Three checks, thirty seconds:
- Is it strategy()-mode? If not, every number is reconstructed. See
Engines. This example is
ema_cross.pine, an indicator script — so the baseline is already in “modelled” territory. - Does the profit survive a bigger cost assumption? Double the commission in your head. If the result flips sign, it was a cost artefact.
- Does it survive removing the best few trades? A profit concentrated in two trades out of forty is not a result.
4 — Search
Now the optimiser.
| Setting | Value | Why |
|---|---|---|
| Method | TPE | The default, and the right general-purpose choice |
| Budget | 1 000 trials | Enough for a 4-dimensional space to converge |
| Seed | Fixed | Reproducibility — you will re-run this later |
| Space | The four input.* values | Discovered from the source automatically |
You should see: trials completing in parallel, and the best-objective curve rising and then flattening.
What a good best-objective curve looks like
A healthy run shows three phases:
- Rise — early trials mostly miss; the curve climbs steeply.
- Improve — TPE has enough observations to build a usable model, and the curve keeps climbing but flattens.
- Plateau — the curve goes flat. The optimiser has converged.
A curve that rises forever means the space has not converged and you should raise the budget. A curve that is flat from trial 1 means either the space is too small to matter, or the metric is not responding to these parameters.
5 — Triage
This is where most self-deception happens, and the most important stage.
Look at the Parameter Analysis section, not just the best row.
| What you see | Verdict |
|---|---|
| A broad, connected region of good results | Promising — a real basin |
| One spike, with ordinary results around it | Noise. Not a finding. |
| Several disconnected spikes of similar height | Multiple-comparisons artefact |
| Good results only at parameter-space boundaries | The optimum is outside the space — widen it |
Why one spike means nothing
Suppose every parameter set produces a result drawn from pure noise around a true value of 1.0, and run 1 000 trials. You will find trials near 2.5 — the maximum of 1 000 noisy samples. The bigger your search, the more certain you become of finding a spectacular result that means nothing.
This is why the optimiser is a tool for finding regions, not maxima. Ask “is this area of the space better than the rest?” rather than “what is the single best number?”.
6 — Stress
The step most people skip, and the one that catches the most false positives.
Run each of these and require that the region survives:
| Test | How | A result that fails this is… |
|---|---|---|
| Different seed | Same config, different seed | Fitting the random sequence |
| Different period | A market segment the run never saw | Fitted to one regime |
| Different market | A related pair | Coin-specific accident |
| Wider cost | Higher commission and slippage | Living on an unpriced edge |
7 — Decide
Three possible outcomes. All three are legitimate.
| Outcome | What it means |
|---|---|
| Edge confirmed | Survives every stress test, and the result is stable across seeds |
| No edge | The space contains nothing reliable. This is the most common real result |
| Inconclusive | Too few trades, or the evidence points both ways. Widen the evidence, not the claims |
8 — Record
Write down four things. This takes two minutes and saves the next two hours:
- The question, exactly as framed in stage 1.
- The configuration: seed, budget, method, market, period, cost assumptions.
- The result, including a negative one, with the seeds you tried.
- What would change your mind — the specific evidence you would need.
Why the negative results matter more
An optimiser can produce a positive result for almost any strategy, given enough trials. It cannot produce a reproducible one for a strategy that has none.
So a log of “this was tested, under these conditions, and it did not work” is worth more than a log of wins — because it is the only part of the process that reliably distinguishes signal from noise. Keep it.
The same loop, for the other track
The Python command-line track is the same workflow with a config file instead of a UI, and it is the right choice when a sweep is large or must be reproducible.
# 1. See the grid before you spend anything on it.
python orchestrator/sweep.py --config examples/sweep_quick.yaml --dry-run
# 2. Run it.
python orchestrator/sweep.py --config examples/sweep_quick.yaml
# 3. Rank with a floor on trade count, so luck does not rank first.
python orchestrator/rank.py --summary results/<run-id>/summary.parquet \
--metric profit_factor --min-trades 20 --out ranked.csv
# 4. Read the report.
python orchestrator/report.py --summary results/<run-id>/summary.parquet --out report.htmlThe stages map one to one: baseline is a single cell, search is the grid, triage is the ranked table with filters, and stress is a second run over a different period.
Full flag reference in CLI reference. Note the two configuration traps that make a sweep lie to you, in Workflow traps below.
Traps
Things that will make this workflow produce a confident wrong answer.
The full list of unapplied configuration
Verified against the code rather than the documentation, because the two disagree in several places:
| Setting | Actual behaviour |
|---|---|
robustness_filter (sweep) | Parsed, echoed to config.yaml, never enforced |
direction: minimize (hyperopt) | Sets the Optuna study direction; the score function still always maximises, so it does not minimise |
min_train_trades | Enforced, in hyperopt.py only |
LUXALGO_PROPFIRMS_URL | Never read — the URL is a module constant |
BACKTEST_MAX_WORKERS | Never read — use max_workers in the YAML |
BACKTEST_RESULTS_DIR | Never read — output path is fixed relative to the repo root |
unknown objective (hyperopt) | Silently falls back to sharpe |
See Environment variables and CLI reference.
Next
- Algorithms — the mechanism behind each search method.
- Limitations — the full list of what this product does not establish.