Execution

Every algorithm is driven by the same loop. Once a candidate parameter set is proposed, the question is who runs it and how many run at once.

Screenshot pending

awaiting screenshots/optimizer-config.png in public/

Optimizer configuration. Parameters, algorithm, budget and seed are captured when the run launches.

The trial loop


while (!optimizer.shouldStop()) {
  const params = optimizer.suggest()
  if (!params) break
  const v = validateValues(searchSpace, params.values)
  if (!v.ok) { reject(params, v); continue }
  const result = await backtest.evaluate(params.values)
  await commit(params, result)
}

Three things fall out of this shape:

suggest() returns one candidate, not a list. It is called once per trial.
report() only sees accepted candidates. A proposal rejected by validateValues never reaches report(), so it does not count against the trial budget.
A failed backtest is reported, not thrown. It comes back as pnl: 0 with an error string, and only pnl and error reach the algorithm.

Two execution paths

There are two ways to run a trial, and the choice is made per run:

Path	Used when	Throughput
Serial	The algorithm has no `suggestBatch`	One trial at a time, in the coordinator
Batched	The algorithm exposes `suggestBatch` and the adapter exposes `evaluateBatch`	Bounded worker pool

The batched path is only taken when both conditions hold. This is the single most surprising thing about the optimiser’s performance:

Why batched evaluation was nearly shippable-but-broken

Parallel evaluation was rejected on measurement and then shipped anyway, because the measurement was wrong.

The first benchmark reported the pool running at 0.02×–0.11× of sequential speed and concluded that parallelism was worthless. Two tells in its own output said the harness was broken, and both were visible:

Per-trial cost came out flat (0.16–0.47 ms) across a 166× range of bar counts.
The 20 000-bar row measured faster per trial than the 5 000-bar row — physically backwards.

Root cause: the benchmark passed strategySource and values, but the backtest entry point destructures source and inputs. The Pine engine received no source at all, so every trial failed in ~0.2 ms without doing any work.

The benchmark now refuses to report timings when nothing actually reached the engine. Corrected numbers, at the app’s default depth of 5 000 bars: 518 ms per trial, break-even at roughly 13 trials, best observed speedup 3.33×.

Pool sizing

The pool size is calculated, not guessed.

Setting	Value	Purpose
`uiCoreShare`	`0.15`	Reserve 15 % of the machine for the UI thread
`hardWorkerBackstop`	`64`	Corruption guard for hosts reporting absurd thread counts
`minBarsForPool`	`1500`	Below this many bars, parallelism cannot pay for itself
`maxBatchPerWorker`	`16`	Trials dispatched per round
`poolMemoryShare`	`0.30`	Cap the pool at 30 % of the memory budget
`bytesPerBar`	`160`	Estimated bytes per candle held in a worker
`safetyFactor`	`1.25`	Headroom on every estimate
`deviceMemoryCapGiB`	`8`	Never assume more than 8 GiB
`memoryBudgetDefaultBytes`	2 GiB	Used when the device does not report memory

The CPU ceiling is:


cores - max(1, ceil(cores × 0.15))

Why not a fixed worker count

A fixed 1-of-4 reservation looks reasonable on a four-core machine and is badly wrong on a workstation: 25 % of 4 is 1 core; 25 % of 24 is 6 cores. Reserving a fraction rather than a count keeps the intent — “leave the UI responsive” — true on both.

Two deliberate exceptions:

A manual worker count is not capped by the CPU ceiling. The app does not second-guess an explicit choice.
hardWorkerBackstop: 64 is not a policy cap. It is a guard against hosts reporting nonsense thread counts.

The memory estimator was wrong by up to 4×

The original model estimated worker memory as 48 MB + 160 B/bar. It had never been measured, and it drifted.

When someone finally measured it, the first attempt produced negative marginal memory per bar — because it sampled after terminate(), i.e. it was measuring memory that had already been reclaimed. The failure mode is silent and plausible: a negative number looks like a bug, but a plausible-looking one does not.

The estimator now carries a fixed per-worker runtime overhead and a separate per-bar runtime cost, and the device-memory reading is handled defensively — some environments report a fractional, absent or absurd deviceMemory, and the two code paths (reported vs. defaulted) are kept identical so the two answers cannot drift apart.

The depth gate

Parallelism is only offered when the run is long enough to amortise worker startup.


bars < 1500  →  sequential
bars ≥ 1500  →  pool available

Determinism under parallelism

Batching is where seeded reproducibility usually breaks, and this is the invariant that guards it:

The mechanism is that the optimizer and all seeded state live in the coordinator; workers are stateless with respect to the search. Batch boundary, generation advance, and RNG ownership never cross into a worker.

What is deliberately not enabled

Design rules that are not negotiable

Rule	Why
The optimizer never enters a worker	A search in a worker cannot be cancelled or snapshotted
`eval-pool.js` must never enter the main-thread bundle	A pool reachable from the main thread is a pool that can freeze the tab
`market` stays `{ symbol, timeframe }`	Adding a `symbolInfo` field activates a module-level, order-populated timezone Map inside the vendor bundle
Each worker builds its own engine and cache	Sharing would reintroduce cross-talk between trials
A batch never spans a generation boundary	The silent-determinism failure above

Cancellation and backpressure

Cancel terminates only the run’s own worker. Completed trials are retained; the in-flight evaluation is discarded.
The worker is deliberately not terminated on cancel. It must stay alive to answer on-demand trial detail requests from the Analysis surface.
Progress frames are not persisted. They are transient UI state.

Two things that look like bugs and are not

runner.flush() does not exist. A workspace smoke originally asserted against it, having carried over a contract from the upstream toolchain that this implementation never had.
The worker survives cancel. Terminating it on cancel looks like tidiness and breaks the Analysis detail path, which asks a live worker for one trial’s full data on demand.

Algorithms — what each search strategy actually does.
Search space — how parameters become dimensions.