Skip to Content
Limitations

Limitations

Read this before drawing a conclusion from anything in this documentation. A backtesting tool that does not say what it cannot show you is not worth much.

What the results do not establish

Offline execution does not establish TradingView parity. Pine semantics in this environment are a transpiler’s interpretation. Complex scripts can behave differently from TradingView even when they compile.

It does not establish live-market accuracy. The fixtures are synthetic or historical, and no claim is made about how results transfer to a live feed.

It does not establish a trading edge. A good-looking parameter set found by searching thousands of configurations is, if anything, evidence of overfitting. The optimizer is a search tool, not a strategy generator.

It does not establish Pine compatibility for arbitrary complex strategies. The four shipped scripts are verified. Your new one is not, until you run it. See Architecture decisions.

These are one limitation, not four

Parity is a claim about two implementations agreeing on the same inputs. An offline run gives you one implementation and one fixture feed — a transpiler’s interpretation of Pine, over synthetic or historical data. There is no observation inside that run which could separate a correct reading of a script from a plausible one, which is why the first three statements cannot be discharged by running the code harder.

The fourth narrows the same gap to your own script. Four shipped scripts are verified; the one you write is a draft until you run it, and that run tells you it works in this environment.

The fills caveat

The single most important limitation is not a bug — it is a property of Pine. Indicator-mode scripts produce markers, not fills, so their PnL is reconstructed rather than simulated. Only one of the four shipped scripts is in strategy() mode. See Engines — real fills vs synthetic.

Three of the four shipped strategies are indicator-mode, and rsi_strategy.pine — the one real-fills script — does not declare its costs. Every number this project produces deserves a discount until you have checked which of the two you are looking at.

No shipped script gives you both

The two problems are disjoint. Three of the four shipped scripts are indicator-mode, so their PnL is reconstructed from plotted signals rather than simulated. rsi_strategy.pine is the one script with real fills, and it is the one that declares no costs. Between them, no script that ships today gives you a simulated position with a declared cost model.

So a metric you are looking at is either reconstructed or cost-free, and which one is decided by the mode of the script in front of you. The indicator-mode half is a property of Pine, and no setting removes it — the route to real fills is a strategy() script. The missing costs are a property of a script, so the fix belongs in the script. See Engines — real fills vs synthetic.

Configuration that is parsed but not applied

WhereWhat happens
robustness_filter in sweep.pyParsed and written back to config.yaml, but never applied
direction: minimize in hyperopt.pySets the Optuna study direction, but the score function always maximises — so it does not actually minimise

These are honest landmines rather than bugs in disguise: they look like they work. See Environment variables and CLI reference.

Optimizer sampling limits

  • Seed coverage is shallow. The recorded baseline is five TPE trials at seed 42. That covers startup sampling, not mature convergence, and it is not every algorithm or strategy.
  • Grid and Brute pass their engine tests, but their full UI runs were not separately repeated.
  • No 10 000-trial stress run has been performed in the browser.
  • The inherited TPE-versus-random quality comparison has varied between runs. It passed the most recent run; its reproducibility concern remains open.

Verification that was not performed

  • No screen-reader session.
  • No physical touch-device test. Narrow-layout checks used desktop Chrome viewport emulation.
  • No blocked-popup browser session.
  • No exhaustive malformed legacy-job test.
  • No forced GPU-failure test.
  • No exhaustive large-job chart-performance or long-duration memory-leak test.
  • IndexedDB quota failure and delayed-write ordering were tested with controlled adapters, not by exhausting the browser’s real storage quota.
  • Representative contrast pairs were checked; not every inherited chart, editor or detached state was.

Deliberately unfixed failures

The worker context-shape assertion. app/__smoke_worker_real_run.cjs expects ctx.trades from getContext("main"); production uses an explicit selection array. The optimizer calculations were not changed to satisfy it, and the assertion was not weakened. It is reported separately.

Six broken hyperopt.py tests. They reflect API drift against the installed Optuna version. Reported, not papered over.

This is deliberate. A test edited until it passes tells you nothing, and an optimizer changed until an outdated assertion agrees with it is worse than a red suite.

Why the red tests stay red

Both are drift, and drift has no honest fix that does not change what is being measured. The smoke assertion expects a context shape that production no longer uses; it was inherited from an earlier code path. The hyperopt tests were written against an Optuna API that the installed version has moved past.

The two available edits are the same in both cases: bend the production code until the old assertion agrees, or weaken the assertion until it stops objecting. The first changes behaviour to satisfy a test, the second makes the test say nothing, and both leave the next person with a green suite and no information. Leaving them red keeps the record.

Coverage gaps in the hygiene test

test_license_hygiene.py scans app, orchestrator, pinets-cli, propfirm and tests. The pinets-cli entry uses a hyphen, while the directory on disk is pinets_cli with an underscore — so that path matches nothing and contributes no coverage.

The check is also an approximation by design: it greps for the distinctive tokens PineTS and PineTSContext, which catches accidental vendoring but is not a licence audit.

Deferred by design

  • Live market integration.
  • Python hyperopt as a replacement for the browser optimizer — it is a separate, slower track today.
  • Storage migrations (explicit job links currently resolve without one).
  • Multi-asset portfolio aggregation.
  • Live order execution, which is a permanent non-goal rather than a deferral.

Documentation caveat

This site describes the post-Phase-F application. The repository’s own README.md and docs/architecture.md describe an earlier phase and disagree with the code in places — including the dataset-hash invariant discussed in Architecture decisions. Where they conflict, the code and this site are correct.

The application UI is also still being polished, so no screenshots are committed yet; figure slots render as labelled placeholders until captures land.

Bottom line

Use this to explore whether a strategy family is worth more of your time. Do not use it to decide what to trade, and do not treat a good optimizer score as evidence of anything.

Last updated on