Backtesting Engine¶

Standard backtesting on higher timeframes has a fundamental flaw: signals are computed on completed bars, and fills execute at bar close. A MACD crossover on a 4-hour bar may trigger 2 hours into the bar at $49,800, but the backtest records the fill at bar close — $51,200. The error grows with timeframe: negligible on 1-minute data, $1,000+ on 4h, $5,000+ on daily (BTC-scale assets). The magnifier technique solves this by simulating intra-bar execution at sub-bar resolution, filling at the actual moment the signal fires.

Two Execution Modes¶

	Standard Mode	Magnifier Mode
Flag	`--no-magnify`	Default (TF > 1m)
Signal computation	Once over full DataFrame	Per sub-bar, progressively
Fill price	Bar close	Sub-bar close at trigger time
Speed	Very fast (single vectorized call)	Moderate (~10K compute calls for 1,000 bars)
Fill realism	Poor on higher TF	Good (fills at sub-bar resolution)
Use case	1m data, parameter sweeps, quick screening	Production backtests on 1h+ timeframes
Compute path	`_compute(df, params)` — pandas	`_compute_fast()` preferred, `_compute()` fallback

flowchart TB
    SRC["PineScript Strategy"] --> COMPILE["Compile<br/>(Tokenizer → Parser → CodeGen)"]
    COMPILE --> DECIDE{Timeframe > 1m<br/>AND magnify enabled?}

    DECIDE -->|Yes| MAG["Magnifier Mode<br/>Sub-bar iteration"]
    DECIDE -->|No| STD["Standard Mode<br/>Single vectorized call"]

    STD --> VBT["vectorbt<br/>Portfolio.from_signals()"]
    MAG --> VBT

    VBT --> RESULT["BacktestResult<br/>Stats + Equity + Trades"]

Magnifier Technique¶

Concept¶

For each chart-timeframe bar, the magnifier iterates through sub-bars at a finer resolution, progressively building a "forming" OHLCV bar. At each sub-bar tick, it runs the strategy with a window of completed bars plus the forming bar. The moment a signal triggers, the entry/exit is recorded at that sub-bar's close price — not the chart bar's close.

Chart bar (4h):     [════════════════════════════════════]
                     Open $50,000                   Close $51,200

Magnifier (15m):    [══|══|══|══|══|══|══|══|══|══|══|══|══|══|══|══]
                                  ↑
                          Tick 4: MACD crossover triggers
                          Fill at $49,800 (sub-bar close)
                          Skip remaining 12 ticks

Detailed Flow¶

sequenceDiagram
    participant B as Backtester
    participant D as DataSource
    participant M as Magnifier Loop
    participant S as Strategy (compute)
    participant V as vectorbt

    B->>D: Load 1m base data
    D-->>B: df_1m (raw candles)
    B->>B: Resample to chart TF (e.g., 4h)
    B->>B: Resample to magnifier TF (e.g., 15m)
    B->>B: Pick resolution: 4h → 15m (16 ticks)

    loop For each chart bar (after warmup)
        B->>M: Process bar [bar_start, bar_end]
        M->>M: Get magnifier ticks within bar
        M->>M: Initialize forming bar (open = first tick open)

        loop For each sub-bar tick
            M->>M: Update forming OHLCV<br/>high = max, low = min, close = tick close
            M->>M: Build window = completed bars + forming bar
            M->>S: compute(window, params)
            S-->>M: (long_entry, long_exit, short_entry, short_exit)

            alt Signal triggered on last bar of window
                M->>M: Record signal at tick index
                M->>M: Update position state (in_long / in_short)
                M->>M: Break — one signal per chart bar
            end
        end
    end

    B->>V: Portfolio.from_signals(mag_close, signals)
    V-->>B: Portfolio with realistic fills
    B->>B: Extract BacktestResult

Algorithm¶

for bar_idx in range(warmup, len(df_chart)):
    bar_start = df_chart.index[bar_idx]
    bar_end = bar_start + chart_timedelta

    # Magnifier ticks within this chart bar
    mag_start = df_mag.index.searchsorted(bar_start, side="left")
    mag_end   = df_mag.index.searchsorted(bar_end,   side="left")

    # Completed chart-TF bars as indicator context
    completed = df_chart.iloc[max(0, bar_idx - warmup*3) : bar_idx]

    # Progressive forming bar
    forming_open = df_mag["open"].iloc[mag_start]
    forming_high, forming_low, forming_vol = -inf, inf, 0.0

    for pos in range(mag_start, mag_end):
        forming_high  = max(forming_high, df_mag["high"].iloc[pos])
        forming_low   = min(forming_low,  df_mag["low"].iloc[pos])
        forming_close = df_mag["close"].iloc[pos]
        forming_vol  += df_mag["volume"].iloc[pos]

        window = concat([completed, forming_bar])
        le, lx, se, sx = strategy.compute(window, params)

        if signal_triggered(le, lx, se, sx):
            record_signal_at(pos)
            break  # One signal per chart bar

Critical: break after signal

Only one signal per chart bar is allowed. After recording a signal, the magnifier breaks out of the sub-bar loop. Without this break, the same bar could generate multiple conflicting entries and exits.

Dynamic Resolution¶

The magnifier picks a sub-bar resolution that divides evenly into the chart timeframe, targeting approximately 10 ticks per bar (maximum ~16):

Chart TF	Magnifier TF	Ticks per Bar
5m	1m	5
15m	1m	15
30m	3m	10
1h	5m	12
4h	15m	16
1d	1h	24

VALID_RESOLUTIONS = [1, 3, 5, 15, 30, 60, 240]  # minutes

def pick_magnifier_resolution(chart_tf, target_ticks=10):
    chart_min = TF_MINUTES[chart_tf]
    if chart_min <= 1:
        return "1m"
    max_ticks = int(target_ticks * 1.6)  # ~16
    best = "1m"
    for res in VALID_RESOLUTIONS:
        if res >= chart_min or chart_min % res != 0:
            continue
        ticks = chart_min // res
        if ticks > max_ticks:
            continue
        if abs(ticks - target_ticks) < best_dist:
            best = minutes_to_tf(res)
    return best

Requirement: 1m base data

The magnifier requires 1-minute candle data as the base resolution. All higher timeframes are resampled from 1m. If 1m data is unavailable, the backtester falls back to standard mode automatically.

Performance Optimization¶

Fast Path vs. Slow Path¶

Path	When Used	Per-Call Overhead
Fast (`_compute_fast`)	Strategy has a NumPy-only implementation	~0.1ms
Slow (`_compute`)	Complex strategies, fallback	~2ms

For a 4h backtest over 1,000 chart bars with 16 magnifier ticks each:

Fast path: 16,000 calls × 0.1ms = 1.6 seconds
Slow path: 16,000 calls × 2ms = 32 seconds

The PineScript compiler generates both paths when possible. The backtester attempts _compute_fast first and falls back to _compute transparently.

Performance Matrix¶

Configuration	Speed	Fill Realism	Best For
Standard, 1m	Fast	Excellent	Production (1m already has bar-level precision)
Standard, 4h	Very fast	Poor	Parameter sweeps, screening
Magnifier, 1h (fast path)	~2s	Good (5m fills)	Production backtests
Magnifier, 4h (fast path)	~4s	Good (15m fills)	Production backtests
Magnifier, 1d (fast path)	~6s	Good (1h fills)	Daily strategies

BacktestResult Model¶

The backtester extracts all results into a structured, JSON-serializable model:

Metadata¶

Field	Type	Description
`strategy_name`	`str`	From `strategy("name", ...)`
`symbol`	`str`	Traded instrument (e.g., `BTCUSDT`)
`exchange`	`str`	Data source exchange
`timeframe`	`str`	Chart timeframe (`1h`, `4h`, `1d`)
`period`	`str`	Date range (`2024-01-01 to 2025-01-01`)
`mode`	`str`	`standard` or `magnifier`
`params`	`dict`	Strategy input parameters used

Performance Metrics¶

Field	Type	Example
`total_return_pct`	`float`	`+28.82`
`annualized_return_pct`	`float`	`+34.15`
`sharpe_ratio`	`float \\| null`	`2.15`
`sortino_ratio`	`float \\| null`	`3.20`
`calmar_ratio`	`float \\| null`	`1.85`
`max_drawdown_pct`	`float`	`-12.50`

Trade Statistics¶

Field	Type	Example
`total_trades`	`int`	`28`
`win_rate_pct`	`float`	`57.14`
`profit_factor`	`float`	`1.85`
`expectancy`	`float`	`106.75`
`best_trade_pct`	`float`	`+8.32`
`worst_trade_pct`	`float`	`-4.15`
`avg_trade_pct`	`float`	`+1.03`

Time Series (sampled)¶

Field	Points	Format
`equity_curve`	~1,000	`[{timestamp, value}, ...]`
`returns`	~1,000	`[{timestamp, return}, ...]`
`drawdown_curve`	~1,000	`[{timestamp, drawdown_pct}, ...]`

Detail Records¶

Field	Type	Key Fields
`trades`	`list[TradeRecord]`	direction, entry/exit time+price, PnL, return%, duration
`orders`	`list[OrderRecord]`	timestamp, side, price, size, fees
`drawdowns`	`list[DrawdownRecord]`	peak_time, valley_time, drawdown_pct, duration

Chart Data¶

Field	Points	Purpose
`ohlcv_bars`	~5,000	OHLCV data for chart rendering
`trade_markers`	per trade	Entry/exit overlay markers on the price chart

NaN → null

Many stats return NaN when undefined (Sharpe with zero variance, Profit Factor with no losing trades). The result extractor converts all NaN and Inf values to null before JSON serialization. The frontend displays null metrics as —, never as 0.

Data Sources¶

The backtester loads data through a DataSource protocol:

class DataSource(Protocol):
    def load_1m(self, symbol: str, exchange: str,
                start: datetime, end: datetime) -> pd.DataFrame:
        """Load 1-minute OHLCV candles."""
        ...

Source	Status	Storage	Notes
`DiskSource`	Active	`.npz` files (via CCXT)	Compressed NumPy archives, fast local reads
`TimescaleSource`	Planned	PostgreSQL / TimescaleDB	Hypertable with continuous aggregates, shared across users

All higher timeframes are resampled from the 1-minute base:

def resample_ohlcv(df_1m: pd.DataFrame, target_tf: str) -> pd.DataFrame:
    rule = TF_RESAMPLE_RULES[target_tf]  # "5T", "1H", "4H", "1D"
    return df_1m.resample(rule).agg({
        "open": "first",
        "high": "max",
        "low": "min",
        "close": "last",
        "volume": "sum",
    }).dropna()

Usage Examples¶

# Standard magnifier mode (default for TF > 1m)
python -m backtest --script strategies/macd_crossover.pine --timeframe 4h

# Disable magnifier for faster parameter sweeps
python -m backtest --script strategies/supertrend.pine --timeframe 1h --no-magnify

# JSON output for programmatic consumption
python -m backtest --script strategies/rsi_overbought.pine --timeframe 1d --json

# Specify date range and initial capital
python -m backtest --script strategies/bb_squeeze.pine --timeframe 4h \
    --start 2024-01-01 --end 2025-01-01 --capital 50000

# Override strategy parameters
python -m backtest --script strategies/macd_crossover.pine --timeframe 1h \
    --param "Fast Length=8" --param "Slow Length=21"

File Map¶

Concept	File
Backtester class	`backtest/backtester.py` (`Backtester`)
Standard mode execution	`backtest/backtester.py` (`_run_standard`)
Magnifier mode execution	`backtest/backtester.py` (`_run_magnified`)
Result extraction	`backtest/backtester.py` (`_extract_result`)
BacktestResult dataclass	`backtest/strategy.py` (`BacktestResult`)
TradeRecord / OrderRecord / DrawdownRecord	`backtest/strategy.py`
Dynamic resolution picker	`backtest/backtester.py` (`pick_magnifier_resolution`)
DataSource protocol	`backtest/backtester.py`
CLI entry point	`backtest/__main__.py`