# 05. Optimization, tags, and sweeps

Take a working strategy from a single-day backtest to a 700-combo sweep without melting your CPU. You will use [`@day_start`](https://quantchartsllc.com/docs/python/py-decorators.md#day_start), `required_columns`, and tags to control the analyzer's parallelism story end-to-end.

Sweeps run the strategy once per parameter combination per day. A naive strategy that does heavy work inside `calculate()` pays for that work `combos x days` times. The fix is hoisting: anything that does not depend on a parameter goes in `@day_start`, runs once per day per worker, and is reused across every combo. This walkthrough takes the EMA crossover from tutorial 02 and rewrites it for sweep performance.

<a id="step-1-the-five-rules-of-hoisting"></a>
## Step 1. The five rules of hoisting

Read this before writing optimization code. These rules are also documented inline in the bundled [`optimize.py`](https://quantchartsllc.com/docs/templates/templates.md#optimize-py) template.

1. Convert columns to numpy ONCE per day. Bad: `np.asarray(df["close"])` inside `calculate()`. Good: do it inside `@day_start` and read `self._day["close"]`.

2. Run literal-period TA ONCE per day. Bad: [`ta.atr(high, low, close, 14)`](https://quantchartsllc.com/docs/python/py-ta.md#ta-atr) inside `calculate()`. Good: same call inside `@day_start`; reuse `self._day["atr14"]`.

3. Run swept-period TA inside `calculate()` ONLY for the param being swept. Keep all other periods literal and hoisted.

4. Cache day-level scalars (vol_mean, regime label, max_drawdown) in `@day_start`. They are recomputed once per combo otherwise.

5. Use `required_columns=[...]` so SHM only ships what you read. On a 28-day sweep this saves 30-70% of inter-process bandwidth.

<a id="step-2-the-day_start-hook"></a>
## Step 2. The @day_start hook

Decorate a method with `@day_start`. The runner calls it once per day per worker BEFORE the param loop and stashes the return value on `self._day`.

### Example

```python
from quant_charts import day_start

class EMACrossOptimized:
    @day_start
    def prep(self, df):
        close = np.asarray(df["close"], dtype=np.float64)
        high = np.asarray(df["high"], dtype=np.float64)
        low = np.asarray(df["low"], dtype=np.float64)
        atr14 = ta.atr(high, low, close, 14)
        median_atr = float(np.median(atr14[np.isfinite(atr14)]))
        return {
            "close": close,
            "atr14": atr14,
            "median_atr": median_atr,
        }
```

### Notes

- The method name does not matter (here `prep`). Only the `@day_start` decorator does.
- Whatever you return becomes `self._day` in `calculate()`. Use a dict so multiple precomputed arrays/scalars travel together.
- Keep heavy literal-period TA in here (`ta.atr(..., 14)`, [`ta.sma(..., 200)`](https://quantchartsllc.com/docs/python/py-ta.md#ta-sma)). Move only swept-period TA into `calculate()`.

<a id="step-3-the-full-optimized-strategy"></a>
## Step 3. The full optimized strategy

EMA crossover with hoisted ATR-regime filter. Save as `ema_cross_opt.py`.

### Example

```python
import numpy as np
from quant_charts import (
    strategy, day_start, input, ta, cross_above, cross_below,
    define_tag, Timeframe,
)


@strategy(
    name="EMA Cross Optimized",
    description="EMA crossover with ATR-regime filter, hoisted for sweeps.",
    overlay=True,
    timeframe=Timeframe.M1,
    data_mode="ohlc",
    required_columns=["high", "low", "close"],
    uses_sltp=False,
)
class EMACrossOptimized:
    fast = input.int(9, "Fast EMA", min=2, max=200)         # SWEPT
    slow = input.int(21, "Slow EMA", min=2, max=400)        # SWEPT
    regime_mult = input.float(1.5, "ATR Regime Mult",
                              min=0.0, max=10.0, step=0.1)  # SWEPT

    @day_start
    def prep(self, df):
        close = np.asarray(df["close"], dtype=np.float64)
        high = np.asarray(df["high"], dtype=np.float64)
        low = np.asarray(df["low"], dtype=np.float64)
        atr14 = ta.atr(high, low, close, 14)
        finite = atr14[np.isfinite(atr14)]
        median_atr = float(np.median(finite)) if finite.size else float("nan")
        return {"close": close, "atr14": atr14, "median_atr": median_atr}

    def calculate(self, df):
        close = self._day["close"]
        atr14 = self._day["atr14"]
        median_atr = self._day["median_atr"]

        # Only the swept-period TA runs here. Two EMA calls.
        fast = ta.ema(close, self.fast)
        slow = ta.ema(close, self.slow)

        # Regime filter: trade only when current ATR is materially above the
        # day's median (avoids sleepy chop). Vectorized.
        in_regime = atr14 > median_atr * self.regime_mult

        bull_cross = cross_above(fast, slow)
        bear_cross = cross_below(fast, slow)

        define_tag("in_regime", "ATR above day median * mult", color="#22c55e")
        define_tag("out_regime", "ATR below regime threshold", color="#a1a1aa")

        return {
            "entry_long":  bull_cross & in_regime,
            "exit_long":   bear_cross,
            "entry_short": bear_cross & in_regime,
            "exit_short":  bull_cross,
            "in_regime":   in_regime,
            "out_regime":  ~in_regime,
        }
```

<a id="step-4-set-up-the-sweep-in-the-analyzer"></a>
## Step 4. Set up the sweep in the analyzer

The analyzer runs the strategy across a parameter grid in parallel.

1. Open the Analyzer tab from the chart top bar.
2. Mode: Strategy. Strategy: pick "EMA Cross Optimized".
3. Date range: pick the days you want to test (multi-select).
4. Parameter form: each `input.*` field shows up. For each one you want to sweep, switch from "single value" to "range" and set start/stop/step. The analyzer computes the cartesian product unless you switch to fixed-N mode.
5. Click Run. Watch the progress bar; the result table populates as days complete.

### Notes

- `required_columns=["high", "low", "close"]` saves shipping [`volume`](https://quantchartsllc.com/docs/python/py-data.md#volume), [`delta`](https://quantchartsllc.com/docs/python/py-data.md#delta), etc. across the worker boundary on every combo.
- The analyzer reuses the `@day_start` output across every combo for that day, so a 700-combo grid pays the ATR cost ONCE per day.

<a id="step-5-read-the-results"></a>
## Step 5. Read the results

Three views matter: the leaderboard, the per-tag stats, and the 2D scatter.

Leaderboard: ranked rows by total PnL (or Sharpe, sortino, profit factor — toggle in the column picker). Click any row to load that exact (param, day) combo onto the chart.

Per-tag stats panel: shows PnL, win rate, trade count for each tag separately. With this strategy you can see how trades tagged `in_regime` performed vs trades tagged `out_regime` (which by definition have zero entries here, but the panel layout is the same).

2D scatter: pick two parameters from the dropdowns; each dot is one combo. Hover for stats. The scatter exposes parameter-space topography that the leaderboard hides.

3D scatter: same idea with three axes. Useful when sweep dimensionality > 2.

<a id="step-6-preventive-tag-filtering"></a>
## Step 6. Preventive tag filtering

The analyzer can re-run a sweep with a tag filter applied at entry time, not just post-hoc. Same Rust engine, no Python re-execution.

Once you have a sweep with tags, click any tag chip in the per-tag panel. The analyzer rebuilds a `trading_mask` from that tag's pre-computed boolean arrays and re-runs through Rust. Result: a new column on the leaderboard showing what each combo would have done if it had ONLY traded inside that tag's True regions.

This differs from a post-hoc filter (which just hides trades on display) because the position constraint matters: with only one position open at a time, removing some entries cascades into different exits. Preventive filtering simulates that cascade correctly.

### Notes

- See "Preventive Tag Filtering" in the Features tab for the full architecture details.
- Tag-filtered sweeps reuse the original signal bundles, so they cost milliseconds per combo, not seconds.