Market Data Emulator — synthetic OHLCV + L2 orderbook + funding

market-data-emulator is a Python market-data emulator that generates realistic, reproducible OHLCV series for a single symbol or for a bundle of 1..8 correlated symbols. It is a tool for backend testing, strategy backtesting, risk-engine load testing, and ML training data preparation. There is no connection to real exchanges and no attempt to build a convincing microstructure model — this is a controlled synthetic source with a known mathematical model.

The base price model is Ornstein-Uhlenbeck with mean-reversion, drift, and volatility driven by a scenario schedule. OHLCV is built on a 1-minute base timeframe, then deterministically resampled to any target timeframe (1m, 5m, 1h, 1d, 1w, 1mo, plus arbitrary <N><unit>), and finally clipped by market invariants (tick/lot, OHLC invariants, volume rules).

The current release ships four surfaces that make backtesting tractable under realistic conditions:

Multi-pair correlated klines — 1..8 symbols in a single request, Cholesky correlations, shared regime schedule, dedicated symbol column per row.
Funding profile — perpetual-swap funding_history block plus per-bar funding row.
Orderbook L2 per bar — per-bar OrderbookSnapshot from liquidity_profile, usable for both matching simulation and ML features.
Multi-file jsonset export — canonical (topic, symbol) layout with meta.json + manifest.json, clean for downstream consumers with separate indexing.

All four surfaces are in production (see README and regression/expected_hashes.json — the single-pair path remains byte-identical; the DE-029..DE-032 umbrella tasks are closed in Issues/).

The library runs in three modes: standalone CLI (market-emulator), Python API (from market_emulator import Emulator), and HTTP API (FastAPI + uvicorn). All three modes go through the same core/pipeline_orchestrator.py — one path means today's pytest and tomorrow's --out data.json produce the same result.

Quick start

Minimum scenario: install the emulator, generate 1000 1-minute candles, verify determinism.

pip install -r requirements.txt
pip install -e .

market-emulator generate --timeframe 1m --count 1000

market-emulator generate --timeframe 5m --duration 30d --seed 42

market-emulator generate --timeframe 1h --count 720 \
    --format csv --out data.csv

Alternative installation paths (via pyproject.toml, dev dependencies, Docker) — see §Installation.

from market_emulator import Emulator, GenerateRequestV1, to_dataframe
from market_emulator.contracts.enums import Timeframe
from market_emulator.contracts.horizon import HorizonCount

emulator = Emulator()
request = GenerateRequestV1(
    timeframe=Timeframe.FIVE_MINUTES,
    horizon=HorizonCount(count=1000),
    seed=42,
)
result = emulator.generate(request)
df = to_dataframe(result)

print(df.head())
print(f"Candles: {result.meta.normalized.count}")
print(f"Fingerprint: {result.meta.fingerprint}")

The emulator targets Python 3.11+. The GenerateRequestV1 contract is stable across minor releases within the 0.4.x series.

What it is and what it is not

The emulator does:

Generate realistic OHLCV series on an Ornstein-Uhlenbeck process with mean-reversion and scenario-driven drift/volatility.
Support 1..8 correlated symbols per request, with Cholesky-factorised correlation matrix and a shared regime schedule.
Arbitrary <N><unit> timeframes (90m, 2h, 3w, 3mo) — deterministic resampling from the 1-minute base.
71 built-in scenarios, 11 named overlay profiles, 20 composable overlay patterns, 8 truth scenarios.
Per-bar OrderbookSnapshot L2 from liquidity_profile and per-bar funding row from funding_profile for perpetual-swap styles.
Deterministic byte-identical output for three writers: JSON, CSV, JSONL, plus a multi-file jsonset format (meta.json + manifest.json + (topic, symbol)/...json).
Truth Dataset Contract: SHA256 hash of raw data, sanity checks, scenario-specific invariants, export_truth_* serializers.
CLI (market-emulator), Python API, HTTP API (FastAPI + uvicorn).
Batch plans in YAML for generating a series of datasets in one call.

The emulator does not:

Connect to real exchanges or replay real ticks. It is a synthetic source, not a replay tool.
Provide order matching, market-impact simulation, or slippage calculation. The OrderbookSnapshot L2 is generated as a feature, not as a matching result.
Cover options, margined futures, or perpetual-swap funding arbitrage. funding_profile produces metadata (funding_history + per-bar funding), not a virtual wallet.
Run backtests. Generated candles are inputs, not an order manager.
Replace a CI framework. Determinism is guaranteed per (seed, scenario, version); it is useful for integration tests, but it is not an assertion utility.

Requirements

Component	Version
Python	3.11 (minimum; `pyproject.toml` declares `requires-python = ">=3.11"`)
pydantic	>= 2.0
numpy	>= 1.24
pandas	>= 2.0
fastapi	>= 0.100
typer	>= 0.9
pyyaml	>= 6.0
uvicorn	>= 0.20 (optional, for HTTP API)

Dev dependencies (pip install -e ".[dev]"): pytest, pytest-xdist, pytest-cov, hypothesis, httpx, jsonschema, scikit-learn, ruff, mypy, bandit, pip-audit, build.

CI gates: pytest --cov-fail-under=90 (minimum 90% coverage), ruff (E/F/I — E501 ignored), mypy (advisory baseline, does not fail the build), bandit (skip B101).

Architecture

The emulator is a pipeline with explicit stages and isolated RNG streams. The orchestrator (core/pipeline_orchestrator.py) dispatches on req.multi_pair: the single-pair path runs byte-identically, the multi-pair path runs per-symbol OU with shared Cholesky innovations.

Input Request
    ↓
Dispatch on req.multi_pair
    ↓   ┌──────────────────────────────────────────────┐
    ├──>│ single-pair path (req.multi_pair is None)    │
    │   │   Time Grid (1m base)                       │
    │   │       ↓                                     │
    │   │   Price Process (OU)                        │
    │   │       ↓                                     │
    │   │   OHLCV Construction                       │
    │   │       ↓                                     │
    │   │   Resampling (1m → target)                  │
    │   └──────────────────────────────────────────────┘
    │   ┌──────────────────────────────────────────────┐
    └──>│ multi-pair path (req.multi_pair is set)      │
        │   Time Grid (1m base, shared)               │
        │       ↓                                     │
        │   Cholesky innovations (shared)             │
        │       ↓                                     │
        │   Per-symbol OU (shared regime schedule)    │
        │       ↓                                     │
        │   Per-symbol OHLCV Construction (rows       │
        │     carry `symbol`)                          │
        │       ↓                                     │
        │   Resampling (1m → target)                  │
        └──────────────────────────────────────────────┘
    ↓
Optional per-bar orderbook (when liquidity_profile is set)
    ↓
Optional per-bar funding    (when funding_profile is set)
    ↓
Constraints (tick/lot, OHLC invariants)
    ↓
Export (JSON / CSV / JSONL / jsonset)
    ↓
Output

Key design decisions:

1-minute base timeframe. All generation runs on 1m, then resamples. This eliminates drift between long-TF and short-TF pipelines.
Constraints after resampling. Tick/lot rounding and OHLC invariants are applied last, so the base model stays continuous.
Isolated RNG streams. Each pipeline stage receives its own seed, so one stage can be swapped without reshuffling the others.
Multi-pair opt-in. GenerationSpec.multi_pair is None for single-pair. When None, the single-pair path runs byte-identically (regression hashes do not move).

The canonical architecture source is docs/architecture.md and docs/architecture/ARCHITECTURE_MAP.md. Multi-pair design rationale: docs/adr/0006-multi-pair-derivatives-design.md.

Installation

The base scenario (via requirements.txt) is already covered in §Quick start. This section covers alternative paths and the full dev environment.

Via `pyproject.toml`

# Base install
pip install -e .

# Dev dependencies (pytest, ruff, mypy, bandit, hypothesis, ...)
pip install -e ".[dev]"

Dev environment via `requirements-dev.txt`

pip install -r requirements.txt -r requirements-dev.txt
pip install -e .

Docker

Multi-stage Dockerfile on python:3.12-slim:

docker build -t market-data-emulator:local .
docker run --rm market-data-emulator:local \
    market-emulator generate --timeframe 5m --count 1000 --seed 42

docker-compose.yml brings up a service with a mounted data/ directory and pre-built scenarios.

CLI

# 1000 1-minute candles
market-emulator generate --timeframe 1m --count 1000

# 30 days of 5-minute data
market-emulator generate --timeframe 5m --duration 30d --seed 42

# CSV export
market-emulator generate --timeframe 1h --count 720 \
    --format csv --out data.csv

# Arbitrary timeframes <N><unit>
market-emulator generate --timeframe 90m --count 100
market-emulator generate --timeframe 2h  --count 24
market-emulator generate --timeframe 3mo --count 12

# Specific scenario
market-emulator generate --timeframe 5m --count 2000 \
    --scenario crash_then_recover --seed 42

# Funding profile (perpetual-swap metadata)
market-emulator generate --timeframe 1m --count 240 --seed 42 \
    --scenario baseline_mix \
    --config funding_profile='{"pattern": "extreme_positive", "magnitude": 0.001}'

# Liquidity profile (per-bar orderbook L2)
market-emulator generate --timeframe 1m --count 240 --seed 42 \
  --scenario low_liquidity_silent \
  --config liquidity_profile='{"levels_per_side": 10, "base_depth_usd": 500.0, "depth_shape": "power_law", "spread_bps_mean": 20.0, "spread_bps_std": 4.0}'

# List registered scenarios
market-emulator generate --list-scenarios

# Batch plan (multiple datasets from a YAML)
market-emulator batch --plan examples/batch_plan.yaml \
    --now 2026-01-31T00:00:00Z

funding_extreme_positive_240, funding_extreme_negative_240, and correlated_xrp_btc_eth are preset profiles registered in src/market_emulator/scenarios/profile_registry.py. Compose via MarketProfileRegistry.get_overlay(name) or via --config funding_profile='{...}'; both paths are equivalent.

Python API

from market_emulator import Emulator, GenerateRequestV1, to_dataframe
from market_emulator.contracts.enums import Timeframe
from market_emulator.contracts.horizon import HorizonCount

emulator = Emulator()
request = GenerateRequestV1(
    timeframe=Timeframe.FIVE_MINUTES,
    horizon=HorizonCount(count=1000),
    seed=42,
)
result = emulator.generate(request)

df = to_dataframe(result)
print(df.head())

print(f"Candles: {result.meta.normalized.count}")
print(f"Scenario: {result.meta.scenario_name}")
print(f"Fingerprint: {result.meta.fingerprint}")

Emulator is the primary entry point. GenerateRequestV1 is the canonical Pydantic v2 input schema. to_dataframe adapts to pandas. The result metadata (result.meta.*) exposes fingerprint, sanity checks, scenario flags, and truth-dataset artifacts.

Multi-pair generation

1..8 correlated symbols in a single request via MultiPairSpec. Under the hood, req.multi_pair routes to the multi-pair path with Cholesky innovations; when None, the single-pair path runs byte-identically.

from market_emulator import Emulator, GenerateRequestV1, to_dataframe
from market_emulator.contracts.enums import Timeframe
from market_emulator.contracts.horizon import HorizonCount
from market_emulator.contracts.spec import (
    CorrelationMatrix,
    MultiPairSpec,
    SymbolSpec,
)

emulator = Emulator()

# BTC, ETH, XRP with the canonical correlations (ADR-0006)
request = GenerateRequestV1(
    timeframe=Timeframe.ONE_MINUTE,
    horizon=HorizonCount(count=240),
    seed=42,
    multi_pair=MultiPairSpec(
        symbols=[
            SymbolSpec(symbol="BTCUSDT"),
            SymbolSpec(symbol="ETHUSDT"),
            SymbolSpec(symbol="XRPUSDT"),
        ],
        correlation_matrix=CorrelationMatrix(
            symbols=["BTCUSDT", "ETHUSDT", "XRPUSDT"],
            matrix=[
                [1.00, 0.85, 0.55],
                [0.85, 1.00, 0.50],
                [0.55, 0.50, 1.00],
            ],
        ),
        shared_regime_schedule=True,
    ),
)

result = emulator.generate(request)
df = to_dataframe(result)  # rows carry a `symbol` column

mp = result.meta.multi_pair
print(f"symbols: {mp.symbols}")
print(f"bundle fingerprint: {mp.bundle_fingerprint}")

The preset correlated_xrp_btc_eth is registered as an overlay profile in src/market_emulator/scenarios/profile_registry.py. Compose it on top of any base scenario via MarketProfileRegistry.get_overlay("correlated_xrp_btc_eth") or inline via MultiPairSpec — both paths are equivalent.

Truth Dataset Contract

The "verifiable datasets" contract: SHA256 of raw data, sanity checks, scenario-specific invariants.

from market_emulator import Emulator, GenerateRequestV1
from market_emulator.contracts.enums import Timeframe
from market_emulator.contracts.horizon import HorizonCount

emulator = Emulator()
request = GenerateRequestV1(
    timeframe=Timeframe.ONE_MINUTE,
    horizon=HorizonCount(count=1000),
    seed=42,
    scenario='baseline_mix',  # triggers artifacts
)

result = emulator.generate(request)

print(f"Raw hash (SHA256): {result.meta.raw_data_hash}")
print(f"Sanity checks: {result.meta.dataset_sanity_checks}")
print(f"Truth version: {result.meta.truth_dataset_version}")

# Determinism verification
result2 = emulator.generate(request)
assert result.meta.raw_data_hash == result2.meta.raw_data_hash
print("✓ Determinism verified")

The canonical contract is docs/contracts/truth-dataset.md. Truth exporters live in src/market_emulator/exporters/truth_exporter.py (export_truth_json, export_truth_meta_json, export_truth_jsonl, export_truth_csv). All four writers produce byte-identical output for a given (seed, scenario, version) triple.

Output formats

Format	CLI	File layout
`json`	`--format json --out data.json`	Self-contained: data + metadata in one file
`csv`	`--format csv --out data.csv`	`data.csv` + `data.meta.json` (sidecar)
`jsonl`	`--format jsonl --out data.jsonl`	Streaming-friendly: `data.jsonl` + `data.meta.json`
`jsonset`	`--format jsonset --out-dir <dir>`	Multi-file: `meta.json` + `manifest.json` + `(topic, symbol)/<symbol>.json`

jsonset (canonical layout, shipped):

<dir>/
├── meta.json                  # overall metadata
├── manifest.json              # topic → file index
├── ohlcv/<symbol>.json        # per-symbol OHLCV
├── funding/<symbol>.json      # per-symbol funding history
├── orderbook/<symbol>.json    # per-symbol orderbook L2
└── truth/<symbol>.json        # truth target (truth scenarios only)

Implementation: src/market_emulator/exporters/jsonset_exporter.py. Test matrix: tests/test_jsonset_cli.py, tests/test_jsonset_exporter.py, tests/test_jsonset_schema_conformance.py.

Example responses

Real outputs from examples/. All examples use seed=42, --now 2026-01-31T00:00:00Z, BTC-like profile.

JSON (`--format json`)

Command:

market-emulator generate \
    --timeframe 1m --count 5 --seed 42 \
    --now 2026-01-31T00:00:00Z --format json \
    --out data.json

data.json (a single self-contained file with metadata and candles):

{
  "meta": {
    "generation_id": "b81d494e-25da-4909-a3ac-0e54926d2a8a",
    "timeframe": "1m",
    "normalized": {
      "start": "2026-01-30T23:55:00Z",
      "end": "2026-01-31T00:00:00Z",
      "count": 5,
      "step_seconds": 60,
      "step_kind": "fixed"
    },
    "seed": 42,
    "profile_used": {
      "symbol_profile": "btc_like",
      "price_start": null,
      "price_precision": null,
      "tick_size": null,
      "lot_size": null
    },
    "market_used": {
      "market_regime": "mix",
      "regime_schedule": [],
      "market_session": "us"
    },
    "format": "json",
    "schema": "ohlcv",
    "include": ["ohlcv"],
    "warnings": [],
    "debug": {
      "regimes_summary": {"trend_up": 5},
      "price_preview": {
        "first": 50000.00000000001,
        "last": 50888.80646166614,
        "count": 5
      },
      "constraints": {
        "tick_size": 0.01,
        "lot_size": 0.0001,
        "precision": 2,
        "min_notional": 5.0,
        "repair": {
          "high_adjusted": 0,
          "low_adjusted": 0,
          "low_clamped_to_tick": 0,
          "high_low_swapped": 0
        }
      }
    },
    "base_timeframe": null,
    "target_timeframe": null,
    "resampled": false,
    "dropped_partial_windows": 0,
    "scenario_name": null,
    "scenario_version": null,
    "scenario_hash": null,
    "engine_version": null,
    "meta_hash": "4186f70a87513395ce4aaaf7032cd47002adc228e3e706a21ca76bc6f2afa26d",
    "data_hash": "9e7479605c9fe81dbcd5e9295d9a52b9f4513a8310076b6534f70ca2db517252",
    "fingerprint": "36e4dc7fd7c4435d442aa76540117858f63de1aa72b569ebc88ab77c81d3019d",
    "truth_dataset_version": null,
    "raw_data_hash": null,
    "dataset_sanity_checks": null,
    "emulator_artifacts": null,
    "funding_history": null,
    "orderbook_context": null,
    "multi_pair": null,
    "partial_last_bar": null,
    "analyzer_meta_hash": null
  },
  "data": [
    {
      "ts_open": "2026-01-30T23:55:00Z",
      "ts_close": "2026-01-30T23:56:00Z",
      "open": 50180.28,
      "high": 51138.92,
      "low": 48548.44,
      "close": 50000.0,
      "volume": 273.9469
    },
    {
      "ts_open": "2026-01-30T23:56:00Z",
      "ts_close": "2026-01-30T23:57:00Z",
      "open": 50000.0,
      "high": 50792.6,
      "low": 49130.56,
      "close": 50524.22,
      "volume": 251.9564
    },
    {
      "ts_open": "2026-01-30T23:57:00Z",
      "ts_close": "2026-01-30T23:58:00Z",
      "open": 50524.22,
      "high": 51719.0,
      "low": 48550.17,
      "close": 51405.81,
      "volume": 376.1741
    },
    {
      "ts_open": "2026-01-30T23:58:00Z",
      "ts_close": "2026-01-30T23:59:00Z",
      "open": 51405.81,
      "high": 54167.23,
      "low": 49056.3,
      "close": 52610.45,
      "volume": 513.139
    },
    {
      "ts_open": "2026-01-30T23:59:00Z",
      "ts_close": "2026-01-31T00:00:00Z",
      "open": 52610.45,
      "high": 55316.8,
      "low": 50184.69,
      "close": 50888.81,
      "volume": 558.4578
    }
  ]
}

meta.fingerprint is a SHA256 of the configuration (excluding generation_id and generated_at), used as a key for caching and regression checks. meta.data_hash is the SHA256 of the candle data itself. meta.regimes_summary shows the per-regime bar distribution (trend_up: 5 means all 5 bars fall inside a trend-up phase).

CSV (`--format csv`)

Command:

market-emulator generate \
    --timeframe 5m --count 3 --seed 42 \
    --now 2026-01-31T00:00:00Z --format csv \
    --out data.csv

data.csv:

ts_open,ts_close,open,high,low,close,volume
2026-01-30T23:45:00Z,2026-01-30T23:50:00Z,50180.28,55316.8,48548.44,50888.81,1973.6744
2026-01-30T23:50:00Z,2026-01-30T23:55:00Z,50888.81,53366.41,47545.55,51518.33,1886.6701
2026-01-30T23:55:00Z,2026-01-31T00:00:00Z,51518.33,58363.88,49798.85,54907.27,2010.3284

data.meta.json (sidecar, same meta structure as the JSON output):

{
  "generation_id": "65b257a0-5642-4d65-981c-51688fec8fc5",
  "timeframe": "5m",
  "normalized": {
    "start": "2026-01-30T23:45:00Z",
    "end": "2026-01-31T00:00:00Z",
    "count": 3,
    "step_seconds": 300,
    "step_kind": "fixed"
  },
  "seed": 42,
  "profile_used": {"symbol_profile": "btc_like"},
  "market_used": {"market_regime": "mix", "market_session": "us"},
  "format": "csv",
  "schema": "ohlcv",
  "resampled": true,
  "base_timeframe": "1m",
  "target_timeframe": "5m",
  "data_hash": "a654cc13c7603f35784d74ddb2faf17a4e2c1703c1699eb79091ef31ecaa1746",
  "meta_hash": "8505f335dbceb16517ce7f67fd743fb23e97fb6cc24b82a8ee20826dd00afb39",
  "fingerprint": "09bcff37290b76955e51b5318a83ba6e16066..."
}

resampled: true means the 1-minute base pipeline was resampled into the 5-minute target; base_timeframe / target_timeframe record the path.

JSONL (`--format jsonl`)

Command:

market-emulator generate \
    --timeframe 5m --count 3 --seed 42 \
    --now 2026-01-31T00:00:00Z --format jsonl \
    --out data.jsonl

data.jsonl — one JSON object per line, no wrapping array:

{"ts_open": "2026-01-30T23:45:00Z", "ts_close": "2026-01-30T23:50:00Z", "open": 50180.28, "high": 55316.8, "low": 48548.44, "close": 50888.81, "volume": 1973.6744}
{"ts_open": "2026-01-30T23:50:00Z", "ts_close": "2026-01-30T23:55:00Z", "open": 50888.81, "high": 53366.41, "low": 47545.55, "close": 51518.33, "volume": 1886.6701}
{"ts_open": "2026-01-30T23:55:00Z", "ts_close": "2026-01-31T00:00:00Z", "open": 51518.33, "high": 58363.88, "low": 49798.85, "close": 54907.27, "volume": 2010.3284}

data.meta.json is written alongside. The JSONL format is for streaming: jq -c '.close' data.jsonl walks 100k+ bars without loading them into memory.

Python API: `to_dataframe`

>>> from market_emulator import Emulator, GenerateRequestV1, to_dataframe
>>> from market_emulator.contracts.enums import Timeframe
>>> from market_emulator.contracts.horizon import HorizonCount
>>> 
>>> emulator = Emulator()
>>> request = GenerateRequestV1(
...     timeframe=Timeframe.ONE_MINUTE,
...     horizon=HorizonCount(count=5),
...     seed=42,
... )
>>> result = emulator.generate(request)
>>> result.meta.normalized.model_dump()
{'start': datetime.datetime(2026, 8, 5, 11, 1, tzinfo=datetime.timezone.utc),
 'end': datetime.datetime(2026, 8, 5, 11, 6, tzinfo=datetime.timezone.utc),
 'count': 5,
 'step_seconds': 60,
 'step_kind': 'fixed'}
>>> result.meta.fingerprint
'38c4736fb1e75de585630001f1ffdaf613a6125546bed13938235d9d48841580'
>>> df = to_dataframe(result)
>>> df.head()
                    ts_open                  ts_close      open      high       low     close    volume symbol orderbook funding universe  is_partial_last_bar regime_label quality_flags context
0 2026-08-05 11:01:00+00:00 2026-08-05 11:02:00+00:00  50180.28  51138.92  48548.44  50000.00  273.9469   None      None    None     None                False         None            []    None
1 2026-08-05 11:02:00+00:00 2026-08-05 11:03:00+00:00  50000.00  50792.60  49130.56  50524.22  251.9564   None      None    None     None                False         None            []    None
2 2026-08-05 11:03:00+00:00 2026-08-05 11:04:00+00:00  50524.22  51719.00  48550.17  51405.81  376.1741   None      None    None     None                False         None            []    None
3 2026-08-05 11:04:00+00:00 2026-08-05 11:05:00+00:00  51405.81  54167.23  49056.30  52610.45  513.1390   None      None    None     None                False         None            []    None
4 2026-08-05 11:05:00+00:00 2026-08-05 11:06:00+00:00  52610.45  55316.80  50184.69  50888.81  558.4578   None      None    None     None                False         None            []    None

to_dataframe returns a pandas DataFrame with columns: ts_open, ts_close, open, high, low, close, volume (base OHLCV) plus symbol, orderbook, funding, universe, is_partial_last_bar, regime_label, quality_flags, context (optional, populated when the corresponding overlays are set). The orderbook / funding and universe columns are populated per-bar under the liquidity_profile / funding_profile / multi-pair path.

What the key `meta` fields mean

fingerprint — SHA256 of the configuration (includes timeframe, seed, scenario, profile, overlays). Used for caching and regression checks.
data_hash — SHA256 of the candle data itself (excluding generated_at).
meta_hash — SHA256 of the entire meta structure.
resampled — true when the target timeframe is not 1m. base_timeframe / target_timeframe record the path.
scenario_name / scenario_version / scenario_hash — null when no scenario is specified; otherwise a triplet for tracking scenario changes.
normalized.start / normalized.end — the final output range (UTC). With --now it matches the expected range; without --now it is computed from the current moment.
debug.regimes_summary — distribution of bars by regime ({"trend_up": 5} means all 5 bars fall inside a trend-up phase).
truth_dataset_version / raw_data_hash / dataset_sanity_checks — populated when scenario='baseline_mix' (or another scenario) via the Truth Dataset Contract.

Determinism

The same inputs always produce the same output:

market-emulator generate --timeframe 1m --count 1000 --seed 42 \
    --now 2026-01-31T00:00:00Z --format json > run1.json
market-emulator generate --timeframe 1m --count 1000 --seed 42 \
    --now 2026-01-31T00:00:00Z --format json > run2.json

# Byte-identical (except generation_id and generated_at)
diff run1.json run2.json

Important: --now is mandatory for count and duration horizons — without it, time runs from the current moment, so a rerun in a different second produces different timestamps. For HorizonRange (with start/end), --now is not needed.

The regression package tests/test_regression_pack.py pins SHA256 hashes for 10 canonical scenarios. This catches any unintended drift in the pipeline (from refactoring to format changes) on CI.

Scenario catalogue

The detailed catalogue is docs/scenarios.md. This is the compact index with the division principle.

Division principle

All 71 built-in scenarios are organized along two axes:

RegimeType (src/market_emulator/contracts/enums.py::RegimeType) — which market phase is modelled: mix, trend_up, trend_down, range, crash, technical_pattern, wyckoff, event_driven, behavioral. This enum propagates into meta.scenario_regime for every result and is read by downstream analyzers.
Mechanism — what the scenario actually does to the price process: pure drift + σ, regime switching, multi-segment drift_segments, session-aware windows, gap (open_jump), open funding/liquidity overlays, attachment to a TA pattern, attachment to a Wyckoff phase, attachment to a crypto event.

The built-in scenarios fall into 6 historical waves (§24-profile catalogue in docs/scenarios.md):

Wave	Region	Count	What it adds
Historical 18	MIX/TREND/RANGE	18	Baseline: baselines, trends, ranges, crashes, stress, high-vol
Profile (Categories 2, 3, 5)	All types	14	Coverage of `trend` × `vol` × `liquidity` × `session` matrix
Crash-spike	CRASH sub-category	7	Liquidation, squeeze, blow-off, capitulation
Technical-pattern	TECHNICAL_PATTERN	7	H&S, triangles, double/триple top/bottom, cup+handle
Wyckoff	WYCKOFF	6	Accumulation, distribution, spring, UTAD, SOS/SOW
Crypto-event	EVENT_DRIVEN	8	Token unlocks, listings, exploits, governance, depeg
Calendar + behavioral	MIX, BEHAVIORAL	11	Halving, options expiry, daily open/close, FOMO, panic

Sum: 18 + 14 + 7 + 7 + 6 + 8 + 11 = 71. Regression hashes for the first 18 in regression/expected_hashes.json are unchanged — the multi-pair path, overlays, and new waves are byte-identical for legacy scenarios.

Group A: Canonical baseline (18)

The baseline; the regression hashes were collected against these. If any change breaks their hash, tests/test_regression_pack.py catches it.

Baseline. baseline_mix — MIX, σ=0.02, normal noise. Balanced regime mix; the canonical "normal market" for regression testing.

Trend. trend_up_soft (drift +0.0003, σ=0.015), trend_up_hard (drift +0.0008, σ=0.025), trend_down_soft (drift -0.0003, σ=0.015), trend_down_hard (drift -0.0008, σ=0.025).

Range. range_tight (drift=0, σ=0.01), range_wide (drift=0, σ=0.025).

High-volatility. high_vol_spiky — MIX, σ=0.04, Student-t (df=5) for fat tails, vol clustering.

Market events. crash_then_recover (drift -0.0002, σ=0.035, Student-t df=4), pump_then_dump (drift +0.0002, σ=0.035, Student-t df=4).

Stress & edge cases. extreme_volatility_burst (σ ramp 0.02→0.15→0.02 over 60+180 bars, volume 10x), gap_event (open_jump +5% @ bar 100, -5% @ bar 200), flash_crash_recovery (-20% over 10 bars @ bar 80, recovery over 20).

Funding profile (requires funding_profile). funding_extreme_positive_240, funding_extreme_negative_240 — sustained funding rates, registered as overlay profiles.

Low-liquidity (requires liquidity_profile). low_liquidity_silent — RANGE + tight OHLCV + low-liquidity orderbook.

Regime transition. regime_transition_smooth — 60 bars in A, 60 bars in smooth transition, 60 bars in B, 60 bars back.

Pullback / breakout. trending_with_pullbacks (uptrend + 3-4 pullbacks 1-3% with recovery), ranging_with_breakout_attempts (100 bars range, failed breakout, 100 bars range, successful breakout).

Volume distribution. high_volume_no_price_move (200 bars, volume 5x, drift=0).

Group B: Profile gap-fill (14)

Closes the QA-program audit for the trend × vol × liquidity × session matrix. Each scenario is a parameterized version of a historical one with additional metadata (market_session, liq, vol).

Breakout (3): breakout_clean (BREAKOUT, medium vol, normal liq), breakout_failed (range with fake breakout, medium vol), breakout_chaotic_vol (BREAKOUT, extreme vol, thin liq).

Mean reversion (2): mean_reversion_high_vol (mean reversion, high vol, normal liq), mean_reversion_low_vol_deep (mean reversion, low vol, deep liq, asia session).

Session diversity (5): asia_uptrend (uptrend, asia), europe_downtrend (downtrend, europe), cross_session_breakout (breakout, cross, high vol), cross_session_chaotic (chaotic, extreme vol, thin liq, cross), asia_low_vol_range (range, low vol, asia).

Mixed combos (4): breakout_low_vol_normal (breakout, low vol), mean_reversion_medium_vol_thin (mean reversion, medium vol, thin), chaotic_medium_vol_normal (chaotic, medium vol), chaotic_high_vol_deep (chaotic, high vol, deep liq, cross).

Group C: Crash / spike sub-category (7)

The CRASH sub-category — liquidation, short squeeze, blow-off, capitulation. All multi-phase via drift_segments + open_jumps.

liquidation_cascade — 3 successive drops + 2 failed relief rallies.
short_squeeze — accelerating uptrend (drift increases per window).
dead_cat_bounce — sharp drop + small bounce + continued decline.
parabolic_blow_off_top — accelerating uptrend + sharp reversal.
capitulation_wick — single -10% open-gap @ bar 100 + V-recovery.
vol_climax_reversal — 100 bars quiet + 20 bars massive-volume reversal.
black_swan — single -15% open-gap, extreme σ, no recovery.

Group D: Technical patterns (TECHNICAL_PATTERN, 7)

All carry market_regime = TECHNICAL_PATTERN. Multi-segment drift_segments to construct TA figures.

head_and_shoulders_top — 7 phases: left peak → head (higher) → right peak (lower) → neckline break.
inverse_head_and_shoulders — mirror: triple bottom.
ascending_triangle_breakout — rising lows + horizontal resistance + breakout.
descending_triangle_breakdown — mirror.
cup_and_handle — U-shape + handle consolidation + breakout.
double_top — two peaks at similar levels + breakdown.
double_bottom — mirror.

Group E: Wyckoff (WYCKOFF, 6)

market_regime = WYCKOFF. Classical Wyckoff phases.

wyckoff_accumulation — PS / SC / AR / ST + Spring + markup.
wyckoff_distribution — mirror: PSY / BC / AR / ST + UTAD + markdown.
wyckoff_spring — range → false breakdown → sharp recovery → markup.
wyckoff_upthrust — range → false breakout up → reaction → markdown.
wyckoff_markup_phase — SOS (sign of strength) + LPS (last point of support) + continuation.
wyckoff_markdown_phase — SOW + LPSY + continuation down.

Group F: Crypto events (EVENT_DRIVEN, 8)

market_regime = EVENT_DRIVEN. Event-driven scenarios for crypto markets.

token_unlock_cliff — 5% cliff unlock: pre-accumulation + cliff dump + post-consolidation.
token_linear_unlock — constant negative drift, 240 bars, 30-day linear vesting.
exchange_listing_pump — pre-listing quiet + +15% open_jump + post-listing fade.
exchange_delisting_dump — pre-warning + -10% open_jump + continued dump.
bridge_exploit — pre-exploit + -40% open_jump (TVL drain) + post-exploit drift, t_df=3 fat tails.
governance_attack — pre-vote + +20% open_jump (whale vote pass) + post-vote fade.
stablecoin_peg_break — pre-depeg + -15% open_jump (USDC depeg) + partial recovery.
flash_loan_manipulation — pre-attack + -15% open_jump + revert (drift +0.18 over 1 bar) + stabilization.

Group G: Calendar + behavioral (11)

Calendar (6) — MIX regime, focus on time-of-day / day-of-week / cyclical.

bitcoin_halving_cycle — compressed 4-year cycle: pre-halving rally + post-halving top + bear + capitulation + new cycle.
monthly_options_expiry — pre-expiry elevated σ + max-pain pinning + post-expiry compression.
daily_open_burst — opening auction: directional drift + elevated σ, bars 0-14.
daily_close_lull — lull bars 210-239 with reduced σ.
weekend_thin_market — tighter range, reduced base_volume.
asia_session_round_trip — Asia range + EU breakout + US continuation.

Behavioral (5) — market_regime = BEHAVIORAL. Introduces a new enum so analyzers can distinguish FOMO, panic, hope bounces, throw-overs, and capitulation-markup paths from trend/range/crash/technical-pattern/wyckoff/event-driven.

fomo_late_parabolic — quiet accumulation + parabolic acceleration + blow-off + crash.
panic_selling_climax — sustained drop + vol expansion (Student-t df=4).
hope_relief_bounce — crash + 3 progressively weaker bounces + continued decline.
throw_over_pattern — accumulation + +8% open_jump + fast return + continued downtrend.
capitulation_then_markup — -10% open_jump + accumulation + markup.

Overlay profiles (11)

Overlay profiles are configurations of existing scenarios with their overlay fields populated. Registered via MarketProfileRegistry.get_overlay(name).

Profile	Base scenario	Overlay field
`deep_book_uptrend`	`trend_up_hard`	`liquidity_class=deep`
`deep_book_downtrend`	`trend_down_hard`	`liquidity_class=deep`
`deep_book_breakout`	`ranging_with_breakout_attempts`	`liquidity_class=deep`
`deep_book_chaotic`	`extreme_volatility_burst`	`liquidity_class=deep`
`deep_book_range`	`high_volume_no_price_move`	`liquidity_class=deep`
`funding_extreme_positive_240`	`baseline_mix`	`FundingProfile(pattern="extreme_positive")`
`funding_extreme_negative_240`	`baseline_mix`	`FundingProfile(pattern="extreme_negative")`
`funding_periodic_oscillating_240`	`baseline_mix`	`FundingProfile(pattern="regime_dependent")`
`multi_pair_btc_eth_correlated_up`	`trend_up_hard`	`MultiPairSpec(BTCUSDT, ETHUSDT, SOLUSDT)`
`correlated_xrp_btc_eth`	`baseline_mix`	`MultiPairSpec(XRPUSDT, BTCUSDT, ETHUSDT)`
`alt_rotation_jan_feb`	`baseline_mix`	`MultiPairSpec(8 alts)`

Composability patterns (20)

These are fields of LiquidityProfile, FundingProfile, MultiPairSpec that compose on top of any base scenario. Not standalone scenarios — overlay knobs.

LiquidityProfile (10): levels_per_side, base_depth_usd, spread_bps_mean, spread_bps_std, depth_shape; overlay knobs: liquidity_dry_up, orderbook_imbalance, stop_cascade, iceberg_absorption, v2 overlays: spoofing, layering, stop_hunt, twap_execution, vwap_execution, iceberg_variation.

FundingProfile (5): base 5 patterns (extreme_positive, extreme_negative, regime_dependent, periodic, oscillating); overlay knobs (B5-B7): funding_flip, funding_squeeze, stable_coin_depeg_shock; volatility-surface annotations: volatility_regime, term_structure, funding_rate_cap, premium_index.

MultiPairSpec (3): lead_lag, correlation_breakdown, cross_asset_contagion.

Truth scenarios (8)

Registered in TRUTH_SCENARIO_SUMMARIES, invoked via Emulator.generate_truth(TruthGenerateRequestV1(...)). Return TruthDatasetResultV1 with target arrays and truth_meta.target for QA assertions.

simple_test — minimal sanity scenario.
linear_trend / scenario_a — deterministic linear trend (or piecewise via drift_change_points). Ground truth for slope-based models.
markov_regime / scenario_b — Markov regimes.
hard_pattern / scenario_c — injection of patterns A and B with target modes (multiclass/binary).
periodic / scenario_d / sum_of_sinusoids — sum-of-sinusoids with a known spectrum.
leak_trap / scenario_e — anti-leak dataset; if a model achieves accuracy > 0.6, the pipeline has a bug.
change_point_injection — AR(1) with deterministic level shift @ bar 100, mean shifts +5σ. Ground truth for change_point_detection.
mean_reversion_oscillation — OU process (θ=0.1, σ=1.0, 240 steps). Ground truth for mean_reversion.

Examples

The examples/ directory holds working scripts:

# Python quickstart
python examples/quickstart_module.py

# CLI scripts
bash examples/generate_csv_10d_1m.sh
bash examples/generate_json_30d_5m.sh

# Comprehensive pass over every documented CLI surface
bash examples/demo_cli.sh

demo_cli.sh is a single end-to-end script that walks every documented CLI surface (basic generation, date range, duration, scenarios, custom profile, export formats, multi-timeframe, batch, determinism, --now override, JSONL streaming with jq) and writes artifacts to data/.

# Batch plan
mkdir -p data
market-emulator batch --plan examples/batch_plan.yaml \
    --now 2026-01-31T00:00:00Z

batch_plan.yaml documents four cases: 1m baseline, 5m trending, 1d by date range, micro-5m for a week. All four use fixed seeds and produce byte-identical output.

Documentation map

docs/ is split into three tiers:

User-facing references (consumer documentation):

docs/cli.md — full CLI reference.
docs/module_api.md — Python API.
docs/contracts.md — the canonical Pydantic surface and output schemas.
docs/scenarios.md — scenario catalogue (incl. truth scenarios A, E, C).
docs/determinism.md — reproducibility guarantees.
docs/resampling.md — timeframe resampling rules.
docs/constraints.md — market invariants.

Architecture meta (internal design):

docs/architecture.md — pipeline and stages.
docs/architecture/ARCHITECTURE_MAP.md — cross-component map.
docs/architecture/DESIGN_PRINCIPLES.md — four load-bearing rules.
docs/architecture/ARCHITECT_PROJECT_CONTEXT.md — domain overlay.
docs/components/ — per-component descriptions (api, cli, config, contracts, core, exporters, generators, scenarios, trainer, truth, utils).
docs/contracts/ — boundary contracts (api-http, batch-plan, deployment-data-dir, generate-request, scenario-registry, truth-dataset).
docs/contract_spec/ — external consumer contract, compatibility rules, fixtures, machine-readable schemas.
docs/adr/ — accepted and planned architecture decisions (ADR-0001, 0002, 0006, 0007).

Operational:

docs/ROADMAP.md — active wave and plan.
docs/RUNBOOK.md — failure triage.
docs/contracts/truth-dataset.md — cryptographically verifiable artifacts.

Testing

# Full run
pytest

# Without slow/perf
pytest -m "not slow and not perf"

# Regression pack
pytest tests/test_regression_pack.py

# Scenarios + jsonset
pytest tests/test_jsonset_cli.py tests/test_jsonset_exporter.py \
    tests/test_jsonset_schema_conformance.py

pytest is configured with --cov-fail-under=90 (minimum 90% coverage). pytest-xdist runs tests in 8 workers. The slow and perf markers are filtered out via -m "not slow and not perf".

The regression pack checks SHA256 hashes for 10 canonical scenarios. Any unintended drift in the pipeline (from refactoring to format changes) is caught by this pack on CI.

CONTRIBUTING.md documents the dev environment, code style, and the change submission process.

Sources

github.com/elriseio/market-data-emulator — MIT.
docs/architecture.md — internal pipeline.
docs/contracts.md — canonical Pydantic contracts.
docs/scenarios.md — scenario catalogue.
docs/adr/0006-multi-pair-derivatives-design.md — multi-pair design rationale.
docs/contracts/truth-dataset.md — Truth Dataset Contract.
Pydantic v2 docs — base for contracts/.
FastAPI docs — HTTP API.
Typer docs — CLI.

market-data-emulator — synthetic OHLCV data generator

Quick start

What it is and what it is not

Requirements

Architecture

Installation

Via pyproject.toml

Dev environment via requirements-dev.txt

Docker

CLI

Python API

Multi-pair generation

Truth Dataset Contract

Output formats

Example responses

JSON (--format json)

CSV (--format csv)

JSONL (--format jsonl)

Python API: to_dataframe

What the key meta fields mean

Determinism

Scenario catalogue

Division principle

Group A: Canonical baseline (18)

Group B: Profile gap-fill (14)

Group C: Crash / spike sub-category (7)

Group D: Technical patterns (TECHNICAL_PATTERN, 7)

Group E: Wyckoff (WYCKOFF, 6)

Group F: Crypto events (EVENT_DRIVEN, 8)

Group G: Calendar + behavioral (11)

Overlay profiles (11)

Composability patterns (20)

Truth scenarios (8)

Examples

Documentation map

Testing

Sources

Via `pyproject.toml`

Dev environment via `requirements-dev.txt`

JSON (`--format json`)

CSV (`--format csv`)

JSONL (`--format jsonl`)

Python API: `to_dataframe`

What the key `meta` fields mean