Skip to content

Fix: don't downgrade map/factor-file provider based only on USA equity data#653

Merged
Martin-Molinero merged 3 commits into
QuantConnect:masterfrom
AlexCatarino:fix-futures-mapfile-provider-downgrade
Jun 23, 2026
Merged

Fix: don't downgrade map/factor-file provider based only on USA equity data#653
Martin-Molinero merged 3 commits into
QuantConnect:masterfrom
AlexCatarino:fix-futures-mapfile-provider-downgrade

Conversation

@AlexCatarino

Copy link
Copy Markdown
Member

Problem

A CME-futures-only backtest (e.g. a continuous future /NQ) silently produces future.Mapped == None for every bar, selects no contract, makes no trades, and ends with Sequence contains no elements in result analysis — with no error raised.

Root cause is in lean-cli (not LEAN). LeanRunner._handle_data_providers downgrades the global map-file-provider (and factor-file-provider) from the Zip provider to the Disk provider based solely on the state of equity/usa/map_files:

self._force_disk_provider_if_necessary(lean_config, "map-file-provider",
    "...LocalZipMapFileProvider", "...LocalDiskMapFileProvider",
    data_dir / "equity" / "usa" / "map_files")   # gate is USA-equity only

The provider is a single global engine setting applied to every market, and LocalDiskMapFileProvider reads only loose .csv files — it silently ignores map_files_yyyyMMdd.zip archives. Futures map files (e.g. nq.csv) ship only inside that zip. So a futures-only user (fresh future/cme/map_files/*.zip, but no fresh equity/usa data) has the global provider swapped to Disk, and the continuous future never maps.

Fix

Gate the downgrade on every market's auxiliary folder (<type>/<market>/{map_files,factor_files}/) instead of just equity/usa. Only fall back to the Disk provider when there is no recent zip to lose for any market. Futures-only users keep LocalZipMapFileProvider; equity-sample users (loose csv, no zips) still downgrade as before.

Testing

  • Added regression tests in tests/components/docker/test_lean_runner.py: futures-only data keeps the Zip providers; loose-csv-only data downgrades to Disk; all-stale-zip data downgrades to Disk.
  • Verified end-to-end: a CME continuous-future backtest with no USA-equity map files now maps correctly (Mapped: NQ ...) with the stock engine image; previously Mapped: None.
  • Full test_lean_runner.py and test_backtest.py suites pass.

🤖 Generated with Claude Code

AlexCatarino and others added 2 commits June 20, 2026 20:56
…y data

`_handle_data_providers` downgraded the global `map-file-provider` (and
`factor-file-provider`) from the Zip provider to the Disk provider based
solely on the state of `equity/usa/map_files`. The provider is a single
global engine setting and the Disk provider silently ignores
`map_files_*.zip` archives, so a futures-only data folder (whose map
files ship only inside the zip) had its provider swapped out and
continuous futures never mapped (`Mapped: None`) with no error raised.

Gate the downgrade on every market's auxiliary folder instead of just
`equity/usa`: only fall back to the Disk provider when there is no recent
zip to lose for any market. Adds regression tests covering futures-only
data, loose-csv-only data, and stale-zip data.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Initialize newest_zip_date to datetime.min so the "no zip found" case
naturally falls through the >7-days-old downgrade check, removing both
explicit None checks.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…add debug logging

- Glob the auxiliary zips directly (*/*/<dir>/*.zip) instead of walking dirs/files.
- Return as soon as any market has a zip within the freshness window instead of
  scanning all markets for the newest date (equivalent decision, fewer iterations).
- Hoist datetime.now() out of the loop.
- Add debug logs for both outcomes (which provider is used and why).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Martin-Molinero Martin-Molinero merged commit 57ec0ef into QuantConnect:master Jun 23, 2026
38 checks passed
@AlexCatarino AlexCatarino deleted the fix-futures-mapfile-provider-downgrade branch June 23, 2026 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants