Skip to content

Speed up HEIC decoding and reduce RGB memory use#9

Merged
laurenspriem merged 8 commits into
ente:mainfrom
laurens-pilot:speed_improvements
Jul 16, 2026
Merged

Speed up HEIC decoding and reduce RGB memory use#9
laurenspriem merged 8 commits into
ente:mainfrom
laurens-pilot:speed_improvements

Conversation

@laurens-pilot

Copy link
Copy Markdown

What changed

  • compile decoder tracing and hot-path diagnostics behind the opt-in decoder-tracing feature
  • decode independent HEIF grid tiles in deterministic, memory-bounded parallel batches
  • add direct RGB8 decode APIs that avoid materializing RGBA before discarding alpha
  • add exact AArch64 NEON kernels for dequantization and residual add/clamp
  • document the profiling evidence, benchmark results, correctness gates, and memory tradeoffs

Why

HEIC decode accounts for nearly all Rust-controlled, non-inference time in Ente's ML indexing pipeline. Grid tiles were decoded serially, RGB-only consumers had to retain a full RGBA allocation while creating RGB output, and AArch64 used scalar implementations for profiled HEVC hot paths. Production builds also retained diagnostic synchronization and counters that had no consumer unless tracing was enabled.

Impact

  • benchmarked grid fixtures decode about 2.1–2.2x faster with bounded parallelism
  • direct RGB8 reduced median peak RSS by about 50% and improved measured decode-plus-conversion latency
  • profiled NEON kernels produced a modest 1.2–1.3% end-to-end improvement on the tested fixtures
  • existing RGBA APIs and the image integration contract remain unchanged
  • validation, error selection, and paste order remain deterministic, with sequential/scalar paths retained as correctness oracles

Validation

  • cargo fmt --all -- --check
  • cargo clippy --all-targets --all-features --locked
  • cargo test --all-features --locked (66 tests passed)
  • scripts/heic_tests.sh verify --quick (60 files, 0 failures)
  • branch development also completed the full differential verifier documented in SPEED_IMPROVEMENTS.md

@laurenspriem
laurenspriem marked this pull request as ready for review July 16, 2026 16:48
@laurenspriem
laurenspriem merged commit be1d2b7 into ente:main Jul 16, 2026
3 checks passed
@laurens-pilot
laurens-pilot deleted the speed_improvements branch July 16, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants