Skip to content

mip: Support optional per-entry native code compatibility tags. - #1144

Open
o-murphy wants to merge 1 commit into
micropython:masterfrom
o-murphy:feat/mip-natmod-support
Open

mip: Support optional per-entry native code compatibility tags.#1144
o-murphy wants to merge 1 commit into
micropython:masterfrom
o-murphy:feat/mip-natmod-support

Conversation

@o-murphy

@o-murphy o-murphy commented Jul 27, 2026

Copy link
Copy Markdown

Summary

Related to #1140 / #19478 / #19479 / #19532: those add {MPY_ARCH}/{MPY_VERSION} URL-templating for single-file natmod installs, but don’t help when a package’s package.json needs to serve different files per architecture from one manifest (e.g. a natmod .mpy alongside a pure-Python wrapper, or multiple prebuilt arch variants of the same natmod).

This adds an optional third element to entries in hashes/urls: a raw sys.implementation._mpy-shaped integer. If present, the entry is only installed when it matches the running device; entries without it install unconditionally (fully backward compatible - see Testing).

Match semantics (_mpy_tag_ok): bits 0-15 (version/sub-version/arch) must match exactly; bits 16+ (optional arch-flags, e.g. RV32 Zba/Zcmp extensions) are treated as required ⊆ available, not equality - a variant that doesn’t need an extension must still install on hardware that happens to support it. This addresses the concern raised at micropython/micropython#19479 (comment) that naive exact-int-equality doesn’t work for arch flags.

{
  "urls": [
    ["mymodule.py", "mymodule.py"],
    ["mymodule.mpy", "mymodule_mpy5.mpy", 5],
    ["mymodule.mpy", "mymodule_mpy6.mpy", 6],
    ["mymodule.mpy", "https://.../armv7m_6.3.mpy", 1543],
    ["mymodule.mpy", "https://.../rv32imc_6.3.mpy", 2822]
  ]
}

No schema version bump needed - this is length-checked (len(entry) > 2), not a new required field, so every manifest published today keeps working unchanged on both old and new mip.

Testing

  • Unit-tested _mpy_tag_ok standalone (base exact-match, flags-subset in both directions, differing-base rejection, None/absent-_mpy handling) - not yet run against real hardware or the unix port end-to-end via mip.install().
  • Not yet tested: an actual multi-variant package.json fetched and installed on a real board. I’d want to verify this on at least unix port + one arch (e.g. rp2/armv6m or esp32/xtensawin) before this is mergeable - will follow up with results, or happy to have a reviewer with hardware on hand try it against a throwaway manifest.

Trade-offs and Alternatives

  • Alternative encodings discussed in RFC: Distribute native modules for easy installation micropython#19479: named arch/mpy_version string fields (more human-readable manifest, but requires keeping an arch-name table in sync with mpyfiles.rst on both publisher and mip sides) vs. this raw-int approach (opaque in the manifest, but the check is ~4 lines and has nothing to keep in sync - the value is whatever the device already reports).
  • Doing the arch/version substitution entirely in mpremote instead (per @agatti’s suggestion in that thread) was also considered - that helps the “publish once” case but doesn’t cover packages resolved through the on-device mip.install() API directly (no mpremote/PC in the loop at all), which this does.
  • Small code-size increase: one constant, one ~5-line function, and both hashes/urls loops change from fixed tuple-unpacking to indexed access (needed regardless of encoding, to support any optional trailing field without breaking 2-element entries).

Generative AI

I used generative AI tools when creating this PR, but a human has checked the
code and is responsible for the code and the description above.

@o-murphy o-murphy changed the title feat: add support for natmod mpy to mip mip: Support optional per-entry native code compatibility tags. Jul 27, 2026
@o-murphy
o-murphy force-pushed the feat/mip-natmod-support branch 4 times, most recently from 58e0a34 to fcddea1 Compare July 28, 2026 08:01
@o-murphy
o-murphy force-pushed the feat/mip-natmod-support branch from fcddea1 to 5381e44 Compare July 28, 2026 08:08
@o-murphy
o-murphy marked this pull request as ready for review July 28, 2026 08:13
@o-murphy
o-murphy force-pushed the feat/mip-natmod-support branch from 5381e44 to b11c452 Compare July 28, 2026 08:22
Related to micropython#1140 / micropython#19478 / micropython#19479: those add
URL-templating for single-file natmod installs, but don't help when
a single manifest needs to serve different files per architecture.

Adds an optional third element to hashes/urls entries: a raw
sys.implementation._mpy-shaped integer. Entries without it install
unconditionally (backward compatible); entries with it only install
if _mpy_tag_ok() matches - exact match on version/sub-version/arch,
subset match on arch-flags (a variant that doesn't require an
extension must still install on hardware that supports it).

Signed-off-by: o-murphy <thehelixpg@gmail.com>
@o-murphy
o-murphy force-pushed the feat/mip-natmod-support branch from b11c452 to 815d15e Compare July 28, 2026 11:27
@o-murphy

Copy link
Copy Markdown
Author

Follow UP
micropython/micropython#19479 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant