[TrimmableTypeMap][NativeAOT] Build plumbing fixes#11989
Open
simonrozsival wants to merge 9 commits into
Open
Conversation
…ID inner build path _ResolveAssemblies always dispatches a per-RID inner build with AppendRuntimeIdentifierToOutputPath=true, so ILC writes each *.scan.dgml.xml under the RID-nested obj/<cfg>/<rid>/native/ path -- for both a single explicit RuntimeIdentifier and a RuntimeIdentifiers list. _CollectTrimmableNativeAotDgmlFiles runs in the outer build, whose NativeIntermediateOutputPath has no RID segment. The single-RuntimeIdentifier branch collected from that flat path (obj/<cfg>/native/), which never exists, so _GenerateTrimmableTypeMapProguardConfiguration failed with XA4321 across every single-RID NativeAOT build (e.g. IncrementalBuildDifferentDevice and the other MockPrimaryCpuAbi tests). Reconstruct the RID-nested path for both the single-RuntimeIdentifier and the RuntimeIdentifiers cases (they now share one item expression); the flat NativeIntermediateOutputPath remains only as the defensive no-RID fallback. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 1512449)
…ath shapes) The previous XA4321 fix assumed ILC's *.scan.dgml.xml always lives at the RID-nested $(IntermediateOutputPath)<rid>/native/. That is only true when the outer $(IntermediateOutputPath) has no RID segment (a $(RuntimeIdentifiers) list, or a RID assigned late by _GetPrimaryCpuAbi). For a single explicit $(RuntimeIdentifier) set early, the SDK already appended the RID to the outer path, so reconstructing <rid>/native/ produced a doubled RID (obj/Release/android-arm64/android-arm64/native/) and XA4321. Emit both candidate paths ($(NativeIntermediateOutputPath) and the RID-nested form), Exists()-filtered, and consume whichever ILC actually produced. This is correct for single-RID (either output-path shape), multi-RID, and no-RID. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 6ea9395)
…L is absent ILC only emits the scan graph (*.scan.dgml.xml) when its scanner phase runs (optimized/Release builds). Unoptimized NativeAOT builds - e.g. a Debug configuration, as produced when a solution is built without an explicit Release configuration (AllProjectsHaveSameOutputDirectory) - emit only the codegen graph (*.codegen.dgml.xml). The trimmable proguard-keep generator then failed with XA4319 "No NativeAOT DGML files were provided". The codegen graph carries the same "Type metadata: [...]" nodes the generator reads, so collect it at the same candidate locations and use it only when no scan graph was found (the scan graph is smaller, hence preferred). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit e58cb04)
…T path Two regressions surfaced by ManifestPlaceholders once the trimmable typemap became the NativeAOT default: * Placeholder values kept literal backslashes (e.g. "a=b\c"). The legacy pipeline re-encodes the substituted manifest through aapt2, which rewrites '\' to '/' on Unix; the trimmable generator writes the merged manifest directly, so normalize placeholder values to Path.DirectorySeparatorChar in ManifestGenerator.ApplyPlaceholders to keep the output identical. * The legacy manifest merger has no _ManifestMerger step (that target only runs for manifestmerger.jar), so with AndroidManifestMerger=legacy nothing copied the already-merged trimmable manifest to obj/<cfg>/android/AndroidManifest.xml and _ReadAndroidManifest failed with a FileNotFoundException. Copy it into place on the trimmable path when the merger is not manifestmerger.jar. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 8af5b62)
…e toggle (default off) The ILC DGML files used to drive Java Callable Wrapper trimming are very large (hundreds of MB) and dominate NativeAOT build time. Until ILC/illink expose a leaner typemap dump, gate that work behind a new property: * _AndroidTrimmableTypemapTrimJavaCode (default false): skip DGML generation (IlcGenerateDgmlFile is no longer forced) and skip DGML collection, and have GenerateNativeAotProguardConfiguration emit a -keep rule for every ACW in the ACW map so R8 keeps them all. This costs a few hundred kB of extra dex but removes the huge DGML and its parsing/scan from every build. * Set to true, the previous behavior is restored: ILC emits the DGML and the keep rules are computed from the DGML-retained subset. GenerateNativeAotProguardConfiguration gains a TrimJavaCallableWrappers switch; when false it no longer requires a DGML (NativeAotDgmlFiles is now optional). Unit tests cover both the DGML-trimmed path and the keep-all path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit f157df9)
…ava sources A component with a non-default android:process (e.g. [BroadcastReceiver(Process=":remote")]) makes the manifest generator emit an extra runtime provider (NativeAotRuntimeProvider_1) and return its name in AdditionalProviderSources. On the legacy/LLVM-IR path GenerateAdditionalProviderSources writes the matching Java source, but the trimmable path only surfaced the item and never generated the .java, so NativeAotRuntimeProvider_1 was missing from classes.dex (Desugar failed on NativeAOT). Extract the provider-source writing into a shared GenerateAdditionalProviderSources.WriteAdditionalRuntimeProviderSources helper and call it from GenerateNativeAotBootstrapSources (which already runs on the trimmable path), passing @(_AdditionalProviderSources). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 934cd3b)
On the trimmable NativeAOT path R8 was given only a comment for proguard_project_primary.cfg, so user-authored AndroidJavaSource types (Bind != true) — which have no managed peer and are absent from the acw-map — were tree-shaken out of classes.dex. Among other things this dropped large unreferenced sources, so an app that required multidex no longer did and classes2.dex was never produced (BuildAfterMultiDexIsNotRequired failed on NativeAOT). Emit the GetUserJavaTypes() -keep rules into proguard_project_primary.cfg on the trimmable path too (the ACW keeps still come from proguard_project_references.cfg). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 8cfd0fd)
… path
Accessing a resource id under the trimmable type map (the NativeAOT default) threw at runtime:
System.TypeInitializationException
---> System.IO.FileNotFoundException: _Microsoft.Android.Resource.Designer
Root cause: the _Microsoft.Android.Resource.Designer assembly IS added to the publish set
(_AddResourceDesignerToPublishFiles marks it PostprocessAssembly=true, so the SDK adds it as an
IlcReference), but it is left trimmable (IsTrimmable=true). Resource ids are resolved via
reflection at runtime (Android.Runtime.ResourceIdManager -> ResourceDesignerAttribute), which
the trimmer cannot follow, so ILC trims the whole designer assembly away and any reflection-only
resource-id access (e.g. an NUnit [TestCaseSource] static field) fails to load it.
The non-trimmable (LlvmIr) path avoids this because PreTrimmingFixLegacyDesigner rewrites legacy
field loads and, more importantly, it roots all publish assemblies for ILC. That rewrite is a
no-op for modern assemblies (whose Resource already derives from the designer), so it is not the
right fix here.
Instead, simply keep the designer assembly: add it to TrimmerRootAssembly on the trimmable path.
TrimmerRootAssembly is honored by both ILLink (CoreCLR) and ILC (NativeAOT, emitted as --root:);
ILC's TrimMode-based rooting can't be used because _AndroidComputeIlcCompileInputs clears
@(_IlcManagedInputAssemblies). No IL rewriting is required.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
(cherry picked from commit 449e2a1)
Contributor
There was a problem hiding this comment.
Pull request overview
This PR tightens up build plumbing for the trimmable typemap + NativeAOT path, focusing on robustness and (optional) performance tradeoffs: improved DGML discovery/collection, fixing manifest placeholder behavior, generating per-process runtime provider Java sources, and ensuring shrinking keeps user-authored Java and the resource designer assembly.
Changes:
- Add a
TrimJavaCallableWrapperstoggle to allow skipping DGML parsing and keeping all JCWs when desired. - Improve NativeAOT DGML file collection (inner-build/RID-nested outputs + scan/codegen fallback) and wire it into Proguard config generation.
- Ensure generated manifests/placeholders and runtime-provider Java sources behave consistently across trimmable/legacy build paths; root resource designer assembly for trimming.
Show a summary per file
| File | Description |
|---|---|
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/GenerateTrimmableTypeMapTests.cs | Adds a regression test ensuring Proguard config generation works without DGML when trimming is disabled. |
| src/Xamarin.Android.Build.Tasks/Tasks/R8.cs | Emits explicit R8 -keep rules for user-authored AndroidJavaSource types on the trimmable NativeAOT path. |
| src/Xamarin.Android.Build.Tasks/Tasks/GenerateNativeAotProguardConfiguration.cs | Introduces TrimJavaCallableWrappers to optionally bypass DGML and keep all ACWs from the map. |
| src/Xamarin.Android.Build.Tasks/Tasks/GenerateNativeAotBootstrapSources.cs | Adds support for generating additional per-process runtime provider Java sources on the trimmable path. |
| src/Xamarin.Android.Build.Tasks/Tasks/GenerateAdditionalProviderSources.cs | Refactors additional runtime-provider source emission into a shared helper usable by both legacy and trimmable flows. |
| src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.TypeMap.Trimmable.targets | Wires additional provider sources into NativeAOT bootstrap generation; ensures the merged manifest lands in the expected intermediate location; roots the resource designer assembly. |
| src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.TypeMap.Trimmable.NativeAOT.targets | Adds the _AndroidTrimmableTypemapTrimJavaCode toggle, gates DGML generation/collection on it, and adds scan/codegen DGML fallback logic. |
| src/Microsoft.Android.Sdk.TrimmableTypeMap/Generator/ManifestGenerator.cs | Normalizes backslashes in manifest placeholder values to match legacy output behavior. |
Copilot's findings
- Files reviewed: 8/8 changed files
- Comments generated: 3
Comment on lines
+20
to
24
| <!-- Only ask ILC for the DGML when JCW trimming is enabled; it is only consumed to compute the keep rules. --> | ||
| <IlcGenerateDgmlFile Condition=" '$(_AndroidTrimmableTypemapTrimJavaCode)' == 'true' and '$(AndroidLinkTool)' != '' and '$(IlcGenerateDgmlFile)' == '' ">true</IlcGenerateDgmlFile> | ||
| <_UseTrimmableNativeAotProguardConfiguration Condition=" '$(_UseTrimmableNativeAotProguardConfiguration)' == '' ">true</_UseTrimmableNativeAotProguardConfiguration> | ||
| <_CompileToDalvikDependsOnTargets>$(_CompileToDalvikDependsOnTargets);_GenerateTrimmableTypeMapProguardConfiguration</_CompileToDalvikDependsOnTargets> | ||
| </PropertyGroup> |
Comment on lines
118
to
122
| <GenerateNativeAotProguardConfiguration | ||
| NativeAotDgmlFiles="@(_TrimmableNativeAotDgmlFiles)" | ||
| AcwMapFile="$(IntermediateOutputPath)acw-map.txt" | ||
| TrimJavaCallableWrappers="$(_AndroidTrimmableTypemapTrimJavaCode)" | ||
| OutputFile="$(_ProguardProjectConfiguration)" /> |
ILC only loads assemblies passed to it as references; _Microsoft.Android.Resource.Designer is a compile-time reference but not in @(IlcReference), so rooting it via TrimmerRootAssembly alone fails: ilc error 'Failed to load assembly _Microsoft.Android.Resource.Designer'. Add it to @(IlcReference) too, driven off @(ReferencePath) filtered to the designer so nothing is referenced/rooted when the designer assembly was not generated. @(IlcReference) is ignored by ILLink, so the CoreCLR trimmable path only gets the root. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
/review |
|
❌ Android PR Reviewer failed to deliver outputs. Please review the logs for details. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
🤖 Android PR Review — NativeAOT trimmable typemap plumbing
Verdict: ✅ LGTM — with one correctness caveat (
Well-scoped plumbing extracted from #11822, and the per-path comments make the intent easy to follow. I read each of the 8 changed files in full and traced the wiring rather than trusting the description. What I verified:
Verified correct
- R8 user-Java keep rules — the new trimmable branch keeps user-authored
AndroidJavaSource(Bind != true) that has no managed peer and is therefore absent from the acw/DGML map;JavaSourceFilesis fed from@(_R8KeepJavaSource)inXamarin.Android.D8.targets. Without it R8 would shrink those sources away (the multidex regression the comment describes is real). TrimJavaCallableWrapperstoggle (GenerateNativeAotProguardConfiguration) — when off, the DGML is never consulted and every ACW is kept; anullretainedTypeKeyscleanly means "keep all". Dropping the[Required]attribute is consistent with the new default.ManifestGeneratorbackslash normalization —Replace ('\\', Path.DirectorySeparatorChar)matches the legacy aapt2 behavior and is already covered by the existingManifestTest.ManifestPlaceholderstest (which runs on the NativeAOT path). Not a regression.GenerateAdditionalProviderSourcesrefactor — extraction into the sharedWriteAdditionalRuntimeProviderSourcesis behavior-preserving; the newAdditionalProviderSourcesoutput onGenerateNativeAotBootstrapSourcesis wired through@(_AdditionalProviderSources).- Designer-assembly rooting — referencing the assembly before rooting it for ILC is the correct ordering.
_GenerateTrimmableTypeMapProguardConfiguration doesn't include _AndroidTrimmableTypemapTrimJavaCode in its Inputs (line 116). On a true → false toggle flip during an incremental build, _CollectTrimmableNativeAotDgmlFiles no longer runs (it's gated on the toggle), so @(_TrimmableNativeAotDgmlFiles) is empty and the only remaining input (acw-map.txt) is unchanged and older than the existing output — MSBuild skips the target and leaves the previously-trimmed proguard_project_references.cfg in place, so R8 keeps shrinking ACWs the user now expects to be retained. (false → true is fine: the freshly-emitted DGML forces a rebuild.) The toggle defaults off and is internal, so severity is low, but it's worth folding the toggle into the target's incremental key (e.g. a small stamp file that encodes its value) before the flag is promoted or documented.
Suggestions (inline)
- 💡 Sort the user Java types in
R8.csfor reproducible generated output, mirroring the ordinal sort on the neighboring acw-map branch. - 💡 Add a
TrimJavaCallableWrappers = falsewith a DGML present test case to lock in that the DGML is ignored when trimming is disabled.
CI — Azure DevOps dotnet-android legs are green so far; a few device-test legs were still in progress at review time. No failures observed.
Nice work. 🎯
Generated by Android PR Reviewer for #11989 · 633.9 AIC · ⌖ 26.7 AIC · ⊞ 6.8K
Comment /review to run again
| // (e.g. dropping large unreferenced sources so an app that needs multidex no longer does). | ||
| using (var appcfg = File.CreateText (ProguardGeneratedApplicationConfiguration)) { | ||
| appcfg.WriteLine ("# ACW keep rules are generated from NativeAOT ILC metadata."); | ||
| foreach (var java in GetUserJavaTypes ()) { |
There was a problem hiding this comment.
🤖 💡 Patterns — GetUserJavaTypes () yields in @(AndroidJavaSource) item order, which is glob/filesystem-dependent, so these -keep rules (and thus the generated proguard_project_application.cfg) have no stable order across machines. The acw-map branch just below already sorts its Java types via javaTypes.Sort (StringComparer.Ordinal) — consider collecting and sorting the user types the same way for reproducible generated output. Impact is low (R8 does not depend on order), but it keeps the artifact deterministic.
{Rule: Deterministic generated output — sort emitted identifiers with .Ordinal}
| BuildEngine = new MockBuildEngine (TestContext.Out), | ||
| AcwMapFile = acwMapFile, | ||
| OutputFile = outputFile, | ||
| TrimJavaCallableWrappers = false, |
There was a problem hiding this comment.
🤖 💡 Testing — Good to see the disabled-toggle path covered. Consider also adding the disabled + DGML-present quadrant: provide a NativeAotDgmlFiles DGML and set TrimJavaCallableWrappers = false, then assert every ACW is still kept (e.g. that other.Type survives). RunTask () skips the entire DGML-loading block when the toggle is off, so this locks in that the DGML is ignored when trimming is disabled — the current disabled test cannot catch a future regression that reads the DGML regardless of the toggle, because it supplies no DGML.
{Rule: Test edge cases — happy path only misses the disabled + DGML-present case}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Independent build-plumbing fixes for the trimmable type map on NativeAOT (the opt-in
-p:_AndroidTypeMapImplementation=trimmable+_AndroidRuntime=NativeAOTpath). These stand alone and do not depend on making the trimmable type map the NativeAOT default (#11822) — extracting them keeps that PR focused.Fixes
ManifestPlaceholderson the trimmable NativeAOT path — placeholder substitution in the generated manifest now works (ManifestGenerator)._AndroidTrimmableTypemapTrimJavaCodetoggle (default off) — optionally skip DGML parsing and keep all ACWs, trading a little dex size for skipping the very large DGML files that dominate NativeAOT build time.GenerateNativeAotProguardConfigurationno longer requires DGML input when disabled.GenerateAdditionalProviderSources/GenerateNativeAotBootstrapSources).AndroidJavaSourceunder R8 — user-authored Java (no managed peer) is absent from the ACW map, so emit explicit-keeprules; otherwise R8 shrinks it away (which can, e.g., drop sources so an app that needs multidex no longer does)._Microsoft.Android.Resource.Designerassembly — resource ids are resolved via reflection (ResourceIdManager), which the trimmer can't follow; root the designer viaTrimmerRootAssembly(honored by both ILLink and ILC) so it isn't trimmed away (otherwise resource-id access throwsFileNotFoundException: _Microsoft.Android.Resource.Designer).Testing
Microsoft.Android.Sdk.TrimmableTypeMapgenerator +GenerateTrimmableTypeMapTestsbuild clean; full validation via CI (device tests) since these exercise the on-device NativeAOT path.Extracted from #11822.
Issue references
Partially addresses #11774. Contributes to #10790 and #10793.