Export: trimming enhancements by DuBento · Pull Request #3530 · thought-machine/please

DuBento · 2026-05-01T14:45:16Z

Enhancements to plz export, moving from a basic target-level trimming (using gc.RewriteFile) to build statement-level trimming, including only the required build rules and subincludes.
For consistency, we format all the exported BUILD files.

Changelog:

Introduced PackageMetadata to track the relationship between BUILD file statements, the targets they generate, and the subincludes they require. Made optional to avoid the overhead for most ops and enabled for the export.
Introduced ScopeMetadata to track object origins (subincluded labels) and statement tracking during the interpreter phase. Made optional to avoid the overhead for most ops and enabled for the export.
Refactored src/export/export.go to enforce better separation of the DefaultExporter (for trimming) and NoTrimExporter.
Added logic to parse BUILD files and selectively write back statements based on whether the generated targets are part of the export set.
Implemented "minimal subinclude" generation, which rewrites subinclude() calls to only include labels actually used by the exported targets.
Update and added some of the e2e to reflect the changes in implementation.
Moved trimming from a GC-based target removal to a full AST statement walk.

… build targets

setting subincludes at package level instead of at target level

- register subinclude statements in the package metadata - filter subincludes label - export all non build_target related statements

…dary build def with sources the updated test uses non-standard child naming to validate new trimming logic

- comments - rename structs, files and interfaces - package parsing - notrim testing

…by build_defs + test cases

previous implementation was append only meaning that it would mark a subinclude as required if it was visited before even if not exported.

…a to also avoid tracking inside subincludes

…Parser is set. This wasn't an issue before since we would force close all the channels and fail during the first parse, but now we will attempt to wait for parsed package during an export.

… package level

…ents (including non-target generators). The reason for pivoting is that the previous implementation failed to identify necessary subincludes for variable declarations or other builtin methods (string join)

…r traversal.

…ls to interpretStatements with the package level scope (e.g. for loop)

…y if blocks that have interpreted stmts (var redeclaration)

duarte added 30 commits April 29, 2026 19:28

Package level metadata that includes a mapping of build statements to…

0ef91bc

… build targets

export using BuildFileMetadata logic

63f1b0d

register caller scope for callstack-like traversal

419bd56

bufio writer for build statements

8e1bd00

rewrite export logic into explicit flow

258b170

export targets related to build statement

4110735

enrich target with subincludes by looping though all scopes

a4252c9

separate buildstmt register from adding target

82f117f

select and write subincludes

eb657de

setting subincludes at package level instead of at target level

skip statement for preloaded subincludes

6535794

test: trim subincludes

8e6c9c0

rename file

9f0b4fe

use slices.collect and sort interface

0e597c6

rewrite export with 2 concrete interfaces: default and notrim

de76e2c

notrim: full copy BUILD file and visit sources

03ed361

double new lines between targets

717aba1

suppress diff output when enforcing repo differences

94160bf

export by filtering the original BUILD file

582319e

- register subinclude statements in the package metadata - filter subincludes label - export all non build_target related statements

subincludes use label short string

e6a5293

Simplify package filtering method into a more explicit "switch" case

ea329c3

skip subrepos and internal packages when writing package file

1a47c22

fix reusing err var resulted in failed filtering

a35119a

doc strings for package metadata

9c1522c

export missing doc strings

9395a59

test: named go_repo and change testify for slimmer UUID

3c3d3d0

export dependencies of subrepos - 13/14 tests passing

13e95cb

test: internal repo test using temp directory to avoid stale data

28b5441

fix: 0 label subinclude

781af3a

adjusting dependency lookup and adjacent target test to include secon…

2f8355c

…dary build def with sources the updated test uses non-standard child naming to validate new trimming logic

test: add custom tool to native test

3e89c85

DuBento force-pushed the export-from-package-metadata branch 2 times, most recently from e584818 to 6b9f0c4 Compare June 10, 2026 18:07

DuBento added 11 commits June 10, 2026 19:08

rename exporter source files

299b259

apply feedback from multiple comments

21f3775

- comments - rename structs, files and interfaces - package parsing - notrim testing

local subrepos: fixes (relative filenames) and test

681a664

export transitive subincludes, fixing unexported targets subincluded …

1bea58f

…by build_defs + test cases

export gitignore

acb81e6

test: glob for sources

f1a0abe

Package parsing via WaitForPackage

5d3073b

test: move trim test files into data

c38eba6

test: trim subinclude in third_party/go test

c387e50

export label set

2446dad

track used objects as a stack

7152109

previous implementation was append only meaning that it would mark a subinclude as required if it was visited before even if not exported.

DuBento force-pushed the export-from-package-metadata branch from f6fbf1a to 7152109 Compare June 17, 2026 10:01

DuBento added 16 commits June 17, 2026 18:27

scopeMetadata: private, rename and comments

50a8240

rename active to required subincludes

51cbb35

imp: skip scope and package metadata tracking for external subrepos

d491c84

remove doc reference to required target naming for successful export

8a1a702

Only pop symbols from stack in package scope + rework newScopeMetadat…

131b489

…a to also avoid tracking inside subincludes

lazy initialize symbol origins

f437f8f

Close package wait channels on error. Introduced a deadlock when Keep…

b0f200e

…Parser is set. This wasn't an issue before since we would force close all the channels and fail during the first parse, but now we will attempt to wait for parsed package during an export.

Support method calls as arguments by popping the symbol stack only at…

9c6cf87

… package level

refactor: record all statements and respective origins for all statem…

349e3f8

…ents (including non-target generators). The reason for pivoting is that the previous implementation failed to identify necessary subincludes for variable declarations or other builtin methods (string join)

Package level scope metadata; Tracking done at package level only.

1c67f8b

Package metadata lookup table in interpreter to avoid recursive calle…

7d6ba12

…r traversal.

Trimmer: always keep for expression but trim stmts inside

0af4d97

glob statement: register files required by statements

1e09063

Checkpoint and restore symbol and file stack to support different cal…

2204124

…ls to interpretStatements with the package level scope (e.g. for loop)

move to BuildLabels in export and metadata to avoid pressure on GC

e6c8fee

track interpreted statements (mark as 0 targets) to correctly identif…

e4aa82b

…y if blocks that have interpreted stmts (var redeclaration)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Export: trimming enhancements#3530

Export: trimming enhancements#3530
DuBento wants to merge 109 commits into
thought-machine:masterfrom
DuBento:export-from-package-metadata

DuBento commented May 1, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

DuBento commented May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

DuBento commented May 1, 2026 •

edited

Loading