Def 51195 truncate request body#18
Open
quirky4 wants to merge 2 commits into
Open
Conversation
Two fork-only tests (both from cloudlinux/coraza 7e054cf) fail in CI but pass under plain `go test`, so the sync merge did not surface them. initcol_test.go: gate with `//go:build !tinygo`. The test lives in package actions_test and uses the `md` mock, which is only defined in setvar_persistence_test.go — a file already gated `!tinygo`. Under TinyGo that file drops out and `md` is undefined, breaking compilation. The test exercises the SESSION persistent collection, the same feature family the fork already excludes from TinyGo, so gating it matches existing intent. expirevar_test.go: give each parallel subtest its own action instance. The test got one expirevarFn via Get("expirevar") and shared it across eight t.Parallel() subtests, each calling Init(), which writes the receiver's collection/key/ttl fields — a data race caught by `go test -race`. It is a test-only defect: in production each rule gets its own action, Init runs once at compile time, and Evaluate only reads. Also renamed the three duplicate "missing ttl" subtests for readable output. Verified: `go vet -tags tinygo ./internal/...` clean; `go test -race` on internal/actions clean under both CI tag sets. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Connectors that keep a transaction open between the request and logging phases (waiting for the upstream response) pin the full buffered request body for the lifetime of the parked transaction, although those phases never re-read it. TruncateRequestBody(limit) releases the body right after request-phase analysis, keeping at most limit bytes as a preview for audit log part C. The in-memory branch copies the prefix into a fresh buffer instead of bytes.Buffer.Truncate, which retains the backing array (and transaction pooling would keep that capacity alive past Close). The file-backed branch truncates the temp file in place, or closes and removes it for limit=0. The REQUEST_BODY variable is cut to the same prefix via strings.Clone so the original backing string is released too; REQUEST_BODY_LENGTH keeps the original received length and parsed derivatives (ARGS_POST, JSON/XML matches) stay intact. Readers obtained via RequestBodyReader before truncation are invalidated and fail with types.ErrBodyTruncated: their positions may point beyond the shrunk buffer, and quietly serving the prefix or EOF would turn a contract violation (holding a reader across truncation) into silent data loss. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
quirky4
marked this pull request as ready for review
July 23, 2026 11:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add Transaction.TruncateRequestBody: release request-body memory after analysis
Problem
Connectors that proxy to an upstream keep a transaction open between the request phase and the logging phase — for a slow backend that's easily dozens of seconds. For that entire window the transaction pins the full buffered request body, although phases 3–5 never re-read it:
SecRequestBodyLimitdefaults to 128 MiB andSecRequestBodyInMemoryLimitdefaults to the same value, so it never spills to disk);REQUEST_BODYvariable holds a second full copy as a string.In practice a parked transaction with an 8 MB body costs ~12 MB of RSS (buffer growth overshoot + variable copy). Under many concurrent slow-backend requests this adds up to the dominant memory consumer, and
SecRequestBodyInMemoryLimitdoesn't help: the body processor rebuilds the fullREQUEST_BODYstring from disk anyway.Proposal
Releases the buffered request body once request-phase analysis is done, keeping at most
limitbytes as a preview for audit log part C.limit=0discards the body entirely. Connectors call it right afterProcessRequestBody(); a parked transaction then costs ~90 KB +limitregardless of body size.Design notes
bytes.Buffer.Truncate, which retains the full backing array. The file-backed branch truncates the temp file in place, or closes and removes it forlimit=0.REQUEST_BODYis cut to the same prefix viastrings.Clone(a plain substring would pin the original backing string).REQUEST_BODY_LENGTHkeeps the original received length — it reports what arrived, and audit consumers may want it next to the preview. Parsed derivatives (ARGS_POST, JSON/XML matches) stay intact: they are small and phase 3–5 rules may legitimately use them.limitbytes — a raw prefix; it may cut mid-token.RequestBodyReader()before truncation fail withtypes.ErrBodyTruncated: their positions may point beyond the shrunk buffer, and quietly serving the prefix or EOF would turn a contract violation (holding a reader across truncation) into silent data loss.ProcessRequestBodyis an error, except when the rule engine is off or the transaction is already interrupted — states in whichlastPhasenever advances and request-body analysis will never run, so truncation cannot hide anything from rules.os.File.Truncateleaves length and readers untouched; a failingClosein the discard path still removes the temp file (at that point nothing else — includingReset()— would ever clean it up).Also included
fix: repair fork-only tests failing under TinyGo and -race— makes the branch's fullgo test -race ./...green, including the pre-existingTestExpirevarrace.Verification
REQUEST_BODY/REQUEST_BODY_LENGTH/ARGS_POSTstate, double-call idempotency, early-call error, engine-off/interrupted acceptance, stale-reader invalidation,Close/Truncatefailure pathsgo test -race ./...green, golangci-lint v2.6.2 clean