fix(http): force identity encoding so compressed API responses parse

[annextuckner]

Problem

The package_files / file-list tool fails on packages whose manifest comes back compressed:

Error fetching file list for pkg:npm/lodash@4.17.21:
Unexpected token '�', "�bJDT�z!"... is not valid JSON

httpRequest from @socketsecurity/lib@6.0.6 advertises Accept-Encoding: gzip, br on every request, but its main response path (request-attempt.js, the res.on("end") handler) builds the response straight from the raw buffer and never decompresses — .json()/.text() just toString("utf8") the still-compressed bytes. The decodeBody decompressor exists only on the unused readIncomingResponse bypass.

The Socket file-list endpoint returns content-encoding: br, so the brotli stream reaches JSON.parse as raw bytes. Confirmed against the live endpoint: the 8b 62 4a 44 54 prefix of the response is exactly the �bJDT in the error, and brotli-decompressing it yields valid {"files":[...]}.

Fix

Add a socketHttpRequest wrapper in lib/http-helpers.ts that forces Accept-Encoding: identity, and route all seven Socket-bound httpRequest call sites through it (alerts, threat-feed, files, depscore, blob, both OAuth calls). The server then returns uncompressed bodies the lib reads correctly. The override is appended last so it wins the lib's header merge regardless of order.

This is a workaround for the upstream lib defect; once httpRequest decompresses on its main path, the override can be dropped and gzip, br restored for the bandwidth win.

Verification

• pnpm run test:tsc clean; 67 tests pass across the touched modules.
• Live endpoint with Accept-Encoding: identity → no content-encoding, plain JSON, 1054 files parsed.
• Wire-level check via the lib's onRequest hook confirms only Accept-Encoding: identity is sent, with no leftover gzip, br and no duplicate header key.

[Bret Comnes (bcomnes)]

Top-level function getApiKeyInteractively (export) is out of alphanumeric order within its visibility group. Expected to come before `socketHttpR

John-David Dalton (@jdalton) does this have a lint autofix available?

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	@​socketsecurity/​lib@​6.0.7

View full report

[socket-security-staging]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	@​socketsecurity/​lib@​6.0.7

View full report

[John-David Dalton (jdalton)]

Thanks for the thorough diagnosis here — your root-cause analysis was spot on: @socketsecurity/lib@6.0.6's httpRequest advertised Accept-Encoding: gzip, br but never decompressed on its main response path, so compressed file-list bodies reached JSON.parse as raw brotli bytes.

We've resolved this at the source rather than via the Accept-Encoding: identity workaround. @socketsecurity/lib@6.0.7 decompresses by Content-Encoding on the buffered response path before exposing .json()/.text(), so the underlying defect is gone and we keep the gzip, br bandwidth win.

Landed in 2dcfa2a (fix: bump @socketsecurity/lib to 6.0.7 and drop accept-encoding workaround), with a gzip/br regression test covering fetchFileList.

Released in v0.0.18.

Closing this out since the upstream fix supersedes the workaround. Appreciate the careful write-up and the wire-level verification — it made confirming the fix straightforward.