Skip to content

feat(js): add provider-prefixed and structural secret rules#361

Open
TBX3D wants to merge 1 commit into
vmfunc:mainfrom
TBX3D:feat/js-secret-provider-coverage
Open

feat(js): add provider-prefixed and structural secret rules#361
TBX3D wants to merge 1 commit into
vmfunc:mainfrom
TBX3D:feat/js-secret-provider-coverage

Conversation

@TBX3D

@TBX3D TBX3D commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

extends the secret bank with eleven providers it does not cover: pypi, openai (legacy and project/service-account), square, mailgun, discord (bot tokens and incoming webhooks), new relic, cloudinary, jwts, and scheme://user:pass@host connection strings.

the last two have no fixed prefix, so shape alone is not proof. a validate hook runs after the entropy and digit gates: the jwt rule decodes the header segment and requires alg and typ, which drops arbitrary dot-separated base64url blobs, and the connection-string rule rejects a userinfo password drawn from the usual documentation placeholders.

originally this added nineteen rules. main has since grown its own bank covering gitlab, anthropic, npm, sendgrid, slack webhooks, github fine-grained pats, google oauth, shopify and stripe (whose (?:sk|rk)_live_ shape already covers restricted keys), so every rule that would have duplicated one of those is dropped rather than shipped a second time under a different name. that also settles the #344 overlap noted in the original description.

TestProviderRulesDoNotDuplicateExistingCoverage pins that no credential is reported twice by two prefixed rules. it excludes the generic assignment rule on purpose: that rule claims any quoted high-entropy value behind a token/password/secret keyword, so it already doubles up with every prefixed rule on main. that is pre-existing and out of scope here, but worth flagging.

@TBX3D
TBX3D requested a review from vmfunc as a code owner July 9, 2026 23:31
@github-actions github-actions Bot added size/l <500 lines changed scan changes to scan engine modules changes to scan modules tests test changes labels Jul 9, 2026
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

pr summary

2 files changed (+297 -1)

category files
go source 2
tests 1

@codecov-commenter

codecov-commenter commented Jul 9, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 64.70588% with 6 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@a38ba0a). Learn more about missing BASE report.

Files with missing lines Patch % Lines
internal/scan/js/secrets.go 64.70% 3 Missing and 3 partials ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #361   +/-   ##
=======================================
  Coverage        ?   64.87%           
=======================================
  Files           ?       88           
  Lines           ?     7875           
  Branches        ?        0           
=======================================
  Hits            ?     5109           
  Misses          ?     2370           
  Partials        ?      396           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@vmfunc vmfunc left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the validate hook is the right shape here. gating jwt and the conn-string on actual structure instead of widening the regex keeps the fp floor where it belongs, and the placeholder-password filter will kill a ton of .env.example noise.

two small things, neither blocking: isStructuredJWT requires "typ" but that's optional per 7515/7519, plenty of real id tokens omit it so you'll miss some. alg alone is the mandatory one. and key-[0-9a-f]{32} for mailgun trips on any cache/redis key named key-, it's the one loose rule in the set. fine to land and tighten later.

the #344 restricted-key / fine-grained-pat overlap is on you two to reconcile whenever the second one lands. in.

extends the secret bank with eleven providers it did not cover: pypi,
openai (legacy and project/service-account), square, mailgun, discord
(bot tokens and incoming webhooks), new relic, cloudinary, jwts and
scheme://user:pass@host connection strings.

the last two have no fixed prefix, so shape alone is not proof: add a
validate hook that runs after the entropy and digit gates. jwt decodes
the header segment and requires alg and typ, which drops arbitrary
dot-separated base64url blobs. connection strings reject a userinfo
password drawn from the usual documentation placeholders.

scoped to what the bank does not already cover: gitlab, anthropic, npm,
sendgrid, slack webhooks, stripe and github fine-grained pats all landed
upstream in the meantime, so the rules that duplicated them are dropped
rather than shipped a second time under a different name.
@TBX3D
TBX3D force-pushed the feat/js-secret-provider-coverage branch from 9aa7489 to 4f82486 Compare July 23, 2026 00:10
@TBX3D TBX3D changed the title feat(js): expand secret rule bank with provider-prefixed and structural rules feat(js): add provider-prefixed and structural secret rules Jul 23, 2026
@TBX3D

TBX3D commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

rebased, and this one needed re-scoping rather than a replay. main grew its own rule bank in the meantime, covering gitlab, anthropic, npm, sendgrid, slack webhooks, github fine-grained pats, google oauth, shopify and stripe.

a straight rebase left eight rules duplicated by name and three more duplicated by shape: main's stripe rule is (?:sk|rk)_live_, so the separate restricted-key rule here had nothing left to add, and "gitlab access token" was main's "gitlab personal access token" regex under a different name. shipping that would have double-reported every one of those credentials.

so this is down to the eleven providers main does not cover, plus the two rules that need validation rather than a shape (jwt, connection strings). that also settles the #344 overlap i flagged in the original description. TestProviderRulesDoNotDuplicateExistingCoverage pins that no credential is claimed by two prefixed rules.

one thing this does not fix, surfacing it rather than quietly leaving it: the generic assignment rule claims any quoted high-entropy value behind a token/password/secret keyword, so const token = "<jwt>" is reported twice, once by the prefixed rule and once by the generic one. that is already true on main for every prefixed rule, so i left it out of scope here, but happy to take it separately if you want it closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

modules changes to scan modules scan changes to scan engine size/l <500 lines changed tests test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants