Use Harper readonly mode for Next.js builds

[cb1kenobi]

Next.js uses child processes to generate static pages. When those pages reference Harper data, it will try to load those databases and error with:

Error: IO error: While lock file: /Users/chris/harper/database/data/LOCK: Resource temporarily unavailable opening database /Users/chris/harper/database/data
    at openRocksDatabase (../harper/resources/databases.ts:162:22)
    at readRocksMetaDb (../harper/resources/databases.ts:407:16)
    at getDatabases (../harper/resources/databases.ts:247:6)
    at database (../harper/resources/databases.ts:734:2)
    at table (../harper/resources/databases.ts:899:20)
    at Object.<anonymous> (../harper/security/auth.ts:27:27)

Harper has a new READONLY environment variable that will allow a RocksDB database to be opened by another process in read-only mode allowing the static pages to be generated.

[kriszyp]

I believe we need to (somehow) flush all the databases so the new process can "see" all the data.

[cb1kenobi]

I believe we need to (somehow) flush all the databases so the new process can "see" all the data.

plugin.ts imports 'harper' and runs on the thread that has write access to the database, so does Harper have any public API that will flush all databases?

[kriszyp]

plugin.ts imports 'harper' and runs on the thread that has write access to the database, so does Harper have any public API that will flush all databases?

No, I think we will need to add one.

[Ethan-Arrowood]

I'm surprised this wasn't prefixed with like ROCKSDB_ or HARPER_, but thats an upstream thing so no concern here.

[cb1kenobi]

Good idea! Sit tight.

[Ethan-Arrowood]

looks like this needs to wait until Harper is updated (or flushDatabases is added to the types export?)

[cb1kenobi]

Yeah, this PR needs to wait a little bit.

[Ethan-Arrowood]

I will convert to draft. Feel free to click the "Ready for Review" button when its good to go again.

[cb1kenobi]

We're just waiting on Harper 5.1.0 to ship.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	harper@​5.0.9 ⏵ 5.1.23	-34		-12	+1
	@​harperfast/​integration-testing@​0.3.1 ⏵ 0.7.0	-1		+1	+5
	@​types/​node@​20.19.39 ⏵ 26.1.1			+1	+2
	@​playwright/​test@​1.59.1 ⏵ 1.62.0

View full report

[socket-security]

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Action	Severity	Alert  (click "▶" to expand/collapse)
Warn		Obfuscated code: npm alasql is 90.0% likely obfuscated Confidence: 0.90 Location: Package overview From: package-lock.json → npm/harper@5.1.23 → npm/alasql@4.17.3 ℹ Read more on: This package | This alert | What is obfuscated code? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/alasql@4.17.3. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.
Warn		Obfuscated code: npm alasql is 90.0% likely obfuscated Confidence: 0.90 Location: Package overview From: package-lock.json → npm/harper@5.1.23 → npm/alasql@4.17.3 ℹ Read more on: This package | This alert | What is obfuscated code? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/alasql@4.17.3. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.
Warn		Obfuscated code: npm brace-expansion is 90.0% likely obfuscated Confidence: 0.90 Location: Package overview From: package-lock.json → npm/harper@5.1.23 → npm/brace-expansion@5.0.4 ℹ Read more on: This package | This alert | What is obfuscated code? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/brace-expansion@5.0.4. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.
Warn		Includes npm shrinkwrap: npm harper Location: Package overview From: package-lock.json → npm/harper@5.1.23 ℹ Read more on: This package | This alert | What is a shrinkwrap file? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Packages should never use npm shrinkwrap files due to the dangers they pose. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/harper@5.1.23. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.
Warn		Obfuscated code: npm lmdb is 90.0% likely obfuscated Confidence: 0.90 Location: Package overview From: package-lock.json → npm/harper@5.1.23 → npm/lmdb@3.5.6 ℹ Read more on: This package | This alert | What is obfuscated code? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/lmdb@3.5.6. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.
Warn		High CVE: Picomatch has a ReDoS vulnerability via extglob quantifiers CVE: GHSA-c2c7-rcm5-vvqj Picomatch has a ReDoS vulnerability via extglob quantifiers (HIGH) Affected versions: >= 4.0.0 < 4.0.4; >= 3.0.0 < 3.0.2; < 2.3.2 Patched version: 2.3.2 From: package-lock.json → npm/harper@5.1.23 → npm/picomatch@2.3.1 ℹ Read more on: This package | This alert | What is a CVE? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Remove or replace dependencies that include known high severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/picomatch@2.3.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.
Warn		Obfuscated code: npm robust-predicates is 90.0% likely obfuscated Confidence: 0.90 Location: Package overview From: package-lock.json → npm/harper@5.1.23 → npm/robust-predicates@3.0.2 ℹ Read more on: This package | This alert | What is obfuscated code? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/robust-predicates@3.0.2. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.
Warn		Deprecated by its maintainer: npm glob Reason: Old versions of glob are not supported, and contain widely publicized security vulnerabilities, which have been fixed in the current version. Please update. Support for old versions may be purchased (at exorbitant rates) by contacting i@izs.me From: package-lock.json → npm/harper@5.1.23 → npm/glob@7.2.3 ℹ Read more on: This package | This alert | What is a deprecated package? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Research the state of the package and determine if there are non-deprecated versions that can be used, or if it should be replaced with a new, supported solution. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/glob@7.2.3. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.
Warn		Deprecated by its maintainer: npm inflight Reason: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful. From: package-lock.json → npm/harper@5.1.23 → npm/inflight@1.0.6 ℹ Read more on: This package | This alert | What is a deprecated package? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Research the state of the package and determine if there are non-deprecated versions that can be used, or if it should be replaced with a new, supported solution. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/inflight@1.0.6. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.
Warn		Deprecated by its maintainer: npm rimraf Reason: Rimraf versions prior to v4 are no longer supported From: package-lock.json → npm/harper@5.1.23 → npm/rimraf@3.0.2 ℹ Read more on: This package | This alert | What is a deprecated package? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Research the state of the package and determine if there are non-deprecated versions that can be used, or if it should be replaced with a new, supported solution. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/rimraf@3.0.2. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

View full report

[dawsontoth]

@cb1kenobi think this is ready for a rebase? Do we have the right stuff published where this can happen?

[cb1kenobi]

@dawsontoth Updated the branch. Dropped Node.js 20 from test matrix, added Node.js 26. Had to update some dependencies to get the tests to run.

[dawsontoth]

Nice updates, though!

[dawsontoth]

@cb1kenobi if we get the GHA for release in before this one, then we can rest real releases.

[cb1kenobi]

Hmm, tests are failing due to "Error: ENOSPC: no space left on device". That can't be good.

[dawsontoth]

can bAIrber take a look at the failing integration tests?

[kriszyp]

I think this would still be great to update and merge.

[dawsontoth]

The wait is over but the blocker moved — #60 targets this branch with the fix, plus the regression test this was missing.

@kriszyp @Ethan-Arrowood on "we'll need to add one" / "wait until Harper is updated": Harper already has flushDatabases, exported from index.ts since v5.1.0, and this branch pins 5.1.23. That's not what's failing.

What's failing is that components don't see the package's exports. Harper hands them a SyntheticModule built from a hardcoded allowlist — getHarperExports() in security/jsLoader.ts — and flushDatabases isn't on it (still isn't on main). A named import of a non-allowlisted symbol fails at link time, which kills the whole component:

SyntaxError: The requested module 'harper' does not provide an export named 'flushDatabases'

So @harperfast/nextjs never loads and no Next.js server starts. That's the current CI failure — 15 of 21 tests, and the 6 that pass only hit Harper's own operations/REST endpoints. It's visible only in the harper-logs-node-* artifacts; Playwright just reports missing elements, which is what makes it look like a Next.js problem. (The earlier ENOSPC was separate and is gone.)

HARPER_READONLY itself works fine in 5.1.23.

The flush does work today — under applications.moduleLoader: native, which bypasses the VM loader and resolves the real package. I verified this rather than assuming it: #60 runs the same fixture on both loaders. On the default loader the prerendered page renders an empty list while GET /Dog/rex returns Rex; under native the flush runs and the row is there. So the data-visibility problem Kris predicted is real and observable, and the flush is the thing that closes it.

Two follow-ups worth deciding:

1. Add flushDatabases to getHarperExports() so the flush works on the default loader. One line in harper. Happy to open it — say the word.
2. applications.moduleLoader: 'none' in our own integrationTests/fixture.ts doesn't mean "no sandbox". importScoped only skips the VM when the value is exactly 'native', so 'none' falls through to the most sandboxed loader. That's why CI hits the synthetic-module path at all. Left as-is in fix: unbreak integration tests on the read-only build branch, and cover it with a test #60 since changing it would alter what the suite covers, but it's probably not what was intended.

Also fixed in #60: --install-links was silently serving stale plugin code. It copies file:../.. rather than symlinking, and npm re-extracts that copy from its cache, so a rebuilt dist/ never reached the fixtures — install:fixtures now clears the copy and npm's hidden .package-lock.json.

---

🤖 Generated by Claude Code (Opus 5), reviewed by @dawsontoth

[kriszyp]

Great to see this coming together! Just a note in there about version requirements.
_{🤖 Reviewed with GPT 5.6}