Skip to content

fix(cli): bump firecrawl dep to 4.26.0 for proxy CONNECT fix - #173

Open
tbontb-iaq wants to merge 1 commit into
firecrawl:mainfrom
tbontb-iaq:fix/proxy-axios-1.16.1-issue-172
Open

fix(cli): bump firecrawl dep to 4.26.0 for proxy CONNECT fix#173
tbontb-iaq wants to merge 1 commit into
firecrawl:mainfrom
tbontb-iaq:fix/proxy-axios-1.16.1-issue-172

Conversation

@tbontb-iaq

@tbontb-iaq tbontb-iaq commented Jul 29, 2026

Copy link
Copy Markdown

What

Bumps the firecrawl SDK dependency from 4.24.0 to 4.26.0 so the bundled axios upgrades from 1.15.2 to 1.18.0.

Fixes #172.

Why

axios@1.15.2 (the exact pin inside firecrawl4.24.0) predates the CONNECT-tunneling fix shipped in axios@1.16.1 (axios/axios#10858, tracked by axios#6320 / axios#6330). Pre-1.16.1 axios sends HTTPS requests through an HTTP proxy in forward-proxy style (POST https://host/path HTTP/1.1) instead of issuing a CONNECT tunnel, so every proxy that only handles CONNECT for HTTPS (xray, squid, tinyproxy, mitmproxy, ...) rejects CLI requests with a non-2xx.

Observed in the wild behind https_proxy=http://127.0.0.1:10808:

$ firecrawl scrape "https://example.com" --json
Error: Request failed with status code 404

Workaround until now: no_proxy=api.firecrawl.dev firecrawl scrape ... (forces the SDK to skip the proxy).

The smallest change that resolves the bug is bumping the firecrawl dep. The axios CONNECT fix first ships in firecrawl@4.25.5 (which bundles axios@1.16.1); 4.26.0 bundles the newer axios@1.18.0 and was chosen to pick up the latest patch in the fix line while keeping the diff minimal. Bumping only to 4.26.0 (rather than straight to 4.31.1) avoids pulling in unrelated changes from later SDK releases.

How I verified

Check Result
pnpm type-check clean
pnpm test (vitest) 390 passed, 1 skipped (no new failures vs baseline)
axios@1.15.2 through local xray proxy ERR status=400 (bug reproduced)
axios@1.18.0 through same proxy 200 OK (CONNECT tunneling works)

Smoke-test capture (ad-hoc script, not included in this PR):

$ node scripts/smoke-proxy.cjs node_modules/.pnpm/axios@1.18.0/node_modules/axios https://example.com
axios version: 1.18.0
OK status=200 bytes=559

$ node scripts/smoke-proxy.cjs node_modules/.pnpm/axios@1.15.2/node_modules/axios https://example.com
axios version: 1.15.2
ERR status=400

Risk

Low. The CLI only consumes public SDK types (Firecrawl, FirecrawlClientOptions, FormatOption, AgentWebhookConfig) and constructor patterns that are stable across 4.24.0 -> 4.26.0. Type-check and full vitest suite pass with no source changes.

Alternatives considered

  • pnpm.overrides forcing axios to >=1.16.1 while keeping firecrawl at 4.24.0. Works, but forks the dep graph away from what the SDK authors publish; a bump is the upstream-aligned fix.
  • Bump straight to latest 4.31.1. Also works, but pulls in seven minor versions of unrelated changes. Bumping only to 4.26.0 keeps the diff minimal and the review surface focused on the proxy bug.

AI assistance disclosure

This pull request was prepared with AI assistance. An AI agent (Sisyphus, running on GLM 5.2 via OhMyOpenCode) performed the dependency analysis, ran the verification steps described above, and authored the initial commit and PR description. The fix itself (a one-line dependency bump) was verified manually: the type-check, test suite, and proxy smoke test were all executed and confirmed by the human author before opening this PR. A separate AI-assisted code review (Oracle subagent) was also run on the diff; its findings (a factual correction to the minimum-version claim, now applied) are reflected in this description.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Bumped firecrawl to 4.26.0 to pull in axios@1.18.0, fixing HTTPS proxy CONNECT tunneling so the CLI works behind HTTP(S) proxies. Fixes #172.

  • Bug Fixes
    • HTTPS requests now tunnel via CONNECT behind proxies (e.g., squid, mitmproxy).
    • The no_proxy=api.firecrawl.dev workaround is no longer needed.

Written for commit 22ad2b6. Summary will update on new commits.

Review in cubic

Fixes firecrawl#172.

The firecrawl SDK pinned axios at exactly 1.15.2, which predates the
CONNECT-tunneling fix shipped in axios@1.16.1 (axios/axios#10858).
Pre-1.16.1 axios forwards HTTPS requests to an HTTP proxy in
forward-proxy style (POST https://host/path HTTP/1.1) instead of
issuing a CONNECT tunnel, so any proxy that only handles CONNECT for
HTTPS (xray, squid, tinyproxy, mitmproxy, ...) rejects every CLI
request with a non-2xx (observed 404 on xray, 400 on others).

firecrawl@4.26.0 already ships axios@1.18.0, so bumping the dep is
the smallest change that resolves the bug without touching CLI source
code or pulling in unrelated SDK changes from later releases.

Verified:
- pnpm type-check clean
- 390 vitest tests pass
- axios@1.15.2 via local xray proxy -> ERR status=400 (bug reproduced)
- axios@1.18.0 via same proxy -> 200 OK (CONNECT tunneling works)

Workaround no_proxy=api.firecrawl.dev is no longer required.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

firecrawl CLI fails with HTTP 404 through HTTP proxy (bundled axios < 1.16.1, no CONNECT tunneling for HTTPS)

1 participant