diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index 19d476e..356b7ed 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -12,7 +12,7 @@ body: id: version attributes: label: ipcheck version - placeholder: ipcheck 0.8.1 + placeholder: ipcheck 0.8.2 validations: required: true - type: dropdown diff --git a/CHANGELOG.md b/CHANGELOG.md index d79530b..7f18580 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,23 @@ All notable changes to ipcheck are documented here. The project follows ## [Unreleased] +## [0.8.2] - 2026-07-23 + +### Fixed + +- Probe the current ChatGPT Codex and OpenAI Responses protocol routes instead + of generic website/model-list endpoints, choosing the primary route from the + detected Codex login mode when possible. +- Remove unsafe `eval` use while reading proxy environment variables. +- Skip misleading OpenAI/Anthropic probes when Codex or Claude Code uses a + provider-native protocol (including Bedrock, Vertex AI, Foundry, and Mantle) + unless an explicit provider endpoint is supplied; report it as unavailable + instead of scoring the wrong route. +- Clarify that probe TTFB excludes authenticated model generation, document the + P95/jitter calculation, and align `NO_PROXY` handling with current Claude Code. +- Bump JSON `schema_version` to 2 for the new `SKIPPED` service and + `UNAVAILABLE` overall result values. + ## [0.8.1] - 2026-07-23 ### Changed @@ -132,7 +149,8 @@ All notable changes to ipcheck are documented here. The project follows - Homebrew and direct-download packaging. - Median/P95 TTFB, jitter, reference bandwidth, and macOS `networkQuality`. -[Unreleased]: https://github.com/jacklv-coder/ipcheck/compare/v0.8.1...HEAD +[Unreleased]: https://github.com/jacklv-coder/ipcheck/compare/v0.8.2...HEAD +[0.8.2]: https://github.com/jacklv-coder/ipcheck/compare/v0.8.1...v0.8.2 [0.8.1]: https://github.com/jacklv-coder/ipcheck/compare/v0.8.0...v0.8.1 [0.8.0]: https://github.com/jacklv-coder/ipcheck/compare/v0.7.0...v0.8.0 [0.7.0]: https://github.com/jacklv-coder/ipcheck/compare/v0.6.1...v0.7.0 diff --git a/README.md b/README.md index d7d6268..c9cba48 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,8 @@ proxy before it breaks your coding flow. `ipcheck` is a zero-dependency Bash CLI that tests the real service routes used by AI coding clients. It reports reachability, median/P95 time to first byte, jitter, reference download/upload speed, and a plain-language readiness score. -It never reads API keys, sends prompts, or makes billable model requests. +No Codex or Claude login is required. It never reads API keys, sends prompts, +or makes billable model requests. ![Animated ipcheck terminal demo showing healthy and limited networks](assets/ipcheck-demo.gif) @@ -34,7 +35,7 @@ brew upgrade ipcheck ```bash mkdir -p "$HOME/.local/bin" -curl -fsSL https://raw.githubusercontent.com/jacklv-coder/ipcheck/v0.8.1/bin/ipcheck \ +curl -fsSL https://raw.githubusercontent.com/jacklv-coder/ipcheck/v0.8.2/bin/ipcheck \ -o "$HOME/.local/bin/ipcheck" chmod +x "$HOME/.local/bin/ipcheck" ``` @@ -71,10 +72,16 @@ interactive bottlenecks directly. | Client | Configuration detected | Route tested | | --- | --- | --- | -| Codex | `$CODEX_HOME/config.toml`, `model`, `openai_base_url`, custom provider | ChatGPT/OpenAI defaults or configured `/v1/responses` | +| Codex | `$CODEX_HOME/config.toml`, login mode, `model`, `openai_base_url`, custom provider | ChatGPT Codex or OpenAI `/responses`, matching the detected login when possible | | Claude Code | `settings.json`, `ANTHROPIC_BASE_URL`, `ANTHROPIC_MODEL` | Configured `${ANTHROPIC_BASE_URL}/v1/messages` | | Custom | `--endpoint`, `IPCHECK_ENDPOINTS` | User-provided HTTP/HTTPS endpoint | +Direct OpenAI/ChatGPT, Anthropic, and compatible gateway routes are auto-probed. +Provider-native routes need an explicit credential-free endpoint: use +`CODEX_NETWORK_ENDPOINTS` for Codex on Bedrock, or `CLAUDE_NETWORK_ENDPOINTS` +for Claude Code provider-native protocols such as Bedrock, Vertex AI, Foundry, +or Mantle. `ipcheck` warns instead of silently testing the wrong provider. + ## Common commands | Command | Purpose | @@ -99,10 +106,15 @@ Run `ipcheck --help` for every option and environment variable. | `FAIR` | Usable, but delayed, intermittent, rate-limited, or temporarily unhealthy | | `POOR` | Very slow, unstable, mostly unavailable, or using an invalid API route | | `BLOCKED` | No service response, or the proxy rejected the request first | +| `UNAVAILABLE` | A provider-specific route was not measured without an explicit safe endpoint | HTTP `401` and `403` count as reachable because DNS, proxying, TLS, and HTTP reached the API route. HTTP `407` means the proxy blocked the request. +TTFB is measured from a credential-free protocol request. It reflects DNS, +proxy, TLS, network, and gateway ingress—not authenticated model generation or +time to the model's first token. + The 0–100 score is a transparent rule, not a user percentile. It prioritizes service reachability and latency over peak bandwidth. See [Scoring and result rules](docs/scoring.md) for the exact calculation. diff --git a/README.zh-CN.md b/README.zh-CN.md index 4461c2e..0940274 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -11,7 +11,8 @@ `ipcheck` 是一个零依赖 Bash CLI,检测 AI 编程客户端实际使用的服务路径, 提供可达性、TTFB 中位数/P95、抖动、参考上下行速度和直白的开发适配分。 -它不会读取 API Key、发送 Prompt,也不会产生模型调用费用。 +无需登录 Codex 或 Claude;它不会读取 API Key、发送 Prompt,也不会产生 +模型调用费用。 ![ipcheck 健康网络与受限网络动态演示](assets/ipcheck-demo-zh.gif) @@ -33,7 +34,7 @@ brew upgrade ipcheck ```bash mkdir -p "$HOME/.local/bin" -curl -fsSL https://raw.githubusercontent.com/jacklv-coder/ipcheck/v0.8.1/bin/ipcheck \ +curl -fsSL https://raw.githubusercontent.com/jacklv-coder/ipcheck/v0.8.2/bin/ipcheck \ -o "$HOME/.local/bin/ipcheck" chmod +x "$HOME/.local/bin/ipcheck" ``` @@ -66,10 +67,15 @@ chmod +x "$HOME/.local/bin/ipcheck" | 客户端 | 自动识别的配置 | 检测路径 | | --- | --- | --- | -| Codex | `$CODEX_HOME/config.toml`、`model`、`openai_base_url`、自定义 provider | ChatGPT/OpenAI 默认路径或 `/v1/responses` | +| Codex | `$CODEX_HOME/config.toml`、登录方式、`model`、`openai_base_url`、自定义 provider | 尽可能按登录方式选择 ChatGPT Codex 或 OpenAI `/responses` | | Claude Code | `settings.json`、`ANTHROPIC_BASE_URL`、`ANTHROPIC_MODEL` | `${ANTHROPIC_BASE_URL}/v1/messages` | | 自定义 | `--endpoint`、`IPCHECK_ENDPOINTS` | 用户指定的 HTTP/HTTPS 地址 | +OpenAI/ChatGPT、Anthropic 直连与兼容网关会自动探测。provider 原生路径需要 +明确提供无凭据检测地址:Codex on Bedrock 使用 `CODEX_NETWORK_ENDPOINTS`; +Claude Code 的 Bedrock、Vertex AI、Foundry、Mantle 等 provider 原生协议使用 +`CLAUDE_NETWORK_ENDPOINTS`。`ipcheck` 会给出警告,而不是静默检测错误的 provider。 + ## 常用命令 | 命令 | 用途 | @@ -94,10 +100,14 @@ chmod +x "$HOME/.local/bin/ipcheck" | `FAIR` | 可以使用,但有延迟、波动、限流或临时服务异常 | | `POOR` | 非常慢、不稳定、多数不可用,或 API 路径配置错误 | | `BLOCKED` | 没有服务响应,或请求先被代理拒绝 | +| `UNAVAILABLE` | provider 专用路径因缺少明确的安全检测地址而未被测量 | HTTP `401` 和 `403` 代表链路可达,因为 DNS、代理、TLS 和 HTTP 已经到达 API 路径;HTTP `407` 表示请求被代理拦截。 +TTFB 来自无凭据协议请求,反映 DNS、代理、TLS、网络与网关入口耗时;它不 +包含认证后的模型生成时间,也不是模型首个 Token 的到达时间。 + 0–100 分是透明的规则评分,不代表用户百分位,并且服务可达性和交互延迟 比峰值带宽更重要。精确计算方式见[评分与结论规则](docs/scoring.zh-CN.md)。 diff --git a/assets/ipcheck-demo-zh.gif b/assets/ipcheck-demo-zh.gif index b7f91e1..e3eed44 100644 Binary files a/assets/ipcheck-demo-zh.gif and b/assets/ipcheck-demo-zh.gif differ diff --git a/assets/ipcheck-demo.gif b/assets/ipcheck-demo.gif index fc8c002..3d2371e 100644 Binary files a/assets/ipcheck-demo.gif and b/assets/ipcheck-demo.gif differ diff --git a/assets/ipcheck-preview-zh.png b/assets/ipcheck-preview-zh.png index 53fea8a..d7bf781 100644 Binary files a/assets/ipcheck-preview-zh.png and b/assets/ipcheck-preview-zh.png differ diff --git a/assets/ipcheck-preview.png b/assets/ipcheck-preview.png index ba0ae1e..ec4fce6 100644 Binary files a/assets/ipcheck-preview.png and b/assets/ipcheck-preview.png differ diff --git a/bin/ipcheck b/bin/ipcheck index 02047a2..c6f677a 100755 --- a/bin/ipcheck +++ b/bin/ipcheck @@ -10,8 +10,8 @@ unset LC_ALL LC_NUMERIC=C export LC_NUMERIC -VERSION="0.8.1" -SCHEMA_VERSION=1 +VERSION="0.8.2" +SCHEMA_VERSION=2 SAMPLES=3 TIMEOUT=15 SERVICE_MODE="auto" @@ -84,6 +84,13 @@ is_integer() { esac } +is_truthy() { + case "$1" in + 1|true|TRUE|yes|YES) return 0 ;; + *) return 1 ;; + esac +} + validate_service() { case "$1" in auto|codex|claude|all) ;; @@ -316,9 +323,13 @@ localized_reason() { "The API route is reachable, but at least one sample was rate limited (HTTP 429).") printf 'API 路由可以访问,但至少一次测试触发了限流(HTTP 429)。' ;; "The API route is reachable, but at least one sample returned a server error.") printf 'API 路由可以访问,但至少一次测试返回服务器错误。' ;; "The service path is reachable, but one or more sampled requests failed.") printf '服务链路可以访问,但一次或多次测试请求失败。' ;; - "The service path is reachable with acceptable first-byte latency and jitter.") printf '服务链路正常,首字节延迟和抖动都在适合开发的范围内。' ;; - "The service path is usable, but interactive responses may feel delayed.") printf '服务链路可用,但交互响应可能会感觉偏慢。' ;; - "The service path is reachable but has high latency or unstable responses.") printf '服务链路可以访问,但延迟较高或响应不稳定。' ;; + "The protocol path is reachable with low probe TTFB and stable samples.") printf '协议链路可以访问,探测首字节延迟较低且样本稳定。' ;; + "The protocol path is usable, but network or gateway ingress may add noticeable delay.") printf '协议链路可以使用,但网络或网关入口可能带来明显等待。' ;; + "The protocol path is reachable but has high probe latency or unstable samples.") printf '协议链路可以访问,但探测延迟较高或样本不稳定。' ;; + "Codex is configured for Amazon Bedrock; automatic OpenAI protocol probing was skipped. Set CODEX_NETWORK_ENDPOINTS to test the provider route explicitly.") printf 'Codex 已配置 Amazon Bedrock;已跳过 OpenAI 协议自动探测。请设置 CODEX_NETWORK_ENDPOINTS 明确指定要测试的服务路径。' ;; + "Claude Code is configured for Amazon Bedrock; automatic Anthropic protocol probing was skipped. Set CLAUDE_NETWORK_ENDPOINTS to test the provider route explicitly.") printf 'Claude Code 已配置 Amazon Bedrock;已跳过 Anthropic 协议自动探测。请设置 CLAUDE_NETWORK_ENDPOINTS 明确指定要测试的服务路径。' ;; + "Claude Code is configured for Google Vertex AI; automatic Anthropic protocol probing was skipped. Set CLAUDE_NETWORK_ENDPOINTS to test the provider route explicitly.") printf 'Claude Code 已配置 Google Vertex AI;已跳过 Anthropic 协议自动探测。请设置 CLAUDE_NETWORK_ENDPOINTS 明确指定要测试的服务路径。' ;; + "Claude Code is configured for a provider-specific protocol; automatic Anthropic protocol probing was skipped. Set CLAUDE_NETWORK_ENDPOINTS to test the provider route explicitly.") printf 'Claude Code 已配置 provider 专用协议;已跳过 Anthropic 协议自动探测。请设置 CLAUDE_NETWORK_ENDPOINTS 明确指定要测试的服务路径。' ;; *) printf '%s' "$1" ;; esac } @@ -331,7 +342,11 @@ localized_warning() { case "$1" in "Claude Code does not support SOCKS proxies; configure an HTTP/HTTPS proxy or gateway.") printf 'Claude Code 不支持 SOCKS 代理;请配置 HTTP/HTTPS 代理或网关。' ;; "Claude Code does not document ALL_PROXY; Claude probes ignore it unless HTTPS_PROXY or HTTP_PROXY is also configured.") printf 'Claude Code 未声明支持 ALL_PROXY;除非同时配置 HTTPS_PROXY 或 HTTP_PROXY,否则 Claude 测试会忽略它。' ;; - "A macOS system proxy is configured, but Claude Code expects HTTPS_PROXY/HTTP_PROXY; Claude probes use the direct path.") printf 'macOS 已配置系统代理,但 Claude Code 需要 HTTPS_PROXY/HTTP_PROXY;当前 Claude 测试使用直连链路。' ;; + "A macOS HTTP proxy is configured, but Claude Code documents HTTPS_PROXY/HTTP_PROXY; the probe does not apply the macOS proxy explicitly, while OS/VPN routing may still carry the traffic.") printf 'macOS 已配置 HTTP 代理,但 Claude Code 官方声明使用 HTTPS_PROXY/HTTP_PROXY;当前探测不会显式应用 macOS 代理,但系统或 VPN/TUN 路由仍可能承载流量。' ;; + "Codex is configured for Amazon Bedrock; automatic OpenAI protocol probing was skipped. Set CODEX_NETWORK_ENDPOINTS to test the provider route explicitly.") printf 'Codex 已配置 Amazon Bedrock;已跳过 OpenAI 协议自动探测。请设置 CODEX_NETWORK_ENDPOINTS 明确指定要测试的服务路径。' ;; + "Claude Code is configured for Amazon Bedrock; automatic Anthropic protocol probing was skipped. Set CLAUDE_NETWORK_ENDPOINTS to test the provider route explicitly.") printf 'Claude Code 已配置 Amazon Bedrock;已跳过 Anthropic 协议自动探测。请设置 CLAUDE_NETWORK_ENDPOINTS 明确指定要测试的服务路径。' ;; + "Claude Code is configured for Google Vertex AI; automatic Anthropic protocol probing was skipped. Set CLAUDE_NETWORK_ENDPOINTS to test the provider route explicitly.") printf 'Claude Code 已配置 Google Vertex AI;已跳过 Anthropic 协议自动探测。请设置 CLAUDE_NETWORK_ENDPOINTS 明确指定要测试的服务路径。' ;; + "Claude Code is configured for a provider-specific protocol; automatic Anthropic protocol probing was skipped. Set CLAUDE_NETWORK_ENDPOINTS to test the provider route explicitly.") printf 'Claude Code 已配置 provider 专用协议;已跳过 Anthropic 协议自动探测。请设置 CLAUDE_NETWORK_ENDPOINTS 明确指定要测试的服务路径。' ;; *) printf '%s' "$1" ;; esac } @@ -411,6 +426,7 @@ INDEXED_PROBES_FILE="$WORK_DIR/indexed-probes.tsv" ENDPOINT_SUMMARY_FILE="$WORK_DIR/endpoint-summary.tsv" ENDPOINT_JSON_FILE="$WORK_DIR/endpoints.jsonl" SERVICE_RESULT_FILE="$WORK_DIR/service-results.tsv" +SKIPPED_SERVICES_FILE="$WORK_DIR/skipped-services.tsv" WARNINGS_FILE="$WORK_DIR/warnings.txt" : > "$SERVICES_FILE" : > "$PROBES_FILE" @@ -418,6 +434,7 @@ WARNINGS_FILE="$WORK_DIR/warnings.txt" : > "$ENDPOINT_SUMMARY_FILE" : > "$ENDPOINT_JSON_FILE" : > "$SERVICE_RESULT_FILE" +: > "$SKIPPED_SERVICES_FILE" : > "$WARNINGS_FILE" json_escape() { @@ -511,7 +528,10 @@ json_setting() { file="$1" key="$2" [ -f "$file" ] || return 0 - sed -n -E 's/.*"'"$key"'"[[:space:]]*:[[:space:]]*"([^"]*)".*/\1/p' "$file" | head -n 1 + sed -n -E \ + -e 's/.*"'"$key"'"[[:space:]]*:[[:space:]]*"([^"]*)".*/\1/p' \ + -e 's/.*"'"$key"'"[[:space:]]*:[[:space:]]*(true|false|TRUE|FALSE|1|0).*/\1/p' \ + "$file" | head -n 1 } append_path() { @@ -524,6 +544,28 @@ append_path() { esac } +append_route() { + local base route + base=$(printf '%s' "$1" | sed 's#/*$##') + route="$2" + case "$base" in + */"$route") printf '%s' "$base" ;; + *) printf '%s/%s' "$base" "$route" ;; + esac +} + +proxy_env_value() { + case "$1" in + HTTPS_PROXY) printf '%s' "${HTTPS_PROXY-}" ;; + https_proxy) printf '%s' "${https_proxy-}" ;; + HTTP_PROXY) printf '%s' "${HTTP_PROXY-}" ;; + http_proxy) printf '%s' "${http_proxy-}" ;; + ALL_PROXY) printf '%s' "${ALL_PROXY-}" ;; + all_proxy) printf '%s' "${all_proxy-}" ;; + *) return 1 ;; + esac +} + validate_http_url() { local url authority url="$1" @@ -596,7 +638,12 @@ if [ "$SERVICE_MODE" = "auto" ]; then [ -n "${CODEX_NETWORK_ENDPOINTS-}" ] && codex_detected=1 command -v claude >/dev/null 2>&1 && claude_detected=1 [ -f "$CLAUDE_SETTINGS" ] && claude_detected=1 - [ -n "${ANTHROPIC_BASE_URL-}${ANTHROPIC_MODEL-}" ] && claude_detected=1 + [ -n "${ANTHROPIC_BASE_URL-}${ANTHROPIC_MODEL-}${ANTHROPIC_BEDROCK_BASE_URL-}${ANTHROPIC_VERTEX_BASE_URL-}${ANTHROPIC_FOUNDRY_BASE_URL-}${ANTHROPIC_AWS_BASE_URL-}${ANTHROPIC_GOOGLE_CLOUD_BASE_URL-}${ANTHROPIC_BEDROCK_MANTLE_BASE_URL-}" ] && claude_detected=1 + if is_truthy "${CLAUDE_CODE_USE_BEDROCK-}" || is_truthy "${CLAUDE_CODE_USE_VERTEX-}" || \ + is_truthy "${CLAUDE_CODE_USE_FOUNDRY-}" || is_truthy "${CLAUDE_CODE_USE_ANTHROPIC_AWS-}" || \ + is_truthy "${CLAUDE_CODE_USE_ANTHROPIC_GOOGLE_CLOUD-}" || is_truthy "${CLAUDE_CODE_USE_MANTLE-}"; then + claude_detected=1 + fi [ -n "${CLAUDE_NETWORK_ENDPOINTS-}" ] && claude_detected=1 if [ "$codex_detected" -eq 1 ] && [ "$claude_detected" -eq 1 ]; then SERVICE_MODE="all" @@ -637,52 +684,172 @@ else codex_source="config.toml openai_base_url" fi + codex_api_key_present=0 + if [ "${CODEX_API_KEY+x}" = x ] && [ "${#CODEX_API_KEY}" -gt 0 ]; then + codex_api_key_present=1 + elif [ "${OPENAI_API_KEY+x}" = x ] && [ "${#OPENAI_API_KEY}" -gt 0 ]; then + codex_api_key_present=1 + fi + codex_login_status="" + if [ "$codex_api_key_present" -eq 0 ] && command -v codex >/dev/null 2>&1; then + # Login status only needs Codex's stored configuration. Keep ambient + # bearer/API credentials out of the child process used for detection. + codex_login_status=$(env \ + -u CODEX_API_KEY -u OPENAI_API_KEY -u CODEX_ACCESS_TOKEN \ + -u AWS_BEARER_TOKEN_BEDROCK \ + codex login status 2>&1 || true) + fi + codex_auth_mode="unknown" + if [ "$codex_api_key_present" -eq 1 ]; then + codex_auth_mode="api_key" + else + case "$codex_login_status" in + *"Amazon Bedrock"*|*"amazon bedrock"*) codex_auth_mode="bedrock" ;; + *[Cc]hatGPT*) codex_auth_mode="chatgpt" ;; + *"API key"*|*"api key"*) codex_auth_mode="api_key" ;; + esac + fi + codex_special_provider="" + case "$codex_provider" in + amazon-bedrock|aws) codex_special_provider="bedrock" ;; + esac + if [ -z "$codex_base" ] && [ "$codex_auth_mode" = "bedrock" ]; then + codex_special_provider="bedrock" + fi + if [ -n "${CODEX_NETWORK_ENDPOINTS-}" ]; then add_service "codex" "Codex" "$codex_model" "custom endpoints" "CODEX_NETWORK_ENDPOINTS" add_endpoint_list "codex" "$CODEX_NETWORK_ENDPOINTS" "Codex endpoint" + elif [ "$codex_special_provider" = "bedrock" ]; then + codex_provider_warning="Codex is configured for Amazon Bedrock; automatic OpenAI protocol probing was skipped. Set CODEX_NETWORK_ENDPOINTS to test the provider route explicitly." + add_service "codex" "Codex" "$codex_model" "Amazon Bedrock (explicit endpoint required)" "provider configuration" + printf '%s\t%s\n' "codex" "$codex_provider_warning" >> "$SKIPPED_SERVICES_FILE" + printf '%s\n' "$codex_provider_warning" >> "$WARNINGS_FILE" elif [ -n "$codex_base" ]; then add_service "codex" "Codex" "$codex_model" "$codex_base" "$codex_source" add_probe "codex" "Codex Responses API" "$(append_path "$codex_base" "responses")" "openai" "primary" else - add_service "codex" "Codex" "$codex_model" "https://chatgpt.com + https://api.openai.com" "$codex_source" - add_probe "codex" "ChatGPT" "https://chatgpt.com/" "head" "primary" - add_probe "codex" "OpenAI API" "https://api.openai.com/v1/models" "head" "secondary" + codex_chatgpt_base=$(toml_top_string "$CODEX_CONFIG" "chatgpt_base_url") + [ -n "$codex_chatgpt_base" ] || codex_chatgpt_base="https://chatgpt.com/backend-api" + codex_chatgpt_base=$(printf '%s' "$codex_chatgpt_base" | sed 's#/*$##') + case "$codex_chatgpt_base" in + */codex/responses) codex_chatgpt_responses="$codex_chatgpt_base" ;; + */codex) codex_chatgpt_responses=$(append_route "$codex_chatgpt_base" "responses") ;; + *) + codex_chatgpt_api=$(append_route "$codex_chatgpt_base" "codex") + codex_chatgpt_responses=$(append_route "$codex_chatgpt_api" "responses") + ;; + esac + codex_openai_responses="https://api.openai.com/v1/responses" + if [ "$codex_auth_mode" = "api_key" ]; then + codex_source="Codex API key login" + add_service "codex" "Codex" "$codex_model" "$codex_openai_responses + $codex_chatgpt_responses" "$codex_source" + add_probe "codex" "OpenAI Responses API" "$codex_openai_responses" "openai" "primary" + add_probe "codex" "ChatGPT Codex API" "$codex_chatgpt_responses" "openai" "secondary" + else + [ "$codex_auth_mode" != "chatgpt" ] || codex_source="Codex ChatGPT login" + add_service "codex" "Codex" "$codex_model" "$codex_chatgpt_responses + $codex_openai_responses" "$codex_source" + add_probe "codex" "ChatGPT Codex API" "$codex_chatgpt_responses" "openai" "primary" + add_probe "codex" "OpenAI Responses API" "$codex_openai_responses" "openai" "secondary" + fi fi fi if [ "$SERVICE_MODE" = "claude" ] || [ "$SERVICE_MODE" = "all" ]; then - claude_base="${ANTHROPIC_BASE_URL-}" - claude_model="${ANTHROPIC_MODEL-}" - claude_source="environment" - if [ -z "$claude_base" ]; then - claude_base=$(json_setting "$CLAUDE_SETTINGS" "ANTHROPIC_BASE_URL") - claude_source="$CLAUDE_SETTINGS_SOURCE" - fi - if [ -z "$claude_model" ]; then - claude_model=$(json_setting "$CLAUDE_SETTINGS" "ANTHROPIC_MODEL") - fi - if [ -z "$claude_base" ]; then - claude_base="https://api.anthropic.com" - claude_source="Claude Code default" + claude_use_bedrock="${CLAUDE_CODE_USE_BEDROCK-}" + claude_use_vertex="${CLAUDE_CODE_USE_VERTEX-}" + claude_use_foundry="${CLAUDE_CODE_USE_FOUNDRY-}" + claude_use_anthropic_aws="${CLAUDE_CODE_USE_ANTHROPIC_AWS-}" + claude_use_anthropic_google="${CLAUDE_CODE_USE_ANTHROPIC_GOOGLE_CLOUD-}" + claude_use_mantle="${CLAUDE_CODE_USE_MANTLE-}" + claude_bedrock_base="${ANTHROPIC_BEDROCK_BASE_URL-}" + claude_vertex_base="${ANTHROPIC_VERTEX_BASE_URL-}" + claude_foundry_base="${ANTHROPIC_FOUNDRY_BASE_URL-}" + claude_anthropic_aws_base="${ANTHROPIC_AWS_BASE_URL-}" + claude_anthropic_google_base="${ANTHROPIC_GOOGLE_CLOUD_BASE_URL-}" + claude_mantle_base="${ANTHROPIC_BEDROCK_MANTLE_BASE_URL-}" + [ -n "$claude_use_bedrock" ] || claude_use_bedrock=$(json_setting "$CLAUDE_SETTINGS" "CLAUDE_CODE_USE_BEDROCK") + [ -n "$claude_use_vertex" ] || claude_use_vertex=$(json_setting "$CLAUDE_SETTINGS" "CLAUDE_CODE_USE_VERTEX") + [ -n "$claude_use_foundry" ] || claude_use_foundry=$(json_setting "$CLAUDE_SETTINGS" "CLAUDE_CODE_USE_FOUNDRY") + [ -n "$claude_use_anthropic_aws" ] || claude_use_anthropic_aws=$(json_setting "$CLAUDE_SETTINGS" "CLAUDE_CODE_USE_ANTHROPIC_AWS") + [ -n "$claude_use_anthropic_google" ] || claude_use_anthropic_google=$(json_setting "$CLAUDE_SETTINGS" "CLAUDE_CODE_USE_ANTHROPIC_GOOGLE_CLOUD") + [ -n "$claude_use_mantle" ] || claude_use_mantle=$(json_setting "$CLAUDE_SETTINGS" "CLAUDE_CODE_USE_MANTLE") + [ -n "$claude_bedrock_base" ] || claude_bedrock_base=$(json_setting "$CLAUDE_SETTINGS" "ANTHROPIC_BEDROCK_BASE_URL") + [ -n "$claude_vertex_base" ] || claude_vertex_base=$(json_setting "$CLAUDE_SETTINGS" "ANTHROPIC_VERTEX_BASE_URL") + [ -n "$claude_foundry_base" ] || claude_foundry_base=$(json_setting "$CLAUDE_SETTINGS" "ANTHROPIC_FOUNDRY_BASE_URL") + [ -n "$claude_anthropic_aws_base" ] || claude_anthropic_aws_base=$(json_setting "$CLAUDE_SETTINGS" "ANTHROPIC_AWS_BASE_URL") + [ -n "$claude_anthropic_google_base" ] || claude_anthropic_google_base=$(json_setting "$CLAUDE_SETTINGS" "ANTHROPIC_GOOGLE_CLOUD_BASE_URL") + [ -n "$claude_mantle_base" ] || claude_mantle_base=$(json_setting "$CLAUDE_SETTINGS" "ANTHROPIC_BEDROCK_MANTLE_BASE_URL") + claude_special_provider="" + is_truthy "$claude_use_bedrock" && claude_special_provider="bedrock" + is_truthy "$claude_use_vertex" && claude_special_provider="vertex" + is_truthy "$claude_use_foundry" && claude_special_provider="Microsoft Foundry" + is_truthy "$claude_use_anthropic_aws" && claude_special_provider="Anthropic on AWS" + is_truthy "$claude_use_anthropic_google" && claude_special_provider="Anthropic on Google Cloud" + is_truthy "$claude_use_mantle" && claude_special_provider="Amazon Bedrock Mantle" + [ -z "$claude_bedrock_base" ] || claude_special_provider="bedrock" + [ -z "$claude_vertex_base" ] || claude_special_provider="vertex" + [ -z "$claude_foundry_base" ] || claude_special_provider="Microsoft Foundry" + [ -z "$claude_anthropic_aws_base" ] || claude_special_provider="Anthropic on AWS" + [ -z "$claude_anthropic_google_base" ] || claude_special_provider="Anthropic on Google Cloud" + [ -z "$claude_mantle_base" ] || claude_special_provider="Amazon Bedrock Mantle" + + if [ -n "$claude_special_provider" ] && [ -z "${CLAUDE_NETWORK_ENDPOINTS-}" ]; then + if [ "$claude_special_provider" = "bedrock" ]; then + claude_provider_warning="Claude Code is configured for Amazon Bedrock; automatic Anthropic protocol probing was skipped. Set CLAUDE_NETWORK_ENDPOINTS to test the provider route explicitly." + claude_provider_route="Amazon Bedrock (explicit endpoint required)" + elif [ "$claude_special_provider" = "vertex" ]; then + claude_provider_warning="Claude Code is configured for Google Vertex AI; automatic Anthropic protocol probing was skipped. Set CLAUDE_NETWORK_ENDPOINTS to test the provider route explicitly." + claude_provider_route="Google Vertex AI (explicit endpoint required)" + else + claude_provider_warning="Claude Code is configured for a provider-specific protocol; automatic Anthropic protocol probing was skipped. Set CLAUDE_NETWORK_ENDPOINTS to test the provider route explicitly." + claude_provider_route="$claude_special_provider (explicit endpoint required)" + fi + claude_model="${ANTHROPIC_MODEL-}" + [ -n "$claude_model" ] || claude_model=$(json_setting "$CLAUDE_SETTINGS" "ANTHROPIC_MODEL") + [ -n "$claude_model" ] || claude_model="default" + add_service "claude" "Claude Code" "$claude_model" "$claude_provider_route" "provider configuration" + printf '%s\t%s\n' "claude" "$claude_provider_warning" >> "$SKIPPED_SERVICES_FILE" + printf '%s\n' "$claude_provider_warning" >> "$WARNINGS_FILE" + continue_claude_probe=0 + else + continue_claude_probe=1 fi - [ -n "$claude_model" ] || claude_model="default" - if [ -n "${CLAUDE_NETWORK_ENDPOINTS-}" ]; then - add_service "claude" "Claude Code" "$claude_model" "custom endpoints" "CLAUDE_NETWORK_ENDPOINTS" - add_endpoint_list "claude" "$CLAUDE_NETWORK_ENDPOINTS" "Claude endpoint" - else - add_service "claude" "Claude Code" "$claude_model" "$claude_base" "$claude_source" - add_probe "claude" "Claude Messages API" "$(append_path "$claude_base" "messages")" "anthropic" "primary" - if [ "$claude_base" = "https://api.anthropic.com" ]; then - add_probe "claude" "Claude account auth" "https://claude.ai/" "head" "secondary" - add_probe "claude" "Claude Console auth" "https://platform.claude.com/" "head" "secondary" + if [ "$continue_claude_probe" -eq 1 ]; then + claude_base="${ANTHROPIC_BASE_URL-}" + claude_model="${ANTHROPIC_MODEL-}" + claude_source="environment" + if [ -z "$claude_base" ]; then + claude_base=$(json_setting "$CLAUDE_SETTINGS" "ANTHROPIC_BASE_URL") + claude_source="$CLAUDE_SETTINGS_SOURCE" + fi + if [ -z "$claude_model" ]; then + claude_model=$(json_setting "$CLAUDE_SETTINGS" "ANTHROPIC_MODEL") + fi + if [ -z "$claude_base" ]; then + claude_base="https://api.anthropic.com" + claude_source="Claude Code default" + fi + [ -n "$claude_model" ] || claude_model="default" + + if [ -n "${CLAUDE_NETWORK_ENDPOINTS-}" ]; then + add_service "claude" "Claude Code" "$claude_model" "custom endpoints" "CLAUDE_NETWORK_ENDPOINTS" + add_endpoint_list "claude" "$CLAUDE_NETWORK_ENDPOINTS" "Claude endpoint" + else + add_service "claude" "Claude Code" "$claude_model" "$claude_base" "$claude_source" + add_probe "claude" "Claude Messages API" "$(append_path "$claude_base" "messages")" "anthropic" "primary" + if [ "$claude_base" = "https://api.anthropic.com" ]; then + add_probe "claude" "Claude account auth" "https://claude.ai/" "head" "secondary" + add_probe "claude" "Claude Console auth" "https://platform.claude.com/" "head" "secondary" + fi fi fi fi fi [ -s "$SERVICES_FILE" ] || die "no services selected" -[ -s "$PROBES_FILE" ] || die "no endpoints configured" +[ -s "$PROBES_FILE" ] || [ -s "$SKIPPED_SERVICES_FILE" ] || die "no endpoints configured" # curl does not automatically use macOS System Settings proxies. For Codex and # custom probes, fall back to the configured system HTTPS proxy when no proxy @@ -703,7 +870,7 @@ any_proxy_env_found=0 claude_http_proxy="" proxy_scheme="" for proxy_name in HTTPS_PROXY https_proxy HTTP_PROXY http_proxy ALL_PROXY all_proxy; do - eval "proxy_value=\${$proxy_name-}" + proxy_value=$(proxy_env_value "$proxy_name") if [ -n "$proxy_value" ]; then any_proxy_env_found=1 case "$proxy_name" in @@ -730,7 +897,7 @@ if awk -F '\t' '$1 == "claude" { found=1 } END { exit !found }' "$SERVICES_FILE" printf '%s\n' "Claude Code does not document ALL_PROXY; Claude probes ignore it unless HTTPS_PROXY or HTTP_PROXY is also configured." >> "$WARNINGS_FILE" fi if [ -n "$system_curl_proxy" ] && [ "$any_proxy_env_found" -eq 0 ]; then - printf '%s\n' "A macOS system proxy is configured, but Claude Code expects HTTPS_PROXY/HTTP_PROXY; Claude probes use the direct path." >> "$WARNINGS_FILE" + printf '%s\n' "A macOS HTTP proxy is configured, but Claude Code documents HTTPS_PROXY/HTTP_PROXY; the probe does not apply the macOS proxy explicitly, while OS/VPN routing may still carry the traffic." >> "$WARNINGS_FILE" fi fi @@ -742,7 +909,7 @@ proxy_summary() { local name value found route_interface found=0 for name in HTTPS_PROXY https_proxy HTTP_PROXY http_proxy ALL_PROXY all_proxy; do - eval "value=\${$name-}" + value=$(proxy_env_value "$name") if [ -n "$value" ]; then [ "$found" -eq 0 ] || printf ', ' printf '%s=%s' "$name" "$(redact_proxy "$value")" @@ -952,7 +1119,7 @@ EOF fi done < "$PROBES_FILE" -[ "$endpoint_count" -gt 0 ] || die "no endpoints configured" +[ "$endpoint_count" -gt 0 ] || [ -s "$SKIPPED_SERVICES_FILE" ] || die "no endpoints configured" bandwidth_speed=0 bandwidth_code=000 @@ -1189,11 +1356,20 @@ good_services=0 fair_services=0 poor_services=0 blocked_services=0 +skipped_services=0 +measured_service_seen=0 overall_rank=0 while IFS="$TAB" read -r service display model base source; do service_count=$((service_count + 1)) + skipped_reason=$(awk -F '\t' -v service="$service" '$1 == service { print $2; exit }' "$SKIPPED_SERVICES_FILE") + if [ -n "$skipped_reason" ]; then + skipped_services=$((skipped_services + 1)) + printf '%s\t%s\t%s\t0\t0\t0\t0\n' "$service" "skipped" "$skipped_reason" >> "$SERVICE_RESULT_FILE" + continue + fi primary_index=$(awk -F '\t' -v service="$service" '$2 == service && $5 == "primary" { print $1; exit }' "$ENDPOINT_SUMMARY_FILE") [ -n "$primary_index" ] || continue + measured_service_seen=$((measured_service_seen + 1)) primary_sample_file="$WORK_DIR/endpoint_${primary_index}.tsv" service_ok=$(awk '$1 == 0 && $2 != "000" && $2 != "407" { count++ } END { print count + 0 }' "$primary_sample_file") service_usable=$(awk '$1 == 0 && $2 != "000" && $2 != "407" && $2 != "404" && $2 != "429" && $2 !~ /^5[0-9][0-9]$/ { count++ } END { print count + 0 }' "$primary_sample_file") @@ -1235,15 +1411,15 @@ while IFS="$TAB" read -r service display model base source; do service_rank=1 elif awk -v ttfb="$service_ttfb" -v jitter="$service_jitter" 'BEGIN { exit !(ttfb > 0 && ttfb < 800 && jitter < 1000) }'; then service_result="good" - service_reason="The service path is reachable with acceptable first-byte latency and jitter." + service_reason="The protocol path is reachable with low probe TTFB and stable samples." service_rank=0 elif awk -v ttfb="$service_ttfb" 'BEGIN { exit !(ttfb > 0 && ttfb < 3000) }'; then service_result="fair" - service_reason="The service path is usable, but interactive responses may feel delayed." + service_reason="The protocol path is usable, but network or gateway ingress may add noticeable delay." service_rank=1 else service_result="poor" - service_reason="The service path is reachable but has high latency or unstable responses." + service_reason="The protocol path is reachable but has high probe latency or unstable samples." service_rank=2 fi fi @@ -1254,7 +1430,7 @@ while IFS="$TAB" read -r service display model base source; do blocked) blocked_services=$((blocked_services + 1)) ;; esac [ "$service_rank" -gt "$overall_rank" ] && overall_rank="$service_rank" - if [ "$service_count" -eq 1 ]; then + if [ "$measured_service_seen" -eq 1 ]; then primary_ttfb="$service_ttfb" primary_jitter="$service_jitter" primary_success_rate="$service_rate" @@ -1263,7 +1439,10 @@ while IFS="$TAB" read -r service display model base source; do "$service" "$service_result" "$service_reason" "$service_rate" "$service_ttfb" "$service_jitter" "$service_usable" >> "$SERVICE_RESULT_FILE" done < "$SERVICES_FILE" -if [ "$blocked_services" -eq "$service_count" ]; then +measured_service_count=$((service_count - skipped_services)) +if [ "$measured_service_count" -eq 0 ]; then + overall="unavailable" +elif [ "$blocked_services" -eq "$measured_service_count" ]; then overall="blocked" elif [ "$blocked_services" -gt 0 ]; then overall="poor" @@ -1277,7 +1456,7 @@ fi if [ "$service_count" -eq 1 ]; then overall_reason=$(awk -F '\t' 'NR == 1 { print $3 }' "$SERVICE_RESULT_FILE") else - overall_reason="$good_services good, $fair_services fair, $poor_services poor, $blocked_services blocked service path(s)." + overall_reason="$good_services good, $fair_services fair, $poor_services poor, $blocked_services blocked, $skipped_services skipped service path(s)." fi case "$overall" in @@ -1296,10 +1475,10 @@ case "$overall" in readiness_ready=true readiness_level="with_caution" readiness_answer_en="YES, WITH CAUTION" - readiness_summary_en="Development is possible, but responses may feel slower than ideal." + readiness_summary_en="Development is possible, but the measured network or gateway path may add delay." readiness_recommendation_en="You can keep working. If pauses become distracting, try a lower-latency proxy route." readiness_answer_zh="可以,但会有些慢" - readiness_summary_zh="当前可以开发,但响应速度可能不够理想。" + readiness_summary_zh="当前可以开发,但测得的网络或网关链路可能带来额外等待。" readiness_recommendation_zh="可以继续工作;如果等待明显影响节奏,建议切换到延迟更低的代理节点。" else readiness_ready=false @@ -1322,6 +1501,28 @@ case "$overall" in readiness_summary_zh="当前网络很可能影响 AI 辅助开发的连续性。" readiness_recommendation_zh="建议先切换代理节点,或排查高延迟和请求失败,再开始较长时间的开发。" ;; + unavailable) + readiness_ready=false + readiness_level="unavailable" + readiness_answer_en="UNKNOWN" + readiness_summary_en="The configured provider route was not measured automatically." + skipped_codex=0 + skipped_claude=0 + awk -F '\t' '$1 == "codex" { found=1 } END { exit !found }' "$SKIPPED_SERVICES_FILE" && skipped_codex=1 + awk -F '\t' '$1 == "claude" { found=1 } END { exit !found }' "$SKIPPED_SERVICES_FILE" && skipped_claude=1 + if [ "$skipped_codex" -eq 1 ] && [ "$skipped_claude" -eq 1 ]; then + readiness_recommendation_en="Set CODEX_NETWORK_ENDPOINTS and CLAUDE_NETWORK_ENDPOINTS to credential-free provider URLs, then run ipcheck again." + readiness_recommendation_zh="请将无需凭据即可探测的 provider URL 分别设置到 CODEX_NETWORK_ENDPOINTS 与 CLAUDE_NETWORK_ENDPOINTS 后重新运行。" + elif [ "$skipped_codex" -eq 1 ]; then + readiness_recommendation_en="Set CODEX_NETWORK_ENDPOINTS to a credential-free provider URL, then run ipcheck again." + readiness_recommendation_zh="请将无需凭据即可探测的 provider URL 设置到 CODEX_NETWORK_ENDPOINTS 后重新运行。" + else + readiness_recommendation_en="Set CLAUDE_NETWORK_ENDPOINTS to a credential-free provider URL, then run ipcheck again." + readiness_recommendation_zh="请将无需凭据即可探测的 provider URL 设置到 CLAUDE_NETWORK_ENDPOINTS 后重新运行。" + fi + readiness_answer_zh="暂无法判断" + readiness_summary_zh="当前配置的 provider 路径未被自动测量。" + ;; *) readiness_ready=false readiness_level="blocked" @@ -1393,6 +1594,7 @@ score_success_rate=0 score_ttfb_value=0 score_jitter_value=0 while IFS="$TAB" read -r service result reason success_rate ttfb jitter service_usable_count; do + [ "$result" != "skipped" ] || continue case "$result" in good) result_rank=0 ;; fair) result_rank=1 ;; @@ -1487,6 +1689,10 @@ else readiness_score_label_en="LIMITED" readiness_score_label_zh="受限" fi +if [ "$overall" = "unavailable" ]; then + readiness_score_label_en="UNAVAILABLE" + readiness_score_label_zh="未测量" +fi if [ "$UI_LANG" = "zh" ]; then readiness_answer="$readiness_answer_zh" @@ -1496,7 +1702,7 @@ if [ "$UI_LANG" = "zh" ]; then if [ "$service_count" -eq 1 ]; then localized_overall_reason=$(localized_reason "$overall_reason") else - localized_overall_reason="$good_services 条良好、$fair_services 条一般、$poor_services 条较差、$blocked_services 条阻断。" + localized_overall_reason="$good_services 条良好、$fair_services 条一般、$poor_services 条较差、$blocked_services 条阻断、$skipped_services 条已跳过。" fi else readiness_answer="$readiness_answer_en" @@ -1558,11 +1764,13 @@ localized_service_result() { zh:fair) printf '一般' ;; zh:poor) printf '较差' ;; zh:blocked) printf '阻断' ;; + zh:skipped) printf '已跳过' ;; *) printf '%s' "$(printf '%s' "$1" | tr '[:lower:]' '[:upper:]')" ;; esac } render_primary_diagnosis() { + [ "$overall" != "unavailable" ] || return 0 diagnosis_count=0 if [ "$score_availability" -lt 35 ]; then diagnosis_count=$((diagnosis_count + 1)) @@ -1575,9 +1783,9 @@ render_primary_diagnosis() { if [ "$score_median_ttfb" -lt 30 ] && awk -v value="$score_ttfb_value" 'BEGIN { exit !(value >= 1500) }'; then diagnosis_count=$((diagnosis_count + 1)) if [ "$UI_LANG" = "zh" ]; then - printf ' %s!%s 首字节中位延迟 %s,AI 回复启动会明显变慢。\n' "$C_YELLOW" "$C_RESET" "$(format_ui_latency "$score_ttfb_value")" + printf ' %s!%s 首字节中位延迟 %s,网络或网关入口可能带来明显等待。\n' "$C_YELLOW" "$C_RESET" "$(format_ui_latency "$score_ttfb_value")" else - printf ' %s!%s Median TTFB is %s; AI responses will start noticeably slower.\n' "$C_YELLOW" "$C_RESET" "$(format_ui_latency "$score_ttfb_value")" + printf ' %s!%s Median TTFB is %s; network or gateway ingress may add noticeable delay.\n' "$C_YELLOW" "$C_RESET" "$(format_ui_latency "$score_ttfb_value")" fi fi if [ "$score_jitter" -lt 7 ] && awk -v value="$score_jitter_value" 'BEGIN { exit !(value >= 500) }'; then @@ -1623,10 +1831,16 @@ render_human() { 6[5-9]|7[0-9]|8[0-9]) score_color="$C_YELLOW" ;; *) score_color="$C_RED" ;; esac - if [ "$readiness_ready" = true ]; then verdict_icon="✓"; else verdict_icon="✕"; fi + if [ "$overall" = "unavailable" ]; then + verdict_icon="?" + elif [ "$readiness_ready" = true ]; then + verdict_icon="✓" + else + verdict_icon="✕" + fi case "$overall" in good) verdict_color="$C_GREEN" ;; - fair) verdict_color="$C_YELLOW" ;; + fair|unavailable) verdict_color="$C_YELLOW" ;; *) verdict_color="$C_RED" ;; esac @@ -1689,9 +1903,18 @@ render_human() { if [ "$UI_LANG" = "zh" ]; then section_title "AI 服务延迟" printf ' %sTTFB:中位/P95 越低越好;抖动越低越稳定。%s\n' "$C_DIM" "$C_RESET" + printf ' %s这里测量无凭据协议响应,不包含模型生成时间。%s\n' "$C_DIM" "$C_RESET" else section_title "AI service latency" printf ' %sTTFB: lower median/P95 is better; lower jitter is more stable.%s\n' "$C_DIM" "$C_RESET" + printf ' %sMeasures credential-free protocol responses, not model generation time.%s\n' "$C_DIM" "$C_RESET" + fi + if [ "$endpoint_count" -eq 0 ]; then + if [ "$UI_LANG" = "zh" ]; then + printf ' %s当前没有可安全自动探测的协议端点。%s\n' "$C_DIM" "$C_RESET" + else + printf ' %sNo protocol endpoint could be measured safely and automatically.%s\n' "$C_DIM" "$C_RESET" + fi fi while IFS="$TAB" read -r index service label endpoint probe_type priority; do sample_file="$WORK_DIR/endpoint_${index}.tsv" @@ -1726,14 +1949,14 @@ EOF result_separator="" printf ' ' while IFS="$TAB" read -r service result reason success_rate ttfb jitter service_usable_count; do - case "$result" in good) result_color="$C_GREEN" ;; fair) result_color="$C_YELLOW" ;; *) result_color="$C_RED" ;; esac + case "$result" in good) result_color="$C_GREEN" ;; fair|skipped) result_color="$C_YELLOW" ;; *) result_color="$C_RED" ;; esac printf '%s%s●%s %s %s%s%s' "$result_separator" "$result_color" "$C_RESET" "$(service_display "$service")" "$result_color" "$(localized_service_result "$result")" "$C_RESET" result_separator=" · " done < "$SERVICE_RESULT_FILE" printf '\n' else while IFS="$TAB" read -r service result reason success_rate ttfb jitter service_usable_count; do - case "$result" in good) result_color="$C_GREEN" ;; fair) result_color="$C_YELLOW" ;; *) result_color="$C_RED" ;; esac + case "$result" in good) result_color="$C_GREEN" ;; fair|skipped) result_color="$C_YELLOW" ;; *) result_color="$C_RED" ;; esac if [ "$EXPLAIN_SCORE" -eq 1 ]; then printf ' %s●%s %-12s %s%-7s%s %s\n' "$result_color" "$C_RESET" "$(service_display "$service")" "$result_color" "$(localized_service_result "$result")" "$C_RESET" "$(localized_reason "$reason")" else @@ -1770,10 +1993,12 @@ EOF printf '\n' fi - if [ "$UI_LANG" = "zh" ]; then - printf '\n %sHTTP 401/403 代表链路可达;不使用凭据,不产生模型费用。%s\n' "$C_DIM" "$C_RESET" - else - printf '\n %sHTTP 401/403 means reachable; no credentials or billable model calls.%s\n' "$C_DIM" "$C_RESET" + if [ "$endpoint_count" -gt 0 ]; then + if [ "$UI_LANG" = "zh" ]; then + printf '\n %sHTTP 401/403 代表链路可达;不使用凭据,不产生模型费用。%s\n' "$C_DIM" "$C_RESET" + else + printf '\n %sHTTP 401/403 means reachable; no credentials or billable model calls.%s\n' "$C_DIM" "$C_RESET" + fi fi } @@ -1823,9 +2048,11 @@ render_markdown() { fi if [ "$UI_LANG" = "zh" ]; then printf '\n## AI 服务延迟(TTFB)\n\n' + printf '> 测量的是无凭据协议响应,不包含模型生成时间。\n\n' printf '| 客户端 | 端点 | 可达 | HTTP | 成功次数 | TTFB 中位数 | TTFB P95 | 抖动 |\n|---|---|---:|---:|---:|---:|---:|---:|\n' else printf '\n## AI service latency (TTFB)\n\n' + printf '> Measures credential-free protocol responses, not model generation time.\n\n' printf '| Client | Endpoint | Reachable | HTTP | Successful samples | Median TTFB | P95 TTFB | Jitter |\n|---|---|---:|---:|---:|---:|---:|---:|\n' fi while IFS= read -r line; do @@ -1904,5 +2131,7 @@ case "$MODE" in json) render_json ;; esac -[ "$overall" = "blocked" ] && exit 1 +case "$overall" in + blocked|unavailable) exit 1 ;; +esac exit 0 diff --git a/docs/network.md b/docs/network.md index 6dc059b..dc12a33 100644 --- a/docs/network.md +++ b/docs/network.md @@ -8,7 +8,7 @@ | Client | Route | | --- | --- | -| Codex defaults | ChatGPT and OpenAI reachability paths | +| Codex defaults | ChatGPT Codex or OpenAI Responses protocol route, ordered by detected login mode | | Codex custom provider | Configured base URL plus `/v1/responses` when needed | | Claude Code | `${ANTHROPIC_BASE_URL}/v1/messages` | | Custom | Credential-free URL supplied with `--endpoint` or `IPCHECK_ENDPOINTS` | @@ -16,6 +16,17 @@ `CODEX_NETWORK_ENDPOINTS` remains supported for compatibility. Claude-specific overrides can use `CLAUDE_NETWORK_ENDPOINTS`. +Codex's built-in Amazon Bedrock provider uses provider-specific authentication. +Without `CODEX_NETWORK_ENDPOINTS`, ipcheck reports it as `SKIPPED` rather than +probing an unrelated OpenAI route. + +Direct Anthropic and Anthropic-compatible gateways are auto-probed. Provider- +native modes such as Amazon Bedrock, Google Vertex AI, Foundry, and Mantle use +provider-specific authenticated protocols, so `ipcheck` skips automatic Anthropic probing and asks for an explicit +credential-free `CLAUDE_NETWORK_ENDPOINTS` route instead of reporting a result +for the wrong provider. The provider appears as `SKIPPED` and the overall result +is `UNAVAILABLE` when no other client route can be measured. + ## Proxy behavior `HTTPS_PROXY`, `HTTP_PROXY`, `ALL_PROXY`, and lowercase variants are reported @@ -23,10 +34,15 @@ with credentials redacted. - Codex and custom checks can fall back to the macOS system HTTPS proxy when no HTTPS proxy environment variable is configured. -- Claude Code documents `HTTPS_PROXY`, `HTTP_PROXY`, and `NO_PROXY`, but not - `ALL_PROXY` or SOCKS proxies. +- Current Claude Code builds honor `HTTPS_PROXY`, `HTTP_PROXY`, and + `NO_PROXY`/`no_proxy` bypass rules. Anthropic's public corporate-proxy page + may still describe `NO_PROXY` as unsupported. `ALL_PROXY` and SOCKS proxies + are not used for Claude probes. - Claude probes therefore ignore `ALL_PROXY` unless `HTTPS_PROXY` or `HTTP_PROXY` is also configured. +- Claude probes do not explicitly copy the macOS HTTP proxy into curl. System, + VPN, or TUN routing can still carry that traffic, so this is not labeled as a + guaranteed direct connection. - `ipcheck` warns when the detected proxy is incompatible or likely to differ from the client's actual route. diff --git a/docs/network.zh-CN.md b/docs/network.zh-CN.md index 55a50f0..48a7c92 100644 --- a/docs/network.zh-CN.md +++ b/docs/network.zh-CN.md @@ -8,7 +8,7 @@ | 客户端 | 检测路径 | | --- | --- | -| Codex 默认配置 | ChatGPT 和 OpenAI 可达路径 | +| Codex 默认配置 | 尽可能按登录方式选择 ChatGPT Codex 或 OpenAI Responses 协议路径 | | Codex 自定义 provider | 配置的 Base URL,必要时补充 `/v1/responses` | | Claude Code | `${ANTHROPIC_BASE_URL}/v1/messages` | | 自定义 | 通过 `--endpoint` 或 `IPCHECK_ENDPOINTS` 提供的无凭据 URL | @@ -16,6 +16,16 @@ `CODEX_NETWORK_ENDPOINTS` 保持兼容;Claude 可使用 `CLAUDE_NETWORK_ENDPOINTS` 覆盖端点。 +Codex 内置的 Amazon Bedrock provider 使用专用认证协议。未设置 +`CODEX_NETWORK_ENDPOINTS` 时,ipcheck 会将其报告为“已跳过”,不会误测 +无关的 OpenAI 路径。 + +Anthropic 直连与 Anthropic 兼容网关会自动探测。Amazon Bedrock、Google +Vertex AI、Foundry、Mantle 等 provider 原生模式使用各自的认证协议,因此 `ipcheck` 会跳过 Anthropic 自动探测, +并要求通过 `CLAUDE_NETWORK_ENDPOINTS` 明确提供无凭据检测路径,避免对错误的 +provider 给出结论。该 provider 会显示为“已跳过”;没有其他可测客户端链路时, +总结果显示为 `UNAVAILABLE`。 + ## 代理行为 `HTTPS_PROXY`、`HTTP_PROXY`、`ALL_PROXY` 及其小写形式会被显示,其中凭据 @@ -23,10 +33,13 @@ - 没有 HTTPS 代理环境变量时,Codex 和自定义检测可以使用 macOS 系统 HTTPS 代理。 -- Claude Code 官方声明支持 `HTTPS_PROXY`、`HTTP_PROXY` 和 `NO_PROXY`, - 但没有声明支持 `ALL_PROXY` 或 SOCKS。 +- 当前 Claude Code 已实际支持 `HTTPS_PROXY`、`HTTP_PROXY` 与 + `NO_PROXY`/`no_proxy` 绕过规则;Anthropic 公开的企业代理页面可能仍写着 + `NO_PROXY` 不受支持。Claude 探测不使用 `ALL_PROXY` 或 SOCKS。 - 因此没有同时配置 `HTTPS_PROXY` 或 `HTTP_PROXY` 时,Claude 检测会忽略 `ALL_PROXY`。 +- Claude 探测不会把 macOS HTTP 代理显式传给 curl;系统、VPN 或 TUN 路由 + 仍可能承载这部分流量,因此不会再把它笼统标记为“直连”。 - 检测到不兼容或可能与客户端实际链路不同的代理时,`ipcheck` 会明确警告。 详见 Anthropic 的 diff --git a/docs/scoring.md b/docs/scoring.md index 10f9a71..e625c1f 100644 --- a/docs/scoring.md +++ b/docs/scoring.md @@ -4,12 +4,19 @@ `ipcheck` uses two related signals: -- a service result: `GOOD`, `FAIR`, `POOR`, or `BLOCKED`; +- a measured service result: `GOOD`, `FAIR`, `POOR`, or `BLOCKED`, plus + `SKIPPED` when a provider-specific route cannot be probed safely; - a 0–100 development-readiness score calculated with `rule_v2`. The score is a transparent heuristic, not a user percentile or a benchmark of model intelligence. +TTFB comes from credential-free protocol probes. It includes DNS, proxy, TLS, +network, and gateway ingress, but not authentication or model generation. P95 +uses the nearest-rank method; with the default three samples it is effectively +the slowest sample. Jitter is the root-mean-square deviation from the sample +median. Use more samples when comparing close results. + ## Service results | Result | Default rule | @@ -18,6 +25,7 @@ model intelligence. | `FAIR` | Reachable with median TTFB below 3,000 ms, recoverable failures, rate limiting, or server errors | | `POOR` | Primary success below 60%, median TTFB at least 3,000 ms, or HTTP 404 on a configured API route | | `BLOCKED` | No primary HTTP response, or proxy authentication stopped the request with HTTP 407 | +| `SKIPPED` | The provider needs an explicit credential-free endpoint before it can be measured | With multiple clients, results remain independent. If every client is blocked, the overall result is `BLOCKED`; if blocked and reachable clients are mixed, it @@ -25,6 +33,10 @@ is `POOR`; otherwise the least healthy client result is used. The readiness score always uses the lowest-scoring service path, so a healthy Claude route cannot hide a broken Codex route. +Skipped clients do not affect another measured client's score. If every +selected client is skipped, the overall result is `UNAVAILABLE`, the score is +0, and the command exits with status 1. + ## Rule v2 service-path points The selected service path contributes up to 90 points. diff --git a/docs/scoring.zh-CN.md b/docs/scoring.zh-CN.md index 9d91836..e67e840 100644 --- a/docs/scoring.zh-CN.md +++ b/docs/scoring.zh-CN.md @@ -4,11 +4,16 @@ `ipcheck` 同时给出两类信号: -- 服务结论:`GOOD`、`FAIR`、`POOR` 或 `BLOCKED`; +- 已测服务结论:`GOOD`、`FAIR`、`POOR` 或 `BLOCKED`;provider 专用路径无法 + 安全自动探测时为 `SKIPPED`; - 使用 `rule_v2` 计算的 0–100 开发适配分。 分数是透明的启发式规则,不代表用户百分位,也不评价模型智能水平。 +TTFB 来自无凭据协议探测,包含 DNS、代理、TLS、网络与网关入口耗时,但不 +包含认证和模型生成。P95 使用最近秩算法;默认三次采样时,它实际上等于最慢 +样本。抖动是各样本相对中位数的均方根偏差。比较接近的结果时建议增加采样数。 + ## 服务结论 | 结果 | 默认规则 | @@ -17,11 +22,15 @@ | `FAIR` | 网络可达且中位 TTFB 低于 3,000 ms,或出现可恢复失败、限流、服务器异常 | | `POOR` | 主路径成功率低于 60%、中位 TTFB 至少 3,000 ms,或配置的 API 路径返回 404 | | `BLOCKED` | 主路径没有 HTTP 响应,或代理先返回 HTTP 407 | +| `SKIPPED` | 必须先提供明确的无凭据检测地址,才能测量该 provider | 检测多个客户端时,各客户端保持独立结论:全部阻断时总体为 `BLOCKED`;阻断 与可达客户端并存时总体为 `POOR`;否则采用最弱客户端的结论。开发适配分始终 使用得分最低的服务链路,因此健康的 Claude 链路不会掩盖异常的 Codex 链路。 +已跳过的客户端不会拉低其他已测客户端的分数。所有选中客户端均被跳过时, +总体结论为 `UNAVAILABLE`、分数为 0,命令返回状态码 1。 + ## 规则 v2 的服务链路分 选中的最弱服务链路最多贡献 90 分。 diff --git a/package.json b/package.json index 3b65407..37c904a 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "@jacklv-coder/ipcheck", "private": true, - "version": "0.8.1", + "version": "0.8.2", "description": "Diagnose proxy, latency, and API routes used by Codex and Claude Code.", "license": "MIT", "author": "Jintao", diff --git a/test/smoke.sh b/test/smoke.sh index 2b8ee28..3369063 100755 --- a/test/smoke.sh +++ b/test/smoke.sh @@ -5,6 +5,10 @@ set -eu # Keep report-language assertions deterministic regardless of the host locale. IPCHECK_LANG=en export IPCHECK_LANG +unset CLAUDE_CODE_USE_FOUNDRY CLAUDE_CODE_USE_ANTHROPIC_AWS \ + CLAUDE_CODE_USE_ANTHROPIC_GOOGLE_CLOUD CLAUDE_CODE_USE_MANTLE \ + ANTHROPIC_FOUNDRY_BASE_URL ANTHROPIC_AWS_BASE_URL \ + ANTHROPIC_GOOGLE_CLOUD_BASE_URL ANTHROPIC_BEDROCK_MANTLE_BASE_URL PROJECT_DIR=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd) STUB_DIR=$(mktemp -d "${TMPDIR:-/tmp}/ipcheck-test.XXXXXX") @@ -37,6 +41,8 @@ is_score_scenario=0 printf 'env:HTTPS_PROXY=%s\n' "${HTTPS_PROXY-}" >> "${IPCHECK_TEST_CURL_LOG:?}" printf 'env:http_proxy=%s\n' "${http_proxy-}" >> "${IPCHECK_TEST_CURL_LOG:?}" printf 'env:ALL_PROXY=%s\n' "${ALL_PROXY-}" >> "${IPCHECK_TEST_CURL_LOG:?}" +printf 'env:NO_PROXY=%s\n' "${NO_PROXY-}" >> "${IPCHECK_TEST_CURL_LOG:?}" +printf 'env:no_proxy=%s\n' "${no_proxy-}" >> "${IPCHECK_TEST_CURL_LOG:?}" for argument in "$@"; do printf '%s\n' "$argument" >> "${IPCHECK_TEST_CURL_LOG:?}" case "$argument" in @@ -116,6 +122,25 @@ fi EOF chmod +x "$STUB_DIR/curl" +cat > "$STUB_DIR/codex" <<'EOF' +#!/usr/bin/env bash +if [ "${1-}" = "login" ] && [ "${2-}" = "status" ]; then + if [ -n "${IPCHECK_TEST_CODEX_ENV_LOG-}" ] && \ + { [ -n "${CODEX_API_KEY-}" ] || [ -n "${OPENAI_API_KEY-}" ] || \ + [ -n "${CODEX_ACCESS_TOKEN-}" ] || [ -n "${AWS_BEARER_TOKEN_BEDROCK-}" ]; }; then + printf 'credential inherited\n' >> "$IPCHECK_TEST_CODEX_ENV_LOG" + fi + if [ "${IPCHECK_TEST_CODEX_STATUS_STDERR:-0}" -eq 1 ]; then + printf '%s\n' "${IPCHECK_TEST_CODEX_LOGIN_STATUS:-Logged in using ChatGPT}" >&2 + else + printf '%s\n' "${IPCHECK_TEST_CODEX_LOGIN_STATUS:-Logged in using ChatGPT}" + fi + exit 0 +fi +exit 2 +EOF +chmod +x "$STUB_DIR/codex" + MINIMAL_BIN="$STUB_DIR/minimal-bin" mkdir -p "$MINIMAL_BIN" for utility in bash awk sed sort mktemp rm tr date head env; do @@ -178,8 +203,11 @@ EOF run_ipcheck() { env \ -u ANTHROPIC_BASE_URL -u ANTHROPIC_MODEL \ + -u ANTHROPIC_BEDROCK_BASE_URL -u ANTHROPIC_VERTEX_BASE_URL \ + -u CLAUDE_CODE_USE_BEDROCK -u CLAUDE_CODE_USE_VERTEX \ -u IPCHECK_SERVICES -u IPCHECK_ENDPOINTS \ -u CODEX_NETWORK_ENDPOINTS -u CLAUDE_NETWORK_ENDPOINTS \ + -u CODEX_API_KEY -u OPENAI_API_KEY \ -u HTTP_PROXY -u http_proxy -u ALL_PROXY -u all_proxy \ PATH="$STUB_DIR:$PATH" HOME="$FIXTURE_HOME" CODEX_HOME="$CODEX_FIXTURE" CLAUDE_CONFIG_DIR="$CLAUDE_FIXTURE" \ HTTPS_PROXY="http://127.0.0.1:1080" IPCHECK_LANG="${IPCHECK_LANG:-en}" IPCHECK_TEST_CURL_LOG="$CURL_LOG" \ @@ -189,8 +217,11 @@ run_ipcheck() { run_ipcheck_direct() { env \ -u ANTHROPIC_BASE_URL -u ANTHROPIC_MODEL \ + -u ANTHROPIC_BEDROCK_BASE_URL -u ANTHROPIC_VERTEX_BASE_URL \ + -u CLAUDE_CODE_USE_BEDROCK -u CLAUDE_CODE_USE_VERTEX \ -u IPCHECK_SERVICES -u IPCHECK_ENDPOINTS \ -u CODEX_NETWORK_ENDPOINTS -u CLAUDE_NETWORK_ENDPOINTS \ + -u CODEX_API_KEY -u OPENAI_API_KEY \ -u HTTPS_PROXY -u https_proxy -u HTTP_PROXY -u http_proxy -u ALL_PROXY -u all_proxy \ PATH="$STUB_DIR:$PATH" HOME="$FIXTURE_HOME" CODEX_HOME="$CODEX_FIXTURE" CLAUDE_CONFIG_DIR="$CLAUDE_FIXTURE" \ IPCHECK_LANG="${IPCHECK_LANG:-en}" IPCHECK_TEST_CURL_LOG="$CURL_LOG" \ @@ -199,16 +230,71 @@ run_ipcheck_direct() { bash -n "$PROJECT_DIR/bin/ipcheck" "$PROJECT_DIR/bin/ipcheck" --help | grep -q '^ipcheck - diagnose Codex and Claude Code' -[ "$("$PROJECT_DIR/bin/ipcheck" --version)" = "ipcheck 0.8.1" ] +[ "$("$PROJECT_DIR/bin/ipcheck" --version)" = "ipcheck 0.8.2" ] "$PROJECT_DIR/bin/ipcheck" --help | grep -q -- '--explain-score' : > "$CURL_LOG" report=$(ANTHROPIC_AUTH_TOKEN="runtime-secret-must-never-appear" run_ipcheck --samples 3 --no-bandwidth --json) -printf '%s\n' "$report" | grep -q '"schema_version":1' +printf '%s\n' "$report" | grep -q '"schema_version":2' printf '%s\n' "$report" | grep -q '"result":"good"' printf '%s\n' "$report" | grep -q '"reachable_endpoints":3' printf '%s\n' "$report" | grep -q '"id":"codex"' printf '%s\n' "$report" | grep -q '"model":"gpt-test"' +printf '%s\n' "$report" | grep -q '"config_source":"Codex ChatGPT login"' +printf '%s\n' "$report" | grep -q '"label":"ChatGPT Codex API","url":"https://chatgpt.com/backend-api/codex/responses","primary":true' +printf '%s\n' "$report" | grep -q '"label":"OpenAI Responses API","url":"https://api.openai.com/v1/responses","primary":false' + +codex_stderr_status_report=$(IPCHECK_TEST_CODEX_STATUS_STDERR=1 run_ipcheck codex --samples 1 --no-bandwidth --json) +printf '%s\n' "$codex_stderr_status_report" | grep -q '"config_source":"Codex ChatGPT login"' +printf '%s\n' "$codex_stderr_status_report" | grep -q '"label":"ChatGPT Codex API","url":"https://chatgpt.com/backend-api/codex/responses","primary":true' + +codex_stored_api_key_report=$(IPCHECK_TEST_CODEX_STATUS_STDERR=1 IPCHECK_TEST_CODEX_LOGIN_STATUS='Logged in using an API key' run_ipcheck codex --samples 1 --no-bandwidth --json) +printf '%s\n' "$codex_stored_api_key_report" | grep -q '"config_source":"Codex API key login"' +printf '%s\n' "$codex_stored_api_key_report" | grep -q '"label":"OpenAI Responses API","url":"https://api.openai.com/v1/responses","primary":true' + +codex_status_env_log="$STUB_DIR/codex-status-env.log" +codex_mixed_auth_report=$(env \ + -u ANTHROPIC_BASE_URL -u ANTHROPIC_MODEL -u CODEX_API_KEY \ + -u CODEX_NETWORK_ENDPOINTS -u CLAUDE_NETWORK_ENDPOINTS \ + -u HTTP_PROXY -u http_proxy -u ALL_PROXY -u all_proxy \ + PATH="$STUB_DIR:$PATH" HOME="$FIXTURE_HOME" CODEX_HOME="$CODEX_FIXTURE" CLAUDE_CONFIG_DIR="$CLAUDE_FIXTURE" \ + OPENAI_API_KEY="fixture-api-key" IPCHECK_TEST_CODEX_STATUS_STDERR=1 \ + IPCHECK_TEST_CODEX_ENV_LOG="$codex_status_env_log" \ + IPCHECK_TEST_CODEX_LOGIN_STATUS='Logged in using ChatGPT' HTTPS_PROXY="http://127.0.0.1:1080" \ + IPCHECK_LANG=en IPCHECK_TEST_CURL_LOG="$CURL_LOG" \ + "$PROJECT_DIR/bin/ipcheck" codex --samples 1 --no-bandwidth --json) +printf '%s\n' "$codex_mixed_auth_report" | grep -q '"config_source":"Codex API key login"' +printf '%s\n' "$codex_mixed_auth_report" | grep -q '"label":"OpenAI Responses API","url":"https://api.openai.com/v1/responses","primary":true' +[ ! -e "$codex_status_env_log" ] + +codex_access_status_env_log="$STUB_DIR/codex-access-status-env.log" +codex_access_token_report=$(env \ + -u ANTHROPIC_BASE_URL -u ANTHROPIC_MODEL -u CODEX_API_KEY -u OPENAI_API_KEY \ + -u CODEX_NETWORK_ENDPOINTS -u CLAUDE_NETWORK_ENDPOINTS \ + -u HTTP_PROXY -u http_proxy -u ALL_PROXY -u all_proxy \ + PATH="$STUB_DIR:$PATH" HOME="$FIXTURE_HOME" CODEX_HOME="$CODEX_FIXTURE" CLAUDE_CONFIG_DIR="$CLAUDE_FIXTURE" \ + CODEX_ACCESS_TOKEN="fixture-access-token" AWS_BEARER_TOKEN_BEDROCK="fixture-bedrock-token" \ + IPCHECK_TEST_CODEX_STATUS_STDERR=1 IPCHECK_TEST_CODEX_ENV_LOG="$codex_access_status_env_log" \ + IPCHECK_TEST_CODEX_LOGIN_STATUS='Logged in using ChatGPT' HTTPS_PROXY="http://127.0.0.1:1080" \ + IPCHECK_LANG=en IPCHECK_TEST_CURL_LOG="$CURL_LOG" \ + "$PROJECT_DIR/bin/ipcheck" codex --samples 1 --no-bandwidth --json) +printf '%s\n' "$codex_access_token_report" | grep -q '"config_source":"Codex ChatGPT login"' +[ ! -e "$codex_access_status_env_log" ] + +codex_key_trace="$STUB_DIR/codex-key.trace" +env \ + -u ANTHROPIC_BASE_URL -u ANTHROPIC_MODEL -u CODEX_API_KEY \ + -u CODEX_NETWORK_ENDPOINTS -u CLAUDE_NETWORK_ENDPOINTS \ + -u HTTP_PROXY -u http_proxy -u ALL_PROXY -u all_proxy \ + PATH="$STUB_DIR:$PATH" HOME="$FIXTURE_HOME" CODEX_HOME="$CODEX_FIXTURE" CLAUDE_CONFIG_DIR="$CLAUDE_FIXTURE" \ + OPENAI_API_KEY="fixture-key-must-not-appear" HTTPS_PROXY="http://127.0.0.1:1080" \ + IPCHECK_LANG=en IPCHECK_TEST_CURL_LOG="$CURL_LOG" \ + bash -x "$PROJECT_DIR/bin/ipcheck" codex --samples 1 --no-bandwidth --json \ + >/dev/null 2>"$codex_key_trace" +if grep -q 'fixture-key-must-not-appear' "$codex_key_trace"; then + printf 'API key leaked into shell trace\n' >&2 + exit 1 +fi printf '%s\n' "$report" | grep -q '"id":"claude"' printf '%s\n' "$report" | grep -q '"model":"deepseek-v4-flash"' printf '%s\n' "$report" | grep -q 'https://dashscope.aliyuncs.com/apps/anthropic' @@ -233,6 +319,24 @@ if command -v python3 >/dev/null 2>&1; then REPORT_JSON="$report" python3 -c 'import json, os; json.loads(os.environ["REPORT_JSON"])' fi +api_key_report=$(IPCHECK_TEST_CODEX_LOGIN_STATUS="Logged in using an API key" run_ipcheck codex --samples 1 --no-bandwidth --json) +printf '%s\n' "$api_key_report" | grep -q '"config_source":"Codex API key login"' +printf '%s\n' "$api_key_report" | grep -q '"label":"OpenAI Responses API","url":"https://api.openai.com/v1/responses","primary":true' +printf '%s\n' "$api_key_report" | grep -q '"label":"ChatGPT Codex API","url":"https://chatgpt.com/backend-api/codex/responses","primary":false' + +set +e +codex_bedrock_login_report=$(IPCHECK_TEST_CODEX_LOGIN_STATUS="Logged in using Amazon Bedrock API key" run_ipcheck codex --samples 1 --no-bandwidth --json) +codex_bedrock_login_exit=$? +set -e +[ "$codex_bedrock_login_exit" -eq 1 ] +printf '%s\n' "$codex_bedrock_login_report" | grep -q '"result":"unavailable"' +printf '%s\n' "$codex_bedrock_login_report" | grep -q '"result":"skipped"' +printf '%s\n' "$codex_bedrock_login_report" | grep -q 'configured for Amazon Bedrock' +if printf '%s\n' "$codex_bedrock_login_report" | grep -q '"service":"codex"'; then + printf 'Codex Bedrock login was incorrectly probed as an OpenAI route\n' >&2 + exit 1 +fi + mixed_client_quality=$(IPCHECK_TEST_ANTHROPIC_TTFB=4.000 run_ipcheck --samples 1 --no-bandwidth --json) printf '%s\n' "$mixed_client_quality" | grep -q '"result":"poor"' printf '%s\n' "$mixed_client_quality" | grep -q '"score":63,"score_label":"LIMITED","score_method":"rule_v2"' @@ -288,6 +392,34 @@ if printf '%s\n' "$auto_claude_override" | grep -q '"id":"codex"'; then exit 1 fi +disabled_provider_report=$(env \ + -u ANTHROPIC_BASE_URL -u ANTHROPIC_MODEL -u ANTHROPIC_BEDROCK_BASE_URL -u ANTHROPIC_VERTEX_BASE_URL \ + -u CLAUDE_CODE_USE_VERTEX -u IPCHECK_SERVICES -u IPCHECK_ENDPOINTS \ + -u CODEX_NETWORK_ENDPOINTS -u CLAUDE_NETWORK_ENDPOINTS -u CODEX_API_KEY -u OPENAI_API_KEY \ + -u HTTP_PROXY -u http_proxy -u ALL_PROXY -u all_proxy \ + CLAUDE_CODE_USE_BEDROCK=0 PATH="$MINIMAL_BIN" HOME="$AUTO_HOME" \ + CODEX_HOME="$AUTO_HOME/codex" CLAUDE_CONFIG_DIR="$AUTO_HOME/claude" \ + HTTPS_PROXY="http://127.0.0.1:1080" IPCHECK_TEST_CURL_LOG="$CURL_LOG" \ + "$PROJECT_DIR/bin/ipcheck" --samples 1 --no-bandwidth --json) +printf '%s\n' "$disabled_provider_report" | grep -q '"id":"codex"' + +cat > "$AUTO_HOME/claude/settings.json" <<'EOF' +{"CLAUDE_CODE_USE_BEDROCK": true} +EOF +set +e +boolean_provider_report=$(env \ + -u ANTHROPIC_BASE_URL -u ANTHROPIC_MODEL -u ANTHROPIC_BEDROCK_BASE_URL -u ANTHROPIC_VERTEX_BASE_URL \ + -u CLAUDE_CODE_USE_BEDROCK -u CLAUDE_CODE_USE_VERTEX -u CLAUDE_NETWORK_ENDPOINTS \ + -u HTTP_PROXY -u http_proxy -u ALL_PROXY -u all_proxy \ + PATH="$MINIMAL_BIN" HOME="$AUTO_HOME" CODEX_HOME="$AUTO_HOME/codex" CLAUDE_CONFIG_DIR="$AUTO_HOME/claude" \ + HTTPS_PROXY="http://127.0.0.1:1080" IPCHECK_TEST_CURL_LOG="$CURL_LOG" \ + "$PROJECT_DIR/bin/ipcheck" claude --samples 1 --no-bandwidth --json) +boolean_provider_exit=$? +set -e +[ "$boolean_provider_exit" -eq 1 ] +printf '%s\n' "$boolean_provider_report" | grep -q 'configured for Amazon Bedrock' +printf '%s\n' '{}' > "$AUTO_HOME/claude/settings.json" + claude_report=$(run_ipcheck claude --samples 1 --no-bandwidth --json) printf '%s\n' "$claude_report" | grep -q '"id":"claude"' if printf '%s\n' "$claude_report" | grep -q '"id":"codex"'; then @@ -295,6 +427,87 @@ if printf '%s\n' "$claude_report" | grep -q '"id":"codex"'; then exit 1 fi +set +e +bedrock_report=$(env \ + -u ANTHROPIC_BASE_URL -u ANTHROPIC_MODEL -u ANTHROPIC_VERTEX_BASE_URL \ + -u CLAUDE_CODE_USE_VERTEX -u IPCHECK_SERVICES -u IPCHECK_ENDPOINTS \ + -u CODEX_NETWORK_ENDPOINTS -u CLAUDE_NETWORK_ENDPOINTS -u CODEX_API_KEY -u OPENAI_API_KEY \ + -u HTTP_PROXY -u http_proxy -u ALL_PROXY -u all_proxy \ + CLAUDE_CODE_USE_BEDROCK=1 PATH="$STUB_DIR:$PATH" HOME="$FIXTURE_HOME" \ + CODEX_HOME="$CODEX_FIXTURE" CLAUDE_CONFIG_DIR="$CLAUDE_FIXTURE" \ + HTTPS_PROXY="http://127.0.0.1:1080" IPCHECK_TEST_CURL_LOG="$CURL_LOG" \ + "$PROJECT_DIR/bin/ipcheck" claude --samples 1 --no-bandwidth --json) +bedrock_exit=$? +set -e +[ "$bedrock_exit" -eq 1 ] +printf '%s\n' "$bedrock_report" | grep -q 'configured for Amazon Bedrock' +printf '%s\n' "$bedrock_report" | grep -q '"result":"unavailable"' +printf '%s\n' "$bedrock_report" | grep -q '"result":"skipped"' + +set +e +foundry_report=$(CLAUDE_CODE_USE_FOUNDRY=1 run_ipcheck claude --samples 1 --no-bandwidth --json) +foundry_exit=$? +set -e +[ "$foundry_exit" -eq 1 ] +printf '%s\n' "$foundry_report" | grep -q 'Microsoft Foundry (explicit endpoint required)' +printf '%s\n' "$foundry_report" | grep -q 'provider-specific protocol' +printf '%s\n' "$foundry_report" | grep -q '"result":"unavailable"' +if printf '%s\n' "$foundry_report" | grep -q 'https://api.anthropic.com/v1/messages'; then + printf 'Provider-specific Foundry mode fell through to the Anthropic route\n' >&2 + exit 1 +fi + +set +e +bedrock_auto_report=$(env \ + -u ANTHROPIC_BASE_URL -u ANTHROPIC_MODEL -u ANTHROPIC_BEDROCK_BASE_URL -u ANTHROPIC_VERTEX_BASE_URL \ + -u CLAUDE_CODE_USE_VERTEX -u IPCHECK_SERVICES -u IPCHECK_ENDPOINTS \ + -u CODEX_NETWORK_ENDPOINTS -u CLAUDE_NETWORK_ENDPOINTS -u CODEX_API_KEY -u OPENAI_API_KEY \ + -u HTTP_PROXY -u http_proxy -u ALL_PROXY -u all_proxy \ + CLAUDE_CODE_USE_BEDROCK=1 PATH="$MINIMAL_BIN" HOME="$AUTO_HOME" \ + CODEX_HOME="$AUTO_HOME/codex" CLAUDE_CONFIG_DIR="$AUTO_HOME/claude" \ + HTTPS_PROXY="http://127.0.0.1:1080" IPCHECK_TEST_CURL_LOG="$CURL_LOG" \ + "$PROJECT_DIR/bin/ipcheck" --samples 1 --no-bandwidth --json) +bedrock_auto_exit=$? +set -e +[ "$bedrock_auto_exit" -eq 1 ] +printf '%s\n' "$bedrock_auto_report" | grep -q '"result":"unavailable"' +printf '%s\n' "$bedrock_auto_report" | grep -q 'configured for Amazon Bedrock' + +set +e +bedrock_base_report=$(env \ + -u ANTHROPIC_BASE_URL -u ANTHROPIC_MODEL -u ANTHROPIC_VERTEX_BASE_URL \ + -u CLAUDE_CODE_USE_BEDROCK -u CLAUDE_CODE_USE_VERTEX -u IPCHECK_SERVICES -u IPCHECK_ENDPOINTS \ + -u CODEX_NETWORK_ENDPOINTS -u CLAUDE_NETWORK_ENDPOINTS -u CODEX_API_KEY -u OPENAI_API_KEY \ + -u HTTP_PROXY -u http_proxy -u ALL_PROXY -u all_proxy \ + ANTHROPIC_BEDROCK_BASE_URL="https://bedrock.example.com" PATH="$STUB_DIR:$PATH" HOME="$FIXTURE_HOME" \ + CODEX_HOME="$CODEX_FIXTURE" CLAUDE_CONFIG_DIR="$CLAUDE_FIXTURE" \ + HTTPS_PROXY="http://127.0.0.1:1080" IPCHECK_TEST_CURL_LOG="$CURL_LOG" \ + "$PROJECT_DIR/bin/ipcheck" claude --samples 1 --no-bandwidth --json) +bedrock_base_exit=$? +set -e +[ "$bedrock_base_exit" -eq 1 ] +printf '%s\n' "$bedrock_base_report" | grep -q '"result":"unavailable"' +printf '%s\n' "$bedrock_base_report" | grep -q 'configured for Amazon Bedrock' +if printf '%s\n' "$bedrock_base_report" | grep -q 'https://api.anthropic.com/v1/messages'; then + printf 'Provider-specific Bedrock base URL fell through to the Anthropic route\n' >&2 + exit 1 +fi + +bedrock_all_report=$(env \ + -u ANTHROPIC_BASE_URL -u ANTHROPIC_MODEL -u ANTHROPIC_VERTEX_BASE_URL \ + -u CLAUDE_CODE_USE_VERTEX -u IPCHECK_SERVICES -u IPCHECK_ENDPOINTS \ + -u CODEX_NETWORK_ENDPOINTS -u CLAUDE_NETWORK_ENDPOINTS -u CODEX_API_KEY -u OPENAI_API_KEY \ + -u HTTP_PROXY -u http_proxy -u ALL_PROXY -u all_proxy \ + CLAUDE_CODE_USE_BEDROCK=1 PATH="$STUB_DIR:$PATH" HOME="$FIXTURE_HOME" \ + CODEX_HOME="$CODEX_FIXTURE" CLAUDE_CONFIG_DIR="$CLAUDE_FIXTURE" \ + HTTPS_PROXY="http://127.0.0.1:1080" IPCHECK_TEST_CURL_LOG="$CURL_LOG" \ + "$PROJECT_DIR/bin/ipcheck" all --samples 1 --no-bandwidth --json) +printf '%s\n' "$bedrock_all_report" | grep -q 'automatic Anthropic protocol probing was skipped' +if printf '%s\n' "$bedrock_all_report" | grep -q '"service":"claude"'; then + printf 'Bedrock configuration was incorrectly scored as an Anthropic route\n' >&2 + exit 1 +fi + env_precedence_report=$(ANTHROPIC_BASE_URL="https://gateway.example.com/anthropic" ANTHROPIC_MODEL="gateway-model" \ env -u HTTP_PROXY -u http_proxy -u ALL_PROXY -u all_proxy \ PATH="$STUB_DIR:$PATH" HOME="$FIXTURE_HOME" CODEX_HOME="$CODEX_FIXTURE" CLAUDE_CONFIG_DIR="$CLAUDE_FIXTURE" \ @@ -303,6 +516,47 @@ env_precedence_report=$(ANTHROPIC_BASE_URL="https://gateway.example.com/anthropi printf '%s\n' "$env_precedence_report" | grep -q '"model":"gateway-model"' printf '%s\n' "$env_precedence_report" | grep -q 'https://gateway.example.com/anthropic/v1/messages' +cat > "$CODEX_FIXTURE/config.toml" <<'EOF' +model = "bedrock-model" +model_provider = "amazon-bedrock" +EOF +set +e +codex_bedrock_provider_report=$(run_ipcheck codex --samples 1 --no-bandwidth --json) +codex_bedrock_provider_exit=$? +set -e +[ "$codex_bedrock_provider_exit" -eq 1 ] +printf '%s\n' "$codex_bedrock_provider_report" | grep -q '"result":"unavailable"' +printf '%s\n' "$codex_bedrock_provider_report" | grep -q '"model":"bedrock-model"' +printf '%s\n' "$codex_bedrock_provider_report" | grep -q '"result":"skipped"' +printf '%s\n' "$codex_bedrock_provider_report" | grep -q 'Set CODEX_NETWORK_ENDPOINTS' +if printf '%s\n' "$codex_bedrock_provider_report" | grep -q 'Set CLAUDE_NETWORK_ENDPOINTS'; then + printf 'Codex Bedrock guidance referenced the Claude endpoint override\n' >&2 + exit 1 +fi + +mixed_codex_bedrock_report=$(run_ipcheck all --samples 1 --no-bandwidth --json) +printf '%s\n' "$mixed_codex_bedrock_report" | grep -q '"reachable_endpoints":1,"primary_success_rate_pct":100,"primary_ttfb_median_ms":100' +mixed_codex_bedrock_zh=$(IPCHECK_LANG=zh run_ipcheck all --samples 1 --no-bandwidth --markdown) +printf '%s\n' "$mixed_codex_bedrock_zh" | grep -q '1 条已跳过' + +cat > "$CODEX_FIXTURE/config.toml" <<'EOF' +model = "chatgpt-proxy-model" +chatgpt_base_url = "https://chatgpt-proxy.example.com/backend-api/" +EOF +chatgpt_base_report=$(run_ipcheck codex --samples 1 --no-bandwidth --json) +printf '%s\n' "$chatgpt_base_report" | grep -q 'https://chatgpt-proxy.example.com/backend-api/codex/responses' + +cat > "$CODEX_FIXTURE/config.toml" <<'EOF' +model = "chatgpt-proxy-model" +chatgpt_base_url = "https://chatgpt-proxy.example.com/backend-api/codex/responses/" +EOF +chatgpt_full_route_report=$(run_ipcheck codex --samples 1 --no-bandwidth --json) +printf '%s\n' "$chatgpt_full_route_report" | grep -q 'https://chatgpt-proxy.example.com/backend-api/codex/responses' +if printf '%s\n' "$chatgpt_full_route_report" | grep -q 'responses/codex/responses'; then + printf 'Complete ChatGPT Codex route was appended twice\n' >&2 + exit 1 +fi + cat > "$CODEX_FIXTURE/config.toml" <<'EOF' model = 'proxy-model' model_provider = "company" @@ -485,6 +739,21 @@ if printf '%s\n' "$proxy_redaction_report" | grep -Eq 'proxy-user|p@ss|proxy-sec exit 1 fi +proxy_eval_marker="$STUB_DIR/proxy-eval-must-not-run" +malicious_proxy="\$(touch $proxy_eval_marker)" +env \ + -u ANTHROPIC_BASE_URL -u ANTHROPIC_MODEL -u HTTPS_PROXY -u https_proxy \ + -u HTTP_PROXY -u http_proxy -u ALL_PROXY -u all_proxy \ + PATH="$STUB_DIR:$PATH" HOME="$FIXTURE_HOME" CODEX_HOME="$CODEX_FIXTURE" \ + CLAUDE_CONFIG_DIR="$CLAUDE_FIXTURE" HTTPS_PROXY="$malicious_proxy" \ + IPCHECK_TEST_CURL_LOG="$CURL_LOG" \ + "$PROJECT_DIR/bin/ipcheck" --samples 1 --no-bandwidth \ + --endpoint https://proxy-injection.invalid --json >/dev/null +if [ -e "$proxy_eval_marker" ]; then + printf 'proxy environment value was executed as shell code\n' >&2 + exit 1 +fi + : > "$CURL_LOG" proxy_fallback_report=$(env \ -u ANTHROPIC_BASE_URL -u ANTHROPIC_MODEL -u HTTPS_PROXY -u https_proxy -u ALL_PROXY -u all_proxy \ @@ -497,7 +766,7 @@ grep -q '^http://127.0.0.1:1082$' "$CURL_LOG" : > "$CURL_LOG" claude_system_proxy_report=$(run_ipcheck_direct claude --samples 1 --no-bandwidth --json) -printf '%s\n' "$claude_system_proxy_report" | grep -q 'Claude Code expects HTTPS_PROXY/HTTP_PROXY' +printf '%s\n' "$claude_system_proxy_report" | grep -q 'the probe does not apply the macOS proxy explicitly' if grep -q '^--proxy$' "$CURL_LOG"; then printf 'Claude probe incorrectly used the macOS system proxy\n' >&2 exit 1 @@ -527,11 +796,14 @@ claude_https_precedence_report=$(env \ -u ANTHROPIC_MODEL -u HTTP_PROXY -u http_proxy -u https_proxy \ ANTHROPIC_BASE_URL="http://claude-http-gateway.invalid/anthropic" \ HTTPS_PROXY="http://supported-proxy.invalid:8080" ALL_PROXY="http://unsupported-proxy.invalid:8081" \ + NO_PROXY="claude-http-gateway.invalid" no_proxy="claude-http-gateway.invalid" \ PATH="$STUB_DIR:$PATH" HOME="$FIXTURE_HOME" CODEX_HOME="$CODEX_FIXTURE" CLAUDE_CONFIG_DIR="$CLAUDE_FIXTURE" \ IPCHECK_TEST_CURL_LOG="$CURL_LOG" "$PROJECT_DIR/bin/ipcheck" claude --samples 1 --no-bandwidth --json) printf '%s\n' "$claude_https_precedence_report" | grep -q '"result":"good"' grep -q '^env:HTTPS_PROXY=http://supported-proxy.invalid:8080$' "$CURL_LOG" grep -q '^env:ALL_PROXY=$' "$CURL_LOG" +grep -q '^env:NO_PROXY=claude-http-gateway.invalid$' "$CURL_LOG" +grep -q '^env:no_proxy=claude-http-gateway.invalid$' "$CURL_LOG" : > "$CURL_LOG" socks_report=$(env \