Skip to content

Tag session-close, statement-close, and rate-limit errors for telemetry - #424

Merged
prathamesh-db merged 3 commits into
mainfrom
prathamesh/PECOBLR-3537-tag-caller-sites
Jul 29, 2026
Merged

Tag session-close, statement-close, and rate-limit errors for telemetry#424
prathamesh-db merged 3 commits into
mainfrom
prathamesh/PECOBLR-3537-tag-caller-sites

Conversation

@prathamesh-db

@prathamesh-db prathamesh-db commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

What

Tags the remaining live error sources so telemetry reports a specific
error_name instead of the generic error:

Category Site
session_closed failed CloseSession (connection.go, DELETE_SESSION telemetry)
statement_closed failed CloseOperation RPC (connection.go, the 3 CLOSE_STATEMENT sites)
rate_limit_exceeded HTTP 429 that survives all retries (internal/client/client.go retry errorHandler)

All tags are telemetry-only and use the existing WithCategory chaining
(same concrete pointer), so errors.Is/errors.As, Error() strings, and the
errors returned to callers are unchanged:

  • session_closed tags a separate telemetry copy; the returned error stays a
    driver.ErrBadConn (database/sql relies on that for connection-pool eviction).
  • statement_closed tags only the telemetry argument; the raw close error still
    flows to the caller/rows.
  • rate_limit_exceeded is tagged in the retry errorHandler — the only place the
    429 status is visible (net/http discards the response on a transport error).
    Scoped to 429 exactly, so 503 keeps its existing classification. This covers the
    retry-exhausted failure only; a per-attempt 429 counter metric is a TODO follow-up.

Notes for reviewers (by design)

  • statement_closed is only live on the Thrift backend; the kernel backend's close
    never returns an error, so it can't be tagged there.
  • Because the innermost source category wins, a close that fails specifically due to
    a 429 reports rate_limit_exceeded rather than session_closed/statement_closed
    — the more specific category.

Naming note

session_closed has no exact JDBC equivalent (JDBC's nearest is CONNECTION_CLOSED);
kept Go-specific here.


Design doc: https://docs.google.com/document/d/12ufP1eZrgFxWt6xzINfkhfrE-NnhB29zCa5PKokVfp8/edit
Jira: PECOBLR-3537

@prathamesh-db prathamesh-db self-assigned this Jul 23, 2026
@prathamesh-db
prathamesh-db force-pushed the prathamesh/PECOBLR-3537-tag-caller-sites branch from 4a03e2a to f8afecc Compare July 23, 2026 05:11
Tags the remaining live error sources so telemetry reports a specific
error_name instead of the generic "error":

- session_closed: a failed CloseSession is tagged on a telemetry-only copy
  in conn.Close before RecordOperation(DELETE_SESSION). The returned error
  is left untouched so it stays a driver.ErrBadConn (database/sql relies on
  that for connection-pool eviction).
- statement_closed: a failed CloseOperation RPC is tagged via a
  telemetry-only helper at the three RecordOperation(CLOSE_STATEMENT) sites
  (sync exec + the two async close-callback paths). The raw error still
  flows to the caller/rows unchanged.
- rate_limit_exceeded: an HTTP 429 that survives all retries is tagged in
  the retry errorHandler, which is the only place the 429 status is visible
  (net/http discards the response on a transport error). Scoped to 429
  exactly, so 503 keeps its existing classification. This covers the
  retry-exhausted failure only; a per-attempt 429 counter metric is left as
  a TODO follow-up.

All tags are telemetry-only and use WithCategory (same concrete pointer),
so errors.Is/As, driver.ErrBadConn identity, Error() strings, and the
returned errors are unchanged. Tests assert each tagged error yields the
right category and that 503/500 are not tagged rate_limit and the returned
session-close error is still driver.ErrBadConn.

Signed-off-by: Prathamesh Baviskar <prathamesh.baviskar@databricks.com>
@prathamesh-db
prathamesh-db force-pushed the prathamesh/PECOBLR-3537-tag-caller-sites branch from f8afecc to 96c8029 Compare July 23, 2026 12:09
@prathamesh-db
prathamesh-db marked this pull request as ready for review July 26, 2026 14:30

@samikshya-db samikshya-db left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sure you test this e2e as discussed - before merging

Addresses review nit: the changelog line enumerated internal driver
category names, which is too low-level for an external changelog.

Co-authored-by: Isaac
Signed-off-by: Prathamesh Baviskar <prathamesh.baviskar@databricks.com>
Base automatically changed from prathamesh/PECOBLR-3537-tag-remaining-sites to main July 29, 2026 04:43
@prathamesh-db
prathamesh-db enabled auto-merge July 29, 2026 04:49
@prathamesh-db
prathamesh-db added this pull request to the merge queue Jul 29, 2026

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No issues identified by the review bot.

Merged via the queue into main with commit f88a576 Jul 29, 2026
13 checks passed
@prathamesh-db
prathamesh-db deleted the prathamesh/PECOBLR-3537-tag-caller-sites branch July 29, 2026 04:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants