Skip to content

privacy_mode set at module level never reaches the client, so AI wrappers capture full prompts and outputs #768

Description

@toolshedlabs-hash

Summary

Setting posthog.privacy_mode = True on the module has no effect. The AI wrappers then send the
full prompt and the full model output as $ai_input and $ai_output_choices.

posthog/__init__.py:387 declares it:

# Used for our AI observability feature to not capture any prompt or output just usage + metadata
privacy_mode = False  # type: bool

setup() passes 31 keyword arguments to Client(...) and then re-syncs disabled, debug,
before_send, _use_ai_lane, _enable_multimodal_capture and the metrics config onto the
instance. privacy_mode is in neither list, and Client.__init__ does accept it at
client.py:466. The only consumer is posthog/ai/utils.py:682:

def with_privacy_mode(ph_client, privacy_mode, value):
    if ph_client.privacy_mode or privacy_mode:
        return None
    return value

and that is what builds $ai_input and $ai_output_choices at ai/utils.py:717-724.

This is the same omission as #450, which you closed in April with #472. That one was
before_send. This one is still open in the code.

Reproduction

posthog 7.29.0, openai 2.48.0, Python 3.14.0, fresh venv. The OpenAI client points at a loopback
server returning a canned completion, and capture is replaced with a recorder so nothing leaves
the machine. Each row is a separate process.

import posthog
from posthog.ai.openai import OpenAI

posthog.api_key = "phc_probe"
posthog.privacy_mode = True
ph = posthog.setup()

client = OpenAI(api_key="sk-probe", base_url=FAKE, posthog_client=ph)
client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "MY SECRET PROMPT"}],
    posthog_distinct_id="u1",
)
how it was set client.privacy_mode $ai_input on the event
posthog.privacy_mode = True then setup() False [{"role": "user", "content": "MY SECRET PROMPT"}]
Client(..., privacy_mode=True) True null
not set at all False [{"role": "user", "content": "MY SECRET PROMPT"}]
posthog.debug = True then setup() False (debug True) [{"role": "user", "content": "MY SECRET PROMPT"}]

$ai_output_choices follows $ai_input exactly in every row.

The last two rows are there for a reason. Row three shows the prompt is captured when nobody
asked for privacy, so an empty result in row one could not have been mistaken for redaction. Row
four sets a different module setting that setup() does re-sync, and it lands, which is what
makes this specific to privacy_mode rather than a general claim that module settings do not
work.

Suggested fix

Add it to the Client(...) call in setup(), next to the others:

privacy_mode=privacy_mode,

Re-syncing it after construction as well would match how disabled and debug are handled, since
someone can set it after the first capture() has already forced setup().

Two more from the same comparison, smaller

Comparing every module-level global in __init__.py against what setup() forwards turns up
exactly three that Client.__init__ accepts and setup() does not pass: privacy_mode,
project_root, and project_api_key.

project_root feeds in-app stack frame detection for exception autocapture. When it is not
forwarded, Client.__init__ falls back to os.getcwd(), which is usually right and sometimes
not.

project_api_key is the odd one. It is the name of the first parameter of Client.__init__, so
posthog.project_api_key = "phc_..." looks like the correct spelling, and setup() reads
posthog.api_key instead. The client ends up with an empty key, disabled becomes True, and
every module-level call quietly does nothing. That one is worth a warning at minimum.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions