PyTSan

An experimental data race detector for Python programs. PyTSan instruments the CPython interpreter to track shared-memory accesses and synchronization operations, then uses a lockset + happens-before algorithm to detect data races in multi-threaded Python code.

For a detailed explanation of the methodology, see the thesis.

How It Works

PyTSan patches the CPython interpreter at the bytecode evaluation level (Python/ceval.c) to intercept:

• Variable accesses — LOAD_NAME, STORE_NAME, LOAD_ATTR, STORE_ATTR, LOAD_GLOBAL, STORE_GLOBAL, LOAD_DEREF, STORE_DEREF, BINARY_SUBSCR, STORE_SUBSCR, etc.
• Lock operations — threading.Lock.acquire/release, threading.RLock.acquire/release
• Thread lifecycle — thread creation, deletion, and join synchronization
• Object deallocation — to track object ID reuse

These events are fed into a C++ analysis backend (TracedAnalyzer) that implements a lockset + happens-before race detection algorithm.

Building

1. Clone CPython v3.10.0 (commit b494f5935c92951e75597bfe1c8b1f3112fec270):

git clone --branch v3.10.0 --depth 1 https://github.com/python/cpython.git
cd cpython

2. Clone PyTSan into the CPython source tree:

git clone https://github.com/dsrg-uoft/pytsan.git

3. Apply the patch:

git apply pytsan/patches/pytsan.patch

4. Build CPython as usual:

./configure
make

Usage

Run your Python program with pytsan enabled via environment variables:

PYTSAN=1 \
  PYTSAN_ANALYZER_OUT=analyzer.json \
  ./python pytsan/examples/counter_race.py

Environment Variables

Variable	Description	Default
PYTSAN	Enable PyTSan instrumentation. Must be set to any non-empty value.	Disabled
PYTSAN_TRACER_OUT	File path to write the JSON trace of all recorded events.	Disabled
PYTSAN_ANALYZER_OUT	File path to write detected races as JSON.	Disabled
PYTSAN_SLOW_MODE	Create a new segment on every event (slower but more precise). 1 = enabled, 0 = disabled.	1
PYTSAN_IGNORE_FIRST_WRITE	Skip race detection on the first write to a variable. 1 = enabled, 0 = disabled.	1
PYTSAN_NEED_INSTRUMENT_JOIN	Instrument Thread.join() for happens-before ordering. 1 = enabled, 0 = disabled.	1

Output

Race Report (PYTSAN_ANALYZER_OUT)

A JSON array of detected races. Each entry includes:

• id — the shared variable identifier (object ID + attribute/name)
• seg1 / seg2 — the two conflicting access segments
• seg1_is_write / seg2_is_write — whether each access was a write
• context_of_detection — thread ID, PID, opcode, filename, and line number

For a detailed explanation of duplicated reports, see the thesis.

Example output from counter_race.py:

[
  {
    "context_of_detection": {
      "filename": "pytsan/examples/counter_race.py",
      "lineno": 9,
      "opcode_name": "LOAD_GLOBAL"
    },
    "id": "<tid>.2 x",
    "nth_action": 0,
    "seg1": {
      "context": {
        "filename": "pytsan/examples/counter_race.py",
        "lineno": 10,
        "opcode_name": "STORE_GLOBAL"
      },
      "lock_set": ["<lock_addr>"],
      "tid": "<tid>.0"
    },
    "seg1_is_write": true,
    "seg2": {
      "context": {
        "filename": "pytsan/examples/counter_race.py",
        "lineno": 9,
        "opcode_name": "LOAD_GLOBAL"
      },
      "lock_set": ["<lock_addr>"],
      "tid": "<tid>.1"
    },
    "seg2_is_write": false
  }
]

Trace Log (PYTSAN_TRACER_OUT)

A JSON array of all recorded events (variable accesses, lock operations, thread events, segment transitions) for debugging and visualization.

License

Copyright 2024-2026 Chaoyue Gong.

Except where otherwise noted, PyTSan's original source code and examples are licensed under the Apache License 2.0.

This repository also contains separately licensed third-party material, including JSON for Modern C++ and CPython-derived patch and example content. See THIRD_PARTY_NOTICES.md for details and the applicable license texts.