Fix Python bindings version ignored in system tests

[AdityaGupta716]

Problem
Tutorial run.sh scripts create their own .venv and install from requirements.txt (e.g. pyprecice~=3.0), silently overwriting the specific PYTHON_BINDINGS_REF version installed by the Docker image. This means the version argument passed to system tests was completely ignored.
Root cause
The Docker image installs the correct version into /home/precice/venv but never permanently activates it. At container runtime the venv isn't on PATH, so the run script sees no active venv and creates its own. PRECICE_TUTORIALS_NO_VENV (added in #680) was never set in the component templates, so the escape hatch was never triggered.
Fix

Add ENV VIRTUAL_ENV and ENV PATH after venv creation in python_bindings, fenics_adapter, and nutils_adapter Dockerfile stages — permanently activating the correct venv at container runtime
Set PRECICE_TUTORIALS_NO_VENV=1 in python-bindings, fenics-adapter, nutils-adapter, and su2-adapter component templates
Add a sanity check that fails fast with a clear error if precice is not importable when the escape hatch is set
Fix --system-site-packages typo in flow-over-heated-plate/fluid-su2/run.sh

Fixes #584

[precice-bot]

This pull request has been mentioned on preCICE Forum on Discourse. There might be relevant details there:

https://precice.discourse.group/t/gsoc-2026-aditya-gupta/2773/4

[AdityaGupta716]

@MakisH the component templates were never setting PRECICE_TUTORIALS_NO_VENV=1 (the escape hatch added in #680), and the Docker stages weren't permanently activating /home/precice/venv at runtime. This PR closes both gaps for python_bindings, fenics_adapter, nutils_adapter, and su2_adapter also adds a fast-fail sanity check if precice isn't importable when the escape hatch is set, and fixes a system-site-packages typo in flow-over-heated-plate/fluid-su2/run.sh. Please review!

[MakisH]

Looks good and clean, thanks! I did some small modifications, and I still need to test it.

Good catch with the --system-site-package[s] typo!

[Copilot]

Pull request overview

Fixes the issue where the PYTHON_BINDINGS_REF argument was silently ignored by system tests because tutorial run.sh scripts created their own .venv overriding the Docker image's venv. The Docker image now permanently activates the venv via ENV VIRTUAL_ENV/ENV PATH for the Python-based stages, and the system-test component templates set PRECICE_TUTORIALS_NO_VENV=1 so the run scripts skip venv creation. A typo in --system-site-package(s) is also corrected in several tutorial run scripts.

Changes:

• Persist venv activation in python_bindings, fenics_adapter, nutils_adapter, su2_adapter, and micro_manager Docker stages.
• Set PRECICE_TUTORIALS_NO_VENV=1 and add a precice-import sanity check in four component templates.
• Fix --system-site-package → --system-site-packages in four tutorial run.sh scripts.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tools/tests/dockerfiles/ubuntu_2404/Dockerfile	Adds ENV VIRTUAL_ENV / ENV PATH after venv creation in python_bindings, fenics_adapter, nutils_adapter, su2_adapter, and micro_manager stages.
tools/tests/component-templates/python-bindings.yaml	Sets PRECICE_TUTORIALS_NO_VENV=1 and adds a precice-import sanity check before running.
tools/tests/component-templates/fenics-adapter.yaml	Same as python-bindings template.
tools/tests/component-templates/nutils-adapter.yaml	Same as python-bindings template.
tools/tests/component-templates/su2-adapter.yaml	Same as python-bindings template, preserving SU2 env vars.
flow-over-heated-plate/fluid-su2/run.sh	Fixes --system-site-package typo.
elastic-tube-3d/solid-fenics/run.sh	Fixes --system-site-package typo.
channel-transport-reaction/fluid-fenics/run.sh	Fixes --system-site-package typo.
channel-transport-reaction/chemical-fenics/run.sh	Fixes --system-site-package typo.
changelog-entries/745.md	Adds changelog entry for the fix.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

[MakisH]

Current state: The FEniCS installation is not using the right venv.

I am trying to test this locally, but (1) the FEniCS adapter tests fail due to venv issues, and (2) fieldcompare detects differences in the elastic-tube-1d and Nutils cases.

Regarding FEniCS, the error is familiar:

Started on: Sun, 17 May 2026 11:00:49 +0000
ERROR: could not import mpi4py!
ValueError: mpi4py.MPI.Op size changed, may indicate binary incompatibility. Expected 56 from C header, got 40 from PyObject

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/runs/flow-over-heated-plate_fluid-openfoam-solid-fenics_2026-05-17-130000/solid-fenics/solid.py", line 6, in <module>
    from fenics import Function, SubDomain, RectangleMesh, BoxMesh, FunctionSpace, VectorFunctionSpace, Point, \
  File "/usr/lib/python3/dist-packages/fenics/__init__.py", line 7, in <module>
    from dolfin import *
  File "/usr/lib/petsc/lib/python3/dist-packages/dolfin/__init__.py", line 170, in <module>
    from .fem.assembling import (assemble, assemble_system, assemble_multimesh, assemble_mixed,
  File "/usr/lib/petsc/lib/python3/dist-packages/dolfin/fem/assembling.py", line 34, in <module>
    from dolfin.fem.form import Form
  File "/usr/lib/petsc/lib/python3/dist-packages/dolfin/fem/form.py", line 12, in <module>
    from dolfin.jit.jit import dolfin_pc, ffc_jit
  File "/usr/lib/petsc/lib/python3/dist-packages/dolfin/jit/jit.py", line 125, in <module>
    def compile_class(cpp_data, mpi_comm=MPI.comm_world):
                                         ^^^^^^^^^^^^^^
RuntimeError: Error when importing mpi4py

When building the container, it looks like a venv is always created at the time of installing fenicsprecice (./venv/):

#22 [solid-fenics fenics_adapter 2/2] RUN . /home/precice/venv/bin/activate &&     pip3 install git+https://github.com/precice/fenics-adapter.git@697ab6f
#22 0.680 WARNING: Skipping /usr/lib/python3.12/dist-packages/pybind11-2.11.1.dist-info due to invalid metadata entry 'name'
#22 0.692 Collecting git+https://github.com/precice/fenics-adapter.git@697ab6f
#22 0.693   Cloning https://github.com/precice/fenics-adapter.git (to revision 697ab6f) to /tmp/pip-req-build-ycm973o5
#22 0.695   Running command git clone --filter=blob:none --quiet https://github.com/precice/fenics-adapter.git /tmp/pip-req-build-ycm973o5
#22 1.607   WARNING: Did not find branch or tag '697ab6f', assuming revision or ref.
#22 1.613   Running command git checkout -q 697ab6f
#22 1.627   Resolved https://github.com/precice/fenics-adapter.git to commit 697ab6f
#22 1.642   Installing build dependencies: started
#22 3.568   Installing build dependencies: finished with status 'done'
#22 3.570   Getting requirements to build wheel: started
#22 3.789   Getting requirements to build wheel: finished with status 'done'
#22 3.789   Preparing metadata (pyproject.toml): started
#22 4.011   Preparing metadata (pyproject.toml): finished with status 'done'
#22 4.017 Requirement already satisfied: pyprecice>=3.0.0.0 in ./venv/lib/python3.12/site-packages (from fenicsprecice==2.3.0) (3.4.0.post1+git.f40889ab)
#22 4.288 Collecting scipy<1.15.0 (from fenicsprecice==2.3.0)
#22 4.357   Downloading scipy-1.14.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
#22 4.377      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.8/60.8 kB 2.9 MB/s eta 0:00:00
#22 4.391 Requirement already satisfied: numpy<2,>=1.13.3 in /usr/lib/python3/dist-packages (from fenicsprecice==2.3.0) (1.26.4)
#22 4.392 Requirement already satisfied: mpi4py>=3 in ./venv/lib/python3.12/site-packages (from fenicsprecice==2.3.0) (4.1.2)
#22 4.438 Downloading scipy-1.14.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (40.8 MB)
#22 4.981    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.8/40.8 MB 75.4 MB/s eta 0:00:00
#22 5.094 Building wheels for collected packages: fenicsprecice
#22 5.097   Building wheel for fenicsprecice (pyproject.toml): started
#22 5.340   Building wheel for fenicsprecice (pyproject.toml): finished with status 'done'
#22 5.340   Created wheel for fenicsprecice: filename=fenicsprecice-2.3.0-py3-none-any.whl size=29356 sha256=4528af0a4e0105e6c3a1d24210da44100c7e0c8b179d7663627e2241090c701e
#22 5.340   Stored in directory: /tmp/pip-ephem-wheel-cache-73um32wm/wheels/e5/df/fe/75ae9722c228ae687f4fdf5760be5605bc04cc4cb75a306591
#22 5.342 Successfully built fenicsprecice
#22 5.530 WARNING: Skipping /usr/lib/python3.12/dist-packages/pybind11-2.11.1.dist-info due to invalid metadata entry 'name'
#22 5.567 Installing collected packages: scipy, fenicsprecice
#22 9.780 Successfully installed fenicsprecice-2.3.0 scipy-1.14.1
#22 DONE 10.0s

The fieldcompare regressions are generally small, but various reasons could be behind them.

Latest command I used:

python3 systemtests.py --build_args=PLATFORM:ubuntu_2404,PRECICE_REF:57eb8d8,PYTHON_BINDINGS_REF:f40889a,CALCULIX_VERSION:2.20,CALCULIX_ADAPTER_REF:1199399,DUMUX_VERSION:3.7,DUMUX_ADAPTER_REF:101aadd,FENICS_ADAPTER_REF:697ab6f,MICRO_MANAGER_REF:93735d2,OPENFOAM_EXECUTABLE:openfoam2512,OPENFOAM_ADAPTER_REF:4b42bb0,SU2_VERSION:7.5.1,SU2_ADAPTER_REF:5abe79b,TUTORIALS_REF:29eb46100e5df6cd48c0b6d89514fb51acb3eb0e --suites=release_test

I am putting this aside for now, as I am running out of ideas and time. @AdityaGupta716 if you have any suggestions, let me now.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

[AdityaGupta716]

@MakisH Would it make sense to revert fenics_adapter back to FROM precice_dependencies instead of FROM python_bindings, and keep its own --system-site-packages venv as before? That would isolate FEniCS from the global venv change while keeping the fix working for all other Python adapters.

[MakisH]

@AdityaGupta716 That would be an option, but it would then lead to an inconsistent state regarding #584.

I am trying to understand how the FEniCS adapter Docker image handles that. Resources:

• FEniCS adapter build-docker.yml
  • In this example, it looks like RUN python3 -m pip install --user git+https://github.com/precice/fenics-adapter.git@develop finds Requirement already satisfied: pyprecice>=3.0.0.0 in ./home/precice/.local/lib/python3.10/site-packages (from fenicsprecice==2.2.0.post30+git.697ab6f4) (3.1.2+8.gced8a42).
• FEniCS adapter Dockerfile
• The Docker images of the Python bindings are no longer maintained
  • The respective Dockerfile was installing using pip install --user

With this system-wide venv, we are essentially reproducing the behavior of --user, so we could as well re-enable that, and install with pip install --user --break-system-packages.

Would you like to try if that works, or should I?

[AdityaGupta716]

@MakisH I'll take this on after my exams. In the meantime I'll go through the FEniCS venv/mpi4py situation and think through the pip install --user --break-system-packages approach you suggested. will that be ok ?

[MakisH]

Sure! If I find time before that, I might still try that directly, as this should generally be close to merging.

[MakisH]

@AdityaGupta716 I saw you pushed some changes to this branch recently (thank you!). Is this now ready for another review? Does FEniCS now pick up the right Python bindings version in the tutorials?

[AdityaGupta716]

@MakisH hi sry for the late reply i first tried the pip install --user --break-system-packages approach you suggested, and removed PRECICE_TUTORIALS_NO_VENV from fenics-adapter.yaml since FEniCS wasn't using the venv anymore.

Tested it locally and got the same mpi4py error as before. Turns out the install method (--user vs venv) wasn't actually the issue — fenicsprecice pulls in mpi4py as a dependency, and that copy ends up in the venv with a different binary ABI than the system mpi4py FEniCS/PETSc were compiled against. Since the venv sits first on sys.path, Python always loads the conflicting one no matter how fenicsprecice itself gets installed.

Fix that worked: uninstall mpi4py from the venv right after installing fenicsprecice (pip3 uninstall -y mpi4py), so Python falls through to the system one. pyprecice and everything else stay in the venv as-is. Restored PRECICE_TUTORIALS_NO_VENV=1 and the sanity check since the venv approach is fine again with this in place.

Tested with your build args (PYTHON_BINDINGS_REF:f40889a, FENICS_ADAPTER_REF:697ab6f, PRECICE_REF:v3.4.1, PRECICE_PRESET:production-audit) — precice, fenics, and fenicsprecice all import cleanly now, and pyprecice still resolves to the venv (correct PYTHON_BINDINGS_REF). Also checked that su2_adapter and micro_manager still have their own mpi4py untouched, since they share the python_bindings base.

Also ran elastic-tube-3d (fluid-openfoam, solid-fenics) tutorial through systemtests.py . The solver log shows preCICE coupling iterating and reaching "All converged" no mpi4py error anywhere in the run. fieldcompare did fail, but that's expected and already noted in tests.yaml for this case ("Too small values, expected to fail the comparisons"), unrelated to this fix.