Skip to content

feat(integrations): the integration contract, and Harbor as its first frontend - #516

Merged
jdchawla29 merged 3 commits into
mainfrom
integrations-contract
Jul 28, 2026
Merged

feat(integrations): the integration contract, and Harbor as its first frontend#516
jdchawla29 merged 3 commits into
mainfrom
integrations-contract

Conversation

@jdchawla29

@jdchawla29 jdchawla29 commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Supersedes #478.

The contract

An integration is a foreign format's frontend. hud.environment.Integration is two verbs:

class Integration(ABC):
    name: ClassVar[str]
    def load(self, ref) -> Taskset: ...                       # the data
    def environment(self, ref, *, name=None) -> Environment: ...  # the world those rows join

Where the constructor can execute is the format's nature — anywhere for an in-process format (a dataset plus a scorer), only inside its own image for a container format like Harbor. Implementations live outside core; core knows only the interface.

hud serve learned the reference form this needs: hud serve pkg.mod:make_env --arg name=x imports a dotted module and calls a factory, so a programmatic environment serves itself with no source file. The path form is unchanged.

Harbor

from hud.integrations import harbor
from hud.eval import DockerRuntime

await harbor.adapt("./terminal-bench")                                    # one image per build context
job = await harbor.load("./terminal-bench").run(agent, runtime=DockerRuntime())

adapt() packages the environment constructor into a HUD-speaking image whose CMD serves hud.integrations.harbor:environment; load() stamps each row with that image, so the same rows also run on ModalRuntime, DaytonaRuntime, or a deploy of the generated build contexts. That is why the package moved under hud. — the adapted image installs the wheel and imports it. Nothing in the SDK or CLI imports it; a Harbor taskset is loaded and placed explicitly.

Rows now carry what the task declared, not just its identity: [metadata] as Task.columns (difficulty/category/tags), [environment] cpu/memory/gpu as RuntimeConfig.resources, real task descriptions on templates, verifier.env for the verifier. Time budgets deliberately stay off the row — a budget bounds the rollout, not the substrate — and harbor.agent_timeout() exposes them for rollout_timeout.

At serve time the workspace applies the task's declared network isolation, env, workdir and user; tests/test.sh grades in place under the task's verifier timeout.

Refused rather than silently downgraded — a wrong score is worse than a refused task. Adaptation replaces the container's own boot process with the serving command, so anything depending on that boot process is refused: a Dockerfile ENTRYPOINT, healthcheck (nothing would start the services it awaits), and mcp_servers (nothing would start the servers they point at) — alongside network_mode = "allowlist", a verifier with its own environment, non-linux os, TPUs, and multi-step tasks (which load fine, but do not adapt). Environments are grouped by build context and declared policy, with the invariant that anything the constructor consumes per group is either in the group key or refused.

Two core fixes this surfaced

  • Workspace checked whether bwrap exists, not whether it works. An installed bwrap that cannot create namespaces failed every session instead of falling back. It now probes once with a session-shaped argv and falls back when unusable; require_isolation=True refuses to serve rather than silently running unisolated.
  • DockerRuntime did not grant env containers the security options they get when hosted, so an environment isolated in one place and silently did not in the other. Both now start containers the same way.

Verification

764 unit tests, plus real-docker end-to-end runs at every step: six datasets from the Harbor registry (hello-world, aime, gpqa-diamond, quixbugs-java, compilebench, swebench-verified) all graded 1.0 through adapt → DockerRuntime, and a task declaring no-network confirmed to have its egress genuinely severed (curl: (6) Could not resolve host) while its env vars and workdir applied.

Container, workspace and grading mechanics were ported from #478 (co-authored).


Note

Medium Risk
Changes container startup security options and workspace isolation behavior (opt-in nested sandbox, bwrap probing, required isolation), plus a large Harbor adapt/grade path; patterns are fail-closed and covered by extensive tests, but misconfiguration could affect grading fidelity or container containment.

Overview
Introduces hud.environment.Integration (loadTaskset, environmentEnvironment) and ships Harbor as hud.integrations.harbor (package moved in-wheel so adapted images can import it). load_environment and hud serve now resolve dotted module targets and factories via --arg key=value, matching adapted image CMDs like hud.integrations.harbor:environment.

Harbor adapt() writes/builds per-env-group contexts under .hud-adapt/ whose CMD serves the integration constructor; load(..., images=) stamps image refs on rows. Rows also carry [metadata] as columns and [environment] resources as runtime_config; agent timeouts stay off-row (harbor.agent_timeout). Unsupported Harbor declarations (ENTRYPOINT, allowlist network, multi-step adapt, etc.) are refused rather than downgraded.

Workspace probes usable_bwrap() once and supports require_isolation=True (fail closed). DockerRuntime adds opt-in nested_sandbox=True (seccomp/systempaths unconfined) for images that sandbox internally; harbor.docker_runtime() enables it by default. HUDRuntime warns once on soft-ignored runtime_config but still errors on GPU/explicit limits. Shared hud.utils.docker and normalize_environment_name; export path hardened (shell quoting, dockerignore, non-root USER restore, determinism note in docs).

Reviewed by Cursor Bugbot for commit adfb5f0. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread hud/integrations/harbor/_adapt.py
Comment thread hud/integrations/harbor/_load.py
Comment thread hud/integrations/harbor/__init__.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 48aaf7423b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/integrations/harbor/_load.py
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from 48aaf74 to 4fd8456 Compare July 27, 2026 05:49
Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/environment/workspace.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4fd84563ca

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_adapt.py
Comment thread hud/integrations/harbor/_adapt.py
Comment thread hud/integrations/harbor/_adapt.py
Comment thread hud/integrations/harbor/_adapt.py Outdated
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from 4fd8456 to 658de7a Compare July 27, 2026 06:10
Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/integrations/harbor/_load.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 658de7a7b4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_adapt.py
Comment thread hud/integrations/harbor/_load.py Outdated
Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/integrations/harbor/_adapt.py Outdated
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from 658de7a to 15b09fa Compare July 27, 2026 06:42
Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/environment/workspace.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 15b09fa9ae

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/v6/advanced/harbor-convert.mdx Outdated
Comment thread hud/integrations/harbor/__init__.py
Comment thread hud/environment/__init__.py Outdated
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch 2 times, most recently from 6e882a5 to 82cf98c Compare July 27, 2026 07:21
Comment thread hud/integrations/harbor/_adapt.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 82cf98cb1e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/environment/__init__.py Outdated
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from 82cf98c to e488823 Compare July 27, 2026 07:27
Comment thread hud/integrations/harbor/_load.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e4888234da

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_load.py Outdated
Comment thread hud/eval/runtime.py Outdated
Comment thread hud/integrations/harbor/_adapt.py
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch 2 times, most recently from 29f2388 to a44a51f Compare July 27, 2026 08:01

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a44a51f247

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/integrations/harbor/_load.py Outdated
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from a44a51f to 7a5cd86 Compare July 27, 2026 08:27
Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/integrations/harbor/_adapt.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

hud task start '{task}' --args '{args_json}' --url tcp://127.0.0.1:{port} >/dev/null 2>&1 || true

P1 Badge Shell-quote exported task arguments

When a task argument contains an apostrophe, such as {"prompt": "what's next?"}, embedding its JSON inside single quotes produces an invalid hud_entrypoint.sh (sh -n reports an unterminated quoted string), so the exported container exits before serving and grading. The same interpolation is repeated in test.sh; shell-quote the task ID and serialized arguments when formatting both scripts.


# Drop any copied taskset files and the source Dockerfile name we don't use.
for stale in env_out.glob("*.json*"):
stale.unlink()

P1 Badge Preserve application JSON files in exported contexts

For an environment whose Dockerfile consumes a root-level JSON file, this removes required build inputs indiscriminately. A common Node context with package.json and COPY package*.json ./, for example, is copied and then has package.json deleted, causing the exported Harbor image build to fail; remove only the specific taskset source file rather than every *.json* file.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/eval/runtime.py
Comment thread hud/environment/__init__.py Outdated
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from 7a5cd86 to aebe6b4 Compare July 27, 2026 09:01
Comment thread hud/integrations/harbor/_adapt.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aebe6b4d14

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/integrations/harbor/_adapt.py
Comment thread hud/eval/runtime.py
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from aebe6b4 to 6f2a3fe Compare July 27, 2026 17:37
Comment thread hud/integrations/harbor/_adapt.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6f2a3fe783

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/integrations/harbor/_adapt.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 736ff58af3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_adapt.py
Comment thread hud/integrations/harbor/_adapt.py
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from 736ff58 to 4a3fbc8 Compare July 27, 2026 21:52

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a3fbc8aa2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/integrations/harbor/_export.py Outdated
Comment thread hud/integrations/harbor/_load.py
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from 4a3fbc8 to 235429d Compare July 27, 2026 22:28

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 235429d4a3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_export.py
Comment thread hud/integrations/harbor/_adapt.py Outdated
Comment thread hud/integrations/harbor/_export.py Outdated
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from 235429d to c9fec84 Compare July 27, 2026 22:39

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

f'hud_task = "{task}"\n'

P2 Badge Escape task IDs in generated TOML

When a valid HUD template ID contains a double quote, backslash, or newline, interpolating it directly inside this quoted TOML value produces an invalid or structurally altered task.toml. Harbor then cannot load the exported task even though the ID works through the HUD task model and shell commands are already safely quoted; serialize this value with TOML-compatible escaping.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_load.py Outdated
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from c9fec84 to 27b58a3 Compare July 27, 2026 22:49

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 27b58a36ab

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_adapt.py
Comment thread hud/utils/docker.py
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from 27b58a3 to 5d6ebe2 Compare July 28, 2026 00:12
Comment thread hud/integrations/harbor/_load.py
Comment thread hud/integrations/harbor/_adapt.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5d6ebe277c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_load.py
Comment thread hud/integrations/harbor/_load.py Outdated
Comment thread docs/v6/reference/runtime.mdx Outdated
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from 5d6ebe2 to e32924e Compare July 28, 2026 02:38

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e32924ee6f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/environment/__init__.py Outdated
Comment thread hud/integrations/harbor/_export.py Outdated
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from e32924e to a0eae4f Compare July 28, 2026 03:18
Comment thread hud/environment/__init__.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a0eae4f497

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_export.py Outdated
Comment thread hud/environment/__init__.py Outdated
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from a0eae4f to 3c9d70b Compare July 28, 2026 03:33
…re factories

An Integration is a foreign format's frontend: load(ref) -> Taskset rows and
environment(ref) -> the live env those rows join. Two verbs, both universal;
where the constructor can execute is the format's nature (in-process for a
dataset+scorer, inside its image for a container format). Implementations
live outside core — core knows only the interface.

load_environment gains the reference form this needs: an importable dotted
module whose attribute is an Environment, or a factory called with args.
'hud serve pkg.mod:make_env --arg name=x' now works, so a programmatic env
serves itself without a source file. The resolver is one function again —
the CLI wrapper is the colon split plus error printing.
…ontainers match hosted

Workspace checked whether bwrap exists, not whether it works. An installed
bwrap that cannot create namespaces — a container whose seccomp profile
blocks unprivileged userns, or whose /proc is masked — failed every session
instead of falling back. It now probes once with a session-shaped argv
(mounting proc is what fails first) and falls back when unusable;
require_isolation=True refuses to serve rather than silently running
unisolated, for callers that promised a task no network.

DockerRuntime did not grant env containers the security options they get when
hosted, so an environment isolated in one place and silently did not in the
other. Both now start containers the same way.
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from 3c9d70b to 1a703e6 Compare July 28, 2026 03:34

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a703e656f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_adapt.py
Comment thread hud/integrations/harbor/_load.py Outdated
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from 1a703e6 to dde4f7f Compare July 28, 2026 04:20

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit dde4f7f. Configure here.

Comment thread hud/integrations/harbor/_load.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dde4f7f36b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_adapt.py
integrations/ becomes hud.integrations because an adapted image imports it:
the image's CMD serves hud.integrations.harbor:environment, so the module has
to ship in the wheel that image installs. Nothing in the SDK or CLI imports
it — a Harbor taskset is loaded and placed explicitly.

Harbor implements the contract:

- load() carries what a task declares, not just its identity: [metadata] as
  Task.columns (difficulty/category/tags — the platform's facets),
  [environment] cpu/memory/gpu as RuntimeConfig.resources, real descriptions
  on templates, verifier.env for the verifier. Time budgets stay off the row
  — a budget bounds the rollout, not the substrate — and agent_timeout()
  exposes them for rollout_timeout.
- adapt() packages the environment constructor into one HUD-speaking image
  per build context, whose CMD serves that constructor; rows then run on any
  container placement. Images are content-addressed and kept, built once per
  group under a lock.
- environment() is what those images serve: the workspace applies the task's
  declared network isolation, env, workdir and user, healthchecks gate
  serving, URL MCP servers are published as capabilities, and tests/test.sh
  grades in place under the task's verifier timeout.

Behaviour that cannot be reproduced faithfully is refused, not dropped:
allowlist egress, a verifier with its own environment, stdio MCP servers,
non-linux os, TPUs, and multi-step tasks (which load, but do not adapt).
Environments are grouped by build context *and* declared policy, so one
environment always serves one policy.

Container, workspace and grading mechanics were ported from #478.

Co-authored-by: Nancy <najilau@ucsc.edu>
@jdchawla29
jdchawla29 force-pushed the integrations-contract branch from dde4f7f to adfb5f0 Compare July 28, 2026 04:37

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: adfb5f0933

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/integrations/harbor/_load.py
Comment thread hud/integrations/harbor/_load.py
@jdchawla29
jdchawla29 merged commit 0401a89 into main Jul 28, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant