Skip to content

Add external Activity delivery transport for companion stores#3548

Closed
Jiwoon-Kim wants to merge 1 commit into
Automattic:trunkfrom
Jiwoon-Kim:feature/external-activity-delivery
Closed

Add external Activity delivery transport for companion stores#3548
Jiwoon-Kim wants to merge 1 commit into
Automattic:trunkfrom
Jiwoon-Kim:feature/external-activity-delivery

Conversation

@Jiwoon-Kim

Copy link
Copy Markdown
Contributor

Summary

This draft explores a supported outbound transport surface for companion plugins that own their own Actor, Object, Activity, and relationship stores.

It adds an opt-in deliver_activity() API that accepts:

  • a complete ActivityStreams Activity payload;
  • an external URI-backed signing descriptor (actor_uri, key_id, and a non-secret private_key_ref); and
  • explicit recipient Inbox URLs.

The official plugin remains responsible for the transport spool, transient key resolution, HTTP signatures, retries, and delivery. The caller remains authoritative for Activity and relationship state.

API shape question

Before treating this particular API as final, I would appreciate maintainer guidance on the smallest preferred extension shape.

The companion needs only two capabilities that the current Outbox API cannot express:

  1. a URI-backed signing identity that is not necessarily an integer WP_User ID; and
  2. explicit recipient Inbox URLs supplied by the companion's authoritative relationship store.

Would you prefer extending the existing add_to_outbox() / Outbox::add() path to accept those two inputs instead of adding a separate deliver_activity() entry point? Reusing the existing path would be welcome if it can preserve the external Actor identity and avoid deriving recipients from the official follower store.

This draft uses a separate transport-only spool because the current path:

  • gates delivery through user_can_activitypub( $user_id );
  • derives the signing Actor and key identity from an integer user ID; and
  • derives recipients from the official plugin's follower model rather than accepting explicit Inboxes.

That makes a virtual Actor discoverable through the existing Actor filters, but it cannot send as the same URI-backed identity without creating duplicate domain state or a split signing identity.

Implementation

  • Stores the complete bounded payload in an ap_outbox transport row identified idempotently by Activity URI.
  • Stores only a non-secret key reference; private key material is rejected at enqueue time and resolved in memory by activitypub_resolve_external_signing_identity during the worker run.
  • Requires public HTTPS Inbox URLs, sends no cookies, follows no redirects, and uses the existing retry policy.
  • Exposes transport status without making the spool authoritative Activity or relationship state.
  • Leaves all stock publishing and delivery behavior unchanged unless a companion explicitly calls the new API.

Reference composition and E2E

The reference implementation is the Axismundi ActivityPub Bridge 0.0.14 Alpha, together with:

A signed bidirectional Follow/Accept E2E passed against mastodon.social:

  • Mastodon Follow -> official signature/request validation -> existing Inbox action -> Axismundi URI-keyed ledger -> outbound Accept through this transport -> accepted by Mastodon.
  • Axismundi Follow -> this transport's queue/sign/retry path -> Mastodon Accept -> official validation/Inbox action -> Axismundi relationship reached accepted.

The staging Actor is @thaumiel999@designbusan.ai.kr.

Scope of the current E2E evidence is Follow/Accept. Like, Undo, shared-Inbox delivery, and Create delivery remain separate activity-level scenarios.

Validation

  • Added focused PHPUnit coverage for validation, idempotent queueing, transient signing resolution, and worker delivery state.
  • Changed-file PHPCS: 4/4 clean.
  • Axismundi Bridge transport composition: 15/15.
  • Real signed Follow/Accept E2E: passed in both directions against mastodon.social.

Relationship to #3541

#3541 addresses selective module initialization and the inbound composition side. This draft is independent of that branch and contains only the outbound transport surface.

Refs #3533

@pfefferle

Copy link
Copy Markdown
Member

Thanks for splitting this out and for asking the API-shape question directly — that makes this easy to answer honestly: I don't think we need either variant. Not a new deliver_activity(), and not an extended add_to_outbox().

The two capabilities you're missing are already available, just not where you looked:

  1. Signing as a URI-backed identity. Signature::sign_request() is hooked on http_request_args (priority 0) for all WordPress HTTP requests. Any plugin can already get a correctly signed request today by calling wp_safe_remote_post( $inbox, array( ..., 'key_id' => $key_id, 'private_key' => $private_key ) ). Your own worker in this PR does exactly that — which means it works unchanged from inside the Bridge, without any upstream change.
  2. Explicit recipient inboxes. Once you send the request yourself, this is trivially yours: your relationship store is authoritative, so it should also own the recipient list, the spool, and the retry policy. The retry constants you reuse (get_retry_error_codes(), get_retry_delay()) are two small values, not machinery worth coupling to.

So the composition boundary I'd suggest is one step to the left of where this PR draws it: the official plugin owns inbound validation and the signature implementation (both already exposed), and the companion owns everything keyed by its own domain model — including outbound spooling.

There's also a concrete problem with the implementation as-is: the transport rows reuse the ap_outbox post type with status pending, but Scheduler::reprocess_outbox() periodically scans all pending ap_outbox posts and schedules them through the stock activitypub_process_outbox pipeline. Your external rows would get picked up by our Dispatcher as well, so you'd get double processing on top of your own worker. That's a good illustration of why a foreign spool shouldn't live inside our CPT: the invariants of that post type belong to the Outbox/Dispatcher/Scheduler triangle and will keep shifting under you.

If there's an upstream change worth making here, it's documentation: the key_id/private_key request-args signing surface isn't documented as a stable extension point anywhere. I'd happily take a PR that documents it (FEDERATION.md or a doc block), which would give the Bridge something supported to build on without adding a parallel delivery pipeline.

Closing the loop with #3541: this is the same pattern — the extension points you need keep turning out to already exist. That's a good sign for the composition approach, but it means the upstream diffs should be shrinking toward docs, not growing toward new APIs.

@Jiwoon-Kim

Copy link
Copy Markdown
Contributor Author

Thank you. I moved the composition boundary exactly as suggested.

The companion now owns a private ax_ap_delivery spool, explicit recipients, retry policy, and HTTP dispatch. It calls wp_safe_remote_post() with transient key_id / private_key arguments and relies on the existing official http_request_args signer. Private key material is resolved only inside the worker and is never persisted by the Bridge.

I also removed all new External_Delivery API dependencies. Existing experimental rows are migrated non-destructively out of ap_outbox: the source rows are retained and linked, old worker events are cleared, and pending source rows are moved out of the stock Scheduler scan only after an exact Bridge job exists.

In the current branch, the stock Dispatcher did not reach a second HTTP delivery because the experimental rows had post_author = 0; it instead changed them from pending to publish after Actor resolution failed. That still confirms the underlying issue you identified: two state machines were assigning different meanings to the same CPT status. The foreign spool did not belong in ap_outbox.

The revised implementation passes the outbound suite against both the fork and unmodified ActivityPub 9.0.2, including real request signing, a single-job worker lock, URI idempotency, and proof that the stock Outbox query cannot discover a Bridge job. See Jiwoon-Kim/axismundi@ff959fe.

This code PR is no longer needed, so I am closing it. I will prepare the much smaller documentation change for the existing key_id / private_key signing surface separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants