Move task execution architecture docs to contributing-docs

[jason810496]

• related: follow-up to Add Task SDK, Go and Java SDK execution architecture diagrams #69750; addresses review feedback on this PR (Move task execution architecture docs to contributing-docs #70657 (comment))

Why

The "Task execution architecture" section added in #69750 documents worker internals (the Supervisor, the Coordinator layer, and the language subprocess protocols). That is developer-facing material, not something end users need in the core-concepts/overview.rst page, so it belongs in the contributing docs instead.

How

• Move the whole section (Python Task SDK execution, the Coordinator path shared by the Java and Go SDKs, and the standalone Go edge worker) into a new dev-facing page contributing-docs/31_task_execution_architecture.rst, next to the new-language-SDK guide.
• Move the five diagrams to contributing-docs/images/ and extend the generate-airflow-diagrams prek hook to cover that directory. Rewrite the cross-links for GitHub rendering (relative .rst links, .. image:: images/...).
• The two former Java-only diagrams are generalized into shared Coordinator diagrams (Java + Go); the native-language diagram is renamed to the Go edge worker it represents.
• Drop the section from core-concepts/overview.rst; register the new page in the contributing-docs README and cross-link it from 30_new_language_sdk.rst.

---

Was generative AI tooling used to co-author this PR?

• Yes, with help of Claude Code (Opus 4.8) following the guidelines

[uranusjr]

I don’t think we want to put coordinator first. I intentionally put the concept to the back because the user does not need to care how it works, only how to use it. The new writeup emphasis on internals too much. It should instead focus on introducing how the coordinator can be used (by configuration) instead.

[jason810496]

I don’t think we want to put coordinator first. I intentionally put the concept to the back because the user does not need to care how it works, only how to use it. The new writeup emphasis on internals too much. It should instead focus on introducing how the coordinator can be used (by configuration) instead.

Hmmm, then I will need to move the whole paragraph introduce in #69750 to somewhere else.

All the diagrams and descriptions are quite verbose for the users IMO -- http://apache-airflow-docs.s3-website.eu-central-1.amazonaws.com/docs/apache-airflow/stable/core-concepts/overview.html#non-python-language-sdks-go-and-java

[jason810496]

Then would it be better to move them all (starting from "Task execution architecture" section) to airflow-core/docs/core-concepts/task-execution-archciture.rst as a new page instead of overwhelming the core-concepts/overview.rst page?

[uranusjr]

I think they do not belong in user-facing documentation. It would fit if we have a section for advanced concepts or something, but we currently don’t. They do fit in dev-facing docs though (contributing-docs).

[jason810496]

I just moved them all to contributing-docs/31_task_execution_architecture.rst (a new page under contributing-docs that don't make the new task-sdk one too large).

[potiuk]

To be honest, I do not think those are "contributing" docs. Those are "deployment" docs that explain what is the interaction between different components, what processes are running and what to expect.

One of the big issues it explains is what is teh resource usage connected and what processes you can expect to see running when you use the SDK. I think the deployment managers should know that in some cases they can just run a long running process that does everything (golang) , and it some other cases they will have python processes taking 100s of MBs for each task run - this is far more deployment information than contributing documentation IMHO,

[potiuk]

I think we should first agree who is the audience of that docs.

IMHO. Deployment Managers - not contributors.