Improve documentation

.yamllint.yaml

@@ -2,7 +2,9 @@ extends: default
 
 rules:
   line-length:
-    max: 150
+    max: 120
+    ignore: |
+      mkdocs.yml
   truthy:
     check-keys: false
   comments:

README.md

@@ -10,7 +10,7 @@ You need uv to run this, take a look at [this](https://docs.astral.sh/uv/getting
 
 The project uses pre-commit to enforce YAML and Markdown codestyle, install that with:
 
-```bash
+```shell
 uv run pre-commit install
 ```
 
@@ -19,15 +19,15 @@ The expected repos are listed in the mkdocs.yml config file.
 Comment out these parts if you dont need them.
 Otherwise, you can clone and install everything you need with:
 
-```bash
+```shell
 ./setup_subprojects.sh
 ```
 
 Now you can edit the documentation.
 
 To render the documentation locally, run:
 
-```bash
+```shell
 uv run --no-sync mkdocs serve
 ```
 

docs/about.md

@@ -8,14 +8,14 @@ orchestrating order, administration and provisioning processes.
 
 ## Core Principles of WFO
 
-  - Open Source and Community-Driven – developed by and for the NREN community,
-    but also adopted by organisations beyond it.
-  - User-Friendly and Modular – lightweight, opinionated, and easy to integrate
-    into existing systems.
-  - Sustainability and Continuity – working towards a self-sustaining
-    organisation to maintain and enhance the core software.
-  - Support and Collaboration – encouraging adoption and supporting the
-    community through documentation, workshops, and best practices.
+- Open Source and Community-Driven – developed by and for the NREN community,
+  but also adopted by organizations beyond it.
+- User-Friendly and Modular – lightweight, opinionated, and easy to integrate
+  into existing systems.
+- Sustainability and Continuity – working towards a self-sustaining
+  organization to maintain and enhance the core software.
+- Support and Collaboration – encouraging adoption and supporting the
+  community through documentation, workshops, and best practices.
 
 ## Governance and project documentation
 

docs/architecture/framework.md

@@ -0,0 +1,33 @@
+The Workflow Orchestrator programme contains multiple components for both the frontend and backend, as shown below:
+
+![Screenshot](../img/base-orchestrator-setup.png)
+
+## Backend
+
+### Orchestrator Core
+
+The `orchestrator-core` component is an open-source backend component, which defines the ruleset for product modeling and workflows. The `orchestrator-core` is a mandatory component to have a functional workflow orchestrator application. This component is written in Python and makes use of the Fastapi framework. The `orchestrator-core` repo can be found [here](https://github.com/workfloworchestrator/orchestrator-core). The `orchestrator-core` cannot be run standalone, as it contains no definition of any products and workflows.
+
+### Workflow Orchestrator
+
+The `Workflow Orchestrator` is the custom implementation of the orchestrator backend. It is the application which defines all your products, workflows and tasks to create/modify/terminate/validate your product instances, the so-called `subscriptions`. All product modeling, tasks/workflows and subscription details are stored in the `orchestrator-coredb` which is part of the orchestrator-core package. This custom implementation of the workflow orchestrator uses the `orchestrator-core` as sub-module.
+
+With the two backend components set up correctly, you'll have a running Workflow Orchestrator instance accessible via the API. Additionally, with minimal effort, you can have a fully functional frontend application running on top of it.
+
+### Example Orchestrator
+
+An example of the `Workflow Orchestrator` using the `orchestrator-core` with some example products and workflow are available [here](https://github.com/workfloworchestrator/example-orchestrator).
+
+## Frontend
+
+### Workflow Orchestrator UI library
+
+The Workflow Orchestrator UI can also be split into 2 major components. A frontend library called the `components orchestrator-ui` library as available on [npm](https://www.npmjs.com/package/@orchestrator-ui/orchestrator-ui-components). And the out-of-the-box `Workflow Orchestrator UI`, which uses the npm library in it's pages. The example of this frontend is the `example-orchestrator-ui` and can be found [here](https://github.com/workfloworchestrator/example-orchestrator-ui). In most cases the example orchestrator is the best deployment model to start with, as is contains a fully functional userinterface, while you can focus your effort on developing products, workflows and tasks.
+
+### More advanced UI deployment models
+
+By tweaking the `example-orchestrator-ui` it is possible to easily add extra pages, cards on dashboard page, or change the rending of certain resource type. Examples of the possible changes is shown [here](orchestrator-ui.md) . This will leverage the default architecture, like shown below:
+
+![Screenshot](../img/custom-orchestrator-setup.png)
+
+Another approach could be to use individual components from the npm library and build your own application or integrate the components in an existing application.

docs/architecture/input-forms.md

@@ -0,0 +1,235 @@
+# Forms - from a frontend perspective
+
+Orchestrator Core contains a module called Pydantic Forms. Pydantic Forms allows for configuration of input forms to collect user input needed for the execution of a workflow. The module contains a frontend part that displays the forms automatically and handles submission and showing validation errors. This documentation describes what happens on the frontend side of this process.
+
+## Initiating a workflow from frontend
+
+A workflow can be initiated by doing a POST call to `/processes/<workflow_name>`
+
+The steps that happen to initiate a workflow on the frontend are:
+
+- A `POST` request to `/processes/<workflow_name>` with an empty payload
+- The backend determines what input values are missing and sends a response with http status code `510` and a payload containing a [JSON6Schema definition][2] describing the form to display. See [Example JSON6Schema response](#example-json6schema-response)
+- The frontend uses the [pydantic-forms library][1] to parse the JSON response into a form to display
+- The [AutofieldLoader function][3] is called for each of the form.properties in the JSON response. This function uses the properties `type` and `format` to determine what kind of field will be displayed.
+
+!!! info
+    The following documentation is out of date after the migration from `uniforms` to `pydantic-forms`.
+    Development ticket [orchestrator-ui-library#2381](https://github.com/workfloworchestrator/orchestrator-ui-library/issues/2381) will update this.
+
+In the example JSON response below one of the properties is
+
+```json
+"customer_id": {
+    "default": "c9b5e717-0b11-e511-80d0-005056956c1a",
+    "format": "customerId",
+    "title": "Customer Id",
+    "type": "string",
+    "uniforms": {
+        "disabled": true,
+        "value": "c9b5e717-0b11-e511-80d0-005056956c1a"
+    }
+}
+```
+
+In the autoFieldFunction this maps to a CustomerField.
+
+```tsx
+export function autoFieldFunction(
+    props,
+    uniforms,
+) {
+    const { allowedValues, checkboxes, fieldType, field } = props;
+    const { format } = field;
+
+    switch (fieldType) {
+    ...
+        case String:
+            switch (format) {
+                ...
+                case 'customerId':
+                    return CustomerField;
+                ...
+            }
+    ...
+```
+
+The CustomerField is a React component that is provided by the Orchestrator Component Library.
+It's passed the complete property object so it can use them to adjust it's behaviour.
+
+- A `POST` with the form values is made to the same `/processes/<workflow_name>` endpoint
+- The response status code can be:
+  - `400: Form invalid` Invalid values have been detected because the validators the backend runs have failed. An error message is shown.
+  - `510: FormNotComplete` There is another step. This response contains another json response containing a form.
+  - `201: Created` The workflow was initiated successfully. The response contains a workflow id and the user is redirected to the workflow detail page
+
+**Note**. For forms that have multiple steps the user input for each step is accumulated in local frontend state and posted to `/processes/<workflowname>` on each step. The endpoint will receive all available user inputs on each step and determine what other user input it still needs or if it's ready to start the workflow.
+
+**Note 2** The Orchestrator Component library contains fields that are marked as deprecated and live in a folder named `deprecated`. These contain field types that are very specific to workflows that are in use by SURF. There are plans to remove these from the general purpose components library.
+
+**Note 3** There are plans to make it easier to extend this functionality to add custom field types and extend the switch statement in the autoFieldFunction to include these custom `types` or `formats`
+
+## Backend: Creating a workflow that generates a form that asks for user input
+
+Creating workflows is described in other parts of this documentation in more detail. The practical steps and those that are relevant to the frontend
+are these
+
+- A mapping between a function and a `processes/<workflow-name>` endpoint is added to the `workflows/init.py` file
+
+```python
+    LazyWorkflowInstance("surf.workflows.core_link.create_core_link", "create_core_link")
+```
+
+This makes `POST` requests to `processes/create_core_link` call `surf.core_lint.create_core_link` with the `POST` payload
+
+- The `create_core_link` function is decorated with the `create_workflow` decorator. It provides workflow orchestrator functionality.
+
+```python
+@create_workflow("Create Core Link", initial_input_form=initial_input_form_generator)
+def create_core_link() -> StepList:
+    return (
+        begin
+        >> step 1
+        ...
+        >> step last
+    )
+```
+
+- If the POST request contains no `user_input` the value provided to `initial_input_form` is called, in this case the function `initial_input_form_generator`
+
+```python
+def initial_input_form_generator(product_name: str) -> FormGenerator:
+    user_input = yield step_1_form(product_name)
+
+    ...
+
+    user_input_ports = yield step_2_form(product_name, ..)
+
+    return (
+        ... result from step ...
+    )
+```
+
+- The functionality provided by the workflow orchestrator decorator makes every yield statement pass if the required user input data is passed in or return a response to the client with a `510` status code and a payload containing the definition for the form to display in [JSON Schema 6 format](https://json-schema.org/draft-06/json-schema-release-notes)
+
+- An example of what the `step_1_form` function could look like.
+
+```python
+def step_1_form(product_name: str) -> type[FormPage]:
+    class SpeedChoice(Choice):
+        _10000 = ("10000", "10 Gbps")
+        _40000 = ("40000", "40 Gbps")
+        _100000 = ("100000", "100 Gbps")
+        _400000 = ("400000", "400 Gbps")
+
+    class CreateCoreLinkSpeedForm(FormPage):
+        model_config = ConfigDict(title=product_name)
+
+        organization: SurfnetOrganisation
+
+        label_core_link_settings: Label
+        divider_1: Divider
+
+        core_link_service_speed: SpeedChoice = SpeedChoice._400000
+        isis_metric: IsisMetric = 20
+
+    return CreateCoreLinkSpeedForm
+```
+
+The type specified for each property (eg divider_1: Divider) determines what `type` property it gets in the resulting JSON 6 Schema. There are a set number of property types that can be provided and that are automatically handled by the frontend by default. This is extendable.
+
+- The response for a POST call without user input to `<wfo-url>/processes/create_core_link` is
+
+## Example JSON6Schema response:
+
+```json
+{
+    "type": "FormNotCompleteError",
+    "detail": [not relevant]
+    "traceback": [not relevant]
+    "form": {
+        "$defs": {
+            "SpeedChoice": {
+                "enum": [
+                    "10000",
+                    "40000",
+                    "100000",
+                    "400000"
+                ],
+                "options": {
+                    "10000": "10 Gbps",
+                    "100000": "100 Gbps",
+                    "40000": "40 Gbps",
+                    "400000": "400 Gbps"
+                },
+                "title": "SpeedChoice",
+                "type": "string"
+            }
+        },
+        "additionalProperties": false,
+        "properties": {
+            "customer_id": {
+                "default": "c9b5e717-0b11-e511-80d0-005056956c1a",
+                "format": "customerId",
+                "title": "Customer Id",
+                "type": "string",
+                "uniforms": {
+                    "disabled": true,
+                    "value": "c9b5e717-0b11-e511-80d0-005056956c1a"
+                }
+            },
+            "label_corelink_settings": {
+                "anyOf": [
+                    {
+                        "type": "string"
+                    },
+                    {
+                        "type": "null"
+                    }
+                ],
+                "default": null,
+                "format": "label",
+                "title": "Label Corelink Settings",
+                "type": "string"
+            },
+            "divider": {
+                "anyOf": [
+                    {
+                        "type": "string"
+                    },
+                    {
+                        "type": "null"
+                    }
+                ],
+                "default": null,
+                "format": "divider",
+                "title": "Divider",
+                "type": "string"
+            },
+            "corelink_service_speed": {
+                "allOf": [
+                    {
+                        "$ref": "#/$defs/SpeedChoice"
+                    }
+                ],
+                "default": "400000"
+            },
+            "isis_metric": {
+                "default": 20,
+                "maximum": 16777215,
+                "minimum": 1,
+                "title": "Isis Metric",
+                "type": "integer"
+            }
+        },
+        "title": "SN8 Corelink",
+        "type": "object"
+    },
+    "title": [not relevant],
+    "status": 510
+}
+```
+
+[1]: https://www.npmjs.com/package/pydantic-forms
+[2]: https://json-schema.org/draft-06/json-schema-release-notes
+[3]: https://github.com/workfloworchestrator/orchestrator-ui-library/blob/main/packages/orchestrator-ui-components/src/components/WfoForms/AutoFieldLoader.tsx