diff --git a/.github/workflows/gh-pages.yml b/.github/workflows/gh-pages.yml index d175d4f..1fa0cc5 100644 --- a/.github/workflows/gh-pages.yml +++ b/.github/workflows/gh-pages.yml @@ -27,4 +27,4 @@ jobs: run: ./setup_subprojects.sh - name: Build and deploy - run: uv run --no-sync mkdocs gh-deploy --force + run: uv run --no-sync mkdocs gh-deploy --force --strict diff --git a/.github/workflows/run-mkdocs-build.yml b/.github/workflows/run-mkdocs-build.yml new file mode 100644 index 0000000..63b379b --- /dev/null +++ b/.github/workflows/run-mkdocs-build.yml @@ -0,0 +1,29 @@ +name: Test mkdocs build + +on: + pull_request: + branches: ["main"] + +env: + UV_LOCKED: true # Assert that the `uv.lock` will remain unchanged + +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + + - name: Install uv and set the python version + uses: astral-sh/setup-uv@v8.2.0 + with: + # It is considered best practice to pin to a specific uv version. + version: "0.11.26" + + - name: Install site dependencies + run: uv sync + + - name: Set up subprojects + run: ./setup_subprojects.sh + + - name: Test mkdocs build with --strict + run: DISABLE_MKDOCS_2_WARNING=true uv run --no-sync mkdocs build --strict diff --git a/docs/workshops/example-orchestrator/bootstrap.md b/docs/workshops/example-orchestrator/bootstrap.md index c35f001..3a0fe3e 100644 --- a/docs/workshops/example-orchestrator/bootstrap.md +++ b/docs/workshops/example-orchestrator/bootstrap.md @@ -1,24 +1,101 @@ # Bootstrapping the environment +This section describes how you can run the orchestrator-core, orchestrator-ui, and netbox with Docker Compose. We have this all +setup in our docker-compose.yml file so that you don't have to think about how to start the applications required for this workshop! -Here is how you can run the orchestrator-core, orchestrator-ui, and netbox with Docker Compose. We have this all -setup in our docker-compose.yml file so that you don't have to think about how to start the applications required for this workshop! If you want to read more about how to manually install the Workflow Orchestrator, please refer to [the beginner workshop here](../beginner/debian.md). The following Docker images are used in this workshop: +The following Docker images are used in this workshop: * [orchestrator-core](https://github.com/workfloworchestrator/orchestrator-core/pkgs/container/orchestrator-core): The workflow orchestrator step engine. -* [orchestrator-ui](https://github.com/workfloworchestrator/orchestrator-ui/pkgs/container/orchestrator-ui): The - GUI for the orchestrator-core. -* [netbox](https://docs.netbox.dev/en/stable/): A free IPAM and SoT system. +* [example-orchestrator-ui](https://github.com/workfloworchestrator/example-orchestrator-ui/pkgs/container/example-orchestrator-ui): An example GUI implementation for the orchestrator-core. +* [netbox](https://docs.netbox.dev/en/stable/): An open source IPAM and NSoT system. * [postgres](https://hub.docker.com/_/postgres): The PostgreSQL object-relational database system. -* [redis](https://redis.io/): An open source, in-memory data store used by netbox -* Optional: [containerlab](https://containerlab.dev/): A free network topology simulator that uses containerized - network operating systems. +* [redis](https://redis.io/): An open source in-memory data store. +* [containerlab](https://containerlab.dev/): A free network topology simulator that uses containerized + Network Operating Systems. -!!! danger - **To run the workshop with container lab, the host architecture must be x86_64 with virtualization - enabled** +## Prerequisites + +The following software is required on your machine to follow this workshop: + +- [docker-compose](https://docs.docker.com/compose/install/) +- [containerlab](https://containerlab.dev/install/) + + +### macOS on Apple Silicon + +!!! note + This only applies to **ARM / Apple Silicon (M-series) Macs**. On Linux and + Intel machines a standard Docker install works — skip to + [Step 1](#step-1-cloning-the-repo). + +**Docker runtime (colima + virtiofs).** +On Apple Silicon the containerlab SR Linux nodes need a Docker VM that mounts host +paths with **virtiofs** and has enough memory. The default Docker Desktop VM and +colima's legacy `sshfs` mount both break the nodes in two ways: + +- **`virtiofs`, not `sshfs`.** SR Linux generates a TLS keypair at boot and + `chown`s it. `sshfs` forbids `chown` even as root, so the keygen fails, + `sr_linux_mgr` crash-loops, and no management server (JSON-RPC / gNMI / gRPC) + ever comes online. +- **≥ 8 GB VM memory.** Each SR Linux node reserves ~2 GB; on a smaller VM the + kernel OOM-killer keeps killing `sr_mgmt_server` and the nodes never settle. + +Use [colima](https://github.com/abiosoft/colima) as the Docker runtime: + +```shell +brew install colima docker + +# vm-type vz + virtiofs is the Apple Silicon path; 8 GB / 4 CPUs runs the +# compose stack and the 3-node lab together. +colima start --vm-type vz --mount-type virtiofs --memory 8 --cpu 4 +``` + +Confirm the VM before continuing: + +```shell +colima status | grep mountType # -> mountType: virtiofs +colima ssh -- free -h # Mem total should be ~8 GB +``` + +!!! danger "Do not run Docker Desktop at the same time as colima" + Both register a `docker` CLI context and fight over the `/var/run/docker.sock` + symlink and host ports, leaving you with a broken, non-deterministic Docker + setup. Quit Docker Desktop (and disable "Start at login") before using colima. + +!!! warning "`mountType` / `vmType` are fixed when the VM is created" + The `--mount-type` / `--vm-type` flags are silently ignored on an existing VM. + If your colima VM was created with `sshfs`, recreate it — this destroys the + VM's containers, images and volumes: + + ```shell + colima delete && colima start --vm-type vz --mount-type virtiofs --memory 8 --cpu 4 + ``` + + Memory and CPU, by contrast, *can* be changed on a plain restart: + `colima stop && colima start --memory 8 --cpu 4`. + +Once the runtime is up, run the `containerlab` commands in this guide from the +maintainers' container — there is no native `containerlab` binary on Apple Silicon: + +!!! info + For MacOS ARM there is no `containerlab` executable. + You can run the `containerlab` commands in this guide from within this container provided by the maintainers. + ```shell + docker run --rm -it --privileged \ + --network host \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -v /var/run/netns:/var/run/netns \ + -v /var/lib/docker/containers:/var/lib/docker/containers \ + --pid="host" \ + -v $(pwd):$(pwd) \ + -w $(pwd) \ + ghcr.io/srl-labs/clab bash + ``` ## Step 1 - Cloning the repo + The fist step is to clone the Example orchestrator repository using: + ``` git clone https://github.com/workfloworchestrator/example-orchestrator.git ``` @@ -30,128 +107,140 @@ At this point, you have a functional environment to start play with. This includ * An example containerlab topology based on Nokia SRlinux. * Some examples of Ansible playbooks -The directory structure looks like this: +The directory contains at least these files and directories: + ``` -~/example-orchestrator# ls -la -total 144 -drwxr-xr-x 14 root root 4096 May 10 10:29 . -drwx------ 16 root root 4096 May 15 06:42 .. --rw-r--r-- 1 root root 147 May 8 10:35 .env.example -drwxr-xr-x 8 root root 4096 May 15 10:15 .git --rw-r--r-- 1 root root 82 May 8 10:56 .gitignore -drwxr-xr-x 2 root root 4096 May 8 10:35 .pictures --rw-r--r-- 1 root root 50000 May 8 10:56 README.md --rw-r--r-- 1 root root 884 May 8 10:35 alembic.ini -drwxr-xr-x 4 root root 4096 May 9 19:42 ansible <<<< Ansible playbooks -drwxr-xr-x 3 root root 4096 May 15 10:16 clab <<<< Containerlab topology -drwxr-xr-x 9 root root 4096 May 8 10:56 docker <<<< Docker folder for config etc.. --rw-r--r-- 1 root root 4469 May 10 10:29 docker-compose.yml <<<< The docker compoose file to spin up orchestrator stack --rw-r--r-- 1 root root 920 May 8 10:35 main.py -drwxr-xr-x 3 root root 4096 May 8 10:35 migrations -drwxr-xr-x 5 root root 4096 May 8 10:56 products --rw-r--r-- 1 root root 100 May 8 10:35 pyproject.toml --rw-r--r-- 1 root root 41 May 8 10:35 requirements.txt -drwxr-xr-x 2 root root 4096 May 10 10:46 services --rw-r--r-- 1 root root 942 May 8 10:35 settings.py -drwxr-xr-x 2 root root 4096 May 8 10:35 templates -drwxr-xr-x 2 root root 4096 May 8 10:35 translations -drwxr-xr-x 2 root root 4096 May 8 10:35 utils -drwxr-xr-x 7 root root 4096 May 8 10:56 workflows +$ ls -1 +alembic.ini +ansible +clab +db +docker +docker-compose.yml +main.py +migrations +products +README.md +services +settings.py +templates +translations +utils +workflows +wsgi.py ``` -## Step 2 - Editing the environment -Before starting up the stacks, we have to check the file: +## Step 2 - Preparing the environment -``` -docker/orchestrator-ui/orchestrator-ui.env -``` +**Check port availability** -and adjust the LISTENING_IP value: +The environment requires several ports to be free. +Use the command matching your OS to check if any are in use. + +On Linux you can use `netstat` or `ss`: ``` -ENVIRONMENT_NAME="Example Orchestrator" +# net-tools +netstat -tulnp | grep -E ':80|:3000|:4000|:5432|:5678|:8000|:8001|:8080' -ORCHESTRATOR_API_HOST=http://<$LISTENING_IP>:8080 -ORCHESTRATOR_API_PATH=/api -ORCHESTRATOR_GRAPHQL_HOST=http://<$LISTENING_IP>:8080 -ORCHESTRATOR_GRAPHQL_PATH=/api/graphql -NEXTAUTH_SECRET=ToDo +# iproute2 +ss -tulnp | grep -E ':80|:3000|:4000|:5432|:5678|:8000|:8001|:8080' ``` -* if you are running the orchestrator locally, for example on your laptop, use "localhost" -* if you are running the orchestrator on a remote machine, use the IP of that machine, for example 1.2.3.4 -## Step 3 - Starting the environment -Once you edited the file, you can start the docker environment with: +On macOS: -### Remote lab environment with LSO and Containerlab ``` -COMPOSE_PROFILES=lso docker compose up +lsof -nP -iTCP -sTCP:LISTEN | grep -E ':80 |:3000 |:4000 |:5432 |:5678 |:8000 |:8001 |:8080 ' ``` -This will also enable LSO, so network devices will be actually configured. If you don't want this, just use: -### Local environment +No output means that all required ports available. +Otherwise, track down and stop the corresponding processes before continuing. -The environment requires several ports to be free. -Use either command below to check if any are in use. No output means they're available: +**Setup variables** +Load the required environment variables: + +```shell +source .env.workshop ``` -netstat -tulnp | grep -E ':80|:3000|:4000|:5432|:5678|:8000|:8001|:8080' -ss -tulnp|grep -E ':80|:3000|:4000|:5432|:5678|:8000|:8001|:8080' -``` -Start the containers: +**Pull and build images** + +Just in case you have any of the docker images cached on your machine we'll explicitly pull and build them. +```shell +docker compose pull +docker compose build ``` + +## Step 3 - Starting the environment + +Now we can start the containers: + +```shell docker compose up -d ``` -and you should be able to view the -applications here: - -If you are using your laptop: +You should be able to view the applications here: 1. Orchestrator ui: [Frontend: http://localhost:3000](http://localhost:3000) -2. Orchestrator backend: [REST api: http://localhost:8080/api/redoc](http://localhost:8080/api/redoc) and - [Graphql API: http://localbost:8080/api/graphql](http://localbost:8080/api/graphql) +2. Orchestrator backend: [REST api: http://localhost:8080/api/redoc](http://localhost:8080/api/redoc) and + [Graphql API: http://localbost:8080/api/graphql](http://localhost:8080/api/graphql) 3. Netbox (admin|admin): [Netbox: http://localhost:8000](http://localhost:8000) -If you are using a remote machine: +!!! note + Take your time to familiarise with the applications and make sure they are working correctly. -1. Orchestrator ui: Frontend: http://<$IP_ADDRESS_OF_THE_MACHINE>:3000 -2. Orchestrator backend: REST api: http://<$IP_ADDRESS_OF_THE_MACHINE>:8080/api/redoc - Graphql API: http://<$IP_ADDRESS_OF_THE_MACHINE>:8080/api/graphql -3. Netbox (admin|admin): Netbox: http://<$IP_ADDRESS_OF_THE_MACHINE>:8000 + If anything is wrong, inspect the results of these commands: -!!! note - Take your time to familiarise with the applications and make sure they are working correctly. You can then - continue with the following steps. + ```shell + # Check the status of all services + docker compose ps + + # Follow the logs of all services + # It can be helpful to keep this command running in a separate terminal + docker compose logs -tf -n 5 + ``` + +## Step 4 - Containerlab -## Optional: Step 4 Containerlab Now that we have our orchestrator stack running, we can spin up the containerlab topology: ``` cd clab containerlab deploy ``` + At the end of this process we can use `containerlab inspect` to check the status of our topology: ``` -~/example-orchestrator/clab# containerlab inspect -INFO[0000] Parsing & checking topology file: srlinux01.clab.yaml -+---+-----------------------+--------------+-----------------------+------+---------+----------------+--------------+ -| # | Name | Container ID | Image | Kind | State | IPv4 Address | IPv6 Address | -+---+-----------------------+--------------+-----------------------+------+---------+----------------+--------------+ -| 1 | clab-orch-demo-ams-pe | 46ddee7df745 | ghcr.io/nokia/srlinux | srl | running | 172.22.0.11/16 | N/A | -| 2 | clab-orch-demo-lon-pe | fe3f5d6eb35e | ghcr.io/nokia/srlinux | srl | running | 172.22.0.10/16 | N/A | -| 3 | clab-orch-demo-par-p | 4831968e075c | ghcr.io/nokia/srlinux | srl | running | 172.22.0.9/16 | N/A | -+---+-----------------------+--------------+-----------------------+------+---------+----------------+--------------+ +╭───────────────────────┬──────────────────────────────┬─────────┬────────────────╮ +│ Name │ Kind/Image │ State │ IPv4/6 Address │ +├───────────────────────┼──────────────────────────────┼─────────┼────────────────┤ +│ clab-orch-demo-ams-pe │ srl │ running │ 172.18.0.12 │ +│ │ ghcr.io/nokia/srlinux:25.7.1 │ │ N/A │ +├───────────────────────┼──────────────────────────────┼─────────┼────────────────┤ +│ clab-orch-demo-lon-pe │ srl │ running │ 172.18.0.11 │ +│ │ ghcr.io/nokia/srlinux:25.7.1 │ │ N/A │ +├───────────────────────┼──────────────────────────────┼─────────┼────────────────┤ +│ clab-orch-demo-par-p │ srl │ running │ 172.18.0.13 │ +│ │ ghcr.io/nokia/srlinux:25.7.1 │ │ N/A │ +╰───────────────────────┴──────────────────────────────┴─────────┴────────────────╯ ``` +!!! info + You will see a few deprecation notices about type `ixrd2` being deprecated, you can ignore this. + And with the command: + ``` containerlab graph ``` -we can have a nice rendering of the topology served on port 50080 (for example https://localhost:50080). + +This will serve a nice rendering of the topology on port [50080](http://localhost:50080). + +!!! info + When using containerlab's docker image, this does not work. The topology we are going to use is something like this one: @@ -170,10 +259,18 @@ We can start feeding initial data into the environment and run some workflows! ### Resetting Your Environment -To reset the active state of your environment back to scratch, simply use docker compose to delete volumes, like so: +To completely reset your environment and start from scratch, follow these steps. + +If you had containerlab deployed, destroy the deployed nodes: + +```shell +containerlab destroy +``` + +Stop the docker compose stack and remove all volumes: -```bash -jlpicard@ncc-1701-d:~$ docker compose down -v +```shell +docker compose down -v ``` -You can then restart the containers as described above. +You can then restart the environment as described above. diff --git a/docs/workshops/example-orchestrator/create-your-own.md b/docs/workshops/example-orchestrator/create-your-own.md index 3ed76ac..72e8367 100644 --- a/docs/workshops/example-orchestrator/create-your-own.md +++ b/docs/workshops/example-orchestrator/create-your-own.md @@ -5,5 +5,6 @@ Orchestrator provides the user. In this scenario you will create a product that L2VPN product, but constrained to two interfaces. In other words a L2 Point-to-Point circuit. ## L2 Point-to-Point model + {{ external_markdown('https://raw.githubusercontent.com/workfloworchestrator/orchestrator-core/main/docs/architecture/product_modelling/l2_point_to_point.md', '') }} diff --git a/docs/workshops/example-orchestrator/domain-models.md b/docs/workshops/example-orchestrator/domain-models.md index 2a61743..48e6650 100644 --- a/docs/workshops/example-orchestrator/domain-models.md +++ b/docs/workshops/example-orchestrator/domain-models.md @@ -2,7 +2,7 @@ ## Introduction -First read the [Architecture; TL;DR](../../architecture/tldr.md) section of the orchestrator core documentation to get an overview of the +First read the [Architecture; TL;DR](../../orchestrator-core/architecture/tldr.md) section of the orchestrator core documentation to get an overview of the concepts that will be covered. ## Products diff --git a/docs/workshops/example-orchestrator/execute-workflows.md b/docs/workshops/example-orchestrator/execute-workflows.md index 42a7f84..4f451d4 100644 --- a/docs/workshops/example-orchestrator/execute-workflows.md +++ b/docs/workshops/example-orchestrator/execute-workflows.md @@ -1,41 +1,52 @@ The topology in the previous section will be used in the workshop as an example of what a network could look like. -Obviously it is possible to create any "physical" topology you like and build the "logical" topology that matches -using th Workflow Orchestrator. - +Obviously it is possible to create any "physical" topology you like and build the matching "logical" topology +using the Workflow Orchestrator. ### Putting initial data in place -The first thing we are going to do is populate Netbox with some initial data such as Manifacturers and Device types as well as some networks allocated for: - -* Loopback addressing -* Core links addressing +The first thing we will do is populate Netbox with some initial data such as Manufacturers and Device types, +as well as networks allocated for Loopback and Core-links addressing. -This is done using the task "Netbox Bootstrap" under the tasks submenu. +To do this, navigate to the **Tasks** page, click **New task** and then select **Netbox Bootstrap**. +An empty form will show where you can click **Start task**. -Once the workflow has successfully ran, we can login into netbox (admin/admin) and check the situation: we should see some vendors and some network device models. -In the IPAM section we are going to reserve the first address of the loopback newtworks since certain network devices dont like "network addresses" to be used as loopback addresses. +Once the workflow has successfully run, we can login into netbox (admin/admin) and check the situation: within the **Devices** menu you can see that the **Device Types** and **Manufacturers** now contain some definitions. -* from the IPv4 prefix 10.0.127.0/24 we allocate the address 10.0.127.0 marking it with a description "RESERVED" -* from the IPv6 prefix fc00:0:0:127::/64 we allocate the address fc00:0:0:127:: marking it with a description "RESERVED" +In the **IPAM >> Prefixes** menu we are going to reserve the first address of the loopback networks since certain network devices dont like "network addresses" to be used as loopback addresses. -We do this from IPAM >> Prefixes. +* From the IPv4 prefix `10.0.127.0/24` we allocate the address `10.0.127.0/32` with status `Reserved` and description `RESERVED` +* From the IPv6 prefix `fc00:0:0:127::/64` we allocate the address `fc00:0:0:127::/128` with status `Reserved` and description `RESERVED` ### Deploying the nodes -Now we should be able to deploy our routers using the `create node` workflow. This is going to be a new subscription of the product node - specifically a nokia node - and we will have to fill an initial form. +Now we should be able to deploy our routers using the `create node` workflow in the Orchestrator. +To do this, click the **New subscription** button and select **node Nokia**. +This is going to create a new subscription of the "node Nokia" product. +We will have to fill an initial form. -!!! note - Make sure that the node name is the same as the node name in containerlab (clab-orch-demo-ams-pe/clab-orch-demo-lon-pe/clab-orch-demo-par-p) +* **Customer**: SURF +* **Node type**: Select the first option +* **Node role**: Provider +* **Site**: Amsterdam +* **Node status**: active +* **Node name**: Same as the node name in containerlab: `clab-orch-demo-ams-pe`, `clab-orch-demo-lon-pe` and `clab-orch-demo-par-p` + +Click **Next** and, review the summary page, and click **Next** again to start the workflow. + +Once the workflow has successfully ran we can login into the node and take a look at the config: -Once the workflow has successfully ran, we can login into the node just configured and take a look at the config: ``` -ssh clab-orch-demo-ams-pe -l admin ##PWD: NokiaSrl1! +ssh clab-orch-demo-ams-pe -l admin +# Enter password: NokiaSrl1! ``` -We can do the same in Netbox, and we will notice that these nodes have no interfaces, to create them in netbox, we can use a workflow. Specifically the "Update Node Interfaces" workflow that will seed the necessary data into Netbox, so we can re-use it later. -We can practice this deployin all the 3 nodes in the topology. +!!! note + If you deployed containerlab through the docker image, run the ssh commands from the same container. + +We can practice this deploying all the 3 nodes in the topology. ### Deploying core links + Once we have 2 nodes configured, we should be able to deploy a core link between them using the "create core link 10G" workflow. You can login into the router and check the status of ISIS using: diff --git a/docs/workshops/example-orchestrator/generator.md b/docs/workshops/example-orchestrator/generator.md index bf9ef0e..c48509d 100644 --- a/docs/workshops/example-orchestrator/generator.md +++ b/docs/workshops/example-orchestrator/generator.md @@ -6,17 +6,16 @@ At the end of these steps the developer will have all the necessary configuratio the workflow and start developing on implementing the business logic. ### Step 1 - Create the product configuration file + Open the Example Orchestrator directory and list the templates directory. It should look similar to this: ```bash -~/Documents/SURF/projects/example-orchestrator ❯ ls -l templates total 32 -rw-r--r-- 1 boers001 staff 2687 Mar 7 11:28 core_link.yaml -rw-r--r-- 1 boers001 staff 2052 Mar 7 11:28 l2vpn.yaml -rw-r--r-- 1 boers001 staff 2575 Mar 7 11:28 node.yaml -rw-r--r-- 1 boers001 staff 2444 Mar 7 11:28 port.yaml -~/Documents/SURF/projects/example-orchestrator ``` This directory houses all the configuration of the initial products that the example orchestrator provides. It is a @@ -24,10 +23,13 @@ starting point for developing new products. In this workshop we will create a ne L2-Point-to-Point model and workflows by configuring it with this yaml file. ### Step 2 - Configure the YAML + Create a new file in the template directory called `l2-p2p.yaml` + ```bash touch templates/l2-p2p.yaml ``` + This file will contain the Initial product type configuration. Please create a yaml configuration reflecting the product model as described on the [previous page](create-your-own.md). The goal is to configure the generator to reuse as many of the product blocks already existing in the orchestrator as possible. @@ -35,7 +37,7 @@ reuse as many of the product blocks already existing in the orchestrator as poss !!! tip "Inspiration" Take a look at the `l2vpn.yaml` model for inspiration. As you can see this file has been configured in a certain way to reflect the configuration of the product. For more in depth documentation take a look at the [reference - doc](../../reference-docs/cli.md#generate). + doc](../../orchestrator-core/reference-docs/cli.md#generate). !!! danger "What can I do when I encounter errors?" If you get stuck just remove all generated files, edit the yaml and try again. @@ -97,18 +99,22 @@ reuse as many of the product blocks already existing in the orchestrator as poss ``` ### Step 3 - Run the generator functions -To help generate the correct file exec into the running container and run the generator: + +To help generate the correct file exec into the running container and activate the Python venv: ```bash -docker exec -it example-orchestrator-orchestrator-1 /bin/bash +docker compose exec -it orchestrator bash +source .venv/bin/activate ``` #### Product Blocks + Run the command to generate the domain models product blocks: ```bash python main.py generate product-blocks -cf templates/l2-p2p.yaml --no-dryrun ``` + The `--no-dryrun` option will immediately write the files to the `products/product_blocks` folder and create: `l2_p2p_sap.py` and `l2_p2p_virtual_circuit.py`. This file contains the product block configuration for the l2-p2p product and defines the strict hierarchy of virtual circuit and saps. @@ -125,9 +131,15 @@ domain model, fixed inputs and imported the correct product blocks to be used in #### Workflows + Now generate the workflows. This command will always create 4 sets of workflows `create`, `modify`, `terminate` and `validate`. These can be implemeted as the users sees fit. +!!! warning + There is a bug in `generate workflows` which makes it overwrite the contents of workflows/shared.py. + Make sure to backup the contents, or restore it afterwards. + Bug is tracked in https://github.com/workfloworchestrator/orchestrator-core/issues/1757. + Run the command: ```bash python main.py generate workflows -cf templates/l2-p2p.yaml --no-dryrun --force diff --git a/docs/workshops/example-orchestrator/overview.md b/docs/workshops/example-orchestrator/overview.md index 0ad5395..91eeb1c 100644 --- a/docs/workshops/example-orchestrator/overview.md +++ b/docs/workshops/example-orchestrator/overview.md @@ -14,15 +14,15 @@ Workflow Orchestrator. ## Topics -* **Installation** +* **Installation** Detailed instructions are given on how to prepare your environment and install the orchestrator and GUI using docker compose. -* **Start applications** +* **Start applications** Outline how to start the Workflow Orchestrator backend and GUI using docker compose. -* **Bootstrapping the applications and familiarisation** +* **Bootstrapping the applications and familiarisation** Through a simple network node and network circuit scenario, a set of products is created showing how domain models are defined. - * **Domain models** + * **Domain models** Explains the benefits of the use of domain models and shows how the hierarchy of products, product blocks, fixed inputs and resource types are used to create product subscriptions for customers. -* **L2 Point-to-Point product modelling and workflow** +* **L2 Point-to-Point product modelling and workflow** For the L2 Point-to-Point product, we will make the CREATE workflow by using the product generator. The use of input forms is explained as part of defining the create workflow. By using this method you should be able to quickly get up to speed and start coding quickly @@ -39,28 +39,29 @@ This workshop uses the following folder layout: │ ├── product_blocks │ ├── product_types │ └── services -│ └── +│ └── ├── services -│ └── +│ └── ├── templates ├── translations ├── utils -└── workflows -├── +├── workflows +│ └── └── tasks -37 directories, 99 files ``` ## Workshop software architecture + The workshop combines a number of opensource software components that can provision a simulated network running in containerlab. The following diagram shows the logical components of the application and how the data flows. In reality there are a number of extra services like Postgres and Redis that store the application data of the Orchestrator, Netbox and LSO. ### Software used in the workshop -* **The orchestrator**: This includes the UI and python backend that will run all workflows. All data is persisted in a Postgres database. Redis is used for caching and syncronisation purposes. + +* **The orchestrator**: This includes the UI and python backend that will run all workflows. All data is persisted in a Postgres database. Redis is event broadcasting and synchronisation purposes. * **[Netbox](https://docs.netbox.dev/en/stable/)**: Netbox is the source of truth for this network topology. It contains all resources that are known - in the topology: Interfaces, Nodes, IP addresses etc. The Orchestrator will configure Netbox but also retrieve resource from it. + in the topology: Interfaces, Nodes, IP addresses etc. The Orchestrator will configure Netbox but also retrieve resources from it. * **[LSO](https://github.com/workfloworchestrator/lso)**: The Network Resource Manager (NRM) of this topology. This software is an API abstraction on top of ansible that integrates well with the orchestrator. It is responsible for running ansible jobs to provision the topology. * **[Container Lab](https://containerlab.dev/)**: This software will manage the (virtual) network topology running the Network Operating System @@ -69,9 +70,9 @@ the Orchestrator, Netbox and LSO. ![Software topology](../images/Software-topology.drawio.png) ## Workshop topology -Assuming you have installed the example orchestartor with containerlab integration enabled. We need to build the -workshop topology that can be used to actually see packets flow. The workflows that you will run in the following -steps will do the following: + +Once you have installed the example orchestrator with containerlab integration enabled (explained in the next section), +we will build the workshop topology that can be used to actually see packets flow. The workflows that you will run in the following steps will do the following: * Seed Netbox * Provision two PE nodes diff --git a/docs/workshops/example-orchestrator/scenario.md b/docs/workshops/example-orchestrator/scenario.md index 84749ef..aa65d28 100644 --- a/docs/workshops/example-orchestrator/scenario.md +++ b/docs/workshops/example-orchestrator/scenario.md @@ -6,11 +6,12 @@ The products will be just complex enough to show the basic capabilities of produ resource types and workflows in the workflow orchestrator. We will cover nesting product blocks and products together. ## Product hiearchy example + In the diagram below you can see how all products and product blocks relate to each other. The example orchestrator has implemented the following example products and corresponding workflows that can be used to build a basic network topology and customer facing services: -{{ external_markdown('https://raw.githubusercontent.com/workfloworchestrator/example-orchestrator/master/README.md', '### Implemented products') }} +{{ external_markdown('https://raw.githubusercontent.com/workfloworchestrator/example-orchestrator/main/README.md', '### Implemented products') }} ## Customers @@ -95,5 +96,5 @@ flowchart !!! Hint - Take some time to explore the module described in above. It shows how the product modelling is done in Python. - Once you are familiar with the code. Continue with the workshop + Take some time to explore the module described above. It shows how the product modelling is done in Python. + Once you are familiar with the code, continue with the workshop diff --git a/docs/workshops/example-orchestrator/workflow-introduction.md b/docs/workshops/example-orchestrator/workflow-introduction.md index fa1cc74..1e15262 100644 --- a/docs/workshops/example-orchestrator/workflow-introduction.md +++ b/docs/workshops/example-orchestrator/workflow-introduction.md @@ -1,4 +1,4 @@ -> This document assumes you are already familiar with the key safeguards and potential pitfalls outlined in the [Architecture of workflows](../../architecture/application/workflow.md) +> This document assumes you are already familiar with the key safeguards and potential pitfalls outlined in the [Architecture of workflows](../../orchestrator-core/architecture/application/workflow.md) ### Continuing the Workshop