diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..3bfc494 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,32 @@ +name: FlowKit CI, tests + +on: + push: + branches: [ main, master ] + pull_request: + branches: [ main, master ] + +jobs: + build-and-test: + runs-on: ubuntu-latest + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Install dependencies + run: | + sudo apt-get update + sudo apt-get install -y cmake g++-11 + + - name: Configure CMake + run: | + cmake -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++-11 + + - name: Build project + run: cmake --build build --config Release + + - name: Run tests suite + run: | + cd build + ctest --output-on-failure \ No newline at end of file diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..431a2db --- /dev/null +++ b/.gitignore @@ -0,0 +1,54 @@ +# Prerequisites +*.d + +# Compiled Object files +*.slo +*.lo +*.o +*.obj + +# Precompiled Headers +*.gch +*.pch + +# Libraries +*.la +*.a +*.lib +*.ln +*.dll +*.so +*.so.* +*.dylib + +# Executables +*.exe +*.out +*.app + +# CMake build directories +/build/ +/out/ +CMakeSettings.json +CMakeUserPresets.json + +# IDEs and Editors +.vs/ +.vscode/ +.idea/ +*.swp +*.swo +.clang-format + +# Debug symbols +*.pdb +*.dSYM/ + +# Environment / secrets +.env + +# Logs & temp files +*.log +*.tmp +*.temp +*.cache \ No newline at end of file diff --git a/CMakeLists.txt b/CMakeLists.txt new file mode 100644 index 0000000..7e8424e --- /dev/null +++ b/CMakeLists.txt @@ -0,0 +1,11 @@ +cmake_minimum_required(VERSION 3.20) + +project(FlowKit LANGUAGES CXX) + +set(CMAKE_CXX_STANDARD 20) +set(CMAKE_CXX_STANDARD_REQUIRED ON) +set(CMAKE_CXX_EXTENSIONS OFF) + +add_executable(FlowKit + flowkit.cpp +) \ No newline at end of file diff --git a/README.md b/README.md index 031949f..35cfb65 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,72 @@ # FlowKit -A modern C++20 workflow execution framework for building dependency-driven task pipelines. + +A modern, low-overhead C++20 workflow engine for building dependency-driven task pipelines. + +FlowKit provides a clean framework for defining tasks and connecting them into a Directed Acyclic Graph (DAG). The framework automatically handles dependency validation, cycle detection, and topological execution ordering, keeping orchestration completely separate from individual task logic. + +--- + +## Features +- **Fluent Graph API:** Easily declare tasks and define dependency structures. +- **Strict DAG Validation:** Automated runtime validation and cycle detection to prevent broken pipelines. +- **Topological Execution:** Resolves and runs tasks sequentially based on mathematical dependency order. +- **Modern C++ Core:** Employs RAII, clean type abstractions, and standard library graph algorithms. + +--- + +## Architecture & Roadmap +FlowKit is explicitly designed around separation of concerns to support clean software engineering metrics. It is currently implemented as a highly deterministic, single-threaded execution engine (MVP). The clean separation between the `WorkflowGraph` and the `Executor` ensures that a concurrent, thread-pool-based executor can be integrated in the future without modifying user-defined tasks. + +For a deep dive into design choices, see the links below: +- [Project Summary](docs/project-summary.md) +- [Requirements Specification](docs/requirements.md) +- [Architecture & Components](docs/architecture.md) +- [Development & Quality Goals](docs/development.md) + +--- + +## Tech Stack +- **Language:** C++20 +- **Build System:** CMake +- **Testing:** GoogleTest +- **Documentation Diagrams:** Mermaid + +--- + +## Example Usage **(TBD)** + +```cpp +#include +#include + +// Define a custom task by inheriting from the base Task class +class DownloadDataTask : public flowkit::Task { +public: + bool execute() override { + std::cout << "Downloading raw assets...\n"; + return true; // Return success + } +}; + +int main() { + flowkit::WorkflowGraph graph; + + // Register tasks + graph.add_task("download", std::make_unique()); + graph.add_task("process", []() { + std::cout << "Processing assets...\n"; + return true; + }); + + // Define dependencies (Process depends on Download) + graph.add_dependency("download", "process"); + + // Execute the workflow + flowkit::Executor executor; + bool success = executor.run(graph); + + return success ? 0 : 1; +} +``` + +**//TODO add context changing between nodes** \ No newline at end of file diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..ac29617 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,160 @@ +# Architecture + +## Overview + +FlowKit uses a modular architecture that separates workflow definition from workflow execution. The execution engine is currently single-threaded, while the design preserves the structural guarantees necessary to support a future multi-threaded execution model with minimal architectural changes. + +--- + +## High-Level Component Design + +```mermaid +flowchart TD + A[Client Application] -->|Defines Tasks & Edges| B[WorkflowGraph] + B -->|Passed to| C[Executor] + C -->|Step 1: Validate & Sort| C + C -->|Step 2: Run Sequentially| D[Task Nodes] +``` + +--- + +## Core Components + +### 1. WorkflowGraph + +**Responsibility** + +- Represents the workflow as a Directed Acyclic Graph (DAG). +- Stores task nodes. +- Maintains directed dependency edges between tasks. + +**Validation** + +Before execution, the graph validates its structure by performing cycle detection using either: +**(TBD:)** +- Depth-First Search (DFS) +- Kahn's Algorithm +- (Or other) + + Execution is only allowed if the graph is a valid DAG. + +--- + +### 2. Task (Interface) + +**Responsibility** + +Defines the behavioral contract for executable work units. + +**Characteristics** + +- Exposes a single execution method: + +```c++ +execute() +``` + +- Completely independent from graph mechanics. +- Tasks have no knowledge of neighboring nodes. +- Easily extensible for custom task implementations. + +--- + +### 3. Executor + +**Responsibility** + +Coordinates the complete workflow lifecycle. + +**Execution Steps** + +1. Accepts a validated `WorkflowGraph`. +2. Performs topological sorting. +3. Executes tasks sequentially. +4. Tracks execution state. +5. Reports execution failures when encountered. + +The executor owns execution logic while the graph remains a pure structural model. + +--- + +## System Execution Flow + +The following sequence diagram outlines the entire lifecycle of a workflow—from initialization by the client application to structural graph validation, topological sorting, and sequential task execution. + +### High-Level Flow +```mermaid +sequenceDiagram + autonumber + actor Client as User / Client Application + participant Graph as WorkflowGraph + participant Exec as Executor + participant Task as Task Nodes + + Client->>Graph: 1. Define tasks & edges + Client->>Exec: 2. Invoke run(graph) + + activate Exec + Exec->>Graph: 3. Perform structural validation + + alt Graph contains a cycle + Graph-->>Exec: Validation failed + Exec-->>Client: Return validation error + + else Graph is a valid DAG + Graph-->>Exec: Validation passed + + Exec->>Exec: 4. Perform topological sort + + loop For each sorted task + Exec->>Task: 5. execute() + activate Task + Task-->>Exec: 6. Success/Failure + deactivate Task + + alt Task failed + Exec->>Exec: Halt execution + Exec-->>Client: Return failure state + end + end + + Exec-->>Client: 7. Return success state + end + + deactivate Exec + + ``` +--- + +## Design Patterns + +| Pattern | Usage | Benefit | +|----------|-------|---------| +| Command | Task abstraction | Encapsulates executable actions behind a consistent interface without exposing framework internals. | +| Builder / Fluent API | Workflow construction | Provides a readable and expressive API for defining workflows and linking task dependencies. | + +--- + +## Architectural Decision: Shared Context + +The framework intentionally excludes a global, untyped shared context. + +### Rationale + +Global mutable state introduces several long-term issues: + +- Weakens DAG guarantees. +- Couples otherwise independent tasks. +- Creates data-race risks when introducing parallel execution. +- Makes workflows harder to reason about and test. + +### Preferred Approach + +Data should flow explicitly through: + +- Type-safe task inputs and outputs. +- Localized token or message passing between tasks. + +This approach keeps task dependencies explicit, improves maintainability, and allows the execution engine to evolve toward safe concurrent execution without redesigning the core architecture. + +[← Back to README](../README.md) \ No newline at end of file diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..cbb0794 --- /dev/null +++ b/docs/development.md @@ -0,0 +1,26 @@ +# Development + +## Development Methodology + +- Test-Driven Development (TDD) +- Feature branches +- Small, incremental commits + +## Tooling + +- C++20 +- CMake +- GoogleTest +- Mermaid +- GitHub Actions + + +## Quality Goals +- High unit test coverage +- SOLID principles +- RAII +- Separation of concerns +- Clean, maintainable APIs + + +[← Back to README](../README.md) \ No newline at end of file diff --git a/docs/project-summary.md b/docs/project-summary.md new file mode 100644 index 0000000..d81be82 --- /dev/null +++ b/docs/project-summary.md @@ -0,0 +1,13 @@ +# Project Summary + +This project is a lightweight, high-performance workflow execution engine written in modern C++20. It provides a generic framework for defining and executing tasks connected by dependencies in a Directed Acyclic Graph (DAG). Applications define their own task implementations, while FlowKit validates the graph structure, determines a valid execution order, and manages execution. + +The project was created as a portfolio piece to bridge the gap between traditional enterprise object-oriented design and modern, low-overhead C++ paradigms. It demonstrates software architecture, graph algorithms, and clean system design developed during my Bachelor's degree in Systems Development at Malmö University. + +### Architectural Philosophy +Rather than over-engineering the engine with heavy enterprise runtime abstractions, FlowKit focuses on C++ idiomatic practices: +- **Value Semantics & RAII:** Efficient memory management without relying on unnecessary heap allocation or pointer chasing. +- **Compile-Time Safety over Runtime Guessing:** Maximizing type safety and leveraging modern C++ structures to prevent runtime crashes. +- **Incremental Scaling:** Designed as a single-threaded Minimum Viable Product (MVP) with structural provisions to seamlessly transition to a multi-threaded execution pool. + +[← Back to README](../README.md) \ No newline at end of file diff --git a/docs/requirements.md b/docs/requirements.md new file mode 100644 index 0000000..b602067 --- /dev/null +++ b/docs/requirements.md @@ -0,0 +1,40 @@ +# Requirements + +## Project Goals +The primary goal of FlowKit is to showcase modern C++20 development practices through a clean, highly cohesive workflow engine. + +--- + +# Functional Requirements + +## Workflow Definition & Graph Management +The framework shall: +- Allow users to define tasks (as nodes) and construct them into workflows as Directed Acyclic Graphs (DAGs) using a clean programmatic API. +- Validate the graph structure prior to execution, explicitly detecting and rejecting circular dependencies. +- Track structural dependencies between discrete task nodes. + +## Workflow Execution +The framework shall: +- Perform a topological sort to resolve a valid sequential execution order for the graph. +- Execute tasks strictly according to dependency order, ensuring no task runs until its dependencies complete. +- Provide runtime state feedback (e.g., Success, Failure) upon completion of execution. + +## Task System +The framework shall: +- Provide a clear interface/base-class for custom executable tasks. +- Isolate task-specific execution logic entirely from graph management and orchestration logic. + +--- + +# Non-Functional Requirements + +## Architecture & Maintainability +- **High Cohesion:** Keep graph storage, validation, and execution mechanisms strictly separated. +- **Low Overhead:** Avoid unnecessary virtual method dispatch or heavy design pattern abstractions where standard library mechanisms (`std::function`, algorithms) suffice. +- **Modern C++20 Constraints:** Leverage C++20 features (e.g., standard library concepts, improved containers) using CMake as the build system. + +## Testability +- Accommodate comprehensive unit testing via GoogleTest. +- Built using Test-Driven Development (TDD) to ensure code reliability and structural flexibility for future performance optimizations. + +[← Back to README](../README.md) \ No newline at end of file diff --git a/flowkit.cpp b/flowkit.cpp new file mode 100644 index 0000000..33cc891 --- /dev/null +++ b/flowkit.cpp @@ -0,0 +1,6 @@ +#include + +int main() { + std::cout << "Hello, world!" << std::endl; + return 0; +} \ No newline at end of file