[FEATURE] Add SQL Datasource Plugin with Explorer Support

[ZPascal]

Description

This PR introduces a comprehensive SQL datasource plugin for Perses, enabling users to query and visualize data from PostgreSQL, MySQL, and MariaDB databases. The plugin includes a full-featured Explorer mode similar to the Prometheus plugin, providing both table and graph views for SQL query results.

Key Features

SQL Datasource Plugin

• Support for PostgreSQL, MySQL, and MariaDB
• TLS/SSL connection support with certificate validation
• Secret management integration for secure credential storage
• Connection string validation and error handling
• Database-specific driver selection

SQL Time Series Query Plugin

• Rich query editor with SQL syntax highlighting
• SQL macro support for dynamic time-based queries:
  • $__timeFilter(column) - Generates time range filter
  • $__timeFrom - Query start time
  • $__timeTo - Query end time
  • $__interval - Auto-calculated interval
  • $__interval_ms - Interval in milliseconds
• Backend proxy integration for secure query execution
• Full support for Perses variables in queries

SQL Explorer Plugin (New)

• Prometheus-style explorer with dual-tab interface
• Table View: Raw query results with column sorting
• Graph View: Time series visualization using existing Perses panels
• Multi-query editor with add/remove capabilities
• Panel preview functionality
• Seamless integration with dashboard workflow

Architecture

The plugin follows Perses plugin architecture best practices:

Frontend (TypeScript/React)

• Plugin module using Module Federation for dynamic loading
• CUE schema validation for datasource and query configurations
• React components for the datasource editor, query editor, and explorer UI
• Utility functions for SQL macro replacement and time handling

Backend Integration

• Backend proxy handles actual database connections
• Secure credential management via secrets API
• Support for sslmode=disable for development environments
• Detailed error messages for connection and query failures

Testing

Unit Tests (12 tests, all passing)

• SQL macro replacement logic (replace-sql-builtin-variables.test.ts)
• SQL client configuration (sql-client.test.ts)
• Datasource component rendering (SQLDatasource.test.tsx)

Test Data

• Docker Compose setup with PostgreSQL, MySQL, and MariaDB
• Sample databases with realistic time-series data
• Adminer UI for database inspection
• Makefile commands for easy database management

Documentation

• README.md: Comprehensive usage guide with examples
• CUE Schemas: Schema definitions for validation
• Code Comments: Apache License headers and JSDoc documentation

Screenshots

SQL Datasource Configuration

SQL Explorer - Table View

SQL Plugin View

Dashboard with SQL Panel

Checklist

• Pull request has a descriptive title and context useful to a reviewer.
• Pull request title follows the [FEATURE] <commit message> naming convention.
• All commits have DCO signoffs.
• Test the SQL plugin for MariaDB
• Test the SQL plugin for MySQL

UI Changes

• Changes that impact the UI include screenshots and/or screencasts of the relevant changes.
• Code follows the UI guidelines.
• E2E tests are stable and unlikely to be flaky.
  • Note: E2E tests not yet implemented. Unit tests cover business logic, and manual testing has been performed. E2E tests can be added in a follow-up PR if required.

Additional Notes

What's Included

✅ Production-ready features:

• 3 database drivers (PostgreSQL, MySQL, MariaDB)
• Full TLS/SSL support
• SQL macro system
• Explorer mode with table and graph views
• Secret management integration
• Comprehensive documentation
• Unit test coverage for core logic

Future Enhancements (Not Blocking)

• E2E tests for UI workflows
• Additional UI component tests
• ESLint configuration (TypeScript already catches most issues)
• Performance optimizations for large datasets

Testing Instructions

1. Start test databases:

   cd plugins/sql
   make db-up

2. Run unit tests:

   npm test

3. Build the plugin:

   npm run build

4. Manual testing:

   • Configure a SQL datasource in Perses
   • Use the query editor to write SQL queries with macros
   • Open the Explorer to build queries interactively
   • Create a dashboard panel using the SQL datasource

Database Compatibility

Database	Version	Status
PostgreSQL	15+	✅ Fully supported
MySQL	8.0+	✅ Fully supported
MariaDB	10.6+	✅ Fully supported

Migration Notes

This is a new plugin and does not require migration from existing configurations.

Related issues

Add PostgreSQL plugin support

Related PRs

Adapt the proxy output
Handle the MariaDB support for the backend proxy
Handle the MariaDB support for the shared CUE lib

[jgbernalp]

Related to: perses/perses#2930

[rickardsjp]

Thanks for the contribution. Please add the sql plugin to the top level (monorepo root) package.json workspaces list. Otherwise, it will not be included in e.g. the build and lint commands. Once you include it, please take a look at the linter issues, too.

[ZPascal]

Update: I'll further update the source code after this PR is merged.

[v-zhuravlev]

hi, any news on this? looks like perses/perses#3815 is merged now.

[ZPascal]

hi, any news on this? looks like perses/perses#3815 is merged now.

Yes, I'm already working on an update. The new version should be ready at the end of the week.

[ZPascal]

I think, the PR is ready for an other round of review. If someone confirms gives me the go from the design and architectural perspective, I'll start to test the plugin intensive based on my test plan.

Test Plan

# SQL Plugin — Test Plan

Covers manual and automated testing for the SQL datasource plugin across PostgreSQL, MySQL, and MariaDB. Organized by component, then by database where behavior diverges.

---

## Table of Contents

1. [Datasource Configuration](#1-datasource-configuration)
2. [Schema Validation (CUE)](#2-schema-validation-cue)
3. [Builtin Variable Replacement](#3-builtin-variable-replacement)
4. [Time Series Transformation](#4-time-series-transformation)
5. [Query Execution](#5-query-execution)
6. [SQL Explorer UI](#6-sql-explorer-ui)
7. [Database-Specific: PostgreSQL](#7-database-specific-postgresql)
8. [Database-Specific: MySQL](#8-database-specific-mysql)
9. [Database-Specific: MariaDB](#9-database-specific-mariadb)
10. [Cross-Cutting Concerns](#10-cross-cutting-concerns)

---

## 1. Datasource Configuration

### 1.1 Editor — Common Fields

| ID | Test | Expected |
|----|------|----------|
| DS-01 | Create a new datasource, leave **Host** empty, save | Validation error: host is required |
| DS-02 | Create a new datasource, leave **Database** empty, save | Validation error: database is required |
| DS-03 | Enter `localhost:5432` in Host, save, reopen | Value is preserved |
| DS-04 | Enter a database name, save, reopen | Value is preserved |
| DS-05 | Set **Secret** to a valid secret name, save, reopen | Value is preserved |
| DS-06 | Leave **Secret** blank | Datasource is accepted (secret is optional) |
| DS-07 | Change driver from PostgreSQL → MySQL, confirm that the PostgreSQL-specific fields (SSL Mode, Max Connections, Connect Timeout) disappear and MySQL fields (Timeout, Read Timeout, Write Timeout) appear | Driver switch updates the visible fields |
| DS-08 | Change driver from MySQL → MariaDB | Same MySQL-specific fields are shown |
| DS-09 | Change driver from MariaDB → PostgreSQL | PostgreSQL-specific fields reappear |
| DS-10 | Verify `createInitialOptions()` default: driver = `postgres`, host = `""`, database = `""` | Matches the code |

### 1.2 Editor — PostgreSQL-Specific Fields

| ID | Test | Expected |
|----|------|----------|
| DS-PG-01 | Set **SSL Mode** to `require`, save, reopen | `postgres.sslMode = "require"` is persisted |
| DS-PG-02 | Set **SSL Mode** to `verify-full`, save, reopen | `postgres.sslMode = "verify-full"` is persisted |
| DS-PG-03 | Set **Max Connections** to `50`, save, reopen | `postgres.maxConns = 50` is persisted |
| DS-PG-04 | Set **Connect Timeout** to `30s`, save, reopen | `postgres.connectTimeout = "30s"` is persisted |
| DS-PG-05 | Leave all PostgreSQL optional fields blank | Datasource is accepted |

### 1.3 Editor — MySQL / MariaDB-Specific Fields

| ID | Test | Expected |
|----|------|----------|
| DS-MY-01 | Set **Timeout** to `10s`, save, reopen | `mysql.timeout = "10s"` is persisted |
| DS-MY-02 | Set **Read Timeout** to `15s`, save, reopen | `mysql.readTimeout = "15s"` is persisted |
| DS-MY-03 | Set **Write Timeout** to `15s`, save, reopen | `mysql.writeTimeout = "15s"` is persisted |
| DS-MY-04 | Leave all MySQL optional fields blank | Datasource is accepted |

### 1.4 Client Creation

| ID | Test | Expected |
|----|------|----------|
| DS-CLI-01 | Call `createClient` with a valid `proxyUrl` | Returns `{ options: { datasourceUrl: proxyUrl } }` |
| DS-CLI-02 | Call `createClient` with `proxyUrl = undefined` | Throws `"No proxy URL available for SQL datasource"` |

---

## 2. Schema Validation (CUE)

### 2.1 Datasource Schema (`schemas/datasource/sql.cue`)

| ID | Test | Expected |
|----|------|----------|
| CUE-DS-01 | Validate a well-formed `SQLDatasource` object with `driver`, `host`, `database` set | Validation passes |
| CUE-DS-02 | Validate with `kind ≠ "SQLDatasource"` | Validation fails |
| CUE-DS-03 | Validate without required `spec` fields | Validation fails |
| CUE-DS-04 | Validate `#selector` with correct `_kind` | Resolves correctly |

### 2.2 Query Schema (`schemas/sql-time-series-query/sql-time-series-query.cue`)

| ID | Test | Expected |
|----|------|----------|
| CUE-QR-01 | Validate a minimal spec with only `query` set | Validation passes |
| CUE-QR-02 | Validate with all optional fields set (`minStep`, `timeColumn`, `labelColumns`, `valueColumns`, `timeFormat`) | Validation passes |
| CUE-QR-03 | Validate with `kind ≠ "SQLTimeSeriesQuery"` | Validation fails |
| CUE-QR-04 | Validate with `query` missing | Validation fails |
| CUE-QR-05 | Validate with `minStep` as a string instead of int | Validation fails |
| CUE-QR-06 | Validate with `labelColumns` containing non-string elements | Validation fails |
| CUE-QR-07 | Validate `ds.#selector` resolves to the `SQLDatasource` selector | Passes CUE unification |
| CUE-QR-08 | Validate with an extra unknown field in spec | Validation fails (`close({...})`) |

---

## 3. Builtin Variable Replacement

Tests for `replaceSQLBuiltinVariables()`. Time range: `start = 2024-01-23T10:00:00.000Z`, `end = 2024-01-23T11:00:00.000Z`, `intervalMs = 60000`.

### 3.1 ISO 8601 Format (default)

| ID | Variable | Input | Expected Output |
|----|----------|-------|----------------|
| VAR-01 | `$__timeFrom` | `WHERE t >= $__timeFrom` | `WHERE t >= '2024-01-23T10:00:00.000Z'` |
| VAR-02 | `${__timeFrom}` | `WHERE t >= ${__timeFrom}` | `WHERE t >= '2024-01-23T10:00:00.000Z'` |
| VAR-03 | `$__timeTo` | `WHERE t <= $__timeTo` | `WHERE t <= '2024-01-23T11:00:00.000Z'` |
| VAR-04 | `${__timeTo}` | `WHERE t <= ${__timeTo}` | `WHERE t <= '2024-01-23T11:00:00.000Z'` |
| VAR-05 | `$__interval` | `INTERVAL $__interval seconds` | `INTERVAL 60 seconds` |
| VAR-06 | `${__interval}` | `INTERVAL ${__interval} seconds` | `INTERVAL 60 seconds` |
| VAR-07 | `$__interval_ms` | `step = $__interval_ms` | `step = 60000` |
| VAR-08 | `${__interval_ms}` | `step = ${__interval_ms}` | `step = 60000` |
| VAR-09 | `$__timeFilter(col)` | `WHERE $__timeFilter(time)` | `WHERE time BETWEEN '2024-01-23T10:00:00.000Z' AND '2024-01-23T11:00:00.000Z'` |
| VAR-10 | `$__timeFilter` with schema-qualified column | `WHERE $__timeFilter(m.time)` | `WHERE m.time BETWEEN '...' AND '...'` |
| VAR-11 | `$__timeFilter` with quoted column | `WHERE $__timeFilter("Time")` | `WHERE "Time" BETWEEN '...' AND '...'` |
| VAR-12 | `$__timeFilter` with backtick column | `WHERE $__timeFilter(\`time\`)` | `WHERE \`time\` BETWEEN '...' AND '...'` |
| VAR-13 | Multiple variables in one query | `WHERE t >= $__timeFrom AND t <= $__timeTo` | Both replaced correctly |
| VAR-14 | No variables in query | `SELECT * FROM t` | Query unchanged |

### 3.2 Unix Format

| ID | Variable | Input | Expected Output |
|----|----------|-------|----------------|
| VAR-U-01 | `$__timeFrom` | `WHERE ts >= $__timeFrom` | `WHERE ts >= 1705903200` |
| VAR-U-02 | `$__timeTo` | `WHERE ts <= $__timeTo` | `WHERE ts <= 1705906800` |
| VAR-U-03 | `$__timeFilter(col)` | `WHERE $__timeFilter(ts)` | `WHERE ts BETWEEN 1705903200 AND 1705906800` (no quotes) |
| VAR-U-04 | `$__interval` | `step = $__interval` | `step = 60` (seconds) |
| VAR-U-05 | `$__interval_ms` | `step_ms = $__interval_ms` | `step_ms = 60000` |

### 3.3 Word Boundary Handling

| ID | Test | Expected |
|----|------|----------|
| VAR-WB-01 | `$__timeFromX` (no word boundary) | Not replaced — `\b` prevents partial match |
| VAR-WB-02 | `$__timeToX` | Not replaced |
| VAR-WB-03 | `$__intervalX` | Not replaced |
| VAR-WB-04 | `$__interval_msX` | Not replaced |
| VAR-WB-05 | `${__timeFrom}X` | Still replaced — brace syntax is exact |

---

## 4. Time Series Transformation

Tests for `transformToTimeSeries()` and `parseTimeValue()` inside `SQLTimeSeriesQuery.tsx`.

### 4.1 Time Column Detection

| ID | Test | Expected |
|----|------|----------|
| TS-TC-01 | Column named `time` | Detected as time column |
| TS-TC-02 | Column named `timestamp` | Detected as time column |
| TS-TC-03 | Column named `datetime` | Detected as time column |
| TS-TC-04 | Column named `created_at` | Detected as time column |
| TS-TC-05 | Column named `updated_at` | Detected as time column |
| TS-TC-06 | Column named `event_timestamp` | Detected (contains "timestamp") |
| TS-TC-07 | No column matches keywords, `timeColumn` not set in spec | Throws `"No time column found in query result"` |
| TS-TC-08 | `timeColumn` explicitly set in spec | Uses that column regardless of name |
| TS-TC-09 | `timeColumn` set to a column that doesn't exist in result | Produces series with `NaN` timestamps (rows where value is undefined) |

### 4.2 Value and Label Column Selection

| ID | Test | Expected |
|----|------|----------|
| TS-VC-01 | No `valueColumns` or `labelColumns` set | All non-time columns treated as value columns |
| TS-VC-02 | `valueColumns = ["cpu"]`, result has `cpu` and `mem` | Only `cpu` is treated as a value column |
| TS-VC-03 | `labelColumns = ["host"]` | `host` column is excluded from value columns |
| TS-VC-04 | `labelColumns = ["host"]`, `valueColumns = ["cpu"]` | `host` is label, `cpu` is value, other columns ignored |
| TS-VC-05 | `valueColumns` column is missing from result | No data points for that series (undefined values are filtered by `isNaN`) |

### 4.3 Series Grouping by Labels

| ID | Test | Expected |
|----|------|----------|
| TS-SG-01 | Two rows with same label combination | Combined into one series |
| TS-SG-02 | Two rows with different values for label column | Two distinct series |
| TS-SG-03 | `labelColumns = []`, multiple value columns | One series per value column |
| TS-SG-04 | `labelColumns = ["host", "region"]` | Series keyed on both labels; different combinations → different series |
| TS-SG-05 | Label value is `null` in row | Row skipped for that label key (null/undefined check) |
| TS-SG-06 | Label value is `0` or `false` | Included as string `"0"` / `"false"` |

### 4.4 Time Value Parsing (`parseTimeValue`)

| ID | Format | Input | Expected (ms) |
|----|--------|-------|---------------|
| TS-TV-01 | `iso8601` (default) | `"2024-01-23T10:00:00.000Z"` | `1705903200000` |
| TS-TV-02 | `iso8601` | `"2024-01-23 10:00:00"` | Parses as local time |
| TS-TV-03 | `unix` | `"1705903200"` | `1705903200000` |
| TS-TV-04 | `unix` | `1705903200` (number) | `1705903200000` |
| TS-TV-05 | `unix_ms` | `"1705903200000"` | `1705903200000` |
| TS-TV-06 | `unix_ms` | `1705903200000` (number) | `1705903200000` |
| TS-TV-07 | any | `Date` object | `date.getTime()` |
| TS-TV-08 | `iso8601` | Invalid date string `"not-a-date"` | `NaN` (propagates to series values) |

### 4.5 Non-Numeric Value Filtering

| ID | Test | Expected |
|----|------|----------|
| TS-NV-01 | Value column contains `"abc"` | Row skipped (`isNaN` filters it) |
| TS-NV-02 | Value column contains `null` | Row skipped |
| TS-NV-03 | Value column contains `undefined` | Row skipped |
| TS-NV-04 | Value column contains `"3.14"` | Parsed as `3.14`, included |
| TS-NV-05 | Value column contains `0` | Included as `0` |
| TS-NV-06 | Value column contains `""` | `parseFloat("")` → `NaN` → skipped |

### 4.6 Result Ordering

| ID | Test | Expected |
|----|------|----------|
| TS-OR-01 | Rows arrive out of time order | Series values sorted ascending by timestamp |
| TS-OR-02 | Empty rows array | Returns `[]` |
| TS-OR-03 | `rows` is `null` | Returns `[]` |

---

## 5. Query Execution

Tests for the `getTimeSeriesData` function in `SQLTimeSeriesQuery.tsx`.

### 5.1 Precondition Checks

| ID | Test | Expected |
|----|------|----------|
| QE-01 | `spec.query` is empty string | Returns `{ series: [] }` immediately (no fetch) |
| QE-02 | `getDatasourceClient` returns `null` | Throws `"No datasource configured for SQL query"` |
| QE-03 | Datasource client has no `datasourceUrl` | Throws `"No datasource URL available"` |

### 5.2 HTTP Interaction

| ID | Test | Expected |
|----|------|----------|
| QE-04 | Backend returns `200` with valid JSON | Parses response, runs transformation |
| QE-05 | Backend returns `400` | Throws `"SQL query failed: 400 ..."` with response body |
| QE-06 | Backend returns `500` | Throws `"SQL query failed: 500 ..."` |
| QE-07 | Backend returns `401` (unauthorized) | Throws with status |
| QE-08 | Network error (fetch rejects) | Error propagates to caller |
| QE-09 | `abortSignal` is triggered mid-flight | Fetch aborted; caller receives `AbortError` |
| QE-10 | Request body is `{ "query": "<processed SQL>" }` with `Content-Type: application/json` | Correct POST format |

### 5.3 Interval Calculation

| ID | Test | Expected |
|----|------|----------|
| QE-INT-01 | `spec.minStep = 30`, time range = 1 hour | `intervalMs = 30000` (minStep wins) |
| QE-INT-02 | `spec.minStep` unset, time range = 1000 seconds | `interval = max(1, floor(1000/1000)) = 1`, `intervalMs = 1000` |
| QE-INT-03 | `spec.minStep` unset, time range = 3600 seconds | `interval = max(1, floor(3600/1000)) = 3`, `intervalMs = 3000` |
| QE-INT-04 | `spec.minStep` unset, time range < 1000 seconds | `interval = 1`, `intervalMs = 1000` |

### 5.4 Variable Substitution Pipeline

| ID | Test | Expected |
|----|------|----------|
| QE-VS-01 | Query contains `$varName`, dashboard variable `varName = "production"` | Dashboard variable replaced before builtin macros |
| QE-VS-02 | Query contains `$__timeFrom` | Builtin macro replaced after dashboard variables |
| QE-VS-03 | `timeFormat = "unix"` in spec | Builtin macros produce unix timestamps; `timeFormat = 'unix'` passed to `replaceSQLBuiltinVariables` |
| QE-VS-04 | `timeFormat = "unix_ms"` in spec | Builtin macros use iso8601 (only `unix` triggers unix format in replacement); result parsing uses `unix_ms` |
| QE-VS-05 | `dependsOn(spec)` returns variables used in query | Correct variable names extracted from `$varName` patterns |

### 5.5 Return Value

| ID | Test | Expected |
|----|------|----------|
| QE-RV-01 | Successful query with data | Returns `{ series, timeRange, stepMs }` |
| QE-RV-02 | `timeRange` in return matches `context.timeRange` | Same reference |
| QE-RV-03 | `stepMs` equals `intervalMs` | Correct |

---

## 6. SQL Explorer UI

Manual tests for `SQLExplorer.tsx`.

### 6.1 Initial Render

| ID | Test | Expected |
|----|------|----------|
| UI-01 | Navigate to Explore, select SQL Explorer | Explorer loads without error |
| UI-02 | On first load with no query | `MultiQueryEditor` visible, Table/Graph tabs visible, no data shown |
| UI-03 | Active tab defaults to "Table" | Table tab selected |

### 6.2 Query Editor

| ID | Test | Expected |
|----|------|----------|
| UI-04 | Type a SQL query in the editor | Query is stored in local state |
| UI-05 | Click "Run" button | `setData` is called with current queries; panel re-fetches |
| UI-06 | Add a second query | `MultiQueryEditor` shows two query entries |
| UI-07 | `filteredQueryPlugins = ['SQLTimeSeriesQuery']` | Only SQL Time Series Query type shown in query type selector |

### 6.3 Tab Switching

| ID | Test | Expected |
|----|------|----------|
| UI-08 | Click "Graph" tab | `TimeSeriesPanel` renders; `DataTable` unmounts |
| UI-09 | Click "Table" tab | `DataTable` renders; `TimeSeriesPanel` unmounts |
| UI-10 | Switch tabs | Query editor remains visible and unchanged |
| UI-11 | Run query on Table tab, switch to Graph | Both panels pick up the same `queries` from context |

### 6.4 Data Display

| ID | Test | Expected |
|----|------|----------|
| UI-12 | Successful query on Table tab | Time series table renders with data rows |
| UI-13 | Successful query on Graph tab | Time series chart renders |
| UI-14 | Query returns no data | Empty state shown in panel |
| UI-15 | Query fails (e.g. bad SQL) | Error shown in panel |

### 6.5 URL State Persistence

| ID | Test | Expected |
|----|------|----------|
| UI-16 | Switch to Graph tab, refresh page | Graph tab is still active (state in URL via query params) |
| UI-17 | Run query, refresh page | Queries are preserved in URL state |

---

## 7. Database-Specific: PostgreSQL

Requires a running PostgreSQL instance (or TimescaleDB). Configure a `SQLDatasource` with `driver = postgres`.

### 7.1 Connection

| ID | Test | Expected |
|----|------|----------|
| PG-01 | Connect to PostgreSQL with valid credentials | Connection succeeds |
| PG-02 | Connect with `sslMode = require` to a TLS-enabled server | Connects over TLS |
| PG-03 | Connect with `sslMode = verify-full` with valid CA cert in secret | Verifies server certificate |
| PG-04 | Connect with `sslMode = verify-full` with wrong CA cert | Connection rejected with TLS error |
| PG-05 | Connect with wrong password | Connection error returned to UI |
| PG-06 | Connect with unreachable host | Timeout or connection refused error |
| PG-07 | `maxConns = 5`, run 10 concurrent queries | Queries queue; no panic |
| PG-08 | `connectTimeout = 1s`, host responds slowly | Timeout error after 1 second |

### 7.2 Basic Queries

| ID | Test | Expected |
|----|------|----------|
| PG-Q-01 | `SELECT now() AS time, 1 AS value` | Returns one row, time in ISO 8601, value `1` |
| PG-Q-02 | `SELECT EXTRACT(EPOCH FROM now())::bigint AS time, 1 AS value` with `timeFormat = unix` | Returns correct unix timestamp |
| PG-Q-03 | Multi-row time series from a table with `TIMESTAMP WITH TIME ZONE` column | Parsed correctly with iso8601 format |
| PG-Q-04 | Multi-row time series from a table with integer epoch column | Parsed correctly with unix format |
| PG-Q-05 | Query with `$__timeFilter(created_at)` macro | Macro replaced with `created_at BETWEEN 'ISO' AND 'ISO'` |
| PG-Q-06 | Query uses `$__interval` in `time_bucket($__interval * interval '1 second', time)` | Interval is injected as integer |
| PG-Q-07 | SQL syntax error in query | Error message from PostgreSQL returned to UI |
| PG-Q-08 | Query references non-existent table | Relation not found error returned to UI |
| PG-Q-09 | Query returns no rows | Empty series returned, no error |

### 7.3 Label Columns

| ID | Test | Expected |
|----|------|----------|
| PG-L-01 | `SELECT time, host, cpu_pct FROM metrics` with `labelColumns = ["host"]` | One series per distinct `host` value |
| PG-L-02 | Three hosts × one time range | Three distinct series in response |
| PG-L-03 | Labels appear correctly in chart legend | Each series labeled with host value |

### 7.4 TimescaleDB (if available)

| ID | Test | Expected |
|----|------|----------|
| PG-TS-01 | `SELECT time_bucket($__interval * interval '1 second', time), avg(value) FROM hypertable WHERE $__timeFilter(time) GROUP BY 1` | Produces time-bucketed series |
| PG-TS-02 | Time range spans multiple chunks | All data returned correctly |

---

## 8. Database-Specific: MySQL

Requires a running MySQL instance. Configure a `SQLDatasource` with `driver = mysql`.

### 8.1 Connection

| ID | Test | Expected |
|----|------|----------|
| MY-01 | Connect to MySQL with valid credentials | Connection succeeds |
| MY-02 | Connect with wrong password | Authentication error returned |
| MY-03 | Connect with unreachable host | Connection refused / timeout error |
| MY-04 | `timeout = 1s`, host responds slowly | Connection timeout after 1 second |
| MY-05 | `readTimeout = 1s`, query takes > 1 second | Read timeout error |
| MY-06 | `writeTimeout = 1s`, large write query takes > 1 second | Write timeout error |

### 8.2 Basic Queries

| ID | Test | Expected |
|----|------|----------|
| MY-Q-01 | `SELECT NOW() AS time, 1 AS value` | Returns one row with ISO datetime string |
| MY-Q-02 | `SELECT UNIX_TIMESTAMP(NOW()) AS time, 1 AS value` with `timeFormat = unix` | Returns unix timestamp |
| MY-Q-03 | Multi-row series from a table with `DATETIME` column | Parsed correctly |
| MY-Q-04 | Multi-row series from a table with `BIGINT` epoch column | Parsed correctly with unix/unix_ms format |
| MY-Q-05 | `WHERE $__timeFilter(created_at)` macro with iso8601 format | Replaced with ISO quoted BETWEEN clause |
| MY-Q-06 | `WHERE $__timeFilter(ts)` with unix format | Replaced with numeric BETWEEN clause |
| MY-Q-07 | SQL syntax error | MySQL error message returned to UI |
| MY-Q-08 | Query returns no rows | Empty series, no error |
| MY-Q-09 | Query with backtick-quoted column name in macro: `$__timeFilter(\`created_at\`)` | Backtick preserved in output |

### 8.3 Time Aggregation

| ID | Test | Expected |
|----|------|----------|
| MY-A-01 | `FROM_UNIXTIME(FLOOR(UNIX_TIMESTAMP(time) / $__interval) * $__interval) AS time` | Produces time-bucketed intervals |
| MY-A-02 | `$__interval` value matches panel time range resolution | Step is appropriate |

### 8.4 Label Columns

| ID | Test | Expected |
|----|------|----------|
| MY-L-01 | `SELECT time, region, value FROM metrics` with `labelColumns = ["region"]` | One series per distinct region |
| MY-L-02 | Multi-label: `labelColumns = ["region", "host"]` | Series keyed on both labels |

---

## 9. Database-Specific: MariaDB

MariaDB shares the MySQL driver. Tests focus on compatibility differences.

### 9.1 Connection

| ID | Test | Expected |
|----|------|----------|
| MA-01 | Connect to MariaDB with `driver = mariadb` and valid credentials | Connection succeeds |
| MA-02 | Same timeout fields as MySQL function correctly | Timeout fields behave as described in DS-MY-01–04 |

### 9.2 Basic Queries

| ID | Test | Expected |
|----|------|----------|
| MA-Q-01 | `SELECT NOW() AS time, 1 AS value` | Returns one row |
| MA-Q-02 | `WHERE $__timeFilter(time)` with iso8601 format | BETWEEN clause with quoted ISO strings |
| MA-Q-03 | MariaDB-specific: `UNIX_TIMESTAMP()` in projection with unix format | Correctly parsed |
| MA-Q-04 | Query against `TIMESTAMP` column | Parsed correctly |
| MA-Q-05 | MariaDB `SEQUENCE` or JSON columns as value | Non-numeric filtered by `isNaN` |

### 9.3 Behavioral Parity with MySQL

| ID | Test | Expected |
|----|------|----------|
| MA-P-01 | Run the same query on MySQL and MariaDB with same schema | Identical time series returned |
| MA-P-02 | Both share the same timeout configuration fields | Same fields shown in editor |
| MA-P-03 | `$__interval` substitution | Same integer value injected |

---

## 10. Cross-Cutting Concerns

### 10.1 Dashboard Variable Integration

| ID | Test | Expected |
|----|------|----------|
| CC-VAR-01 | Dashboard has a variable `$env = "prod"`, query uses `WHERE env = '$env'` | Variable resolved before sending to backend |
| CC-VAR-02 | `dependsOn` lists `env` | Panel re-fetches when `$env` changes |
| CC-VAR-03 | Query uses `$__timeFrom` AND `$env` | Both dashboard var and builtin replaced correctly |
| CC-VAR-04 | Variable not defined | `$varName` left as-is or resolved to empty per platform behavior |

### 10.2 Time Range Changes

| ID | Test | Expected |
|----|------|----------|
| CC-TR-01 | Change time range from 1h to 6h | Query re-executes with new `$__timeFrom` and `$__timeTo` |
| CC-TR-02 | Zoom in on chart | Time range narrows, query re-executes |
| CC-TR-03 | `$__interval` scales with time range | Larger range → larger interval value |

### 10.3 Abort and Cancellation

| ID | Test | Expected |
|----|------|----------|
| CC-AB-01 | Navigate away while query is executing | Fetch aborted (no dangling request) |
| CC-AB-02 | Trigger a new query while one is in-flight | Previous request aborted |

### 10.4 Security

| ID | Test | Expected |
|----|------|----------|
| CC-SEC-01 | Credentials are stored in a Secret, not in datasource spec | Secret name in spec, not raw password |
| CC-SEC-02 | Backend proxy enforces credential injection | Frontend never sees username/password |
| CC-SEC-03 | SQL query is user-supplied; backend must not allow schema modifications | Validate backend read-only enforcement if applicable |
| CC-SEC-04 | Dashboard variable injection into query | If variable contains SQL injection (`'; DROP TABLE--`), backend must sanitize or use parameterized queries |

### 10.5 Error Handling in UI

| ID | Test | Expected |
|----|------|----------|
| CC-ERR-01 | Backend returns `{ "error": "syntax error" }` | Error message displayed in panel, not generic failure |
| CC-ERR-02 | Backend unreachable | Network error displayed |
| CC-ERR-03 | Query result has no recognizable time column and `timeColumn` not set | Error `"No time column found"` shown in panel |
| CC-ERR-04 | Datasource deleted while panel is on screen | Error shown on next query execution |

### 10.6 Performance

| ID | Test | Expected |
|----|------|----------|
| CC-PERF-01 | Query returning 10,000 rows | Transformed without browser hang; chart renders |
| CC-PERF-02 | 5 panels with SQL queries on same dashboard | 5 concurrent requests; all succeed |
| CC-PERF-03 | Time range spanning 30 days with `minStep = 3600` | ~720 data points returned |

### 10.7 CUE Schema Enforcement (Integration)

| ID | Test | Expected |
|----|------|----------|
| CC-CUE-01 | Save a dashboard panel with `SQLTimeSeriesQuery` and extra unknown spec field | Backend rejects with validation error |
| CC-CUE-02 | Save a datasource with `kind = "SQLDatasource"` and valid spec | Accepted and stored |
| CC-CUE-03 | Save a datasource with wrong `kind` | Rejected by backend |

---

## Appendix A — Test Data Setup

### PostgreSQL

```sql
CREATE TABLE metrics (
  time TIMESTAMPTZ NOT NULL,
  host TEXT,
  region TEXT,
  cpu_pct DOUBLE PRECISION,
  mem_pct DOUBLE PRECISION
);

-- Generate 1 hour of data at 1-minute resolution for 2 hosts
INSERT INTO metrics
SELECT
  generate_series(
    NOW() - INTERVAL '1 hour',
    NOW(),
    INTERVAL '1 minute'
  ) AS time,
  host,
  region,
  random() * 100 AS cpu_pct,
  random() * 100 AS mem_pct
FROM (VALUES ('web-01', 'eu-west'), ('web-02', 'us-east')) AS t(host, region);

MySQL / MariaDB

CREATE TABLE metrics (
  time DATETIME NOT NULL,
  host VARCHAR(64),
  region VARCHAR(64),
  cpu_pct DOUBLE,
  mem_pct DOUBLE
);

-- Insert sample rows manually or via a stored procedure
INSERT INTO metrics VALUES
  ('2024-01-23 10:00:00', 'web-01', 'eu-west', 42.5, 61.0),
  ('2024-01-23 10:01:00', 'web-01', 'eu-west', 44.1, 62.3),
  ('2024-01-23 10:00:00', 'web-02', 'us-east', 30.0, 55.0),
  ('2024-01-23 10:01:00', 'web-02', 'us-east', 31.5, 56.1);

---

Appendix B — Sample Queries for Manual Testing

PostgreSQL — ISO 8601

SELECT time, host, cpu_pct
FROM metrics
WHERE $__timeFilter(time)
ORDER BY time;

PostgreSQL — Unix Timestamps

SELECT
  EXTRACT(EPOCH FROM time)::bigint AS time,
  host,
  cpu_pct
FROM metrics
WHERE time BETWEEN $__timeFrom AND $__timeTo
ORDER BY time;

MySQL / MariaDB — ISO 8601

SELECT time, host, cpu_pct
FROM metrics
WHERE $__timeFilter(time)
ORDER BY time;

MySQL — Unix Timestamps

SELECT UNIX_TIMESTAMP(time) AS time, host, cpu_pct
FROM metrics
WHERE time BETWEEN FROM_UNIXTIME($__timeFrom) AND FROM_UNIXTIME($__timeTo)
ORDER BY time;

Multi-label query (any DB)

SELECT time, host, region, cpu_pct, mem_pct
FROM metrics
WHERE $__timeFilter(time)
ORDER BY time;
-- Set labelColumns = ["host", "region"]
-- Set valueColumns = ["cpu_pct", "mem_pct"]

</details>

[Copilot]

Pull request overview

This PR adds a new SQL plugin package to the Perses plugins monorepo, providing a SQL datasource, a SQL time series query plugin, and an Explorer experience (table + graph) along with schemas, docs, and local test tooling.

Changes:

• Introduces the sql/ workspace package with Module Federation build config, plugin registrations, and public exports.
• Implements SQL datasource + SQL time series query logic (macro replacement + time series transformation) and an Explorer UI.
• Adds CUE schemas, unit tests, and a Docker Compose–based test environment (Postgres/Timescale, MySQL, MariaDB) plus docs and contributor guidance.

Reviewed changes

Copilot reviewed 41 out of 45 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
package.json	Adds sql to the root workspaces list so CI/build tooling discovers the new plugin.
package-lock.json	Registers the new workspace link for @perses-dev/sql-plugin.
sql/package.json	Defines the SQL plugin package, scripts, and Perses plugin registrations (Datasource/Query/Explore).
sql/package-lock.json	Captures workspace dependency lock state for the new package.
sql/tsconfig.json	Adds TypeScript config for the new package (base extends + compiler options).
sql/tsconfig.build.json	Build-only TS config for declaration emit and test/story exclusions.
sql/rsbuild.config.ts	Module Federation build configuration for exposing the SQL plugins.
sql/jest.config.ts	Jest configuration integrating shared config and per-package setup file.
sql/src/setup-tests.ts	Jest test setup (jest-dom + echarts mock).
sql/src/env.d.ts	Adds rsbuild type references.
sql/src/index.ts	Barrel export for datasources/queries/explore and getPluginModule.
sql/src/index-federation.ts	Federation entrypoint that bootstraps the MF runtime.
sql/src/bootstrap.tsx	Dev bootstrap page for running the plugin in isolation.
sql/src/getPluginModule.ts	Reads plugin module metadata/spec from package.json.
sql/src/datasources/index.ts	Datasource barrel export.
sql/src/datasources/sql-datasource/index.ts	SQL datasource plugin barrel exports.
sql/src/datasources/sql-datasource/sql-datasource-types.ts	Type definitions for SQL datasource/proxy spec and client.
sql/src/datasources/sql-datasource/SQLDatasource.tsx	Datasource plugin implementation (proxy-based client).
sql/src/datasources/sql-datasource/SQLDatasourceEditor.tsx	UI editor for configuring SQL proxy parameters.
sql/src/datasources/sql-datasource/SQLDatasource.test.tsx	Unit tests for datasource plugin client creation and defaults.
sql/src/model/index.ts	Model barrel export.
sql/src/model/replace-sql-builtin-variables.ts	SQL macro replacement implementation ($__timeFrom, $__timeTo, $__interval, $__timeFilter, etc.).
sql/src/model/replace-sql-builtin-variables.test.ts	Unit tests for SQL macro replacement behavior.
sql/src/queries/index.ts	Query barrel export.
sql/src/queries/sql-time-series-query/index.ts	SQL time series query barrel exports.
sql/src/queries/sql-time-series-query/sql-time-series-query-types.ts	Type definitions for SQL time series query spec/definition.
sql/src/queries/sql-time-series-query/SQLTimeSeriesQueryEditor.tsx	Query editor UI for SQL time series queries.
sql/src/queries/sql-time-series-query/SQLTimeSeriesQuery.tsx	Query execution via proxy + result-to-timeseries transformation.
sql/src/explore/index.ts	Explorer barrel export.
sql/src/explore/sql-explorer-types.ts	Explorer spec/props typings.
sql/src/explore/SQLExplorer.tsx	Explorer UI with Table/Graph tabs using MultiQueryEditor + Panel previews.
sql/schemas/datasource/sql.cue	CUE schema for SQL datasource validation (shared base spec).
sql/schemas/sql-time-series-query/sql-time-series-query.cue	CUE schema for SQL time series query validation.
sql/cue.mod/module.cue	CUE module definition and dependency on shared CUE lib.
sql/README.md	End-user documentation for setup, macros, explorer usage, and examples.
sql/CONTRIBUTING.md	Contributor/developer workflow and architecture notes for the SQL plugin.
sql/Makefile	Local dev/test helpers (db-up/down, verify, tests, etc.).
sql/test-setup.sh	Quick-start script to bring up DBs and run basic checks/tests.
sql/docker-compose.test.yaml	Local test environment with Postgres/Timescale, MySQL, MariaDB, and Adminer.
sql/test-data/postgres/init.sql	Postgres/Timescale schema + indexes + continuous aggregate setup.
sql/test-data/postgres/sample-data.sql	Postgres sample data generator + continuous aggregate refresh.
sql/test-data/mysql/init.sql	MySQL schema + indexes.
sql/test-data/mysql/sample-data.sql	MySQL sample data generator via stored procedure.
sql/test-data/mariadb/init.sql	MariaDB schema + indexes.
sql/test-data/mariadb/sample-data.sql	MariaDB sample data script (currently empty file).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.