Skip to content

chore: import sap document ai#96

Merged
Schmarvinius merged 97 commits into
mainfrom
chore/import-sap-document-ai
Jul 6, 2026
Merged

chore: import sap document ai#96
Schmarvinius merged 97 commits into
mainfrom
chore/import-sap-document-ai

Conversation

@Schmarvinius

@Schmarvinius Schmarvinius commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Import SAP Document AI CAP Java Plugin (cds-feature-sap-document-ai)

New Features

✨ Introduces the cds-feature-sap-document-ai module — a new CAP Java plugin (Alpha) that integrates SAP Document AI (Document Information Extraction / DIE) into CDS applications.

The plugin exposes a CDS event-based API for submitting documents, manages asynchronous polling against the DIE service via the CDS persistent outbox, and delivers extraction results back to the application through a DocumentExtractionResult outbound event. It supports graceful degraded operation when no DIE service binding is present.

Key capabilities:

  • DocumentExtraction inbound event triggers document submission to DIE
  • DocumentExtractionResult outbound event delivers extraction results to consumer handlers
  • Outbox-driven, self-scheduling polling (PENDING → SUBMITTED → RUNNING → DONE/FAILED state machine)
  • Optimistic locking for concurrent job updates
  • Configurable poll interval via cds.document-ai.polling.interval-seconds
  • Structured exception hierarchy: Connectivity, Request, Processing

Changes

  • cds-feature-sap-document-ai/pom.xml: New Maven module definition with CDS build, code generation (POJOs), JaCoCo coverage enforcement (85%), Spotless, and PMD tooling.
  • cds-feature-sap-document-ai/src/main/resources/cds/...: CDS model files defining DocumentAiService (events), ExtractionJob (persistence entity), and index.cds entry point.
  • DocumentAiServiceConfiguration.java: Plugin entry point via CdsRuntimeConfiguration/ServiceLoader; resolves DIE binding, builds OAuth2 destination via SAP Cloud SDK, wires all handlers.
  • DocumentSubmissionHandler.java: Handles DocumentExtraction events on any ApplicationService; delegates to ExtractionService.
  • ExtractionPollingHandler.java: Outbox-registered handler polling DIE for active jobs; self-reschedules and emits DocumentExtractionResult on completion.
  • ExtractionServiceImpl.java: Core orchestrator — creates jobs, submits via processing service, enforces state machine, schedules polling.
  • DefaultDocumentAiClient.java: HTTP client using Apache HttpClient 5 + SAP Cloud SDK for DIE REST API (submitDocument, getJobResult).
  • StatusTransitionValidator.java, ExtractionStatus.java: Stateless state machine enforcement.
  • service/exceptions/: Typed exceptions (DocumentAiException, ConcurrentJobUpdateException, IllegalStatusTransitionException).
  • src/test/java/.../: Comprehensive unit tests for all production classes using Mockito and AssertJ.
  • integration-tests/spring/: New Spring Boot integration tests covering the full extraction lifecycle, parallel processing, error resilience, event emission, and invalid transition rejection.
  • samples/document-ai-bookshop/: Complete runnable Bookshop sample application demonstrating plugin integration with the CAP Attachments plugin and a Fiori UI.
  • pom.xml (root): Added cds-feature-sap-document-ai as a module and to the dependency management BOM.
  • cds-feature-sap-document-ai/README.md, docs/architecture.md: Full user-facing documentation covering quick start, integration guide, configuration, architecture, and quality tooling.
  • 🔄 Regenerate and Update Summary
PR Bot Information

Version: 1.26.14

  • Output Template: Default Template
  • Event Trigger: pull_request.opened
  • LLM: anthropic--claude-4.6-sonnet
  • Correlation ID: 5d40f3a0-76d7-11f1-86df-2056d2e8c59b
  • File Content Strategy: Full file content
  • Summary Prompt: Default Prompt

Samyuktha Prabhu and others added 30 commits May 5, 2026 10:34
Initialize CAP bookshop sample application
Setup Repo for the SAP Document AI Plugin
remove extra files

rename handler

tiny fixes
remove extra files

rename handler

tiny fixes
#2750 - Integrate Attachment Plugin into Doc AI Plugin
…-setup

# Conflicts:
#	sap-document-ai/pom.xml
#	sap-document-ai/src/main/java/com/sap/cds/handlers/AttachmentEventHandler.java
#	sap-document-ai/src/main/resources/META-INF/services/com.sap.cds.services.runtime.CdsRuntimeConfiguration
#	sap-document-ai/src/test/java/com/sap/cds/AttachmentEventHandlerTest.java
#2753 - Implement attachment-triggered Document AI orchestration
remove extra files

rename handler

tiny fixes

setup orchestrator

initial Extraction Orchestrator setup

async process :)

add tests

bounded thread pool

# Conflicts:
#	sap-document-ai/pom.xml
#	sap-document-ai/src/main/java/com/sap/cds/handlers/AttachmentEventHandler.java
#	sap-document-ai/src/main/resources/META-INF/services/com.sap.cds.services.runtime.CdsRuntimeConfiguration
#	sap-document-ai/src/test/java/com/sap/cds/AttachmentEventHandlerTest.java
#2761 -  Add Document AI processing service
move the file one level lower

attempt to fix failing test

attempt 2 to fix failing test

attempt 3 to fix failing test

attempt 4 to fix failing test

attempt 5 to fix failing workflow

test

clean up

configure pmd, jacoco and mvn spotless

fix version

tests
#2782 - Introduce a CI Pipeline to automatically validate pull requests
…5b733ff3e6d19125371'

git-subtree-dir: cds-feature-sap-document-ai
git-subtree-mainline: de59837
git-subtree-split: 7bb95f5
- Hoist the plugin Maven module up: sap-document-ai/{pom.xml,src} -> cds-feature-sap-document-ai/{pom.xml,src}
- Move imported bookshop sample to samples/document-ai-bookshop/
- Move imported integration test Java sources into integration-tests/spring/
  under com.sap.cds.feature.documentai.integrationtest (Application, config
  and remaining files are handled in later commits).

Structural-only change; no pom or Java content is modified in this commit.
- Delete the imported LICENSE (Apache 2.0 already present at repo root, only
  whitespace-different).
- Delete the imported .github/ workflows and .gitignore; both are superseded
  by the repo-level equivalents in cds-ai.
- Delete .hyperspace/pull_request_bot.json (byte-identical to the copy at
  repo root).
- Delete the leftover cds-feature-sap-document-ai/integration-tests/
  directory: its Java test sources were moved into integration-tests/spring
  in the previous commit; the remaining Application.java, application.yaml,
  logback-test.xml, pom.xml, package.json will be reintegrated into the
  spring integration-tests module in commit 5.
- Set parent to cds-ai-root, drop local version, groupId, dependencyManagement
  and spotless/pmd/spotbugs/compiler configuration (inherit from root).
- Depend on cds-services-api, cds-services-utils, cds4j-core, cds-services-impl
  and connectivity-apache-httpclient5 (versions managed by root parent or
  the SAP Cloud SDK BOM).
- Add hermetic CDS build: cds.install-node and cds.npm-ci executions plus a
  local package.json pinning @sap/cds-dk 9.9.1 (matches cds-feature-ai-core).
- Keep the plugin's existing cds.build layout (src/main/resources/cds/
  com.sap.cds/sap-document-ai) via a workingDirectory-based invocation of
  the cds-maven-plugin cds goal, mirroring cds-feature-ai-core.
- Add cds.generate for basePackage com.sap.cds.feature.documentai.generated.cds4j.
- Add a module spotbugs-exclusion-filter.xml so the spotbugs check inherited
  from the root pluginManagement can resolve its excludeFilterFile.
- Wire module-level jacoco (prepare-agent, report) matching sibling modules.
…rences

- Rewrite the copyright header in every imported Java file (plugin sources,
  moved bookshop handlers and moved integration tests):
  * update contributor line from "cds-feature-sap-document-ai contributors"
    to "cds-ai contributors" so files match the license header enforced by
    the root spotless configuration;
  * fix indentation from "* " / "*/" to " * " / " */" to match the
    canonical style used by the rest of the repository.
- Update the DocumentExtractionHandler error message in the bookshop sample
  to reference the new artifactId cds-feature-sap-document-ai.
…actor

Root pom:
- Register cds-feature-sap-document-ai as a Maven module.
- Add a dependencyManagement entry for cds-feature-sap-document-ai using
  ${revision} so downstream modules pick up the CI-friendly version.

Integration tests (integration-tests/spring):
- Add cds-feature-sap-document-ai (managed) as a test-scope reachable
  dependency so tests can reach the plugin's generated cds4j classes and
  runtime handlers.
- Add org.awaitility:awaitility 4.2.2 (test) used by the moved tests.
- Add "using from 'com.sap.cds/sap-document-ai';" to test-service.cds so
  the CDS build for the integration-tests reactor includes the plugin
  model and the H2 deploy provisions the ExtractionJob table.
- Rename moved *ITest.java files to *Test.java so surefire picks them up
  (the spring integration-tests module runs everything through surefire
  and does not have failsafe wired), and update internal class references.

Samples (samples/document-ai-bookshop):
- Swap plugin coordinates from sap-document-ai:1.0-SNAPSHOT to
  com.sap.cds:cds-feature-sap-document-ai using a version property.
- Pin cds-feature-sap-document-ai.version to 0.0.1-alpha (matches root
  ${revision}) and align cds.services.version to the repo default 4.9.0.
…build blockers

- Set explicit version on cds4j-core dependency (com.sap.cds:cds4j-core is
  not managed by cds-services-bom); use ${cds.services.version} to keep it
  aligned with the rest of the CDS stack.
- Commit package-lock.json so the cds-maven-plugin cds.npm-ci execution can
  run non-interactively (mirrors cds-feature-ai-core and cds-feature-
  recommendations).
- Extend the module-local spotbugs-exclusion-filter.xml to also skip the
  DM_DEFAULT_ENCODING pattern for classes matching *Test / *TestBase. The
  imported unit tests use byte fixtures that intentionally rely on the
  platform default encoding; production code is not affected.
@Schmarvinius Schmarvinius requested a review from a team as a code owner July 3, 2026 12:04

@hyperspace-insights hyperspace-insights Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR introduces a well-structured CAP Java plugin for SAP Document AI with good test coverage, but there are several concrete bugs to address: a NullPointerException risk in mapDieStatus, a documentAiJobId field that is never populated on the outbound DocumentExtractionResult event (causing consumers to always receive null for that field), a URI.resolve misuse that can silently produce wrong URLs when the destination has a non-root base path, and a redundant variable assignment in DefaultDocumentAiProcessingService.

PR Bot Information

Version: 1.26.14

  • Event Trigger: pull_request.opened
  • Correlation ID: 5d40f3a0-76d7-11f1-86df-2056d2e8c59b
  • File Content Strategy: Full file content
  • LLM: anthropic--claude-4.6-sonnet

Ran `mvn spotless:apply` at the repo root. Reformatting affects:

- The moved doc-ai integration tests under integration-tests/spring
  (five files): minor line-break normalization introduced during the
  cds-feature-sap-document-ai import.
- One pre-existing formatting nit in
  cds-feature-ai-core/.../MockAICoreApiHandler.java that the previous
  runs of spotless never caught because the spotless check phase is
  gated by `${spotless.check.skip}` (default true) in the root pom.

No functional changes; unit and integration tests still pass.
The integration-tests/spring module now compile-depends on
com.sap.cds:cds-feature-sap-document-ai:${revision}. The composite
action for the Integration Tests job first installs a small subset of
plugins to ~/.m2 with a -pl list and then runs `mvn clean verify`
against integration-tests/pom.xml. The new plugin was missing from
that list, so dependency resolution failed with:

  Could not find artifact com.sap.cds:cds-feature-sap-document-ai:
    jar:0.0.1-alpha in central

Add cds-feature-sap-document-ai to the -pl list so the plugin gets
built and installed locally before the integration-tests reactor
runs. -am still resolves the transitive dependency graph correctly.
…e cds install

Reproducing the failing CI 'Local MTX Tests' job locally showed that a
fresh 'npm install' in integration-tests/mtx-local/ ends up with three
copies of @sap/cds:

  - mtx-local/node_modules/@sap/cds          @ 10.0.3 (hoisted)
  - mtx-local/node_modules/@sap/cds-dk/node_modules/@sap/cds  @ 9.9.1
  - mtx-local/mtx/sidecar/node_modules/@sap/cds               @ 9.9.2

The sidecar boots with 'cds-serve --profile development' and refuses to
start with:

  ERROR: @sap/cds was loaded from different locations

Root cause:

  * @sap/cds 10.0.3 was published to npm (major bump from 9.x).
  * @sap/cds-mtxs 3.9.5 (latest 3.x) declares peer '@sap/cds: >=9' -
    the loose lower bound. With no top-level pin, npm satisfies that
    peer with the newest matching version and hoists 10.0.3.
  * The sidecar itself declares '@sap/cds: ^9' (strict 9.x) so it
    keeps a nested 9.x copy. Same for @sap/cds-dk, whose own dep
    range '^8.3 || ^9' can not accept 10.x.

Result: three copies of @sap/cds, cds-serve aborts.

Fix: declare '@sap/cds: ^9' at the workspace root as a devDependency.
This gives npm a top-level constraint that pins the hoisted @sap/cds
to 9.x, which simultaneously satisfies cds-mtxs's '>=9', cds-dk's
'^8.3 || ^9' and the sidecar's '^9' - all resolve to the same hoisted
9.x copy. Verified locally with a full 'mvn clean verify -pl
integration-tests/mtx-local/srv -am -P mtx-integration-tests' run:
8 MTX tests pass, sidecar starts cleanly.

The 'CI - PR' job for main was not affected earlier only because @sap/
cds 10 was published between PR 95's run (2026-07-02) and PR 96's run
(2026-07-03).

@lisajulia lisajulia left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this PR!
I have some suggestions, open for discussion :)

  • cds-starter-ai: that should include cds-feature-sap-document-ai as well once it supports multi-tenancy - so we:
    • either add it now, despite the missing multi-tenancy support (I'd favor this)
    • or we document that this is missing now for this reason
  • We could also add a sample in cds-starter-ai in the starter module; then this sample showcases all ai-related plugins
  • Please move the sample located at samples/document-ai-bookshop to inside of cds-feature-sap-document-ai and move the sample at samples/bookshop to inside of cds-feature-recommendations; then add a comment in cds-feature-ai-core: sample is in cds-feature-recommendations.
  • right now, coverage-report/pom.xml accumulates the coverage of cds-feature-ai-core and cds-feature-recommendations, correct? Please add document ai here as well.
  • I noticed that the minimum coverages are 0.0 in coverage-report/pom.xml (e.g. https://github.com/cap-java/cds-ai/blob/main/coverage-report/pom.xml#L146) - that's not intended, no?
  • Update the outer README.md to reflect the addition of sap-document-ai: here I'd suggest to move the samples section and local development section into the respective modules

@Schmarvinius

Copy link
Copy Markdown
Contributor Author

Thanks for the review! Going through each point:

  1. cds-starter-ai + doc-ai: doc-ai isn't fully multi-tenant-ready yet, so I'd rather not include it in the starter until it is. An app pulling cds-starter-ai in a multi-tenant scenario would gain surprises. I'll add a TODO comment in cds-starter-ai/pom.xml and a short note in the root README's Starter section so the intent is tracked.
    2 & 3. Sample layout: I agree with the underlying goal (fewer, more discoverable samples) but disagree on the "per-plugin" location. CAP's idiomatic pattern is one CAP app hosting multiple services that's exactly what integration-tests/spring already does. I'd like to merge samples/document-ai-bookshop INTO samples/bookshop, so the single sample demonstrates AI Core + recommendations + doc-ai side by side. Each plugin's README will link to it. This is less maintenance (one CAP app, one package.json, one pom.xml) and matches CAP best practices. Happy to discuss further if you'd prefer per-module samples.
  2. Coverage report includes doc-ai: Good catch! Adding it to the dependencies, copy-classes resources, and the jacoco-merge fileset.
  3. 0.00: Intentional! SonarQube enforces coverage in the pipeline, so the JaCoCo check goals are effectively no-ops. I'll remove the check-ai-core and check-recommendations executions to make this explicit. (This also means point 4 doesn't need a matching check exec for doc-ai just adding it to merge/report.)
  4. Root README: Will add the doc-ai row to the Plugins table + the starter note from point 1. I'll keep short pointer sections for Samples and Local Development at the root and put the detailed instructions inside the respective module READMEs.

Document the intentional omission of cds-feature-sap-document-ai from
the cds-starter-ai aggregator. The plugin is not fully multi-tenant
ready yet, so bundling it into the starter would surprise multi-tenant
CAP applications.

- Add a TODO comment inside cds-starter-ai/pom.xml so contributors
  see the intent at the point that would need to change.
- Add a short note under the Starter section of the root README so
  consumers know to add cds-feature-sap-document-ai as an explicit
  dependency if they need it today.
…shop

The two sample apps at samples/bookshop and samples/document-ai-bookshop
both called "bookshop" and had overlapping filenames (admin-service.cds,
cat-service.cds, package.json...). Consolidate them into a single CAP
Java sample that demonstrates all AI plugins in the repository side by
side, matching CAP idioms (one app, multiple services).

Changes to samples/bookshop:
- Add cds-feature-sap-document-ai and cds-feature-attachments to the
  dependencyManagement of the parent pom and to the srv module deps.
- Add "com.sap.cds/sap-document-ai" and "com.sap.cds/cds-feature-
  attachments" under "cds.requires" in package.json so the Maven-based
  CDS models are picked up during the cds build.
- Extend .cdsrc.json cdsc.moduleLookupDirectories with "target/cds/"
  so cds finds the models that cds-maven-plugin's resolve goal extracts
  there.
- Add srv/attachments.cds extending Books with an Attachments composition,
  reused from the previous document-ai-bookshop.
- Extend AdminService.Books with a bound extractDocumentData() action.
- Add DocumentExtractionHandler and DocumentExtractionResultHandler under
  customer.bookshop.handlers.
- Merge doc-ai-specific configuration into srv/src/main/resources/
  application.yaml (persistent outbox scheduler, polling interval, logging).

Delete samples/document-ai-bookshop/ entirely.

Update the plugin READMEs (cds-feature-ai-core, cds-feature-recommendations,
cds-feature-sap-document-ai) to point at samples/bookshop as the runnable
sample.
…port

Extend the coverage-report aggregator so the new plugin appears in the
merged JaCoCo report and its coverage feeds into SonarQube:

- Add cds-feature-sap-document-ai as a compile-scope dependency.
- Add the module's target/classes to the copy-classes resource execution
  (jacoco:check requires the classes on the aggregator's classpath).
- Add cds-feature-sap-document-ai/target/jacoco.exec to the jacoco:merge
  fileset that produces the aggregated exec data file.

The coverage check executions (check-ai-core, check-recommendations)
are kept unchanged here; they are removed in a follow-up commit.
The check-ai-core and check-recommendations executions used
<minimum>0.00</minimum> for all counters (INSTRUCTION, BRANCH,
COMPLEXITY), meaning they always passed and provided no build-time
gate. Coverage is enforced by SonarQube in the pipeline, so the
duplicated JaCoCo checks are dead configuration.

Remove both executions. Keep jacoco-merge and jacoco-aggregate-report
which still feed the aggregated coverage data into SonarQube.
- Add cds-feature-sap-document-ai row to the Plugins table.
- Shorten the Samples section to a pointer at samples/bookshop, which
  now unifies the sample apps that demonstrate all plugins together.
- Shorten the Local Development section: keep the top-level build and
  test commands, drop the module-specific AI Core / integration-tests
  invocation and point to the individual module READMEs and to the
  integration-tests/spring module instead.
@lisajulia

Copy link
Copy Markdown
Contributor

Thanks for the review! Going through each point:

  1. cds-starter-ai + doc-ai: doc-ai isn't fully multi-tenant-ready yet, so I'd rather not include it in the starter until it is. An app pulling cds-starter-ai in a multi-tenant scenario would gain surprises. I'll add a TODO comment in cds-starter-ai/pom.xml and a short note in the root README's Starter section so the intent is tracked.
    2 & 3. Sample layout: I agree with the underlying goal (fewer, more discoverable samples) but disagree on the "per-plugin" location. CAP's idiomatic pattern is one CAP app hosting multiple services that's exactly what integration-tests/spring already does. I'd like to merge samples/document-ai-bookshop INTO samples/bookshop, so the single sample demonstrates AI Core + recommendations + doc-ai side by side. Each plugin's README will link to it. This is less maintenance (one CAP app, one package.json, one pom.xml) and matches CAP best practices. Happy to discuss further if you'd prefer per-module samples.
  2. Coverage report includes doc-ai: Good catch! Adding it to the dependencies, copy-classes resources, and the jacoco-merge fileset.
  3. 0.00: Intentional! SonarQube enforces coverage in the pipeline, so the JaCoCo check goals are effectively no-ops. I'll remove the check-ai-core and check-recommendations executions to make this explicit. (This also means point 4 doesn't need a matching check exec for doc-ai just adding it to merge/report.)
  4. Root README: Will add the doc-ai row to the Plugins table + the starter note from point 1. I'll keep short pointer sections for Samples and Local Development at the root and put the detailed instructions inside the respective module READMEs.

Thanks :) - I see you already did all these changes then we're good to go and can merge this

lisajulia
lisajulia previously approved these changes Jul 6, 2026
@Schmarvinius Schmarvinius requested a review from lisajulia July 6, 2026 12:43
@Schmarvinius Schmarvinius merged commit 32c3930 into main Jul 6, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants