feat(ctl): add --dependencies flag to infrahubctl marketplace get#1118
Open
minitriga wants to merge 6 commits into
Open
feat(ctl): add --dependencies flag to infrahubctl marketplace get#1118minitriga wants to merge 6 commits into
minitriga wants to merge 6 commits into
Conversation
Deploying infrahub-sdk-python with
|
| Latest commit: |
cf655db
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://e83bac0c.infrahub-sdk-python.pages.dev |
| Branch Preview URL: | https://marketplace-dependencies-ihs.infrahub-sdk-python.pages.dev |
github-actions
Bot
added
the
type/documentation
Codecov Report❌ Patch coverage is
@@ Coverage Diff @@
## stable #1118 +/- ##
==========================================
+ Coverage 82.16% 82.19% +0.03%
==========================================
Files 138 138
Lines 11896 12107 +211
Branches 1784 1826 +42
==========================================
+ Hits 9774 9951 +177
- Misses 1572 1594 +22
- Partials 550 562 +12
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Resolve a collection's dependencies via the marketplace API and download each schema individually. Prerequisite collections are grouped into their own directories and standalone dependency schemas land in the output root, with transitive, cycle-safe resolution. Collection members (requested or prerequisite) download strictly; loose and transitively-discovered schema dependencies soft-fail with a note so one missing dependency does not abort the bundle. Referenced kinds the marketplace cannot resolve are reported without downloading. Refs: IHS-246, #1117
minitriga
force-pushed
the
marketplace-dependencies-ihs-246
branch
from
cfb9019 to
e02ae4f
Compare
Extend --dependencies beyond collections: `marketplace get <schema> --dependencies` now downloads the requested schema plus its transitive dependency schemas (soft-fail, deduped, cycle-safe) into the output root. The requested schema downloads strictly; a dependency sharing its name in another namespace is disambiguated into a namespace subdirectory rather than overwriting it. Refs: IHS-246, #1117
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 4 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="infrahub_sdk/ctl/marketplace.py">
<violation number="1" location="infrahub_sdk/ctl/marketplace.py:628">
P2: Versioned single-schema downloads resolve dependencies from the latest schema metadata instead of the requested version. When `--version` is used with `--dependencies`, `_download_schema_tree` correctly downloads the requested schema version, but `_resolve_dependency_closure` discards the version and `_read_schema_dependencies` always fetches unversioned schema details and selects `latest_version` dependencies.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| # ``seen`` skips it and only its dependencies are written (soft-fail, to the output root). | ||
| seen: set[tuple[str, str]] = {(namespace, name)} | ||
| seed = [{"namespace": namespace, "name": name, "version": version}] | ||
| closure, unresolved = await _resolve_dependency_closure(client, base_url, seed, status=status) |
There was a problem hiding this comment.
P2: Versioned single-schema downloads resolve dependencies from the latest schema metadata instead of the requested version. When --version is used with --dependencies, _download_schema_tree correctly downloads the requested schema version, but _resolve_dependency_closure discards the version and _read_schema_dependencies always fetches unversioned schema details and selects latest_version dependencies.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At infrahub_sdk/ctl/marketplace.py, line 628:
<comment>Versioned single-schema downloads resolve dependencies from the latest schema metadata instead of the requested version. When `--version` is used with `--dependencies`, `_download_schema_tree` correctly downloads the requested schema version, but `_resolve_dependency_closure` discards the version and `_read_schema_dependencies` always fetches unversioned schema details and selects `latest_version` dependencies.</comment>
<file context>
@@ -583,6 +589,68 @@ async def _download_collection_tree(
+ # ``seen`` skips it and only its dependencies are written (soft-fail, to the output root).
+ seen: set[tuple[str, str]] = {(namespace, name)}
+ seed = [{"namespace": namespace, "name": name, "version": version}]
+ closure, unresolved = await _resolve_dependency_closure(client, base_url, seed, status=status)
+ reserved = {name} if requested_written else None
+ total_written += await _download_schema_set(
</file context>
Avoid duplicating a schema across the output tree when resolving dependencies: before writing, reconcile against schemas already present (e.g. a dependency at the root vs. a copy under a collection directory from a prior download). An already-present schema is kept by default, or overwritten in place with the new `-y`/`--yes` flag; an interactive terminal is prompted. Only files present before the run are considered, and the check is skipped entirely without --dependencies and in --stdout mode, so existing behavior is unchanged. Refs: IHS-246, #1117
Collapse the latest-version dependency lookup in _read_schema_dependencies to a single expression, and extract the duplicated "unresolved dependencies" report line into a shared _print_unresolved helper. No behavior change. Refs: IHS-246, #1117
When resolving a single schema's --dependencies, place each dependency under the directory of the collection it belongs to (via /collections/for-schema), mirroring a collection download; dependencies in no collection stay at the output root. The existing existing-file reconciliation (keep / --yes overwrite, no cross-directory duplicates) continues to apply. Refs: IHS-246, #1117
There was a problem hiding this comment.
1 issue found across 3 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="infrahub_sdk/ctl/marketplace.py">
<violation number="1" location="infrahub_sdk/ctl/marketplace.py:726">
P1: Unsanitized collection name from API used as directory path: potential path traversal</violation>
</file>
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
| if (dep["namespace"], dep["name"]) == (namespace, name): | ||
| continue | ||
| owning = await _owning_collection_name(client, base_url, dep.get("schema_id", "")) | ||
| buckets.setdefault(output_dir / owning if owning else output_dir, []).append(dep) |
There was a problem hiding this comment.
P1: Unsanitized collection name from API used as directory path: potential path traversal
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At infrahub_sdk/ctl/marketplace.py, line 726:
<comment>Unsanitized collection name from API used as directory path: potential path traversal</comment>
<file context>
@@ -689,24 +712,32 @@ async def _download_schema_tree(
+ if (dep["namespace"], dep["name"]) == (namespace, name):
+ continue
+ owning = await _owning_collection_name(client, base_url, dep.get("schema_id", ""))
+ buckets.setdefault(output_dir / owning if owning else output_dir, []).append(dep)
+
+ for target, group in buckets.items():
</file context>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Downloading a marketplace collection only fetched its own member schemas. The schemas those members reference (defined in other collections or as standalone schemas) were left out, so the bundle wouldn't load into Infrahub without manually hunting down each missing dependency. The marketplace UI already advertises
infrahubctl marketplace get <ns>/<name> --collection --dependencies, but the SDK didn't implement the flag — so anyone copying that command hit an error.Goal: ship the
--dependenciesflag so a collection downloads with everything it needs in one command.Non-goals: changing the no-flag behavior; using the marketplace's ZIP
?dependencies=trueendpoint (we resolve via the API and download per-schema for a consistent on-disk layout).Closes #1117
What changed
Behavioral:
infrahubctl marketplace get <ns>/<name> --collection --dependenciesnow downloads the collection's members plus the schemas it depends on.schemas/<collection>/directory; standalone dependency schemas land in the output root. Resolution is transitive, cycle-safe, and deduplicated.What stayed the same:
--dependencies, collection and single-schema downloads are byte-for-byte unchanged.--dependencieson a single schema is a no-op with an informational note.How to review
Focus on
infrahub_sdk/ctl/marketplace.py— the dependency walk and the per-collection layout. Tests intests/unit/ctl/test_marketplace_app.pymirror cases C1–C9 plus dedup/cycle and strict-vs-soft. The.mdxdoc anddev/specs/artifacts are generated/planning.How to test
```bash
uv run pytest tests/unit/ctl/test_marketplace_app.py -q
uv run infrahubctl marketplace get infrahub/routing-protocols --collection --dependencies -o ./schemas
find ./schemas -type f
```
Expected: members under
schemas/routing-protocols/, thebase-schemasprerequisite underschemas/base-schemas/. All 8 live marketplace collections were downloaded successfully (0 failures) during development.Impact & rollout
Checklist
changelog/1117.added.md)dev/specs/)Summary by cubic
Adds
--dependenciestoinfrahubctl marketplace getand-y/--yesoverwrite control. Collections and single schemas now download with their transitive dependencies, cycle-safe and deduped, matching the Marketplace UI. Closes IHS-246.New Features
... --collection --dependenciesdownloads members plus prerequisite collections and standalone dependency schemas. Layout: requested collection under./<collection>/; each prerequisite collection in its own directory; standalone dependencies in the output root. Name collisions are disambiguated by namespace.... --dependenciesdownloads the schema plus its transitive dependencies; dependencies are grouped under their owning collection directory (via/collections/for-schema), or in the output root if none. Same-name conflicts across namespaces are disambiguated into namespace subdirectories.-y/--yes(TTY prompts; non-interactive keeps). Reconciliation only considers files present before the run and is skipped with--stdout. Without--dependencies, behavior is unchanged.Refactors
Written for commit cf655db. Summary will update on new commits.