Create modelrepotest.mdx - #696
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
@Promptless add this as step one in the doc: Install Go (and the command is: brew install go) |
|
Done — added Install Go ( |
| export RUNPOD_GRAPHQL_URL="https://api.runpod.io/graphql" | ||
| export RUNPOD_API_KEY="your-api-key" # replace with your actual API key | ||
|
|
||
| export MODEL_NAME="$(whoami)-test-$(date +%s)" # unique name per test run — reusing the same name uploads a new version, not a new model |
There was a problem hiding this comment.
This can probably be changed to:
export MODEL_NAME="model_name" # unique name per test run — reusing the same name uploads a new version, not a new model
There was a problem hiding this comment.
|
|
||
| --- | ||
|
|
||
| ## Scripted testing |
There was a problem hiding this comment.
We should just remove this section, as it was just for internal testing
There was a problem hiding this comment.
|
|
||
| ```bash | ||
| runpodctl model add \ | ||
| --name "$MODEL_NAME" \ # the name to register the model under in your repo |
There was a problem hiding this comment.
There can't be comments after the \. The \ needs to be the last character on the line. The comments can go before or after.
There was a problem hiding this comment.
| ```bash | ||
| export USER_ID="$(runpodctl user | jq -r '.id')" # your Runpod user ID | ||
| export MODEL_HASH="$(runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash')" # the hash from step 3 | ||
| export MODEL_URL="https://local/${USER_ID}/${MODEL_NAME}:${MODEL_HASH}" # the full model reference URL |
There was a problem hiding this comment.
Let's get rid of this line, as we will build the Model URL and also the reference below
There was a problem hiding this comment.
| export MODEL_URL="https://local/${USER_ID}/${MODEL_NAME}:${MODEL_HASH}" # the full model reference URL | ||
| ``` | ||
|
|
||
| The resulting `MODEL_URL` will look like: |
There was a problem hiding this comment.
Get rid of this too, as we are no longer using this variable
There was a problem hiding this comment.
|
|
||
| ```bash | ||
| runpodctl serverless create \ | ||
| --name "$(whoami)_ctl_test" \ # name for the endpoint |
There was a problem hiding this comment.
This could be something more generic. whoami was confusing some users. It could be changed to:
--name "my_worker" \
There was a problem hiding this comment.
| --gpu-id "AMPERE_24" \ # GPU type | ||
| --workers-max 3 \ # maximum number of active workers | ||
| --workers-min 1 \ # minimum number of workers kept warm | ||
| --model-reference "$MODEL_URL" # attaches your model to the endpoint |
There was a problem hiding this comment.
Let's change this line to these two lines:
--model-reference "https://local/$USER_ID/$MODEL_NAME:$MODEL_HASH" \
--env MODEL_NAME="/runpod/model-store/modelrepo-local/models/$USER_ID/$MODEL_NAME/$MODEL_HASH"
There was a problem hiding this comment.
Also need to add the following to work around a bug in runpodctl:
--min-cuda-version "13.0"
There was a problem hiding this comment.
| ```bash | ||
| runpodctl serverless create \ | ||
| --name "$(whoami)_ctl_test" \ # name for the endpoint | ||
| --template-id "mockworker" \ # worker template to use |
There was a problem hiding this comment.
Let's change this line to:
--hub-id "cm8h09d9n000008jvh2rqdsmb" \
As that will actually do something if they use it, rather than being the mockworker which doesn't do anything
There was a problem hiding this comment.
|
I added some comments in line, but generically, for all of the shell script code blocks, there can't be comments after the last |
|
@Promptless can you address all the pending comments from brosenpod ? |
- Simplify MODEL_NAME example to a placeholder - Remove internal-only scripted testing section - Move inline comments off line-continuation (\) lines in shell blocks - Drop unused MODEL_URL variable and its example - Use generic --name, real --hub-id, and split model reference into --model-reference and --env, plus --min-cuda-version workaround - Mention the web UI as an alternative for sending test requests
|
Done — I've addressed all of @brosenpod's pending review comments and pushed the changes to this PR branch. Summary of what changed in
One additional cleanup: since The updated preview should refresh shortly. |
|
Thanks for the additional comments — we've got all of them. Update on
All of these are being applied together on this branch, and we'll follow up here once the changes are pushed. |
|
Update on all five review comments 👇 We've drafted, reviewed, and verified changes for every tagged comment on
We need a decision from you to publish these. This PR is based on the
Just reply here with your preference and we'll proceed. One more item (not yet tagged for us): the per-file size limit placeholder in the same section was answered in review as "5TB max file size," but that comment wasn't tagged for us, so we left it untouched. If you'd like that fact added too, tag it or let us know here. |
Addresses @brosenpod's review on PR #696: - Use provider-neutral storage wording (drop specific R2 mention) - Document all file types supported with no type checking - Add 5TB per-file maximum size - Note total per-account/per-model storage limit is still being finalized - Document indefinite retention of uploaded model files - Remove resolved internal engineering placeholders
|
@lavanya-gunreddi the write-access issue is resolved and the review feedback is now published — all six edits to
Review the full diff on the dashboard: https://app.gopromptless.ai/suggestions/abeea91e-64c8-483d-9fd6-5258513e7e32 |
|
👋 Thanks for the review — we've picked up your comment on this PR and are working on the requested changes now. We'll follow up here once the update is ready. |
|
Update: we've reviewed all of the requested changes on this PR and scoped the edits to the Model Repo testing page — including the status wording, softening the cold-start claim, tidying the inline comments in the upload and deploy commands, adding the |
- Update status note: alpha -> beta - Soften cold-start claim (models fetched from storage in some cases) - Add --wait-for-hash to upload command; remove redundant hashing wait step - Move inline comments into blocks above bash commands - Renumber steps 1-6
|
Update: the requested changes are now applied and pushed to this PR's branch. This pass covers the six review comments on the Model Repo testing page — the status wording, the softened cold-start note, the |
|
@lavanya-gunreddi all six of your review comments on the Model Repo testing page have been addressed and pushed to
One small adjustment: we used lowercase "beta" to match the convention used elsewhere in these docs (it's not on the always-capitalize list) — let us know if you'd prefer capital "Beta". Note that this pass covered the six comments addressed to us. A few other review threads on the page — the line-47 reword and the overview/security comments — weren't part of this request; happy to pick those up if you'd like. Review the full update here: https://app.gopromptless.ai/suggestions/abeea91e-64c8-483d-9fd6-5258513e7e32 |
|
👋 Thanks — we've picked up your latest review comment and are working on it now. We'll follow up here once the update is ready. |
|
Update: we've reviewed the three new comments on the Model Repo overview page and scoped the edits — clarifying that you write your own handler (Model Repo just mounts the model, nothing special), noting that Model Repo works across multiple datacenters whereas a network volume is tied to a single one, and addressing what migrating from a baked-in-image model may require. We're preparing the update now and will follow up here when it's ready. |
Resolve two engineering placeholders and add multi-datacenter comparison using @brosenpod's review answers: - Remove the handler-wrapping placeholder (Runpod does no special wrapping beyond mounting; existing prose already accurate). - Add a data center availability row and bullet to Model Repo vs network volumes (Model Repo serves a model in any data center; a network volume is tied to one). - Replace the baked-image edge-case placeholder with migration guidance to repoint a hardcoded model path to the Model Repo path.
|
@lavanya-gunreddi all three of your latest review comments on the Model Repo overview page have been addressed and pushed to
Note that this pass covered the three comments tagged for us. The storage-billing placeholder in the comparison table and @ashleyfrith-alt's structural comments (doc placement, diagram, quickstart split) weren't part of this request — happy to pick those up if they're tagged for us. Review the full update here: https://app.gopromptless.ai/suggestions/abeea91e-64c8-483d-9fd6-5258513e7e32 |
…arison, remove worker paths section
…ipted testing section
|
@Promptless # Promptless Prompt: MDX documentation cleanup Apply the following two fixes to this or any any MDX documentation file. Fix 1: Remove em dashes and hyphens used as separators in proseIn prose text (outside of code blocks), remove em dashes (—) and hyphens used as mid-sentence separators. Rewrite the sentence so it reads clearly without them. Do not remove hyphens that are part of a word (e.g. Do not modify anything inside fenced code blocks ( Examples❌ Before: ✅ After: ❌ Before: ✅ After: Fix 2: Move inline comments above multi-line bash commandsIn bash code blocks, inline comments must not appear on lines that end with a backslash continuation ( Move all inline flag comments to a comment block directly above the command. Each comment should be on its own line starting with Examples❌ Before: runpodctl model add \
--name "$MODEL_NAME" \ # the model name
--model-path "$MODEL_PATH" \ # local path to model files
--create-upload \ # creates the upload session
--wait-for-hash # waits for hashing to complete✅ After: # --name: the model name
# --model-path: local path to model files
# --create-upload: creates the upload session
# --wait-for-hash: waits for hashing to complete
runpodctl model add \
--name "$MODEL_NAME" \
--model-path "$MODEL_PATH" \
--create-upload \
--wait-for-hash❌ Before: runpodctl serverless create \
--name "my_worker" \ # endpoint name
--hub-id "abc123" \ # Hub template ID
--gpu-id "AMPERE_24" # GPU type✅ After: # --name: endpoint name
# --hub-id: Hub template ID
# --gpu-id: GPU type
runpodctl serverless create \
--name "my_worker" \
--hub-id "abc123" \
--gpu-id "AMPERE_24"Note: Inline comments on single-line commands (no export RUNPOD_API_KEY="your-api-key" # replace with your actual API key ← leave this alone |
|
👋 Thanks — we've picked up the MDX cleanup request (removing em dashes / separator hyphens in prose and moving inline comments above multi-line bash commands) and are working on it now. We'll follow up here once the update is ready. |
|
Update: we've reviewed the Model Repo doc set against both cleanup rules. The bash-comment formatting (Fix 2) is already satisfied — the multi-line commands already have comment blocks above them, and the remaining inline comments are on single-line commands, which the request says to leave. For Fix 1, we found about eight em-dash/separator instances in prose across the overview, testing, modelrepotest, and security pages, and we're rewriting those now. We'll follow up here when the update is ready. |
Apply reviewer's MDX cleanup (Fix 1): remove em dashes and separator hyphens from prose across the Model Repo pages, rewriting each sentence to read cleanly. Fix 2 (inline comments off backslash-continuation lines) was already satisfied. Code blocks, inline code, word hyphens, and numeric ranges left untouched.
|
@lavanya-gunreddi the MDX cleanup is done and pushed to
The PR description now records what was applied. Review the full update here: https://app.gopromptless.ai/suggestions/abeea91e-64c8-483d-9fd6-5258513e7e32 |
3 participants
Creates the Model Repo documentation set and applies reviewer feedback from this PR.
Review-feedback pass —
serverless/storage/modelrepo/overview.mdxApplied @brosenpod's engineering answers and the requested changes to the overview page. Per-item disposition:
The 5TB figure is documented as the per-file maximum only; the total per-account/per-model limit remains unquantified per the reviewer's "value needs to be decided" note. The other engineering placeholders on this page (handler wrapping, storage billing, baked-images edge cases) belong to separate review threads and were left untouched in this pass.
Review-feedback pass —
serverless/storage/modelrepo/testing.mdxApplied @brosenpod's review comments on the testing page (surfaced via the threaded "@Promptless work on this comment" replies). Per-item disposition:
\is the final character on each continuation line--wait-for-hashto auto-wait for hashingrunpodctlsource)--wait-for-hashMODEL_HASHcomment that referenced the removed step)\is the final character on each continuation lineNot in scope for this pass: the line-47 reword thread (r3633099715) and other unassigned overview/security/modelrepotest comments were left untouched.
Review-feedback pass 2 —
serverless/storage/modelrepo/overview.mdxResolved the two remaining engineering placeholders and added the multi-datacenter comparison, using @brosenpod's threaded answers. Per-item disposition:
[ENGINEERING: Marcin's question…]placeholder. Reviewer confirmed Runpod does no special wrapping beyond mounting the model, which validates the existing prose that the user's handler loads the model; no new claim added[ENGINEERING: …baked images…]placeholder with guidance to repoint a hardcoded model reference to the Model Repo path, noting the effort depends on how the image loads the modelStill intentionally untouched: the storage-billing placeholder on the comparison table (a separate, unassigned thread) and @ashleyfrith-alt's structural comments (doc placement, diagram, quickstart split).
Review-feedback pass 3 — MDX cleanup (em dashes & bash comments)
Applied the "MDX documentation cleanup" Promptless Prompt from @lavanya-gunreddi (comment). Per-item disposition:
overview.mdx(intro, "What you can upload", the data-center bullet, and "Your data"),testing.mdx(Step 1 API-key note and the build-from-source<Note>),security.mdx("In transit" bullet and the storage-infrastructure FAQ answer), andmodelrepotest.mdx(Set-env-vars note and the build-from-source<Note>). Meaning preserved on every line. The data-center bullet was additionally normalized to the canonical term "data centers" and to drop the undefined "DCs" abbreviation.\line-continuation lines in bash blocksrunpodctlcommand on these pages already carries its flag comments in a block above the command; the remaining inline comments sit on single-line commands, which the prompt says to leave.Deliberately left as-is: the em dash inside a bash code-block comment in
modelrepotest.mdx(inside code), the10–15 minutesen-dash numeric range, and the unrelated[ENGINEERING]compliance placeholder insecurity.mdx.