Skip to content

feat(import): scale GitHub imports and add secure private access - #233

Draft
ThisIs-Developer wants to merge 6 commits into
mainfrom
agent/show-all-github-markdown-files
Draft

feat(import): scale GitHub imports and add secure private access#233
ThisIs-Developer wants to merge 6 commits into
mainfrom
agent/show-all-github-markdown-files

Conversation

@ThisIs-Developer

@ThisIs-Developer ThisIs-Developer commented Jul 30, 2026

Copy link
Copy Markdown
Owner

What changed

  • Removed the 30-file cap and retained complete subtree traversal when GitHub truncates a recursive tree response.
  • Added branch, tag, and commit URL resolution, including branch names containing /; every import is pinned to the resolved immutable commit SHA.
  • Shows the repository, resolved ref, and linked short commit in the file-selection step. Non-default imports also include the short commit beside the Explorer repository folder name.
  • Added private-repository imports through optional PAT access:
    • Fine-grained PAT is selected by default and marked Recommended. It supports multiple selected repositories, or all repositories, for one resource owner with Contents: Read-only.
    • Personal access token (classic) is available as an explicitly broader compatibility fallback, with an in-product repo-scope warning.
    • The chosen token type is preserved with remembered encrypted-access metadata so the dialog remains consistent after reload.
    • Public repositories are requested anonymously; authenticated requests and private file downloads go only to https://api.github.com.
    • Session-only access is the default and is never written to application storage.
    • Optional 1-, 7-, or 30-day retention stores AES-256-GCM authenticated ciphertext under a PBKDF2-SHA-256 derived key (250,000 iterations), with a random salt and IV.
    • The protection passphrase is never stored; expiry and manual removal delete the local record.
  • Kept the GitHub URL step at the compact 520px width and made its body scroll independently between a stable header and footer. The expanded private-access card can no longer shrink and clip its lower controls.
  • Kept the existing Select Markdown files to import picker at 760px, with file search, selected/total counts, icon-based select all/deselect all, and collapsible GitHub-style rows.
  • Changed browser file selection, drag-and-drop, Neutralino device import, and GitHub import to save Markdown documents in Explorer without automatically opening tabs.
  • Batched Close all, Close others, Close left, and Close right state updates so large tab sets render and schedule storage once instead of once per tab.
  • Synchronized browser changes into the desktop resources and updated the README, changelog, usage, FAQ, feature, and troubleshooting documentation.

Why

Large repositories were capped at the first 30 Markdown files. The importer assumed the first URL path segment was the complete ref, private repositories had no supported access path, bulk imports flooded the tab bar, and closing hundreds of tabs repeatedly rebuilt the UI. The first PAT UI also presented only fine-grained tokens and allowed its expanded card to shrink inside short viewports, clipping controls above the footer.

Security model

A browser-only application cannot promise zero compromise risk. This implementation minimizes exposure: PATs are never placed in URLs, logs, localStorage, the Markdown Viewer backend, or requests to raw.githubusercontent.com; authenticated fetches reject non-api.github.com destinations and cross-origin redirects. Fine-grained access remains the recommended default because it has a smaller blast radius. Classic PATs are clearly identified as broad-access compatibility credentials. Session-only access remains the safest retention choice.

References:

User impact

  • Every Markdown file in a public or authorized private repository/folder can be found and searched.
  • Default branches, other branches, slash-containing branch names, tags, and commit URLs work, with the exact imported commit visible and linked.
  • Users can choose Fine-grained PAT (Recommended) or Classic PAT (compatibility), with permission guidance tailored to the selection.
  • The compact URL dialog remains usable on short screens: lower credential fields scroll into view while Cancel and Import remain visible.
  • Private access is session-only by default, optionally encrypted with timed local disposal, and manually removable.
  • Bulk imports remain organized in Explorer without flooding the tab bar, and closing hundreds of tabs completes as one batched UI update.

Validation

  • node --check script.js
  • node --check desktop-app/resources/js/script.js
  • git diff --check
  • Browser/desktop script and style resources compared byte-for-byte; GitHub modal markup compared section-for-section.
  • GitHub importer Playwright suite: 8/8 passed
    • all 2,717 mocked Markdown files render, filter, and bulk-select;
    • truncated trees traverse every subtree;
    • slash branches resolve by longest valid ref and show the immutable commit;
    • Fine-grained and Classic choices update labels, guidance, and creation links;
    • session tokens remain absent from Local Storage and IndexedDB;
    • remembered Classic-token records contain ciphertext rather than token/passphrase plaintext and restore the selected type after reload;
    • private content uses authenticated api.github.com Contents requests and never the raw host;
    • 533x535 short-viewport and phone portrait/landscape layouts remain reachable.
  • Shared modal-shell regression: 1/1 passed.
  • Visual QA completed for the scrollable short-screen dialog in light and dark themes.

Known unrelated baseline check

  • npm run build stops because the repository baseline references the absent functions/api/report-issue.js file in tests/helpers/static-build-check.mjs.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 30, 2026

Copy link
Copy Markdown

Deploying markdownviewer with  Cloudflare Pages  Cloudflare Pages

Latest commit: 4a8bc33
Status: ✅  Deploy successful!
Preview URL: https://63c23024.markdownviewer.pages.dev
Branch Preview URL: https://agent-show-all-github-markdo.markdownviewer.pages.dev

View logs

@vercel

vercel Bot commented Jul 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
markdown-viwer Ready Ready Preview Jul 30, 2026 3:05pm

@ThisIs-Developer ThisIs-Developer changed the title fix(github): show every Markdown file in importer fix(import): modernize GitHub picker and prevent bulk tab stalls Jul 30, 2026
@gitguardian

gitguardian Bot commented Jul 30, 2026

Copy link
Copy Markdown

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them.
While these secrets were previously flagged, we no longer have a reference to the
specific commits where they were detected. Once a secret has been leaked into a git
repository, you should consider it compromised, even if it was deleted immediately.
Find here more information about risks.


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@ThisIs-Developer ThisIs-Developer changed the title fix(import): modernize GitHub picker and prevent bulk tab stalls feat(import): scale GitHub imports and add secure private access Jul 30, 2026
@ThisIs-Developer
ThisIs-Developer force-pushed the agent/show-all-github-markdown-files branch from fd41696 to fb0afce Compare July 30, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant