Skip to content

fix: resolve cache markdown crash, inverted viewport ratio, prometheus 500, dead code, and badge URLs#2033

Open
Mohammad-Faiz-Cloud-Engineer wants to merge 4 commits into
unclecode:mainfrom
Mohammad-Faiz-Cloud-Engineer:main
Open

fix: resolve cache markdown crash, inverted viewport ratio, prometheus 500, dead code, and badge URLs#2033
Mohammad-Faiz-Cloud-Engineer wants to merge 4 commits into
unclecode:mainfrom
Mohammad-Faiz-Cloud-Engineer:main

Conversation

@Mohammad-Faiz-Cloud-Engineer

Copy link
Copy Markdown

Bugs

  • Cache markdown crash —> json.loads() on raw markdown text passes parsed
    values (bool, int) straight to CrawlResult, then StringCompatibleMarkdown
    explodes on .raw_markdown access. Missing markdown also triggers Pydantic
    validation error. Rewrote the cache read path to handle every json.loads()
    outcome safely.
  • Viewport ratio inverted —> target_height was target_width * page_width /
    page_height, giving ~4% of the correct height. Fixed to target_width *
    page_height / page_width.
  • Prometheus 500 on all prefixed routes —> Starlette's _IncludedRouter
    lacks .path, crashes the instrumentator middleware before auth runs.
    Monkey-patched _get_route_name to return "unknown" instead of throwing.
  • Duplicate except block —> second except Exception after both try and
    first except already return. Removed.

Dead code

  • Removed first normalize_url() definition (immediately overwritten by the
    real one), normalize_url_tmp() (never called), orphaned adaptive_crawler
    copy.py, and unused html2text.HTML2Text() init.

Typos

  • fa_user_agenr_generator → ua_generator
  • SCREENSHOT_HEIGHT_TRESHOLD → THRESHOLD (backward compat alias kept)
  • Docstring: configure_windows_event_loop import path was wrong

Docs

  • Stripped ~1300 decorative emojis from 46 .md files. Kept only functional
    ones (check marks, warnings, links, locks, shields, brand rocket/robot).
    Code blocks preserved.

Badges

  • nstproxy: www.nstproxy.com/logo.svg returns 2MB of HTML, not an SVG.
    Replaced with the working gist URL.
  • Thor Data: switched from gist.github.com (redirect) to direct
    gist.githubusercontent.com URL.

…s, and cleanup docs emojis

- fix(critical): cache markdown dict collapsing causes AttributeError on cached URLs
- fix(high): viewport aspect ratio formula was inverted, producing wrong sizes
- fix(high): remove unreachable duplicate except block in execute_user_script
- fix(medium): remove dead duplicate normalize_url, normalize_url_tmp definitions
- fix(medium): remove dead html2text.HTML2Text() instantiation
- fix(medium): fix typo fa_user_agenr_generator -> ua_generator
- fix(medium): SCREENSHOT_HEIGHT_TRESHOLD -> THRESHOLD with backward compat
- fix(medium): remove orphaned adaptive_crawler copy.py
- fix(low): fix configure_windows_event_loop import path in docstring
- fix(low): remove duplicate exclude_internal_links docstring
- fix: complete markdown cache read rewrite to handle all json.loads() outcomes
- fix: add type safety in CrawlResult.__init__ for markdown parameter
- fix: acache_url MarkdownGenerationResult() Pydantic v2 crash with missing fields
- docs: remove decorative emojis from 46 .md files, keep only functional ones
…routes

The prometheus_fastapi_instrumentator routing.get_route_name tries to access
route.path on all app.routes entries, but Starlette's _IncludedRouter objects
(created by app.include_router with prefix=) lack a .path attribute.

Monkey-patch _get_route_name to catch AttributeError gracefully, returning
'unknown' for affected routes instead of crashing with HTTP 500.
…stead

www.nstproxy.com/logo.svg returns text/html (2MB, full website)
instead of an SVG image, breaking the badge in light mode and fallback.
Replaced the picture element with a simple img pointing to the known-good
gist SVG at gist.githubusercontent.com.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant