diff --git a/EPIC_NON_HTML_PAGES.md b/EPIC_NON_HTML_PAGES.md index 4ab5bf22a65..346c6474ed9 100644 --- a/EPIC_NON_HTML_PAGES.md +++ b/EPIC_NON_HTML_PAGES.md @@ -648,7 +648,7 @@ Implementation notes (branch `v3/non-html-pages-llms-txt`): > 3. **A long explanatory comment in a config stub is a design smell,** not diligence. If > an option needs paragraphs to explain, the option is usually the problem. -### PR 8 — Documentation & release notes +### PR 8 — Documentation & release notes ✅ Implemented - Document in-code virtual pages, `sitemap: false` front matter, robots/llms config, the container-rebind customization tier for generated pages, and the "user-defined @@ -657,6 +657,23 @@ Implementation notes (branch `v3/non-html-pages-llms-txt`): serve support for sitemap/RSS), breaking changes (build task classes removed/relocated, search.json removed from sitemaps). +Implementation notes (branch `v3/non-html-pages-documentation`): + +- The public InMemoryPage and customization guides document exact non-HTML paths, + navigation and sitemap defaults, all four generated pages, their feature conditions, + robots/llms configuration, generator rebinding, and user-defined route precedence. +- The Build Tasks guide no longer describes sitemap/RSS generation as post-build tasks. + The console command guide now records that `build:sitemap` and `build:rss` compile + registered pages and fail when no matching page is registered. +- The audit found behavior worth documenting beyond the original checklist: llms.txt + reuses sitemap inclusion, its format has no minor/patch compatibility promise while + the proposal evolves, robots.txt controls crawler access while llms.txt does not, + generated pages are hidden from automatic navigation, and sitemap/RSS registration + depends on SimpleXML in addition to their documented content prerequisites. +- `HYDEPHP_V3_PLANNING.md` and `UPGRADE.md` already contain the feature, breaking-change, + and migration entries added with PRs 1–7; this PR verified them rather than duplicating + those notes. + ## Out of scope (noted for later) - Filesystem autodiscovery for verbatim or Blade-processed text files diff --git a/docs/advanced-features/build-tasks.md b/docs/advanced-features/build-tasks.md index b85e9c0c4e7..abb9b8b196c 100644 --- a/docs/advanced-features/build-tasks.md +++ b/docs/advanced-features/build-tasks.md @@ -1,5 +1,5 @@ --- -abstract: "The Build Task API lets you hook into HydePHP's build process to run custom logic, or override built-in tasks like sitemap and RSS generation, whenever the site compiles." +abstract: "The Build Task API lets you hook custom logic into HydePHP's build process before or after the site's pages are compiled." --- # Custom Build Tasks @@ -9,9 +9,10 @@ abstract: "The Build Task API lets you hook into HydePHP's build process to run The Build Task API offers a simple way to hook into the build process. The build tasks are very powerful and allow for limitless customizability. -The built-in Hyde features like sitemap generation and RSS feeds are created using tasks like these. Maybe you want to create your own, to for example upload the site to FTP or copy the files to a public directory? -You can also overload the built-in tasks to customize them to your needs. + +Sitemaps, RSS feeds, robots.txt, and llms.txt are generated as regular pages rather than build tasks. To customize +those files, see [Generated discovery files](customization#generated-discovery-files). ## Good to know before you start diff --git a/docs/advanced-features/in-memory-pages.md b/docs/advanced-features/in-memory-pages.md index f6431891913..eb20b7e149a 100644 --- a/docs/advanced-features/in-memory-pages.md +++ b/docs/advanced-features/in-memory-pages.md @@ -69,6 +69,19 @@ InMemoryPage::make( ); ``` +Non-HTML pages are also excluded from the sitemap by default. You can control sitemap inclusion for any page with +the `sitemap` front matter key. This same setting controls whether a page is listed in Hyde's generated `llms.txt`: + +```php +InMemoryPage::make( + 'api/schema.json', + matter: ['sitemap' => true], + contents: $schema, +); +``` + +For custom page classes, override `showInSitemap()` when the decision cannot be expressed as front matter. + Pass a closure when the contents should be generated lazily during compilation. The closure is invoked again for each compilation, which makes it useful for pages generated from the current application state. @@ -173,6 +186,10 @@ class AppServiceProvider extends ServiceProvider The page will be written to `_site/hello.html` and can be referenced using the `hello` route key. +Hyde's generated `sitemap.xml`, RSS feed, `robots.txt`, and `llms.txt` are also in-memory pages. Registering your own +page with one of those route keys during booting replaces Hyde's generated page, allowing complete control over the +file. For the RSS feed, use the filename configured in `hyde.rss.filename`. + ### In a package extension Package extensions can register the page directly in the page discovery callback. Pages added at this stage are diff --git a/docs/digging-deeper/customization.md b/docs/digging-deeper/customization.md index ab067fb53f4..acb8f6d3c89 100644 --- a/docs/digging-deeper/customization.md +++ b/docs/digging-deeper/customization.md @@ -127,26 +127,99 @@ redirects: docs/old-guide: docs/new-guide ``` -### RSS feed generation +### Generated discovery files -When enabled, an RSS feed containing all your Markdown blog posts will be generated when you compile your static site. -Here are the default settings: +Hyde generates a sitemap, RSS feed, `robots.txt`, and `llms.txt` as regular in-memory pages. They are registered routes, +so they are included in a normal site build, shown by `php hyde route:list`, recorded in the build manifest, and served +by `php hyde serve`. Generated non-HTML pages are excluded from navigation and from the sitemap by default. + +The sitemap and llms.txt require a site URL so they can contain absolute links. RSS additionally requires Markdown blog +posts and the SimpleXML extension; sitemap generation also requires SimpleXML. Robots.txt has no site URL requirement, +and includes a `Sitemap:` line only when the sitemap is available. + +Here are the related default settings: ```php // filepath config/hyde.php +'generate_sitemap' => true, + 'rss' => [ - // Should the RSS feed be generated? 'enabled' => true, - - // What filename should the RSS file use? 'filename' => 'feed.xml', - - // The channel description. 'description' => env('SITE_NAME', 'HydePHP').' RSS Feed', ], + +'robots' => [ + 'enabled' => true, + 'disallow' => [ + // '/private', + // '/*.pdf$', + ], +], + +'llms' => [ + 'enabled' => true, + 'description' => null, +], ``` ->warning Note that this feature requires that a `site_url` is set! +Each robots.txt `disallow` value is written verbatim as a `Disallow:` rule, allowing patterns such as `/*.pdf$`. +The llms.txt description becomes its introductory blockquote. The file groups published pages by type and uses each +page's `abstract` front matter, falling back to `description`, as the link description. A page is included in llms.txt +when it is included in the sitemap, so `sitemap: false` excludes it from both indexes. This does not prevent an AI +crawler from accessing the page; use robots.txt rules for crawler access control. + +>warning Llms.txt is an emerging standard. Hyde may change the generated format in minor or patch releases as the +> specification evolves. Disable it with `hyde.llms.enabled` or replace the page if you need a fixed format. + +#### Customizing generated output + +For small output changes, extend the corresponding generator and bind your implementation in a service provider. Hyde +resolves generators from the service container when the page is compiled, after route discovery is complete: + +```php +// filepath app/Providers/AppServiceProvider.php + +use Hyde\Framework\Features\TextGenerators\RobotsTxtGenerator; +use Illuminate\Support\ServiceProvider; + +class CustomRobotsTxtGenerator extends RobotsTxtGenerator +{ + public function generate(): string + { + return parent::generate()."\nHost: example.com\n"; + } +} + +class AppServiceProvider extends ServiceProvider +{ + public function register(): void + { + $this->app->bind(RobotsTxtGenerator::class, CustomRobotsTxtGenerator::class); + } +} +``` + +The available generators are `SitemapGenerator`, `RssFeedGenerator`, `RobotsTxtGenerator`, and `LlmsTxtGenerator` in +their respective `Hyde\Framework\Features\XmlGenerators` and `Hyde\Framework\Features\TextGenerators` namespaces. +The protected `LlmsTxtGenerator::sections()` method can be overridden to change its page groups. + +To replace a generated file completely, register your own [`InMemoryPage`](in-memory-pages) with +the same route key during a kernel booting callback or extension discovery. User-defined pages take precedence over +Hyde's generators, even when the corresponding feature is disabled. For example: + +```php +use Hyde\Hyde; +use Hyde\Pages\InMemoryPage; +use Hyde\Foundation\HydeKernel; + +Hyde::booting(function (HydeKernel $kernel): void { + $kernel->pages()->addPage(InMemoryPage::make( + 'robots.txt', + contents: "User-agent: *\nDisallow: /private\n", + )); +}); +``` ### Authors diff --git a/docs/getting-started/console-commands.md b/docs/getting-started/console-commands.md index 2f4c8f55be7..32194760945 100644 --- a/docs/getting-started/console-commands.md +++ b/docs/getting-started/console-commands.md @@ -116,6 +116,11 @@ php hyde build:rss Generate the RSS feed +This command compiles the registered RSS page. The page is normally registered when RSS generation is enabled, a site +URL is configured, at least one Markdown post exists, and the SimpleXML extension is available. The command exits with +an error if no RSS page is registered. A custom page registered at the configured `hyde.rss.filename` route is compiled +instead of Hyde's generated page. + ## Generate the `docs/search.json` file @@ -136,6 +141,11 @@ php hyde build:sitemap Generate the `sitemap.xml` file +This command compiles the registered `sitemap.xml` page. The page is normally registered when sitemap generation is +enabled, a site URL is configured, and the SimpleXML extension is available. The command exits with an error if no +sitemap page is registered. A custom page registered at the `sitemap.xml` route is compiled instead of Hyde's generated +page. + ## Scaffold a new Markdown, Blade, or documentation page file