Enforce the LSP session lifecycle

[Widthdom]

Summary

• enforce one thread-safe LSP lifecycle across normal dispatch, cancellation, and overload paths
• reserve lifecycle decisions in receive order, synchronize initialize response publication, and reject out-of-phase requests with the required JSON-RPC errors
• make shutdown wait for active dispatches and dispose owned query resources exactly once
• add transcript, concurrency, overload, cancellation, and publication-race coverage

Root cause

Lifecycle booleans were updated only as handler side effects. The transport and alternate dispatch paths did not share a receive-ordered state machine, so queued or concurrent traffic could observe the wrong phase and shutdown did not provide one coordinated resource boundary.

Validation

• dotnet restore CodeIndex.sln
• dotnet build CodeIndex.sln -c Release --no-restore --nologo — 0 warnings, 0 errors
• dotnet test tests/CodeIndex.Tests/CodeIndex.Tests.csproj -c Release --no-build --no-restore --filter "FullyQualifiedName~LspServerTests|FullyQualifiedName~LspServerBudgetTests" — 115 passed on net8.0 and 115 passed on net9.0
• Issue Enforce the LSP initialize, shutdown, and exit lifecycle #4849 filter — 8 passed on each target framework
• dotnet format CodeIndex.sln --verify-no-changes --no-restore
• dotnet run --project tools/CodeIndex.Changelog -- check
• full Release suite before the final targeted race fix: net9.0 passed 10,125 with 419 skipped; net8.0 passed 10,597 with 7 skipped and one unrelated MCP parallel-suite flake (ProcessFrameAsync_BatchInitializeDoesNotFlowBackIntoTimedOutPriorGeneration_Issue4540_Issue4545), which passed on immediate isolated rerun
• Codex adversarial review: three first-round findings and one final publication-boundary finding were fixed and covered by regression tests

Documentation and changelog

• updated the English and Japanese LSP lifecycle sections in USER_GUIDE.md and DEVELOPER_GUIDE.md
• added changelog.d/unreleased/4849.fixed.md, including the LSP shutdown and exit notifications do not stop the server loop #2830 regression relationship

Fixes #4849