Add Vulkan rendering backend, with HDR10 output and raytraced shadows#7554
Add Vulkan rendering backend, with HDR10 output and raytraced shadows#7554The-E wants to merge 58 commits into
Conversation
BMagnu
left a comment
BMagnu
left a comment
There was a problem hiding this comment.
Check out SDR_FLAG_TONEMAPPING_LINEAR_OUT.
It's basically poor-man's HDR output in the OpenGL pass, currently only used for the OpenXR pass, since the headset swapchain seems to do its own tonemap, or at the very least expects non-SDR input (it's highly possible that proper HDR input is in fact the correct thing to forward here, instead of truly linear data).
This should very likely be merged into this work, even if it means supporting OpenGL to some degree, rather than exist as a weird, second, half-baked HDR pass. Especially since proper handling across not only gameplay but also menus will fix #7181.
I'm not yet fully familiar with most of the Vulkan PR, but I assume that before this change, there is no handling of Cmdline_window_res, which this new buffer could then do in the future as well? I also assume that this PR is going to be the proper place to mirror #7484 for the SDL3 upgrade. If so, we should at least prepare / design this in a way that makes it easy to retrofit.
The-E
changed the title
The-E
added
graphics
c6dbb22 to
1d6a74b
Compare
|
Gave this a try on my Mac. Most of my known Vulkan bugs appear to be gone (tex flicker, MSAA, fog, loadout flicker). The texture corruption in massive_battle2 looks to be substantially reduced, if not entirely eliminated. New issues that I noticed: Shadows do not appear to work correctly. Or at least not like they did before. I'm not seeing/noticing self-shadowing in the techroom or lab. I see shadows in-mission, but they can move or completely vanish depending on my view direction even if nothing else in the mission is moving. I cleaned out the cache just to be sure but there was no visual change. HDR works but certain graphical elements flicker (like text and highlighted ui controls), and there is some easily producible yet kind of random graphics corruption. The corruption was visible on some ui screens but not others. In the techroom when viewing the Ulysses there is corruption, but not when viewing the Herc. I have not yet played a mission with HDR enabled. |
|
Another known bug, for the record: |
can you post logs (with the vulkan log filter enabled) and screenshots of this? |
|
Here are logs, screenshots, and a vid. I'm not sure if the screenshot or vid will actually show the HDR, but in the game it's very noticeable and nice looking on my MBP screen. For the log I made sure to enable "Vulkan" and "vulkanstate", launched the game, went to the techroom, switch the model from Ulysses to Hec, went back to mainhall and exited. The pilot select screen had ui flickering and corruption, the mainhall had ui flickering but no real corruption, the Ulysses had both, the Herc had ui flicker only. I had trouble with the screenshots as one frame appears to have corruption but show none of the ui elements that flicker, or show the ui elements but not the corruption. So most of the screenshots, particularly from the techroom, never showed any corruption. As such I'm just including a single screenshot that demonstrated the corruption as well as a shot of my in-game settings. The video is just enough to show the various issues. However while recording the video the Herc actually showed corruption, so I just cut it before the switch. 
vulkan-hdr.mp4I should probably also note that I'm unable to use fullscreen here. I tried, to see if there was any difference, but it always crashes. On the Mac it must do a mode switch to work, but that was disabled in the Vulkan PR. Mode switching should work better when SDL3 hits though so we can probably re-enable it. |
ebc3677 to
8f027ba
Compare
The only things that jumped out at me during a cursory review are fixed. I'm going to leave actual approval to BMagnu however, since I'm not qualified to properly review the bulk of the changes.
The-E
changed the title
Squashed snapshot of the Vulkan backend as vendored into this branch (ce2d1e8..6ce5394, authored by Mara van der Laan with build/CI fixes by Taylor Richards). This work has its own upstream PR scp-fs2open#7553 (notimaginative:vulkan-pr-new) and is not otherwise original to true-hdr.
Squashed snapshot of true-hdr's own integration work: true HDR framebuffer output, wiring the shadow overhaul into the Vulkan backend (raytraced + cascaded shadow maps), logging refactors, log filter UI, and CI/build fixes.
- Moved SMAA-specific logic into `VulkanSMAA` class for better separation of concerns. - Simplified `initSMAA`, `shutdownSMAA`, and `executeSMAA` logic by leveraging `VulkanLDR` and `PostProcessContext` abstractions. - Improved initialization and cleanup code with clearer resource lifecycle management. - Updated debug markers and messages to reference `VulkanSMAA`. - Enhanced flexibility of fullscreen triangle rendering API with additional parameters (`sampleCount`, `bindGlobalSet`).
- Adjusted viewport height to negative value to account for G-buffer render pass's negative Y-axis convention. - Ensured proper alignment of point and tube light volumes with actual light positions.
- Added gamma correction to HDR10 path, applying user settings before PQ/BT.2020 encode. - Replaced the SDR blit-only path with an encode pass that applies gamma correction, ensuring consistent behavior across devices. - Refactored VulkanPostProcessor and associated shaders to support gamma slider adjustments for both SDR and HDR output.
…ering - Added uniform buffer setup and default material uniforms in `vulkan_draw_sphere`. - Disabled face culling in material definition for sphere rendering to align behavior with OpenGL.
…rier for MoltenVK compatibility.
… diagnostics Groundwork for the renderer-review fix series (.agents/plans/ vulkan-renderer-review-fixes.md, Phase 0): - Register a VK_EXT_debug_utils messenger (preferred) instead of only the deprecated VK_EXT_debug_report callback, which is kept as a fallback for loaders without debug-utils. The messenger config is also chained into instance creation so vkCreateInstance-time messages are captured. Validation errors/warnings now go to the main log via mprintf. - New -gr_sync_validation cmdline flag (implies -gr_debug): chains VkValidationFeaturesEXT with synchronization validation into instance creation when VK_LAYER_KHRONOS_validation is available. This is the regression net for the synchronization fixes in the following commits. - Frame stats now report descriptor sets/writes allocated per frame and the total pipeline count (descriptor-manager counters; DescriptorWriter::flush moved out-of-line to report into the manager). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…per-frame UBO ring Phase 1 of the renderer-review fix series: shared helpers that the correctness fixes build on. - New VulkanRenderer::beginTrackedRenderPass(PassBeginDesc): every render-pass begin on the frame command buffer now records vkCmdBeginRenderPass and updates the state tracker (pass, color attachment count, sample count, render area, viewport) in one place. Converted: setupFrame, beginSceneRendering (both branches), resumeSceneRendering, endSceneRendering, beginRenderTarget (viewport kept for gr_set_viewport), resumeSwapChainPass, plus new resumeScenePassAfterCopy() shared by copyEffectTexture/copySceneDepthForParticles and rebeginSwapChainPassAfterReadback() shared by both readbackFramebuffer exits (~200 lines of duplicated begin/clear/viewport blocks removed). - The state tracker records the active render area; all four vkCmdClearAttachments sites clamp their rects against it and skip empty results. Previously gr_screen-sized clear rects could exceed a smaller off-screen render target's render area, which is invalid API usage. - New PerFrameUboRing utility (VulkanPerFrameUbo.h/.cpp): a persistently mapped uniform buffer with one slot region per frame in flight and a self-flushing alloc(). Used by the next commit to fix the cross-frame UBO races. Deliberate behavior deltas (all safe-direction): the helper always sets colorAttachmentCount/sampleCount (some sites previously inherited stale values), and resumeSwapChainPass/resumeSceneRendering now also reset the standard flipped viewport instead of leaving the previous one. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 2 of the renderer-review fix series: with two frames in flight, several host-visible resources were rewritten by the CPU while the previous frame's GPU work could still be reading them. - The post-processing scratch UBO is now a PerFrameUboRing (per-frame-in- flight slot regions, persistently mapped, self-flushing alloc). All the per-pass map/unmap/cursor bookkeeping in bloom/LDR/fog/SMAA is deleted; the cursor resets exactly once per frame via the new VulkanPostProcessor::beginFrame() hook in setupFrame. This also fixes two latent bugs: (a) fog consumed scratch slots mid-scene and bloom's later same-frame cursor reset overwrote them before submission whenever fog+bloom were both active; (b) volumetric_fog_data (288 bytes) overflowed the 256-byte slots - the memcpy spilled into the neighboring slot and the shader's bound range ended before the aspect/fov/noise fields. Slots are now 512 bytes (static_assert at the fog call site) and the per-frame budget is 32 slots. - The dedicated tonemap, output-encode, and MSAA-resolve UBOs (each a single instance rewritten per frame, i.e. racy; the MSAA one also never flushed) are deleted entirely - those passes build their structs on the stack and allocate from the scratch ring. - Static-buffer rewrites orphan instead of racing: getVkBuffer() stamps a last-used frame number, and a full-content updateBufferData() on a Static buffer used within the last MAX_FRAMES_IN_FLIGHT frames defers the old VkBuffer to the deletion queue and writes a fresh one. Offset writes (GPU-heap appends to fresh regions) and PersistentMapping buffers (the engine holds their mapped pointer and syncs via gr_sync) are exempt. - gr_sync_wait now honors its contract: VulkanRenderFrame::waitForFinish takes a timeout and reports expiry without consuming the fence; VulkanRenderer::waitForFrame returns false for fences taken during the still-recording frame (previously it waited on a stale or idle slot and reported success) and resolves the frame->slot mapping correctly. This makes UniformBufferManager's segment-fence throttling actually protective on Vulkan for the first time. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ing logic, improved resource management across Vulkan post-processing subsystems.
…tion updates - Unified Vulkan validation flag handling in setup, separating engine debug output from synchronization validation. - Specified access masks and stage flags to address cross-frame WRITE_AFTER_WRITE vulnerabilities, notably in G-buffer, post-processing, and MSAA resolve paths. - Adjusted subpass dependencies and image layout transitions to prevent read-after-write and write-after-write hazards, detected using `-gr_sync_validation`. - Fixed uniform buffer and barrier transitions to ensure proper ordering in render passes across frames.
…and future testing - Clarified dependency logic for cross-pass and cross-frame synchronization, ensuring explicit stage and access flag comments match observed behavior. - Adjusted subpass dependencies to decouple render-pass compatibility definitions from dependency assumptions. - Highlighted MoltenVK-specific issues and retained workarounds with notes for re-testing on macOS hardware. - Updated synchronization barriers and validation notes to address historical inconsistencies across platforms.
…aphics queue - Removed transfer queue handling, simplifying buffer and queue logic to use the graphics queue for all transfer operations. - Updated buffer sharing modes and removed related code paths and parameters for transfer queue families. - Introduced exclusive buffer access, noting async transfer and queue ownership transfers are future work. - Limited enabled device features to only those actively used by the renderer, reducing unnecessary feature requests (e.g., robustBufferAccess).
- Removed `endFrame()` from `VulkanDescriptorManager`, replacing it with `setCurrentFrame(frameIndex)` for explicit frame synchronization. - Centralized frame index advancement within `VulkanRenderer` to ensure consistency across components. - Invalidated cached descriptor set bindings at render-pass boundaries to handle raw recorder staleness correctly. - Optimized pipeline descriptor rebind logic by removing redundant clears during pipeline changes.
…for improved hashing and memoization efficiency - Replaced inefficient shift-based hashing in `PipelineConfig::hash` with a robust `hashCombine` strategy to minimize bucket collisions. - Introduced `maskBits` helper to streamline bitmask creation for boolean fields. - Implemented memoization for Global (Set 0) descriptor sets, reducing redundant rebuilds across frames and improving performance. - Added dirty state tracking (`m_globalSetDirty`) and invalidation logic for Global set updates following relevant input changes (e.g., TLAS or UBO modifications).
…eToSwapChainPass` method to streamline HDR and SDR output paths.
- Added memoization for Material (Set 1) and PerDraw (Set 2) descriptor sets to reuse cached sets across frames if inputs remain unchanged. - Implemented precise input tracking for Material and PerDraw descriptor sets to detect and rebuild only when necessary. - Reduced redundant Vulkan descriptor allocations and writes, improving performance on repeated or batched draw calls.
- Added `eColorAttachmentRead` to access masks in multiple barriers to ensure proper synchronization for load operations (e.g., `loadOp=eLoad`) across render passes. - Prevented `READ_AFTER_WRITE` synchronization validation issues in deferred and multi-pass rendering scenarios. - Updated comments to clarify the reasoning behind the access mask changes and load dependencies.
- Consolidated destruction queue methods in `VulkanDeletionQueue` using a shared `queueHandle` template to reduce redundancy. - Introduced `ModelDrawParams` structure and `computeModelDrawParams` function to unify indexed draw parameters for models and shadows. - Simplified draw logic by eliminating repeated calculation of indexed draw parameters and improving code reuse.
…ion texture arrays - Introduced `releaseAnimationSlotRef` to manage reference counting for shared animation texture-array slots, preventing premature destruction of shared resources. - Updated `flushTextures` and related methods to integrate `releaseAnimationSlotRef` for shared slot cleanup. - Refactored staging buffer creation into `createStagingBuffer` for 3D texture uploads and shared path consistency. - Enhanced texture upload logic with improved tracking and logging for re-upload cases, aiding preloading gap analysis. - Added instrumentation for on-demand mid-frame texture uploads in `FrameStats`.
- Updated smart pointer usage for consistency across Vulkan resource managers (`std::make_unique`). - Removed redundant `cubeImageView` member and related cleanup logic; unified sampling view management. - Added per-instance debug log counters (avoiding static state) for Vulkan texture uploads and bindings, improving visibility during initialization and run-time. - Optimized `VulkanRenderer` property/feature getters by caching physical device properties and features during device creation. - Improved memory region flushing logic for out-of-date offsets, aligning with Vulkan Memory Allocator behavior. - Refactored render pass state tracking to reapply dynamic state reliably after mid-frame render passes. - Enhanced validation and error handling for mismatched update texture formats.
…dering in Vulkan - Corrected bitmap lock type from `BMP_TEX_OTHER` to `BMP_AABITMAP` for YUV textures during on-demand loading. - Updated movie rendering to bind alpha from the `MovieData` UBO (set 2, binding 4) for Vulkan, aligning behavior with OpenGL implementation.
4 participants
Add Vulkan rendering backend, with HDR10 output and raytraced shadows
Summary
Adds a Vulkan rendering backend to FSO, reaching feature parity with the existing OpenGL backend (including the cascaded-shadow-map overhaul from #7529) and adding two capabilities that are currently Vulkan-only: true HDR10 (PQ / BT.2020) display output and hardware-raytraced shadows. OpenGL behavior is unchanged;
-vulkan/Graphics.APIopts into the new backend,-opengl(or simply not requesting Vulkan) keeps the old path.The original Vulkan port is Mara van der Laan's (laanwj); this PR carries it, and the shadow/HDR/raytracing work built on top of it, to mergeable state. Additional porting work by taylor, Swifty, BMagnu and The_E (credits as shown in qtFRED's About dialog).
Command line options
-vulkan-opengl-vulkandefault without editing settings.-rt_shadows-gr_sync_validation-gr_debugoverlays.New in-game options (Options → Graphics, Advanced level)
Also adds two new screens under SCP Options: an HDR Calibration screen (status banner, live paper-white/peak-luminance sliders, grayscale ramp and RGB/white patches for checking the BT.709→BT.2020 mapping) and a Log Filters window (toggle log categories, browse the session log in-game).
What's included
Vulkan renderer core — swap chain/device setup, pipeline and descriptor-set management (with per-frame memoization), a per-frame UBO ring, an explicit cross-frame sync model with proper access masks (opt-in validation via
-gr_sync_validation), centralized framebuffer/render-target lifecycle management, and a debug-utils messenger withGR_DEBUG_SCOPEmarkers throughout.HDR10 output (
VulkanRendererSetup.cpp) — negotiates anA2B10G10R10/HDR10_ST2084swap chain when requested (falling back cleanly to SDR/sRGB otherwise), submitsVkHdrMetadataEXT, and routes all rendering through an intermediate fp16 composition buffer with a final encode pass (SDR passthrough or HDR10 PQ encode). Gamma correction now applies consistently across both output paths.Raytraced shadows — new
VulkanRaytracingManager(BLAS caching keyed to model lifecycle, per-frame TLAS build/rebuild with dynamic capacity growth) and the buffer/device-address plumbing acceleration structures need. Exposed as an alternateShadow Methodalongside the existing cascaded shadow maps; gated behind aCAPABILITY_RAYTRACED_SHADOWSquery so it only appears where the backend/GPU actually supports it.Vulkan shadow-map parity — brings Vulkan up to par with the OpenGL cascaded-shadow-map overhaul from #7529: dynamic cascade counts, cockpit-geometry-specific cascades, and hardware PCF via
sampler2DArrayShadow(replacing the old VSM approach), sharing cascade-count defines and shader code with the OpenGL path.SMAA on Vulkan — Vulkan previously had no SMAA implementation at all; selecting an SMAA preset while running Vulkan silently produced no anti-aliasing. All three passes (edge detection, blending weights, neighborhood blending) are now implemented. FXAA and SMAA also now work correctly with HDR10 output active (their luma thresholds assume
[0,1]input, so edge/luma detection reads a separately tonemapped proxy buffer while blending still uses the real extended-range colors).HDR-safe framebuffer readback — adds an fp16 → BGRA8 conversion path so screenshots and other 8-bit-assuming consumers keep working with HDR display output enabled.
qtFRED — builds against the same renderer abstraction; "Ported to Vulkan" credits added to the About dialog. qtFRED itself still always initializes OpenGL regardless of
-vulkan(its Qt-based windowing has no Vulkan surface path yet — documented inqtfred/README.mdfor whoever picks that up next).Logging — Vulkan-side logging calls unified on
mprintf/scopednprintfcategories, feeding the new in-game Log Filters window.Fixes picked up along the way
gr_uniform_buffer_managers_init()was only called from the OpenGL init path, leavingUniformBufferManagernull under Vulkan and crashing on the first draw needing a uniform buffer.#ifndef VULKAN.ccache: No such file or directory— the workflow hardcoded the Intel Homebrew ccache path, which doesn't exist on Apple Silicon runners;configure_cmake.shnow falls back to a PATH-resolvedccache.Notes / limitations
QT_QPA_PLATFORM=xcbworks around it. Both are pre-existing qtFRED limitations, not regressions from this PR.Credits