Compare commits

...

31 Commits

Author SHA1 Message Date
CamilleLaVey 8d19ee6dc5 What if we just rely entirely on SGSR common than using EdgeDir? 2026-08-26 19:06:23 -04:00
CamilleLaVey 3e91569d36 Another intent 2026-08-26 18:40:10 -04:00
CamilleLaVey 6010f60803 Just cries internally 2026-08-26 18:05:03 -04:00
CamilleLaVey c14b1de71e Meow. 2026-08-26 17:47:12 -04:00
CamilleLaVey 14cc49e727 Adjustments on upload for textures 2026-08-26 17:00:18 -04:00
CamilleLaVey e88923331c Fix build 2026-08-26 15:04:18 -04:00
CamilleLaVey 78918e995d Fix msvc build 2026-08-26 14:50:42 -04:00
CamilleLaVey 522440bcb2 [TEST] Bring the unswizzle work from tiled-gpu-v2 x2 + revert of vertex, indexes from AHB path 2026-08-26 14:49:01 -04:00
CamilleLaVey c1d8b7dd42 Just a quick test 2026-08-26 05:09:12 -04:00
CamilleLaVey 28d7c97d71 Added vertex, uniform and indexes on the upload path 2026-08-26 04:52:52 -04:00
CamilleLaVey 8a8763d975 Meowly The Build Destroyer 2026-08-26 04:11:59 -04:00
CamilleLaVey 54d8d5b0a8 Another intent on write/read on the fly 2026-08-26 04:06:25 -04:00
CamilleLaVey b9b9a0d704 Changes on the maps counts again 2026-08-26 03:12:06 -04:00
CamilleLaVey b8ce711575 Step back + keeping fixes from unified download + shared mem refactor 2026-08-25 21:36:17 -04:00
CamilleLaVey 46cd1e5d65 another take 2026-08-25 21:36:17 -04:00
CamilleLaVey ae561d49aa Keep AHB limitations 2026-08-25 21:36:17 -04:00
CamilleLaVey 9f45d357e9 [TEST] Another increased on windows size 2026-08-25 21:36:17 -04:00
CamilleLaVey 2c8c3062c4 Increased windows size 2026-08-25 21:36:17 -04:00
CamilleLaVey 51bdb62d74 [TEST] Increased window limiter on AHB 2026-08-25 21:36:17 -04:00
CamilleLaVey ed3838ccbb [TEST] Adjustment AHB to tiled-gpu-v2 state 2026-08-25 21:36:17 -04:00
CamilleLaVey e8c253f511 [memory, vulkan] Initial implementation for Unified Memory 2026-08-25 21:36:17 -04:00
CamilleLaVey 0295dc5fff [vulkan] Removal of QCOM sampler limiters + CustomBorderColor and ColorBorderSwizzle adjustments (#4301)
This PR removes the artificial limit added for QCOM drivers based on a sampler budget limit (based on #3280 work), removes the ban on CustomBorderColor/BorderColorSwizzle for also QCOM driver, reduce the amount of combination required to created a sampler with custom border color/ color border swizzle; adds cases on R16 formats not able to swizzle on BGR565 formats; degrades samplers and color combinations when there's no color border swizzle available; fixes the srgb (LUT) values from legacy and now all the color variations are cached in the same key, instead of having 7 different handlers for them. In resume, rather than performance (due to the reduced amount of process on duplicated/ synced code on pipeline/texture cache), it's a graphica accuracy work that will improve image quality at the cost of almost 0 performance hit, rather than certain hitch if the color combination wasn't cached before. This applies to all platform.

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4301
Reviewed-by: Samuel <lizzie@eden-emu.dev>
Reviewed-by: MaranBr <maranbr@eden-emu.dev>
2026-08-25 20:03:18 +02:00
CamilleLaVey 60a474b8df [vulkan, qcom] Fix shader float controls on QCOM driver (#4297)
Finishes my torture to find the culprit behind the broken behavior with float controls, even if device does support flush denorm on fp32 it doesn't really support it and provokes bad rounding modes by not flushing correctly denorms; this also returns the other working rounding modes on QCOM drivers.

_Special Thanks_

1.- Smoly The Big (@Gidoly)

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4297
Reviewed-by: Samuel <lizzie@eden-emu.dev>
Reviewed-by: MaranBr <maranbr@eden-emu.dev>
2026-08-24 17:57:12 +02:00
lizzie d9336ce6ce [common, settings] Fixes misaligned Aniso levels (#4295)
PR/Commit Owner: CamilleLaVey

Just quick fixes to ANISO bugged level selector, removes x32 and x64 from choices, also fixes the none value to actully use default.

Co-authored-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4295
Reviewed-by: Maufeat <sahyno1996@gmail.com>
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-24 00:06:11 +02:00
PavelBARABANOV 4eb8dd1458 partial revert a41a98028a (#4292)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------
This commit fix launch Assassin's creed 3

changes nvdrv:a - break launch with fw 20+
changes ldr:ro - break skip loading menu with all fw

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4292
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-by: Maufeat <sahyno1996@gmail.com>
2026-08-24 00:01:21 +02:00
PavelBARABANOV 4a3cc9a3c2 [android] Fix Nvdec None (#4288)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------
Nvdec None is working now; I removed the unnecessary comments — the variable names make everything clear enough.

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4288
Reviewed-by: Samuel <lizzie@eden-emu.dev>
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-23 23:58:50 +02:00
Samuel 86ba79b1ba [vulkan, sgsr] Adjustments to SGSR implementation (#4293)
PR/ Commit Owner: CamiileLaVey

This change has specified fixes on the SGSR initial implementation, it brought a wider sharpening level limits and better image quality.

Co-authored-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4293
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-by: crueter <crueter@eden-emu.dev>
2026-08-23 23:36:24 +02:00
lizzie b610b03d29 [common/logging] eliminate uneeded std::string{} allocations per each logging (#4273)
Signed-off-by: lizzie <lizzie@eden-emu.dev>

- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

the idea is simple, `std::string{}` adds pressure to the memory allocator
so whats the best next thing we can do? well of course use our stack
its plenty, but we shouldn't be greedy either, BUFSIZ should be a fair amount of
space for any would-be messages anyways

the main idea behind this PR is to remove std::string{} allocations
done with libfmt, that way we have 0-allocs per logging entry
this should 100% remove pressure uneeded from the memory allocator
we should only allocate things that are important, we can use our
trusty fast stack for any string manip we need to do
not await/global lock or do evil things with the memory allocator

obviously stack is thread local already sooo... this is even better
than having to deal with malloc()/free() in any capacity whatsoever

and no clang can't heap ellide this (how would you even?)

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4273
Reviewed-by: Maufeat <sahyno1996@gmail.com>
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-23 22:37:17 +02:00
lizzie 421e0b834c Revert "[dynarmic] Remove Ignore Global Monitor from CPU Accuracy Auto (#3846)" (#4276)
This reverts commit 3a823de605.

- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

kills performance and doesn't fix graphical glitches
(according to gido)
"It improved nothing but kills performance on default setting"

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4276
Reviewed-by: Maufeat <sahyno1996@gmail.com>
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-23 22:10:22 +02:00
John 55acf5d260 [Vulkan] Apply MK8D Buffer Fix to Linux (#4289)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

Credit: MaranBR
- Adds the Rainbow Graphics Bug Fix to Linux as the game needs MAX_STREAM_BUFFER_SIZE = 256_MiB

![image](/attachments/722d4014-9449-47a0-be47-730b99e0b464)

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4289
Reviewed-by: Maufeat <sahyno1996@gmail.com>
Reviewed-by: Samuel <lizzie@eden-emu.dev>
2026-08-23 17:43:35 +02:00
crueter 3e07b466eb [vulkan] Adjustments on MSAA and BlitHelpers (#4287)
PR/Commit Owner: CamilleLaVey

This PR contains changes complementary to the MSAA refactor from some weeks ago, adds proper shader convert to depth, stencil for msaa and non msaa convertions; removes redundant helpers and unify paths on the resolve functions, just to make the readability and maintainability better; includes fixes for the blit operations on Nvidia, Intel/Windows (previously banned) and ensures Linux retain the fix without extra burden; fixes the QCOM driver bug on resolution upscaling on any games above x1 (includes turnip on A8xx series) and fixes crashes/ resolution upscaled bugs on certain games that used to have wrong viewporting effect on screen or directly device loss on Vulkan. Adds fixes for regressions caused by previous MSAA refactor on games as Fire Emblem: Engage.

Special Thanks

1.- Big Smolio (@Gidoly)

Co-authored-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4287
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-by: Samuel <lizzie@eden-emu.dev>
2026-08-23 17:31:09 +02:00
91 changed files with 4730 additions and 1112 deletions
@@ -27,6 +27,7 @@ enum class BooleanSetting(override val key: String) : AbstractBooleanSetting {
RENDERER_ASYNCHRONOUS_GPU_EMULATION("use_asynchronous_gpu_emulation"),
RENDERER_ASYNC_PRESENTATION("async_presentation"),
RENDERER_ASYNCHRONOUS_SHADERS("use_asynchronous_shaders"),
RENDERER_UNIFIED_MEMORY("use_unified_memory"),
RENDERER_REACTIVE_FLUSHING("use_reactive_flushing"),
ENABLE_BUFFER_HISTORY("enable_buffer_history"),
USE_OPTIMIZED_VERTEX_BUFFERS("use_optimized_vertex_buffers"),
@@ -624,6 +624,7 @@ abstract class SettingsItem(
IntSetting.FSR_SHARPENING_SLIDER,
titleId = R.string.fsr_sharpness,
descriptionId = R.string.fsr_sharpness_description,
max = 200,
units = "%"
)
)
@@ -788,6 +789,13 @@ abstract class SettingsItem(
descriptionId = R.string.renderer_asynchronous_shaders_description
)
)
put(
SwitchSetting(
BooleanSetting.RENDERER_UNIFIED_MEMORY,
titleId = R.string.renderer_unified_memory,
descriptionId = R.string.renderer_unified_memory_description
)
)
put(
SingleChoiceSetting(
IntSetting.FAST_GPU_TIME,
@@ -344,6 +344,7 @@ class SettingsFragmentPresenter(
add(BooleanSetting.FIX_BLOOM_EFFECTS.key)
add(BooleanSetting.EMULATE_BGR565.key)
add(BooleanSetting.RENDERER_ASYNCHRONOUS_SHADERS.key)
add(BooleanSetting.RENDERER_UNIFIED_MEMORY.key)
add(IntSetting.ANDROID_PIPELINE_WORKERS.key)
add(BooleanSetting.RENDERER_ASYNCHRONOUS_GPU_EMULATION.key)
add(BooleanSetting.RENDERER_ASYNC_PRESENTATION.key)
@@ -1182,7 +1182,7 @@ class EmulationFragment : Fragment(), SurfaceHolder.Callback {
container,
IntSetting.FSR_SHARPENING_SLIDER,
minValue = 0,
maxValue = 100,
maxValue = 200,
units = "%"
)
}
@@ -1107,7 +1107,6 @@
<string name="theme_mode_light">فاتح</string>
<string name="theme_mode_dark">داكن</string>
<string name="multiplier_none">لا شيء</string>
<!-- Black backgrounds theme -->
<string name="use_black_backgrounds">خلفيات سوداء</string>
@@ -991,7 +991,6 @@ Wirklich fortfahren?</string>
<string name="theme_mode_light">Hell</string>
<string name="theme_mode_dark">Dunkel</string>
<string name="multiplier_none">Keine</string>
<!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Schwarze Hintergründe</string>
@@ -1092,7 +1092,6 @@
<string name="theme_mode_light">Claro</string>
<string name="theme_mode_dark">Oscuro</string>
<string name="multiplier_none">Nada</string>
<!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Fondos oscuros</string>
@@ -808,9 +808,6 @@
<string name="multiplier_x4">x4</string>
<string name="multiplier_x8">x8</string>
<string name="multiplier_x16">x16</string>
<string name="multiplier_x32">x32</string>
<string name="multiplier_x64">x64</string>
<string name="multiplier_none">None</string>
<!-- Black backgrounds theme -->
<string name="use_black_backgrounds">پس‌زمینه مشکی</string>
@@ -1004,7 +1004,6 @@
<string name="theme_mode_light">Lumineux</string>
<string name="theme_mode_dark">Sombre</string>
<string name="multiplier_none">Aucun</string>
<!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Arrière-plan noir</string>
@@ -935,7 +935,6 @@
<string name="theme_mode_light">Jasny</string>
<string name="theme_mode_dark">Ciemny</string>
<string name="multiplier_none">Brak</string>
<!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Czarne tła</string>
@@ -891,7 +891,6 @@
<string name="theme_mode_light">Claro</string>
<string name="theme_mode_dark">Escuro</string>
<string name="multiplier_none">Nenhum</string>
<!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Planos de fundo pretos</string>
@@ -1071,7 +1071,6 @@
<string name="theme_mode_light">Светлая</string>
<string name="theme_mode_dark">Темная</string>
<string name="multiplier_none">Отключено</string>
<!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Чёрный фон</string>
@@ -1053,7 +1053,6 @@
<string name="theme_mode_light">Світла</string>
<string name="theme_mode_dark">Темна</string>
<string name="multiplier_none">Жодного</string>
<!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Чорний фон</string>
@@ -1081,7 +1081,6 @@
<string name="theme_mode_light">浅色</string>
<string name="theme_mode_dark">深色</string>
<string name="multiplier_none"></string>
<!-- Black backgrounds theme -->
<string name="use_black_backgrounds">使用黑色背景</string>
@@ -1006,7 +1006,6 @@
<string name="theme_mode_light">淺色</string>
<string name="theme_mode_dark">深色</string>
<string name="multiplier_none"></string>
<!-- Black backgrounds theme -->
<string name="use_black_backgrounds">黑色背景</string>
+12 -23
View File
@@ -111,43 +111,38 @@
<item>1</item>
</integer-array>
<!-- VRAM USAGE MODE CHOICES -->
<string-array name="vramUsageMethodNames">
<item>@string/vram_usage_conservative</item>
<item>@string/vram_usage_aggressive</item>
</string-array>
<!-- VRAM USAGE MODE VALUES -->
<integer-array name="vramUsageMethodValues">
<item>0</item> <!-- Conservative -->
<item>1</item> <!-- Aggressive -->
<item>0</item>
<item>1</item>
</integer-array>
<!-- ASTC Decoding Method Choices -->
<string-array name="astcDecodingMethodNames">
<item>@string/accelerate_astc_cpu</item>
<item>@string/accelerate_astc_gpu</item>
<item>@string/accelerate_astc_async</item>
</string-array>
<!-- ASTC Decoding Method Values -->
<integer-array name="astcDecodingMethodValues">
<item>0</item> <!-- CPU -->
<item>1</item> <!-- GPU -->
<item>2</item> <!-- CPU Asynchronously -->
<item>0</item>
<item>1</item>
<item>2</item>
</integer-array>
<!-- NVDEC Emulation Choices -->
<string-array name="rendererNvdecNames">
<item>@string/nvdec_emulation_none</item> <!-- Off -->
<item>@string/nvdec_emulation_cpu</item> <!-- Cpu -->
<item>@string/nvdec_emulation_gpu</item> <!-- Gpu -->
<item>@string/nvdec_emulation_none</item>
<item>@string/nvdec_emulation_cpu</item>
<item>@string/nvdec_emulation_gpu</item>
</string-array>
<!-- NVDEC Emulation Values -->
<integer-array name="rendererNvdecValues">
<item>3</item> <!-- Off value -->
<item>1</item> <!-- CPU value -->
<item>2</item> <!-- GPU value -->
<item>0</item>
<item>1</item>
<item>2</item>
</integer-array>
<string-array name="rendererResolutionNames">
@@ -513,9 +508,6 @@
<item>@string/multiplier_x4</item>
<item>@string/multiplier_x8</item>
<item>@string/multiplier_x16</item>
<item>@string/multiplier_x32</item>
<item>@string/multiplier_x64</item>
<item>@string/multiplier_none</item>
</string-array>
<integer-array name="anisoValues">
<item>0</item>
@@ -524,9 +516,6 @@
<item>3</item>
<item>4</item>
<item>5</item>
<item>6</item>
<item>7</item>
<item>8</item>
</integer-array>
<string-array name="verticalAlignmentEntries">
@@ -585,6 +585,8 @@
<string name="rescale_hack_description">Enables a legacy handling for the rescale configuration pass for games by using a quick rescale path</string>
<string name="renderer_asynchronous_shaders">Use asynchronous shaders</string>
<string name="renderer_asynchronous_shaders_description">Compiles shaders asynchronously. This may reduce stutters but may also introduce glitches.</string>
<string name="renderer_unified_memory">Unified memory access</string>
<string name="renderer_unified_memory_description">Allows GPU write buffer readbacks directly into guest memory, skipping the CPU staging copy.</string>
<string name="gpu_unswizzle_settings">GPU Unswizzle Settings</string>
<string name="gpu_unswizzle_settings_description">Configure GPU-based texture unswizzling parameters or disable it entirely. Adjust these settings to balance performance and texture loading quality.</string>
<string name="gpu_unswizzle_enable">Enable GPU Unswizzle</string>
@@ -1246,9 +1248,6 @@
<string name="multiplier_x4" translatable="false">x4</string>
<string name="multiplier_x8" translatable="false">x8</string>
<string name="multiplier_x16" translatable="false">x16</string>
<string name="multiplier_x32" translatable="false">x32</string>
<string name="multiplier_x64" translatable="false">x64</string>
<string name="multiplier_none">None</string>
<!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Black backgrounds</string>
+22 -28
View File
@@ -4,34 +4,17 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <fstream>
#include "common/heap_tracker.h"
#include "common/logging.h"
#include "common/memory_detect.h"
#include "common/assert.h"
namespace Common {
namespace {
s64 GetMaxPermissibleResidentMapCount() {
// Default value.
s64 value = 65530;
// Try to read how many mappings we can make.
std::ifstream s("/proc/sys/vm/max_map_count");
s >> value;
// Print, for debug.
LOG_INFO(HW_Memory, "Current maximum map count: {}", value);
// Allow 20000 maps for other code and to account for split inaccuracy.
return std::max<s64>(value - 20000, 0);
}
} // namespace
HeapTracker::HeapTracker(Common::HostMemory& buffer)
: m_buffer(buffer), m_max_resident_map_count(GetMaxPermissibleResidentMapCount()) {}
: m_buffer(buffer),
m_has_hardware_buffer_backing(!buffer.BackingHardwareBuffers().empty()),
m_max_resident_map_count(static_cast<s64>(GetPermissibleMapCount())) {}
HeapTracker::~HeapTracker() = default;
void HeapTracker::Map(size_t virtual_offset, size_t host_offset, size_t length,
@@ -85,7 +68,8 @@ void HeapTracker::Unmap(size_t virtual_offset, size_t size, bool is_separate_hea
// If resident, erase from resident map.
if (item->is_resident) {
ASSERT(--m_resident_map_count >= 0);
m_resident_map_count -= this->HostMapCount(item->paddr, item->size);
ASSERT(m_resident_map_count >= 0);
m_resident_mappings.erase(m_resident_mappings.iterator_to(*item));
}
@@ -191,7 +175,7 @@ bool HeapTracker::DeferredMapSeparateHeap(size_t virtual_offset) {
// This map is now resident.
it->is_resident = true;
m_resident_map_count++;
m_resident_map_count += this->HostMapCount(it->paddr, it->size);
m_resident_mappings.insert(*it);
}
@@ -213,17 +197,17 @@ void HeapTracker::RebuildSeparateHeapAddressSpace() {
// Despite being worse in theory, this has proven to be better in practice than more
// regularly dumping a smaller amount, because it significantly reduces average case
// lock contention.
std::size_t const desired_count = (std::min)(m_resident_map_count, m_max_resident_map_count) / 2;
std::size_t const evict_count = m_resident_map_count - desired_count;
s64 const desired_count = (std::min)(m_resident_map_count, m_max_resident_map_count) / 2;
auto it = m_resident_mappings.begin();
for (size_t i = 0; i < evict_count && it != m_resident_mappings.end(); i++) {
while (m_resident_map_count > desired_count && it != m_resident_mappings.end()) {
// Unmark and unmap.
it->is_resident = false;
m_buffer.Unmap(it->vaddr, it->size, false);
// Advance.
ASSERT(--m_resident_map_count >= 0);
m_resident_map_count -= this->HostMapCount(it->paddr, it->size);
ASSERT(m_resident_map_count >= 0);
it = m_resident_mappings.erase(it);
}
}
@@ -245,6 +229,7 @@ void HeapTracker::SplitHeapMapLocked(VAddr offset) {
// Cache the original values.
auto* const left = std::addressof(*it);
const size_t orig_size = left->size;
const s64 orig_host_map_count = this->HostMapCount(left->paddr, orig_size);
// Adjust the left map.
const size_t left_size = offset - left->vaddr;
@@ -266,11 +251,20 @@ void HeapTracker::SplitHeapMapLocked(VAddr offset) {
// If resident, also insert into resident map.
if (right->is_resident) {
m_resident_map_count++;
m_resident_map_count += this->HostMapCount(left->paddr, left->size) +
this->HostMapCount(right->paddr, right->size) -
orig_host_map_count;
m_resident_mappings.insert(*right);
}
}
s64 HeapTracker::HostMapCount(PAddr paddr, size_t size) const {
if (!m_has_hardware_buffer_backing) {
return size != 0 ? 1 : 0;
}
return static_cast<s64>(m_buffer.BackingMapCount(paddr, size));
}
HeapTracker::AddrTree::iterator HeapTracker::GetNearestHeapMapLocked(VAddr offset) {
const SeparateHeapMap key{
.vaddr = offset,
+6
View File
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
@@ -82,10 +85,13 @@ private:
AddrTree::iterator GetNearestHeapMapLocked(VAddr offset);
s64 HostMapCount(PAddr paddr, size_t size) const;
void RebuildSeparateHeapAddressSpace();
private:
Common::HostMemory& m_buffer;
const bool m_has_hardware_buffer_backing;
const s64 m_max_resident_map_count;
std::shared_mutex m_rebuild_lock{};
+389 -6
View File
@@ -51,14 +51,65 @@
#endif // ^^^ POSIX ^^^
#include <atomic>
#include <mutex>
#include <random>
#include <vector>
#include "common/alignment.h"
#include "common/assert.h"
#include "common/free_region_manager.h"
#include "common/host_memory.h"
#include "common/logging.h"
#include "common/memory_detect.h"
#include "common/settings.h"
#ifdef __ANDROID__
#include <cerrno>
#include <dlfcn.h>
#include <sys/ioctl.h>
#include <android/hardware_buffer.h>
namespace {
struct NativeHandle {
int version;
int numFds;
int numInts;
int data[1];
};
using PFN_AHardwareBuffer_getNativeHandle = const NativeHandle* (*)(const AHardwareBuffer*);
PFN_AHardwareBuffer_getNativeHandle ResolveGetNativeHandle() {
void* const lib = dlopen("libnativewindow.so", RTLD_NOW);
if (lib == nullptr) {
return nullptr;
}
return reinterpret_cast<PFN_AHardwareBuffer_getNativeHandle>(
dlsym(lib, "AHardwareBuffer_getNativeHandle"));
}
struct DmaBufSync {
u64 flags;
};
constexpr u64 DmaBufSyncRead = 1ULL << 0;
constexpr u64 DmaBufSyncWrite = 1ULL << 1;
constexpr u64 DmaBufSyncStart = 0ULL << 2;
constexpr u64 DmaBufSyncEnd = 1ULL << 2;
void SyncDmaBufCpuAccess(int fd, u64 phase) {
DmaBufSync sync{.flags = phase | DmaBufSyncRead | DmaBufSyncWrite};
while (ioctl(fd, _IOW('b', 0, DmaBufSync), &sync) != 0) {
if (errno != EINTR) {
return;
}
}
}
} // namespace
#endif
#if defined(__ANDROID__) && __ANDROID_API__ < 30
#include <sys/syscall.h>
@@ -75,6 +126,12 @@ namespace Common {
[[maybe_unused]] constexpr size_t PageAlignment = 0x1000;
[[maybe_unused]] constexpr size_t HugePageSize = 0x200000;
static std::atomic<u64> committed_backing_size{};
u64 GetCommittedBackingSize() noexcept {
return committed_backing_size.load(std::memory_order_relaxed);
}
#ifdef _WIN32
// Manually imported for MinGW compatibility
@@ -123,7 +180,7 @@ static void GetFuncAddress(Common::DynamicLibrary& dll, const char* name, T& pfn
class HostMemory::Impl {
public:
explicit Impl(size_t backing_size_, size_t virtual_size_)
explicit Impl(size_t backing_size_, size_t virtual_size_, size_t)
: backing_size{backing_size_}
, virtual_size{virtual_size_}
, process{GetCurrentProcess()}
@@ -229,6 +286,10 @@ public:
UNREACHABLE();
}
bool IsBackingShared() const noexcept {
return true;
}
const size_t backing_size; ///< Size of the backing memory in bytes
const size_t virtual_size; ///< Size of the virtual address placeholder in bytes
@@ -501,9 +562,10 @@ static int shm_open_anon(int flags, mode_t mode) {
class HostMemory::Impl {
public:
explicit Impl(size_t backing_size_, size_t virtual_size_)
explicit Impl(size_t backing_size_, size_t virtual_size_, size_t preferred_offset_)
: backing_size{backing_size_}
, virtual_size{virtual_size_}
, preferred_offset{preferred_offset_}
{}
bool Init() {
@@ -543,10 +605,15 @@ public:
LOG_WARNING(Common_Memory, "Using private mappings instead of shared ones");
backing_base = static_cast<u8*>(mmap(nullptr, backing_size, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0));
if (fd > 0) {
fd = -1;
close(fd);
}
fd = -1;
} else {
#ifdef __ANDROID__
if (InitAhbBacking()) {
return InitVirtual();
}
#endif
backing_base = static_cast<u8*>(mmap(nullptr, backing_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0));
}
if (backing_base == MAP_FAILED) {
@@ -554,7 +621,10 @@ public:
return false;
}
// Virtual memory initialization
return InitVirtual();
}
bool InitVirtual() {
virtual_base = virtual_map_base = static_cast<u8*>(ChooseVirtualBase(virtual_size));
if (virtual_base == MAP_FAILED) {
LOG_CRITICAL(HW_Memory, "mmap failed: {}", strerror(errno));
@@ -567,6 +637,240 @@ public:
return true;
}
#ifdef __ANDROID__
static AHardwareBuffer_Desc MakeBlobDesc(size_t len) {
return AHardwareBuffer_Desc{
.width = static_cast<u32>(len),
.height = 1,
.layers = 1,
.format = AHARDWAREBUFFER_FORMAT_BLOB,
.usage = AHARDWAREBUFFER_USAGE_CPU_READ_OFTEN |
AHARDWAREBUFFER_USAGE_CPU_WRITE_OFTEN |
AHARDWAREBUFFER_USAGE_GPU_DATA_BUFFER,
.stride = 0,
.rfu0 = 0,
.rfu1 = 0,
};
}
static bool ProbeAhbBacking(PFN_AHardwareBuffer_getNativeHandle get_native_handle) {
const AHardwareBuffer_Desc desc = MakeBlobDesc(PageAlignment * 2);
AHardwareBuffer* buffer{};
if (AHardwareBuffer_allocate(&desc, &buffer) != 0 || buffer == nullptr) {
return false;
}
const NativeHandle* const handle = get_native_handle(buffer);
if (handle == nullptr || handle->numFds < 1) {
AHardwareBuffer_release(buffer);
return false;
}
const int probe_fd = handle->data[0];
bool ok = true;
const auto try_map = [&](int prot, off_t offset) {
if (!ok) {
return;
}
void* const ptr = mmap(nullptr, PageAlignment, prot, MAP_SHARED, probe_fd, offset);
if (ptr == MAP_FAILED) {
ok = false;
return;
}
munmap(ptr, PageAlignment);
};
try_map(PROT_READ | PROT_WRITE, 0);
try_map(PROT_READ | PROT_WRITE, static_cast<off_t>(PageAlignment));
#ifdef ARCHITECTURE_arm64
try_map(PROT_READ | PROT_EXEC, 0);
#endif
AHardwareBuffer_release(buffer);
return ok;
}
size_t ComputeAhbBudget(size_t window_size) const {
const u64 total_physical = Common::GetMemInfo().TotalPhysicalMemory;
constexpr u64 BaselineFootprint = 6ULL << 30;
if (total_physical <= BaselineFootprint) {
return 0;
}
const u64 permissible_maps = Common::GetPermissibleMapCount();
if (permissible_maps == 0) {
return 0;
}
u64 budget = (total_physical - BaselineFootprint) / 2;
constexpr u64 MapSlotsPerWindow = 64;
const u64 affordable_windows = permissible_maps / MapSlotsPerWindow;
budget = (std::min)(budget, affordable_windows * window_size);
budget = (std::min)(budget, static_cast<u64>(backing_size) / 2);
budget = Common::AlignDown(budget, window_size);
constexpr u64 MinimumBudget = 256ULL << 20;
if (budget < MinimumBudget) {
return 0;
}
return static_cast<size_t>(budget);
}
bool InitAhbBacking() {
if (!Settings::values.use_unified_memory.GetValue()) {
return false;
}
static const PFN_AHardwareBuffer_getNativeHandle get_native_handle =
ResolveGetNativeHandle();
if (get_native_handle == nullptr) {
return false;
}
constexpr size_t window_size = 256ULL << 20;
const size_t budget = ComputeAhbBudget(window_size);
if (budget == 0) {
return false;
}
if (!ProbeAhbBacking(get_native_handle)) {
return false;
}
const size_t aligned_backing = Common::AlignDown(backing_size, window_size);
const size_t max_windows = (std::min)(budget, aligned_backing) / window_size;
std::vector<AHardwareBuffer*> buffers;
std::vector<int> buffer_fds;
const auto cleanup = [&] {
for (AHardwareBuffer* buffer : buffers) {
AHardwareBuffer_release(buffer);
}
buffers.clear();
buffer_fds.clear();
};
for (size_t i = 0; i < max_windows; ++i) {
const AHardwareBuffer_Desc desc = MakeBlobDesc(window_size);
AHardwareBuffer* buffer{};
if (AHardwareBuffer_allocate(&desc, &buffer) != 0 || buffer == nullptr) {
break;
}
const NativeHandle* const handle = get_native_handle(buffer);
if (handle == nullptr || handle->numFds < 1) {
AHardwareBuffer_release(buffer);
break;
}
const int buffer_fd = handle->data[0];
const off_t buffer_len = lseek(buffer_fd, 0, SEEK_END);
if (buffer_len < static_cast<off_t>(window_size)) {
AHardwareBuffer_release(buffer);
break;
}
buffers.push_back(buffer);
buffer_fds.push_back(buffer_fd);
}
const size_t num_windows = buffers.size();
if (num_windows == 0) {
return false;
}
const size_t region_size = num_windows * window_size;
const size_t region_base = Common::AlignDown(
(std::min)(preferred_offset, aligned_backing - region_size), window_size);
u8* const base = static_cast<u8*>(mmap(nullptr, backing_size, PROT_NONE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0));
if (base == MAP_FAILED) {
cleanup();
return false;
}
const auto map_over_reservation = [&](size_t offset, size_t len, int map_fd,
off_t map_offset) {
if (len == 0) {
return true;
}
if (mmap(base + offset, len, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_FIXED, map_fd,
map_offset) == MAP_FAILED) {
munmap(base, backing_size);
cleanup();
return false;
}
return true;
};
if (!map_over_reservation(0, region_base, fd, 0)) {
return false;
}
for (size_t i = 0; i < num_windows; ++i) {
if (!map_over_reservation(region_base + i * window_size, window_size, buffer_fds[i],
0)) {
return false;
}
}
const size_t tail_offset = region_base + region_size;
if (!map_over_reservation(tail_offset, backing_size - tail_offset, fd,
static_cast<off_t>(tail_offset))) {
return false;
}
backing_base = base;
ahb_windows = std::move(buffers);
ahb_fds = std::move(buffer_fds);
ahb_window_size = window_size;
ahb_base = region_base;
ahb_bytes = region_size;
committed_backing_size.store(region_size, std::memory_order_relaxed);
for (const int window_fd : ahb_fds) {
SyncDmaBufCpuAccess(window_fd, DmaBufSyncStart);
}
return true;
}
void MapBackingRange(size_t virtual_offset, size_t host_offset, size_t length, int prot_flags) {
while (length > 0) {
int map_fd = fd;
off_t map_offset = static_cast<off_t>(host_offset);
size_t chunk = length;
if (host_offset < ahb_base) {
chunk = (std::min)(chunk, ahb_base - host_offset);
} else if (host_offset < ahb_base + ahb_bytes) {
const size_t relative = host_offset - ahb_base;
const size_t window = relative / ahb_window_size;
const size_t local = relative % ahb_window_size;
map_fd = ahb_fds[window];
map_offset = static_cast<off_t>(local);
chunk = (std::min)(chunk, ahb_window_size - local);
}
void* const ret = mmap(virtual_base + virtual_offset, chunk, prot_flags,
MAP_SHARED | MAP_FIXED, map_fd, map_offset);
ASSERT_MSG(ret != MAP_FAILED, "mmap: {}", strerror(errno));
virtual_offset += chunk;
host_offset += chunk;
length -= chunk;
}
}
size_t BackingMapCount(size_t host_offset, size_t length) const noexcept {
if (length == 0) {
return 0;
}
if (ahb_bytes == 0) {
return 1;
}
size_t count = 0;
while (length > 0) {
size_t chunk = length;
if (host_offset < ahb_base) {
chunk = (std::min)(chunk, ahb_base - host_offset);
} else if (host_offset < ahb_base + ahb_bytes) {
const size_t local = (host_offset - ahb_base) % ahb_window_size;
chunk = (std::min)(chunk, ahb_window_size - local);
}
host_offset += chunk;
length -= chunk;
++count;
}
return count;
}
std::span<AHardwareBuffer* const> AhbWindows() const noexcept {
return ahb_windows;
}
size_t AhbWindowSize() const noexcept {
return ahb_bytes != 0 ? ahb_window_size : 0;
}
size_t AhbBase() const noexcept {
return ahb_base;
}
#endif
~Impl() {
Release();
}
@@ -587,6 +891,12 @@ public:
#ifdef ARCHITECTURE_arm64
if (True(perms & MemoryPermission::Execute))
prot_flags |= PROT_EXEC;
#endif
#ifdef __ANDROID__
if (ahb_bytes != 0) {
MapBackingRange(virtual_offset, host_offset, length, prot_flags);
return;
}
#endif
int flags = (fd >= 0 ? MAP_SHARED : MAP_PRIVATE) | MAP_FIXED;
void* ret = mmap(virtual_base + virtual_offset, length, prot_flags, flags, fd, host_offset);
@@ -632,8 +942,18 @@ public:
virtual_base = nullptr;
}
bool IsBackingShared() const noexcept {
#ifdef __ANDROID__
if (ahb_bytes != 0) {
return true;
}
#endif
return fd >= 0;
}
const size_t backing_size; ///< Size of the backing memory in bytes
const size_t virtual_size; ///< Size of the virtual address placeholder in bytes
const size_t preferred_offset;
u8* backing_base{reinterpret_cast<u8*>(MAP_FAILED)};
u8* virtual_base{reinterpret_cast<u8*>(MAP_FAILED)};
@@ -656,6 +976,21 @@ private:
int ret = close(fd);
ASSERT_MSG(ret == 0, "close failed: {}", strerror(errno));
}
#ifdef __ANDROID__
for (const int window_fd : ahb_fds) {
SyncDmaBufCpuAccess(window_fd, DmaBufSyncEnd);
}
for (AHardwareBuffer* buffer : ahb_windows) {
AHardwareBuffer_release(buffer);
}
ahb_windows.clear();
ahb_fds.clear();
if (ahb_bytes != 0) {
committed_backing_size.store(0, std::memory_order_relaxed);
ahb_bytes = 0;
}
#endif
}
void AdjustMap(size_t* virtual_offset, size_t* length) {
@@ -681,11 +1016,19 @@ private:
int fd{-1}; // memfd file descriptor, -1 is the error value of memfd_create
FreeRegionManager free_manager{};
#ifdef __ANDROID__
std::vector<AHardwareBuffer*> ahb_windows;
std::vector<int> ahb_fds;
size_t ahb_window_size{};
size_t ahb_base{};
size_t ahb_bytes{};
#endif
};
#endif // ^^^ POSIX ^^^
HostMemory::HostMemory(size_t backing_size_, size_t virtual_size_)
HostMemory::HostMemory(size_t backing_size_, size_t virtual_size_, size_t preferred_offset_)
: backing_size(backing_size_)
, virtual_size(virtual_size_)
{
@@ -697,7 +1040,7 @@ HostMemory::HostMemory(size_t backing_size_, size_t virtual_size_)
#else
// Try to allocate a fastmem arena.
// The implementation will fail with std::bad_alloc on errors.
impl = std::make_unique<HostMemory::Impl>(AlignUp(backing_size, PageAlignment), AlignUp(virtual_size, PageAlignment) + HugePageSize);
impl = std::make_unique<HostMemory::Impl>(AlignUp(backing_size, PageAlignment), AlignUp(virtual_size, PageAlignment) + HugePageSize, preferred_offset_);
if (impl->Init()) {
backing_base = impl->backing_base;
virtual_base = impl->virtual_base;
@@ -767,6 +1110,46 @@ void HostMemory::ClearBackingRegion(size_t physical_offset, size_t length, u32 f
std::memset(backing_base + physical_offset, fill_value, length);
}
std::span<AHardwareBuffer* const> HostMemory::BackingHardwareBuffers() const noexcept {
#ifdef __ANDROID__
return impl ? impl->AhbWindows() : std::span<AHardwareBuffer* const>{};
#else
return {};
#endif
}
size_t HostMemory::BackingMapCount(size_t host_offset, size_t length) const noexcept {
#ifdef __ANDROID__
return impl ? impl->BackingMapCount(host_offset, length) : (length != 0 ? 1 : 0);
#else
return length != 0 ? 1 : 0;
#endif
}
size_t HostMemory::BackingHardwareBufferWindowSize() const noexcept {
#ifdef __ANDROID__
return impl ? impl->AhbWindowSize() : 0;
#else
return 0;
#endif
}
bool HostMemory::IsBackingShared() const noexcept {
#if defined(__OPENORBIS__) || defined(__managarm__)
return false;
#else
return impl && impl->IsBackingShared();
#endif
}
size_t HostMemory::BackingHardwareBufferBase() const noexcept {
#ifdef __ANDROID__
return impl ? impl->AhbBase() : 0;
#else
return 0;
#endif
}
void HostMemory::EnableDirectMappedAddress() {
#if !(defined(__OPENORBIS__) || defined(__managarm__))
if (impl) {
+20 -1
View File
@@ -8,12 +8,17 @@
#include <memory>
#include <optional>
#include <span>
#include "common/common_funcs.h"
#include "common/common_types.h"
#include "common/virtual_buffer.h"
struct AHardwareBuffer;
namespace Common {
[[nodiscard]] u64 GetCommittedBackingSize() noexcept;
enum class MemoryPermission : u32 {
Read = 1 << 0,
Write = 1 << 1,
@@ -28,7 +33,7 @@ DECLARE_ENUM_FLAG_OPERATORS(MemoryPermission)
*/
class HostMemory {
public:
explicit HostMemory(size_t backing_size_, size_t virtual_size_);
explicit HostMemory(size_t backing_size_, size_t virtual_size_, size_t preferred_offset_ = 0);
~HostMemory();
/**
@@ -62,6 +67,20 @@ public:
return backing_base;
}
[[nodiscard]] size_t BackingSize() const noexcept {
return backing_size;
}
[[nodiscard]] size_t BackingMapCount(size_t host_offset, size_t length) const noexcept;
[[nodiscard]] std::span<AHardwareBuffer* const> BackingHardwareBuffers() const noexcept;
[[nodiscard]] size_t BackingHardwareBufferWindowSize() const noexcept;
[[nodiscard]] size_t BackingHardwareBufferBase() const noexcept;
[[nodiscard]] bool IsBackingShared() const noexcept;
[[nodiscard]] u8* VirtualBasePointer() noexcept {
return virtual_base;
}
+56 -45
View File
@@ -39,6 +39,19 @@
namespace Common::Log {
/// @brief A log entry. Log entries are store in a structured format to permit more varied output
/// formatting on different frontends, as well as facilitating filtering and aggregation.
struct Entry {
char const* message = nullptr;
size_t message_len = 0;
std::chrono::microseconds timestamp;
Class log_class{};
Level log_level{};
const char* filename = nullptr;
const char* function = nullptr;
uint32_t line_num = 0;
};
namespace {
/// @brief Returns the name of the passed log class as a C-string. Subclasses are separated by periods
@@ -70,8 +83,6 @@ const char* GetLevelName(Level log_level) {
}
}
}
// Some IDEs prefer <file>:<line> instead, so let's just do that :)
std::string FormatLogMessage(const Entry& entry) noexcept {
if (!entry.filename) return "";
@@ -79,10 +90,9 @@ std::string FormatLogMessage(const Entry& entry) noexcept {
auto const time_fractional = uint32_t(entry.timestamp.count() % 1000000);
auto const class_name = GetLogClassName(entry.log_class);
auto const level_name = GetLevelName(entry.log_level);
return fmt::format("[{:4d}.{:06d}] {} <{}> {}:{}:{}: {}", time_seconds, time_fractional, class_name, level_name, entry.filename, entry.line_num, entry.function, entry.message);
return fmt::format("[{:4d}.{:06d}] {} <{}> {}:{}:{}: {}\n", time_seconds, time_fractional, class_name, level_name, entry.filename, entry.line_num, entry.function, entry.message);
}
namespace {
template <typename It>
Level GetLevelByName(const It begin, const It end) {
for (u32 i = 0; i < u32(Level::Count); ++i) {
@@ -127,25 +137,6 @@ bool ParseFilterRule(Filter& instance, Iterator begin, Iterator end) {
instance.SetClassLevel(log_class, level);
return true;
}
} // Anonymous namespace
void Filter::ParseFilterString(std::string_view filter_view) {
auto clause_begin = filter_view.cbegin();
while (clause_begin != filter_view.cend()) {
auto clause_end = std::find(clause_begin, filter_view.cend(), ' ');
// If clause isn't empty
if (clause_end != clause_begin) {
ParseFilterRule(*this, clause_begin, clause_end);
}
if (clause_end != filter_view.cend()) {
// Skip over the whitespace
++clause_end;
}
clause_begin = clause_end;
}
}
namespace {
/// @brief Trims up to and including the last of ../, ..\, src/, src\ in a string
/// do not be fooled this isn't generating new strings on .rodata :)
@@ -208,7 +199,7 @@ struct ColorConsoleBackend final : public Backend {
}());
SetConsoleTextAttribute(console_handle, color);
auto const df = GetDirectFormatArgs(entry);
std::fprintf(stdout, CCB_PRINTF_FMT "\n", df.time_seconds, df.time_fractional, df.class_name, df.level_name, entry.filename, entry.line_num, entry.function, entry.message.c_str());
std::fprintf(stdout, CCB_PRINTF_FMT "\n", df.time_seconds, df.time_fractional, df.class_name, df.level_name, entry.filename, entry.line_num, entry.function, entry.message);
}
}
void Flush() noexcept override {}
@@ -220,22 +211,24 @@ struct ColorConsoleBackend final : public Backend {
~ColorConsoleBackend() noexcept override {}
void Write(const Entry& entry) noexcept override {
if (enabled) {
#define ESC "\x1b"
auto const color_str = [&entry]() -> const char* {
switch (entry.log_level) {
#define CCB_MAKE_COLOR_FMT(X) ESC X CCB_PRINTF_FMT ESC "[0m\n"
case Level::Debug: return CCB_MAKE_COLOR_FMT("[0;36m"); // Cyan
case Level::Info: return CCB_MAKE_COLOR_FMT("[0;37m"); // Bright gray
case Level::Warning: return CCB_MAKE_COLOR_FMT("[1;33m"); // Bright yellow
case Level::Error: return CCB_MAKE_COLOR_FMT("[1;31m"); // Bright red
case Level::Critical: return CCB_MAKE_COLOR_FMT("[1;35m"); // Bright magenta
default: return CCB_MAKE_COLOR_FMT("[1;30m"); // Grey
#undef CCB_MAKE_COLOR_FMT
case Level::Debug: return "[0;36m"; // Cyan
case Level::Info: return "[0;37m"; // Bright gray
case Level::Warning: return "[1;33m"; // Bright yellow
case Level::Error: return "[1;31m"; // Bright red
case Level::Critical: return "[1;35m"; // Bright magenta
default: return "[1;30m"; // Grey
}
}();
auto const df = GetDirectFormatArgs(entry);
std::fprintf(stdout, color_str, df.time_seconds, df.time_fractional, df.class_name, df.level_name, entry.filename, entry.line_num, entry.function, entry.message.c_str());
#undef ESC
// more restrictive, because take for example this simple prelude:
// [ 50.872256] Config <Info> common/settings.cpp:142:LogSettings:
char buffer[128];
auto result = fmt::format_to_n(buffer, sizeof(buffer) - 1, "\x1b{}[{:4d}.{:06d}] {} <{}> {}:{}:{}: ", color_str, df.time_seconds, df.time_fractional, df.class_name, df.level_name, entry.filename, entry.line_num, entry.function, entry.message);
std::fwrite(buffer, 1, (std::min)(sizeof(buffer) - 1, result.size), stdout);
std::fwrite(entry.message, 1, entry.message_len, stdout);
std::fwrite("\x1b[0m\n", 1, sizeof("\x1b[0m\n"), stdout);
}
}
void Flush() noexcept override {}
@@ -246,7 +239,7 @@ struct ColorConsoleBackend final : public Backend {
#ifndef __OPENORBIS__
/// @brief Backend that writes to a file passed into the constructor
struct FileBackend final : public Backend {
explicit FileBackend(const std::filesystem::path& filename) noexcept {
explicit FileBackend(const std::filesystem::path filename) noexcept {
auto old_filename = filename;
old_filename += ".old.txt";
// Existence checks are done within the functions themselves.
@@ -261,7 +254,7 @@ struct FileBackend final : public Backend {
if (!enabled)
return;
auto message = FormatLogMessage(entry).append(1, '\n');
auto message = FormatLogMessage(entry);
#ifndef __ANDROID__
if (Settings::values.censor_username.GetValue()) {
// This must be a static otherwise it would get checked on EVERY
@@ -269,8 +262,7 @@ struct FileBackend final : public Backend {
static std::string username = []() -> std::string {
// in order of precedence
// LOGNAME usually works on UNIX, USERNAME on Windows
// Some UNIX systems suck and don't use LOGNAME so we also
// need USER :(
// Some UNIX systems suck and don't use LOGNAME so we also need USER :(
for (auto const var : { "LOGNAME", "USERNAME", "USER", })
if (auto const s = ::getenv(var); s != nullptr)
return std::string{s};
@@ -280,7 +272,7 @@ struct FileBackend final : public Backend {
boost::replace_all(message, username, "user");
}
#endif
bytes_written += file->WriteString(message);
bytes_written += file->WriteSpan(std::span<const char>{message.begin(), message.end()});
// Option to log each line rather than 4k buffers
if (Settings::values.log_flush_line.GetValue())
@@ -308,14 +300,13 @@ private:
bool enabled = true;
};
#endif
#ifdef _WIN32
/// @brief Backend that writes to Visual Studio's output window
struct DebuggerBackend final : public Backend {
explicit DebuggerBackend() noexcept = default;
~DebuggerBackend() noexcept override = default;
void Write(const Entry& entry) noexcept override {
::OutputDebugStringW(UTF8ToUTF16W(FormatLogMessage(entry).append(1, '\n')).c_str());
::OutputDebugStringW(UTF8ToUTF16W(FormatLogMessage(entry)).c_str());
}
void Flush() noexcept override {}
};
@@ -338,7 +329,7 @@ struct LogcatBackend : public Backend {
}
}();
auto const df = GetDirectFormatArgs(entry);
__android_log_print(android_log_priority, "YuzuNative", CCB_PRINTF_FMT, df.time_seconds, df.time_fractional, df.class_name, df.level_name, entry.filename, entry.line_num, entry.function, entry.message.c_str());
__android_log_print(android_log_priority, "YuzuNative", CCB_PRINTF_FMT, df.time_seconds, df.time_fractional, df.class_name, df.level_name, entry.filename, entry.line_num, entry.function, entry.message);
}
void Flush() noexcept override {}
};
@@ -377,7 +368,23 @@ struct Impl {
#endif
std::chrono::steady_clock::time_point time_origin{std::chrono::steady_clock::now()};
};
} // namespace
} // Anonymous namespace
void Filter::ParseFilterString(std::string_view filter_view) {
auto clause_begin = filter_view.cbegin();
while (clause_begin < filter_view.cend()) {
auto clause_end = std::find(clause_begin, filter_view.cend(), ' ');
// If clause isn't empty
if (clause_end != clause_begin) {
ParseFilterRule(*this, clause_begin, clause_end);
}
if (clause_end != filter_view.cend()) {
// Skip over the whitespace
++clause_end;
}
clause_begin = clause_end;
}
}
// Constructor shall NOT depend upon Settings() or whatever
// it's ran at global static ctor() time... so BE CAREFUL MFER!
@@ -418,10 +425,14 @@ void SetColorConsoleBackendEnabled(bool enabled) {
void FmtLogMessageImpl(Class log_class, Level log_level, const char* filename, unsigned int line_num, const char* function, fmt::string_view format, const fmt::format_args& args) {
if (logging_instance && logging_instance->filter.CheckMessage(log_class, log_level)) {
char buffer[BUFSIZ];
auto result = fmt::vformat_to_n(buffer, sizeof(buffer) - 1, format, args);
buffer[result.size] = '\0';
auto const flush = ::Settings::values.log_flush_line.GetValue();
logging_instance->ForEachBackend([=](Backend& backend) {
backend.Write(Entry{
.message = fmt::vformat(format, args),
.message = buffer,
.message_len = (std::min)(sizeof(buffer) - 1, result.size),
.timestamp = std::chrono::duration_cast<std::chrono::microseconds>(std::chrono::steady_clock::now() - logging_instance->time_origin),
.log_class = log_class,
.log_level = log_level,
-21
View File
@@ -140,25 +140,4 @@ void Stop();
void SetGlobalFilter(const Filter& filter);
void SetColorConsoleBackendEnabled(bool enabled);
/// @brief A log entry. Log entries are store in a structured format to permit more varied output
/// formatting on different frontends, as well as facilitating filtering and aggregation.
struct Entry {
std::string message;
std::chrono::microseconds timestamp;
Class log_class{};
Level log_level{};
const char* filename = nullptr;
const char* function = nullptr;
unsigned int line_num = 0;
};
/// Formats a log entry into the provided text buffer.
std::string FormatLogMessage(const Entry& entry) noexcept;
/// Prints the same message as `PrintMessage`, but colored according to the severity level.
void PrintColoredMessage(const Entry& entry) noexcept;
/// Formats and prints a log entry to the android logcat.
void PrintMessageToLogcat(const Entry& entry) noexcept;
} // namespace Common::Log
+28
View File
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
@@ -17,6 +20,9 @@
#endif
#endif
#include <cstdio>
#include <cstdlib>
#include "common/memory_detect.h"
namespace Common {
@@ -69,4 +75,26 @@ const MemoryInfo& GetMemInfo() {
return mem_info;
}
u64 GetPermissibleMapCount() {
constexpr u64 DefaultMapCount = 65530;
constexpr u64 ReservedMaps = 20000;
u64 count = DefaultMapCount;
#ifdef __linux__
if (std::FILE* const file = std::fopen("/proc/sys/vm/max_map_count", "re")) {
char line[32];
if (std::fgets(line, sizeof(line), file) != nullptr) {
const u64 parsed = std::strtoull(line, nullptr, 10);
if (parsed != 0) {
count = parsed;
}
}
std::fclose(file);
}
#endif
if (count <= ReservedMaps) {
return 0;
}
return count - ReservedMaps;
}
} // namespace Common
+5
View File
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
@@ -18,4 +21,6 @@ struct MemoryInfo {
*/
[[nodiscard]] const MemoryInfo& GetMemInfo();
[[nodiscard]] u64 GetPermissibleMapCount();
} // namespace Common
+3
View File
@@ -645,6 +645,9 @@ struct Values {
SwitchableSetting<bool> use_asynchronous_shaders{linkage, false, "use_asynchronous_shaders",
Category::RendererHacks};
SwitchableSetting<bool> use_unified_memory{linkage, false, "use_unified_memory",
Category::RendererHacks};
SwitchableSetting<GpuUnswizzleSize> gpu_unswizzle_texture_size{linkage,
GpuUnswizzleSize::Large,
"gpu_unswizzle_texture_size",
+1 -1
View File
@@ -128,7 +128,7 @@ ENUM(TimeZone, Auto, Default, Cet, Cst6Cdt, Cuba, Eet, Egypt, Eire, Est, Est5Edt
GmtPlusZero, GmtMinusZero, GmtZero, Greenwich, Hongkong, Hst, Iceland, Iran, Israel, Jamaica,
Japan, Kwajalein, Libya, Met, Mst, Mst7Mdt, Navajo, Nz, NzChat, Poland, Portugal, Prc, Pst8Pdt,
Roc, Rok, Singapore, Turkey, Uct, Universal, Utc, WSu, Wet, Zulu);
ENUM(AnisotropyMode, Automatic, Default, X2, X4, X8, X16, X32, X64, None);
ENUM(AnisotropyMode, Automatic, Default, X2, X4, X8, X16);
ENUM(AstcDecodeMode, Cpu, Gpu, CpuAsynchronous);
ENUM(AstcRecompression, Uncompressed, Bc1, Bc3);
ENUM(FramePacingMode, Target_Auto, Target_30, Target_60, Target_90, Target_120);
@@ -286,6 +286,7 @@ void ArmDynarmic32::MakeJit(Common::PageTable* page_table) {
config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_UnfuseFMA;
config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_IgnoreStandardFPCRValue;
config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_InaccurateNaN;
config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_IgnoreGlobalMonitor;
break;
// Paranoia mode for debugging optimizations
case Settings::CpuAccuracy::Paranoid:
@@ -338,6 +338,7 @@ void ArmDynarmic64::MakeJit(Common::PageTable* page_table, std::size_t address_s
config.unsafe_optimizations = true;
config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_UnfuseFMA;
config.fastmem_address_space_bits = 64;
config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_IgnoreGlobalMonitor;
break;
// Paranoia mode for debugging optimizations
case Settings::CpuAccuracy::Paranoid:
+5 -1
View File
@@ -119,6 +119,7 @@ struct System::Impl {
is_multicore = Settings::values.use_multi_core.GetValue();
extended_memory_layout = Settings::values.memory_layout_mode.GetValue() != Settings::MemoryLayout::Memory_4Gb;
unified_memory = Settings::values.use_unified_memory.GetValue();
core_timing.SetMulticore(is_multicore);
core_timing.Initialize([&system]() { system.RegisterHostThread(); });
@@ -146,7 +147,8 @@ struct System::Impl {
!device_memory.has_value() ||
is_multicore != Settings::values.use_multi_core.GetValue() ||
extended_memory_layout != (Settings::values.memory_layout_mode.GetValue() !=
Settings::MemoryLayout::Memory_4Gb);
Settings::MemoryLayout::Memory_4Gb) ||
unified_memory != Settings::values.use_unified_memory.GetValue();
if (!must_reinitialize) {
return;
@@ -157,6 +159,7 @@ struct System::Impl {
is_multicore = Settings::values.use_multi_core.GetValue();
extended_memory_layout =
Settings::values.memory_layout_mode.GetValue() != Settings::MemoryLayout::Memory_4Gb;
unified_memory = Settings::values.use_unified_memory.GetValue();
Initialize(system);
}
@@ -503,6 +506,7 @@ struct System::Impl {
std::atomic_bool is_powered_on{};
bool is_multicore : 1 = false;
bool extended_memory_layout : 1 = false;
bool unified_memory : 1 = false;
bool exit_locked : 1 = false;
bool exit_requested : 1 = false;
bool nvdec_active : 1 = false;
+13 -1
View File
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
@@ -12,9 +15,18 @@ constexpr size_t VirtualReserveSize = 1ULL << 38;
constexpr size_t VirtualReserveSize = 1ULL << 39;
#endif
namespace {
size_t ApplicationPoolOffset() {
using Init = Kernel::Board::Nintendo::Nx::KSystemControl::Init;
const size_t dram_size = Init::GetIntendedMemorySize();
const size_t application_pool_size = Init::GetApplicationPoolSize();
return dram_size > application_pool_size ? dram_size - application_pool_size : 0;
}
}
DeviceMemory::DeviceMemory()
: buffer{Kernel::Board::Nintendo::Nx::KSystemControl::Init::GetIntendedMemorySize(),
VirtualReserveSize} {}
VirtualReserveSize, ApplicationPoolOffset()} {}
DeviceMemory::~DeviceMemory() = default;
+35
View File
@@ -20,6 +20,8 @@
#include "common/scratch_buffer.h"
#include "common/virtual_buffer.h"
struct AHardwareBuffer;
namespace Core {
constexpr size_t DEVICE_PAGEBITS = 12ULL;
@@ -95,6 +97,34 @@ public:
ApplyOpOnPAddr(address, buffer, operation);
}
u8* GetPhysicalBase() noexcept {
return reinterpret_cast<u8*>(physical_base);
}
const u8* GetPhysicalBase() const noexcept {
return reinterpret_cast<const u8*>(physical_base);
}
size_t GetPhysicalSize() const noexcept {
return physical_size;
}
std::span<AHardwareBuffer* const> GetBackingHardwareBuffers() const noexcept {
return ahb_windows;
}
size_t GetBackingHardwareBufferWindowSize() const noexcept {
return ahb_window_size;
}
size_t GetBackingHardwareBufferBase() const noexcept {
return ahb_base;
}
bool IsBackingShared() const noexcept {
return backing_is_shared;
}
PAddr GetPhysicalRawAddressFromDAddr(DAddr address) const {
PAddr subbits = PAddr(address & page_mask);
auto paddr = tracked_entries[(address >> page_bits)].compressed_physical_ptr;
@@ -171,6 +201,11 @@ private:
std::unique_ptr<DeviceMemoryManagerAllocator<Traits>> impl;
const uintptr_t physical_base;
const size_t physical_size;
const std::span<AHardwareBuffer* const> ahb_windows;
const size_t ahb_window_size;
const size_t ahb_base;
const bool backing_is_shared;
DeviceInterface* device_inter;
struct TrackedEntry {
+5
View File
@@ -171,6 +171,11 @@ struct DeviceMemoryManagerAllocator {
template <typename Traits>
DeviceMemoryManager<Traits>::DeviceMemoryManager(const DeviceMemory& device_memory_)
: physical_base{uintptr_t(device_memory_.buffer.BackingBasePointer())}
, physical_size{device_memory_.buffer.BackingSize()}
, ahb_windows{device_memory_.buffer.BackingHardwareBuffers()}
, ahb_window_size{device_memory_.buffer.BackingHardwareBufferWindowSize()}
, ahb_base{device_memory_.buffer.BackingHardwareBufferBase()}
, backing_is_shared{device_memory_.buffer.IsBackingShared()}
, device_inter{nullptr}
, compressed_device_addr(1ULL << ((Settings::values.memory_layout_mode.GetValue() == Settings::MemoryLayout::Memory_4Gb ? physical_min_bits : physical_max_bits) - Memory::YUZU_PAGEBITS))
, tracked_entries(device_as_size >> Memory::YUZU_PAGEBITS)
+6 -1
View File
@@ -105,7 +105,12 @@ void LoopProcess(Core::System& system) {
auto server_manager = std::make_unique<ServerManager>(system);
auto module = std::make_shared<Module>(system);
server_manager->RegisterNamedService("nvdrv", std::make_shared<NVDRV>(system, module, "nvdrv"));
server_manager->RegisterNamedService("nvdrv:a", std::make_shared<NVDRV>(system, module, "nvdrv:a"));
const auto NvdrvInterfaceFactoryForApplets = [&, module] {
return std::make_shared<NVDRV>(system, module, "nvdrv:a");
};
server_manager->RegisterNamedService("nvdrv:a", NvdrvInterfaceFactoryForApplets);
server_manager->RegisterNamedService("nvdrv:s", std::make_shared<NVDRV>(system, module, "nvdrv:s"));
server_manager->RegisterNamedService("nvdrv:t", std::make_shared<NVDRV>(system, module, "nvdrv:t"));
server_manager->RegisterNamedService("nvmemp", std::make_shared<NVMEMP>(system));
+6 -1
View File
@@ -588,7 +588,12 @@ void LoopProcess(Core::System& system) {
auto server_manager = std::make_unique<ServerManager>(system);
auto ro = std::make_shared<RoContext>();
server_manager->RegisterNamedService("ldr:ro", std::make_shared<RoInterface>(system, "ldr:ro", ro, NrrKind::User));
const auto RoInterfaceFactoryForUser = [&, ro] {
return std::make_shared<RoInterface>(system, "ldr:ro", ro, NrrKind::User);
};
server_manager->RegisterNamedService("ldr:ro", std::move(RoInterfaceFactoryForUser));
server_manager->RegisterNamedService("ro:1", std::make_shared<RoInterface>(system, "ro:1", ro, NrrKind::JitPlugin));
server_manager->RegisterNamedService("ro:dmnt", std::make_shared<IDebugMonitorInterface>(system));
ServerManager::RunServer(std::move(server_manager));
+2 -3
View File
@@ -227,6 +227,8 @@ std::unique_ptr<TranslationMap> InitializeTranslations(QObject* parent) {
tr("Preserves GPU-modified data by reading it back before uploading.\nSome games require this to render certain effects properly."));
INSERT(Settings, use_asynchronous_shaders, tr("Enable asynchronous shader compilation"),
tr("May reduce shader stutter."));
INSERT(Settings, use_unified_memory, tr("Enable unified memory access"),
tr("Lets the GPU write buffer readbacks directly into guest memory."));
INSERT(Settings, gpu_clock, tr("GPU Clocks"),
tr("Makes the game believe GPU work finishes faster than it does, so it stops lowering "
"resolution and render distance to fit the Switch's clocks."));
@@ -522,9 +524,6 @@ std::unique_ptr<ComboboxTranslationMap> ComboboxEnumeration(QObject* parent) {
PAIR(AnisotropyMode, X4, tr("4x")),
PAIR(AnisotropyMode, X8, tr("8x")),
PAIR(AnisotropyMode, X16, tr("16x")),
PAIR(AnisotropyMode, X32, tr("32x")),
PAIR(AnisotropyMode, X64, tr("64x")),
PAIR(AnisotropyMode, None, tr("None")),
}});
translations->insert(
{Settings::EnumMetadata<Settings::Language>::Index(),
@@ -335,7 +335,7 @@ void SetupDenormControl(const Profile& profile, const IR::Program& program, Emit
if (info.uses_fp32_denorms_flush && info.uses_fp32_denorms_preserve) {
LOG_DEBUG(Shader_SPIRV, "Fp32 denorm flush and preserve on the same shader");
} else if (info.uses_fp32_denorms_flush) {
if (profile.support_fp32_denorm_flush) {
if (profile.support_fp32_denorm_flush && !profile.has_broken_fp32_denorm_flush) {
ctx.AddCapability(spv::Capability::DenormFlushToZero);
ctx.AddExecutionMode(main_func, spv::ExecutionMode::DenormFlushToZero, 32U);
} else {
@@ -13,6 +13,7 @@
namespace Shader::Backend::SPIRV {
namespace {
Id SharedPointer(EmitContext& ctx, Id offset, u32 index_offset = 0) {
offset = ctx.BoundSharedOffset(offset, 4 + index_offset * 4);
const Id shift_id{ctx.Const(2U)};
Id index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift_id)};
if (index_offset > 0) {
@@ -160,7 +161,8 @@ Id EmitSharedAtomicExchange32(EmitContext& ctx, Id offset, Id value) {
Id EmitSharedAtomicExchange64(EmitContext& ctx, Id offset, Id value) {
if (ctx.profile.support_shared_int64_atomics && ctx.uses_explicit_workgroup_layout) {
const Id shift_id{ctx.Const(3U)};
const Id index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift_id)};
const Id index{
ctx.OpShiftRightArithmetic(ctx.U32[1], ctx.BoundSharedOffset(offset, 8), shift_id)};
const Id pointer{
ctx.OpAccessChain(ctx.shared_u64, ctx.shared_memory_u64, ctx.u32_zero_value, index)};
const auto [scope, semantics]{AtomicArgs(ctx)};
@@ -31,6 +31,7 @@ std::pair<Id, Id> ExtractArgs(EmitContext& ctx, Id offset, u32 mask, u32 count)
} // Anonymous namespace
Id EmitLoadSharedU8(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 1);
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{
ctx.OpAccessChain(ctx.shared_u8, ctx.shared_memory_u8, ctx.u32_zero_value, offset)};
@@ -42,6 +43,7 @@ Id EmitLoadSharedU8(EmitContext& ctx, Id offset) {
}
Id EmitLoadSharedS8(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 1);
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{
ctx.OpAccessChain(ctx.shared_u8, ctx.shared_memory_u8, ctx.u32_zero_value, offset)};
@@ -53,6 +55,7 @@ Id EmitLoadSharedS8(EmitContext& ctx, Id offset) {
}
Id EmitLoadSharedU16(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 2);
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u16, ctx.shared_memory_u16, offset, 1)};
return ctx.OpUConvert(ctx.U32[1], ctx.OpLoad(ctx.U16, pointer));
@@ -63,6 +66,7 @@ Id EmitLoadSharedU16(EmitContext& ctx, Id offset) {
}
Id EmitLoadSharedS16(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 2);
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u16, ctx.shared_memory_u16, offset, 1)};
return ctx.OpSConvert(ctx.U32[1], ctx.OpLoad(ctx.U16, pointer));
@@ -73,6 +77,7 @@ Id EmitLoadSharedS16(EmitContext& ctx, Id offset) {
}
Id EmitLoadSharedU32(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 4);
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32, ctx.shared_memory_u32, offset, 2)};
return ctx.OpLoad(ctx.U32[1], pointer);
@@ -82,6 +87,7 @@ Id EmitLoadSharedU32(EmitContext& ctx, Id offset) {
}
Id EmitLoadSharedU64(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 8);
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32x2, ctx.shared_memory_u32x2, offset, 3)};
return ctx.OpLoad(ctx.U32[2], pointer);
@@ -97,6 +103,7 @@ Id EmitLoadSharedU64(EmitContext& ctx, Id offset) {
}
Id EmitLoadSharedU128(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 16);
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32x4, ctx.shared_memory_u32x4, offset, 4)};
return ctx.OpLoad(ctx.U32[4], pointer);
@@ -113,6 +120,7 @@ Id EmitLoadSharedU128(EmitContext& ctx, Id offset) {
}
void EmitWriteSharedU8(EmitContext& ctx, Id offset, Id value) {
offset = ctx.BoundSharedOffset(offset, 1);
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{
ctx.OpAccessChain(ctx.shared_u8, ctx.shared_memory_u8, ctx.u32_zero_value, offset)};
@@ -123,6 +131,7 @@ void EmitWriteSharedU8(EmitContext& ctx, Id offset, Id value) {
}
void EmitWriteSharedU16(EmitContext& ctx, Id offset, Id value) {
offset = ctx.BoundSharedOffset(offset, 2);
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u16, ctx.shared_memory_u16, offset, 1)};
ctx.OpStore(pointer, ctx.OpUConvert(ctx.U16, value));
@@ -132,6 +141,7 @@ void EmitWriteSharedU16(EmitContext& ctx, Id offset, Id value) {
}
void EmitWriteSharedU32(EmitContext& ctx, Id offset, Id value) {
offset = ctx.BoundSharedOffset(offset, 4);
Id pointer{};
if (ctx.uses_explicit_workgroup_layout) {
pointer = Pointer(ctx, ctx.shared_u32, ctx.shared_memory_u32, offset, 2);
@@ -144,6 +154,7 @@ void EmitWriteSharedU32(EmitContext& ctx, Id offset, Id value) {
}
void EmitWriteSharedU64(EmitContext& ctx, Id offset, Id value) {
offset = ctx.BoundSharedOffset(offset, 8);
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32x2, ctx.shared_memory_u32x2, offset, 3)};
ctx.OpStore(pointer, value);
@@ -159,6 +170,7 @@ void EmitWriteSharedU64(EmitContext& ctx, Id offset, Id value) {
}
void EmitWriteSharedU128(EmitContext& ctx, Id offset, Id value) {
offset = ctx.BoundSharedOffset(offset, 16);
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32x4, ctx.shared_memory_u32x4, offset, 4)};
ctx.OpStore(pointer, value);
@@ -600,6 +600,16 @@ void EmitContext::DefineLocalMemory(const IR::Program& program) {
}
}
Id EmitContext::BoundSharedOffset(Id offset, u32 access_bytes) {
if (shared_memory_declared_bytes == 0) {
return offset;
}
const u32 last_valid{shared_memory_declared_bytes > access_bytes
? shared_memory_declared_bytes - access_bytes
: 0U};
return OpUMin(U32[1], offset, Const(last_valid));
}
void EmitContext::DefineSharedMemory(const IR::Program& program) {
uses_explicit_workgroup_layout =
profile.support_explicit_workgroup_layout &&
@@ -608,8 +618,15 @@ void EmitContext::DefineSharedMemory(const IR::Program& program) {
if (program.shared_memory_size == 0) {
return;
}
const u32 device_limit{profile.max_shared_memory_size};
const u32 shared_memory_size{device_limit != 0 && program.shared_memory_size > device_limit
? device_limit
: program.shared_memory_size};
if (shared_memory_size != program.shared_memory_size) {
shared_memory_declared_bytes = shared_memory_size;
}
const auto make{[&](Id element_type, u32 element_size) {
const u32 num_elements{Common::DivCeil(program.shared_memory_size, element_size)};
const u32 num_elements{Common::DivCeil(shared_memory_size, element_size)};
const Id array_type{TypeArray(element_type, Const(num_elements))};
Decorate(array_type, spv::Decoration::ArrayStride, element_size);
@@ -644,7 +661,7 @@ void EmitContext::DefineSharedMemory(const IR::Program& program) {
std::tie(shared_memory_u32x4, shared_u32x4, std::ignore) = make(U32[4], 16);
return;
}
const u32 num_elements{Common::DivCeil(program.shared_memory_size, 4U)};
const u32 num_elements{Common::DivCeil(shared_memory_size, 4U)};
const Id type{TypeArray(U32[1], Const(num_elements))};
shared_memory_u32_type = TypePointer(spv::StorageClass::Workgroup, type);
@@ -312,6 +312,8 @@ public:
Id local_memory{};
bool uses_explicit_workgroup_layout{};
u32 shared_memory_declared_bytes{};
[[nodiscard]] Id BoundSharedOffset(Id offset, u32 access_bytes);
Id shared_memory_u8{};
Id shared_memory_u16{};
Id shared_memory_u32{};
+5
View File
@@ -86,6 +86,8 @@ struct Profile {
bool has_broken_signed_operations{};
/// Float controls break when fp16 is enabled
bool has_broken_fp16_float_controls{};
/// Declaring fp32 denorm flush to zero miscompiles on some drivers
bool has_broken_fp32_denorm_flush{};
/// Dynamic vec4 indexing is broken on some OpenGL drivers
bool has_gl_component_indexing_bug{};
/// The precise type qualifier is broken in the fragment stage of some drivers
@@ -101,6 +103,9 @@ struct Profile {
u32 gl_max_compute_smem_size{};
/// Largest workgroup shared memory allocation the device accepts, 0 when unconstrained
u32 max_shared_memory_size{};
/// Maxwell and earlier nVidia architectures have broken robust support
bool has_broken_robust{};
+272 -37
View File
@@ -571,7 +571,11 @@ void BufferCache<P>::AccumulateFlushes() {
template <class P>
bool BufferCache<P>::ShouldWaitAsyncFlushes() const noexcept {
return (!async_buffers.empty() && async_buffers.front().has_value());
if (async_buffers.empty()) {
return false;
}
return async_buffers.front().has_value() ||
!pending_downloads.front().unified_copies.empty();
}
template <class P>
@@ -579,6 +583,7 @@ void BufferCache<P>::CommitAsyncFlushesHigh() {
AccumulateFlushes();
if (committed_gpu_modified_ranges.empty()) {
pending_downloads.emplace_back();
async_buffers.emplace_back(std::optional<Async_Buffer>{});
return;
}
@@ -638,27 +643,84 @@ void BufferCache<P>::CommitAsyncFlushesHigh() {
}
committed_gpu_modified_ranges.clear();
if (downloads.empty()) {
pending_downloads.emplace_back();
async_buffers.emplace_back(std::optional<Async_Buffer>{});
return;
}
auto download_staging = runtime.DownloadStagingBuffer(total_size_bytes, true);
boost::container::small_vector<BufferCopy, 4> normalized_copies;
runtime.PreCopyBarrier();
struct QueuedUnifiedCopy {
u64 window;
BufferId buffer_id;
boost::container::small_vector<BufferCopy, 16> copies;
};
AsyncDownloadBatch batch;
boost::container::small_vector<std::pair<BufferCopy, BufferId>, 16> staging_downloads;
boost::container::small_vector<QueuedUnifiedCopy, 4> unified_copy_queue;
boost::container::small_vector<u64, 4> window_ids;
UnifiedWindowGroups groups;
u64 staging_size_bytes = 0;
for (auto& [copy, buffer_id] : downloads) {
copy.dst_offset += download_staging.offset;
const std::array copies{copy};
BufferCopy second_copy{copy};
Buffer& buffer = slot_buffers[buffer_id];
second_copy.src_offset = static_cast<size_t>(buffer.CpuAddr()) + copy.src_offset;
const DAddr orig_device_addr = static_cast<DAddr>(second_copy.src_offset);
const DAddr orig_device_addr = buffer.CpuAddr() + copy.src_offset;
bool unified = false;
if constexpr (USE_UNIFIED_MEMORY) {
if (runtime.HasUnifiedMemory()) {
window_ids.clear();
groups.clear();
unified = ResolveUnifiedWindows(orig_device_addr, copy.src_offset, copy.size,
window_ids, groups);
}
}
BufferCopy record{copy};
record.src_offset = static_cast<size_t>(orig_device_addr);
if (unified) {
async_downloads.Add(orig_device_addr, copy.size);
buffer.MarkUsage(copy.src_offset, copy.size);
for (size_t i = 0; i < window_ids.size(); ++i) {
unified_copy_queue.push_back(
QueuedUnifiedCopy{window_ids[i], buffer_id, std::move(groups[i])});
}
batch.unified_copies.push_back(record);
continue;
}
copy.dst_offset = staging_size_bytes;
constexpr u64 align = 64ULL;
staging_size_bytes += (copy.size + align - 1) & ~(align - 1ULL);
staging_downloads.push_back({copy, buffer_id});
}
std::optional<Async_Buffer> download_staging;
if (!staging_downloads.empty()) {
download_staging = runtime.DownloadStagingBuffer(staging_size_bytes, true);
}
runtime.PreCopyBarrier();
for (auto& [copy, buffer_id] : staging_downloads) {
copy.dst_offset += download_staging->offset;
const std::array copies{copy};
Buffer& buffer = slot_buffers[buffer_id];
BufferCopy record{copy};
record.src_offset = static_cast<size_t>(buffer.CpuAddr()) + copy.src_offset;
const DAddr orig_device_addr = static_cast<DAddr>(record.src_offset);
async_downloads.Add(orig_device_addr, copy.size);
buffer.MarkUsage(copy.src_offset, copy.size);
runtime.CopyBuffer(download_staging.buffer, buffer, copies, false);
normalized_copies.push_back(second_copy);
runtime.CopyBuffer(download_staging->buffer, buffer, copies, false);
batch.staging_copies.push_back(record);
}
if constexpr (USE_UNIFIED_MEMORY) {
for (const auto& queued : unified_copy_queue) {
const std::span<const BufferCopy> group_span(queued.copies.data(),
queued.copies.size());
runtime.CopyToUnifiedMemory(queued.window, slot_buffers[queued.buffer_id], group_span);
}
if (!unified_copy_queue.empty()) {
runtime.FlushUnifiedMemoryCopies();
runtime.UnifiedMemoryHostBarrier();
}
}
runtime.PostCopyBarrier();
pending_downloads.emplace_back(std::move(normalized_copies));
async_buffers.emplace_back(download_staging);
pending_downloads.emplace_back(std::move(batch));
async_buffers.emplace_back(std::move(download_staging));
}
template <class P>
@@ -673,32 +735,49 @@ void BufferCache<P>::PopAsyncFlushes() {
template <class P>
void BufferCache<P>::PopAsyncBuffers() {
if (async_buffers.empty()) {
return;
}
if (!async_buffers.front().has_value()) {
struct Writeback {
DAddr addr;
const u8* src;
u64 size;
};
boost::container::small_vector<Writeback, 8> writebacks;
{
std::scoped_lock lock{mutex};
if (async_buffers.empty()) {
return;
}
auto& batch = pending_downloads.front();
auto& async_buffer = async_buffers.front();
if (async_buffer.has_value()) {
const u8* base = async_buffer->mapped_span.data();
const size_t base_offset = async_buffer->offset;
for (const auto& copy : batch.staging_copies) {
const DAddr device_addr = static_cast<DAddr>(copy.src_offset);
const u64 dst_offset = copy.dst_offset - base_offset;
const u8* read_mapped_memory = base + dst_offset;
async_downloads.ForEachInRange(
device_addr, copy.size, [&](DAddr start, DAddr end, s32) {
writebacks.push_back(
{start, &read_mapped_memory[start - device_addr], end - start});
});
async_downloads.Subtract(device_addr, copy.size, [&](DAddr start, DAddr end) {
gpu_modified_ranges.Subtract(start, end - start);
});
}
async_buffers_death_ring.emplace_back(*async_buffer);
}
for (const auto& copy : batch.unified_copies) {
const DAddr device_addr = static_cast<DAddr>(copy.src_offset);
async_downloads.Subtract(device_addr, copy.size, [&](DAddr start, DAddr end) {
gpu_modified_ranges.Subtract(start, end - start);
});
}
async_buffers.pop_front();
return;
pending_downloads.pop_front();
}
auto& downloads = pending_downloads.front();
auto& async_buffer = async_buffers.front();
u8* base = async_buffer->mapped_span.data();
const size_t base_offset = async_buffer->offset;
for (const auto& copy : downloads) {
const DAddr device_addr = static_cast<DAddr>(copy.src_offset);
const u64 dst_offset = copy.dst_offset - base_offset;
const u8* read_mapped_memory = base + dst_offset;
async_downloads.ForEachInRange(device_addr, copy.size, [&](DAddr start, DAddr end, s32) {
device_memory.WriteBlockUnsafe(start, &read_mapped_memory[start - device_addr],
end - start);
});
async_downloads.Subtract(device_addr, copy.size, [&](DAddr start, DAddr end) {
gpu_modified_ranges.Subtract(start, end - start);
});
for (const auto& wb : writebacks) {
device_memory.WriteBlockUnsafe(wb.addr, wb.src, wb.size);
}
async_buffers_death_ring.emplace_back(*async_buffer);
async_buffers.pop_front();
pending_downloads.pop_front();
}
template <class P>
@@ -1006,11 +1085,25 @@ void BufferCache<P>::BindHostGraphicsStorageBuffers(size_t stage) {
Buffer& buffer = slot_buffers[binding.buffer_id];
TouchBuffer(buffer, binding.buffer_id);
const u32 size = binding.size;
const bool is_written = ((channel_state->written_storage_buffers[stage] >> index) & 1) != 0;
if constexpr (USE_UNIFIED_MEMORY) {
const auto window = TryResolveUnifiedRange(binding.device_addr, size);
if (window && runtime.IsUnifiedStorageRange(size, window->offset)) {
if (is_written) {
memory_tracker.MarkRegionAsCpuModified(binding.device_addr, size);
}
runtime.BindStorageBuffer(runtime.UnifiedWindowBuffer(window->window),
runtime.UnifiedWindowAddress(window->window),
static_cast<u32>(window->offset), size, is_written);
return;
}
}
SynchronizeBuffer(buffer, binding.device_addr, size);
const u32 offset = buffer.Offset(binding.device_addr);
buffer.MarkUsage(offset, size);
const bool is_written = ((channel_state->written_storage_buffers[stage] >> index) & 1) != 0;
if (is_written) {
MarkWrittenBuffer(binding.buffer_id, binding.device_addr, size);
@@ -1699,6 +1792,142 @@ void BufferCache<P>::ImmediateUploadMemory([[maybe_unused]] Buffer& buffer,
}
}
template <class P>
bool BufferCache<P>::ResolveUnifiedWindows(
[[maybe_unused]] DAddr device_addr, [[maybe_unused]] u64 buffer_offset,
[[maybe_unused]] u64 size, [[maybe_unused]] boost::container::small_vector<u64, 4>& window_ids,
[[maybe_unused]] UnifiedWindowGroups& groups) {
if constexpr (USE_UNIFIED_MEMORY) {
const u8* const physical_base = device_memory.GetPhysicalBase();
const u64 unified_base = runtime.UnifiedMemoryBase();
const u64 unified_size = runtime.UnifiedMemorySize();
const u64 window_size = runtime.UnifiedMemoryWindowSize();
if (window_size == 0) {
return false;
}
const auto group_for = [&](u64 window) -> boost::container::small_vector<BufferCopy, 16>& {
for (size_t i = 0; i < window_ids.size(); ++i) {
if (window_ids[i] == window) {
return groups[i];
}
}
window_ids.push_back(window);
groups.emplace_back();
return groups.back();
};
u64 downloaded = 0;
while (downloaded < size) {
const DAddr page_addr = device_addr + downloaded;
const u8* const ptr = device_memory.GetPointer<u8>(page_addr);
if (ptr == nullptr) {
return false;
}
const u64 page_offset = page_addr & Core::DEVICE_PAGEMASK;
u64 chunk = (std::min)(size - downloaded,
static_cast<u64>(Core::DEVICE_PAGESIZE) - page_offset);
const u64 phys_offset = static_cast<u64>(ptr - physical_base);
if (phys_offset < unified_base || phys_offset - unified_base + chunk > unified_size) {
return false;
}
const u64 relative = phys_offset - unified_base;
const u64 window = relative / window_size;
const u64 local_offset = relative % window_size;
chunk = (std::min)(chunk, window_size - local_offset);
auto& group = group_for(window);
if (!group.empty()) {
BufferCopy& last = group.back();
if (last.src_offset + last.size == buffer_offset + downloaded &&
last.dst_offset + last.size == local_offset) {
last.size += chunk;
downloaded += chunk;
continue;
}
}
group.push_back(BufferCopy{
.src_offset = buffer_offset + downloaded,
.dst_offset = local_offset,
.size = chunk,
});
downloaded += chunk;
}
return true;
} else {
return false;
}
}
template <class P>
std::optional<typename BufferCache<P>::UnifiedWindowRange>
BufferCache<P>::TryResolveUnifiedRange([[maybe_unused]] DAddr device_addr,
[[maybe_unused]] u64 size) {
if constexpr (USE_UNIFIED_MEMORY) {
if (size == 0 || !runtime.IsUnifiedMemoryBindable()) {
return std::nullopt;
}
const u64 window_size = runtime.UnifiedMemoryWindowSize();
if (window_size == 0) {
return std::nullopt;
}
const u8* const first = device_memory.GetSpan(device_addr, size);
if (first == nullptr) {
return std::nullopt;
}
const u64 phys_offset = static_cast<u64>(first - device_memory.GetPhysicalBase());
const u64 unified_base = runtime.UnifiedMemoryBase();
if (phys_offset < unified_base) {
return std::nullopt;
}
const u64 relative = phys_offset - unified_base;
const u64 unified_size = runtime.UnifiedMemorySize();
if (relative >= unified_size || unified_size - relative < size) {
return std::nullopt;
}
const u64 local_offset = relative % window_size;
if (window_size - local_offset < size) {
return std::nullopt;
}
if (memory_tracker.IsRegionGpuModified(device_addr, size) ||
IsRegionGpuModified(device_addr, size)) {
return std::nullopt;
}
return UnifiedWindowRange{
.window = static_cast<size_t>(relative / window_size),
.offset = local_offset,
};
} else {
return std::nullopt;
}
}
template <class P>
bool BufferCache<P>::TryUnifiedDownloadMemory([[maybe_unused]] Buffer& buffer,
[[maybe_unused]] std::span<BufferCopy> copies) {
if constexpr (USE_UNIFIED_MEMORY) {
boost::container::small_vector<u64, 4> window_ids;
UnifiedWindowGroups groups;
for (const BufferCopy& copy : copies) {
if (!ResolveUnifiedWindows(buffer.CpuAddr() + copy.src_offset, copy.src_offset,
copy.size, window_ids, groups)) {
return false;
}
}
for (const BufferCopy& copy : copies) {
buffer.MarkUsage(copy.src_offset, copy.size);
}
runtime.PreCopyBarrier();
for (size_t i = 0; i < window_ids.size(); ++i) {
const std::span<const BufferCopy> group_span(groups[i].data(), groups[i].size());
runtime.CopyToUnifiedMemory(window_ids[i], buffer, group_span);
}
runtime.FlushUnifiedMemoryCopies();
runtime.UnifiedMemoryHostBarrier();
runtime.Finish();
return true;
} else {
return false;
}
}
template <class P>
void BufferCache<P>::MappedUploadMemory([[maybe_unused]] Buffer& buffer,
[[maybe_unused]] u64 total_size_bytes,
@@ -1802,6 +2031,12 @@ void BufferCache<P>::DownloadBufferMemory(Buffer& buffer, DAddr device_addr, u64
}
if constexpr (USE_MEMORY_MAPS) {
if constexpr (USE_UNIFIED_MEMORY) {
if (runtime.HasUnifiedMemory() &&
TryUnifiedDownloadMemory(buffer, std::span(copies.data(), copies.size()))) {
return;
}
}
auto download_staging = runtime.DownloadStagingBuffer(total_size_bytes);
const u8* const mapped_memory = download_staging.mapped_span.data();
const std::span<BufferCopy> copies_span(copies.data(), copies.data() + copies.size());
@@ -12,6 +12,7 @@
#include <functional>
#include <memory>
#include <mutex>
#include <optional>
#include <numeric>
#include <span>
#include <vector>
@@ -180,6 +181,7 @@ class BufferCache : public VideoCommon::ChannelSetupCaches<BufferCacheChannelInf
static constexpr bool USE_MEMORY_MAPS = P::USE_MEMORY_MAPS;
static constexpr bool SEPARATE_IMAGE_BUFFERS_BINDINGS = P::SEPARATE_IMAGE_BUFFER_BINDINGS;
static constexpr bool USE_MEMORY_MAPS_FOR_UPLOADS = P::USE_MEMORY_MAPS_FOR_UPLOADS;
static constexpr bool USE_UNIFIED_MEMORY = P::USE_UNIFIED_MEMORY;
#ifdef YUZU_LEGACY
static constexpr s64 TARGET_THRESHOLD = 3_GiB;
@@ -443,6 +445,22 @@ private:
void MappedUploadMemory(Buffer& buffer, u64 total_size_bytes, std::span<BufferCopy> copies);
bool TryUnifiedDownloadMemory(Buffer& buffer, std::span<BufferCopy> copies);
struct UnifiedWindowRange {
size_t window;
u64 offset;
};
std::optional<UnifiedWindowRange> TryResolveUnifiedRange(DAddr device_addr, u64 size);
using UnifiedWindowGroups =
boost::container::small_vector<boost::container::small_vector<BufferCopy, 16>, 4>;
bool ResolveUnifiedWindows(DAddr device_addr, u64 buffer_offset, u64 size,
boost::container::small_vector<u64, 4>& window_ids,
UnifiedWindowGroups& groups);
void DownloadBufferMemory(Buffer& buffer_id);
void DownloadBufferMemory(Buffer& buffer_id, DAddr device_addr, u64 size);
@@ -498,9 +516,14 @@ private:
std::deque<Common::RangeSet<DAddr>> committed_gpu_modified_ranges;
// Async Buffers
struct AsyncDownloadBatch {
boost::container::small_vector<BufferCopy, 4> staging_copies;
boost::container::small_vector<BufferCopy, 4> unified_copies;
};
Common::OverlapRangeSet<DAddr> async_downloads;
std::deque<std::optional<Async_Buffer>> async_buffers;
std::deque<boost::container::small_vector<BufferCopy, 4>> pending_downloads;
std::deque<AsyncDownloadBatch> pending_downloads;
std::optional<Async_Buffer> current_buffer;
std::deque<Async_Buffer> async_buffers_death_ring;
@@ -17,11 +17,14 @@ set(SHADER_FILES
${CMAKE_CURRENT_SOURCE_DIR}/astc_decoder.comp
${CMAKE_CURRENT_SOURCE_DIR}/blit_color_float.frag
${CMAKE_CURRENT_SOURCE_DIR}/block_linear_unswizzle_2d.comp
${CMAKE_CURRENT_SOURCE_DIR}/block_linear_unswizzle_2d_buffer.comp
${CMAKE_CURRENT_SOURCE_DIR}/blit_color_msaa.frag
${CMAKE_CURRENT_SOURCE_DIR}/blit_depth.frag
${CMAKE_CURRENT_SOURCE_DIR}/blit_depth_msaa.frag
${CMAKE_CURRENT_SOURCE_DIR}/blit_depth_stencil_msaa.frag
${CMAKE_CURRENT_SOURCE_DIR}/block_linear_unswizzle_3d.comp
${CMAKE_CURRENT_SOURCE_DIR}/block_linear_unswizzle_3d_bcn.comp
${CMAKE_CURRENT_SOURCE_DIR}/block_linear_unswizzle_3d_buffer.comp
${CMAKE_CURRENT_SOURCE_DIR}/convert_abgr8_to_d24s8.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_abgr8_to_d32f.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_d32f_to_abgr8.frag
@@ -30,8 +33,12 @@ set(SHADER_FILES
${CMAKE_CURRENT_SOURCE_DIR}/convert_float_to_depth.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_msaa_to_non_msaa.comp
${CMAKE_CURRENT_SOURCE_DIR}/convert_msaa_to_non_msaa.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_msaa_to_non_msaa_depth.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_msaa_to_non_msaa_depth_stencil.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_non_msaa_to_msaa.comp
${CMAKE_CURRENT_SOURCE_DIR}/convert_non_msaa_to_msaa.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_non_msaa_to_msaa_depth.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_non_msaa_to_msaa_depth_stencil.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_s8d24_to_abgr8.frag
${CMAKE_CURRENT_SOURCE_DIR}/full_screen_triangle.vert
${CMAKE_CURRENT_SOURCE_DIR}/fxaa.frag
@@ -163,6 +170,44 @@ foreach(SOURCE_FILE IN ITEMS ${SHADER_FILES})
endif()
endforeach()
# Integer variants of the MSAA conversion shaders. They only differ from the float
# source in the sampler and output types, so they are generated from it via defines.
set(SHADER_TYPE_VARIANTS
"convert_msaa_to_non_msaa.frag|sint|isampler2DMS|ivec4"
"convert_msaa_to_non_msaa.frag|uint|usampler2DMS|uvec4"
"convert_non_msaa_to_msaa.frag|sint|isampler2D|ivec4"
"convert_non_msaa_to_msaa.frag|uint|usampler2D|uvec4"
)
foreach(VARIANT IN ITEMS ${SHADER_TYPE_VARIANTS})
string(REPLACE "|" ";" VARIANT_PARTS ${VARIANT})
list(GET VARIANT_PARTS 0 VARIANT_FILENAME)
list(GET VARIANT_PARTS 1 VARIANT_SUFFIX)
list(GET VARIANT_PARTS 2 VARIANT_SAMPLER)
list(GET VARIANT_PARTS 3 VARIANT_TEXEL)
set(VARIANT_SOURCE ${CMAKE_CURRENT_SOURCE_DIR}/${VARIANT_FILENAME})
get_filename_component(VARIANT_STEM ${VARIANT_FILENAME} NAME_WE)
get_filename_component(VARIANT_EXT ${VARIANT_FILENAME} EXT)
string(REPLACE "." "" VARIANT_EXT ${VARIANT_EXT})
set(VARIANT_NAME ${VARIANT_STEM}_${VARIANT_SUFFIX}_${VARIANT_EXT})
string(TOUPPER ${VARIANT_NAME}_SPV VARIANT_VARIABLE_NAME)
set(VARIANT_HEADER_FILE ${SHADER_DIR}/${VARIANT_NAME}_spv.h)
add_custom_command(
OUTPUT
${VARIANT_HEADER_FILE}
COMMAND
${GLSLANGVALIDATOR} -V ${QUIET_FLAG} -I"${FIDELITYFX_INCLUDE_DIR}" ${GLSL_FLAGS}
-DSAMPLER_TYPE=${VARIANT_SAMPLER} -DTEXEL_TYPE=${VARIANT_TEXEL}
--variable-name ${VARIANT_VARIABLE_NAME} -o ${VARIANT_HEADER_FILE} ${VARIANT_SOURCE}
--target-env ${SPIR_V_VERSION}
MAIN_DEPENDENCY
${VARIANT_SOURCE}
)
set(SHADER_HEADERS ${SHADER_HEADERS} ${VARIANT_HEADER_FILE})
endforeach()
foreach(FILEPATH IN ITEMS ${FIDELITYFX_FILES})
get_filename_component(FILENAME ${FILEPATH} NAME)
string(REPLACE "." "_" HEADER_NAME ${FILENAME})
@@ -0,0 +1,12 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 450 core
layout(binding = 0) uniform sampler2D depth_tex;
layout(location = 0) in vec2 texcoord;
void main() {
gl_FragDepth = textureLod(depth_tex, texcoord, 0).r;
}
@@ -8,5 +8,5 @@ layout(binding = 0) uniform sampler2DMS depth_tex;
layout(location = 0) in vec2 texcoord;
void main() {
gl_FragDepth = texelFetch(depth_tex, ivec2(texcoord), 0).r;
gl_FragDepth = texelFetch(depth_tex, ivec2(texcoord), gl_SampleID).r;
}
@@ -10,6 +10,6 @@ layout(binding = 1) uniform usampler2DMS stencil_tex;
layout(location = 0) in vec2 texcoord;
void main() {
gl_FragDepth = texelFetch(depth_tex, ivec2(texcoord), 0).r;
gl_FragStencilRefARB = int(texelFetch(stencil_tex, ivec2(texcoord), 0).r);
gl_FragDepth = texelFetch(depth_tex, ivec2(texcoord), gl_SampleID).r;
gl_FragStencilRefARB = int(texelFetch(stencil_tex, ivec2(texcoord), gl_SampleID).r);
}
@@ -0,0 +1,104 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 430
#extension GL_EXT_shader_16bit_storage : require
#extension GL_EXT_shader_8bit_storage : require
#define BINDING_INPUT_BUFFER 0
#define BINDING_OUTPUT_BUFFER 1
layout(push_constant) uniform PushConstants {
uvec3 dim;
uint bytes_per_block_log2;
uvec3 origin;
uint layer_stride;
uint block_size;
uint x_shift;
uint block_height;
uint block_height_mask;
} pc;
layout(binding = BINDING_INPUT_BUFFER, std430) buffer InputBufferU32 { uint u32data[]; };
layout(binding = BINDING_INPUT_BUFFER, std430) buffer InputBufferU64 { uvec2 u64data[]; };
layout(binding = BINDING_INPUT_BUFFER, std430) buffer InputBufferU128 { uvec4 u128data[]; };
layout(binding = BINDING_OUTPUT_BUFFER, std430) writeonly buffer OutputBuffer {
uint out_u32[];
};
layout(local_size_x = 16, local_size_y = 8, local_size_z = 1) in;
const uint GOB_SIZE_X = 64;
const uint GOB_SIZE_Y = 8;
const uint GOB_SIZE_X_SHIFT = 6;
const uint GOB_SIZE_Y_SHIFT = 3;
const uint GOB_SIZE_SHIFT = GOB_SIZE_X_SHIFT + GOB_SIZE_Y_SHIFT;
const uvec2 SWIZZLE_MASK = uvec2(GOB_SIZE_X - 1u, GOB_SIZE_Y - 1u);
uint SwizzleTable(uint pos) {
const uint t[8] = uint[](
0x12100200, 0x13110301, 0x16140604, 0x17150705,
0x1a180a08, 0x1b190b09, 0x1e1c0e0c, 0x1f1d0f0d
);
const uint i = pos >> 4;
const uint h = (t[i / 4] >> ((i % 4) * 8)) & 0xff;
return (h << 4) | (pos & 0xf);
}
uint SwizzleOffset(uvec2 pos) {
pos = pos & SWIZZLE_MASK;
return SwizzleTable(pos.y * 64u + pos.x);
}
uvec4 ReadTexel(uint offset) {
switch (pc.bytes_per_block_log2) {
case 2u:
return uvec4(u32data[offset / 4u], 0u, 0u, 0u);
case 3u:
return uvec4(u64data[offset / 8u], 0u, 0u);
case 4u:
return u128data[offset / 16u];
}
return uvec4(0u);
}
void main() {
uvec3 coord = gl_GlobalInvocationID;
if (coord.x >= pc.dim.x || coord.y >= pc.dim.y || coord.z >= pc.dim.z) {
return;
}
uvec3 pos = coord + pc.origin;
pos.x <<= pc.bytes_per_block_log2;
uint swizzle = SwizzleOffset(pos.xy);
uint block_y = pos.y >> GOB_SIZE_Y_SHIFT;
uint offset = 0u;
offset += pos.z * pc.layer_stride;
offset += (block_y >> pc.block_height) * pc.block_size;
offset += (block_y & pc.block_height_mask) << GOB_SIZE_SHIFT;
offset += (pos.x >> GOB_SIZE_X_SHIFT) << pc.x_shift;
offset += swizzle;
uvec4 texel = ReadTexel(offset);
uint words = 1u << (pc.bytes_per_block_log2 - 2u);
uint linear_index = coord.x + coord.y * pc.dim.x + coord.z * pc.dim.x * pc.dim.y;
uint out_idx = linear_index * words;
out_u32[out_idx] = texel.x;
if (words > 1u) {
out_u32[out_idx + 1u] = texel.y;
}
if (words > 2u) {
out_u32[out_idx + 2u] = texel.z;
out_u32[out_idx + 3u] = texel.w;
}
}
@@ -0,0 +1,105 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 430
#define BINDING_INPUT_BUFFER 0
#define BINDING_OUTPUT_BUFFER 1
layout(push_constant) uniform PushConstants {
uvec3 dim;
uint bytes_per_block_log2;
uvec3 origin;
uint slice_size;
uint block_size;
uint x_shift;
uint block_height;
uint block_height_mask;
uint block_depth;
uint block_depth_mask;
} pc;
layout(binding = BINDING_INPUT_BUFFER, std430) buffer InputBufferU32 { uint u32data[]; };
layout(binding = BINDING_INPUT_BUFFER, std430) buffer InputBufferU64 { uvec2 u64data[]; };
layout(binding = BINDING_INPUT_BUFFER, std430) buffer InputBufferU128 { uvec4 u128data[]; };
layout(binding = BINDING_OUTPUT_BUFFER, std430) writeonly buffer OutputBuffer {
uint out_u32[];
};
layout(local_size_x = 8, local_size_y = 8, local_size_z = 4) in;
const uint GOB_SIZE_X = 64;
const uint GOB_SIZE_Y = 8;
const uint GOB_SIZE_X_SHIFT = 6;
const uint GOB_SIZE_Y_SHIFT = 3;
const uint GOB_SIZE_SHIFT = GOB_SIZE_X_SHIFT + GOB_SIZE_Y_SHIFT;
const uvec2 SWIZZLE_MASK = uvec2(GOB_SIZE_X - 1u, GOB_SIZE_Y - 1u);
uint SwizzleTable(uint pos) {
const uint t[8] = uint[](
0x12100200, 0x13110301, 0x16140604, 0x17150705,
0x1a180a08, 0x1b190b09, 0x1e1c0e0c, 0x1f1d0f0d
);
const uint i = pos >> 4;
const uint h = (t[i / 4] >> ((i % 4) * 8)) & 0xff;
return (h << 4) | (pos & 0xf);
}
uint SwizzleOffset(uvec2 pos) {
pos = pos & SWIZZLE_MASK;
return SwizzleTable(pos.y * 64u + pos.x);
}
uvec4 ReadTexel(uint offset) {
switch (pc.bytes_per_block_log2) {
case 2u:
return uvec4(u32data[offset / 4u], 0u, 0u, 0u);
case 3u:
return uvec4(u64data[offset / 8u], 0u, 0u);
case 4u:
return u128data[offset / 16u];
}
return uvec4(0u);
}
void main() {
uvec3 coord = gl_GlobalInvocationID;
if (coord.x >= pc.dim.x || coord.y >= pc.dim.y || coord.z >= pc.dim.z) {
return;
}
uvec3 pos = coord + pc.origin;
pos.x <<= pc.bytes_per_block_log2;
uint swizzle = SwizzleOffset(pos.xy);
uint block_y = pos.y >> GOB_SIZE_Y_SHIFT;
uint offset = 0u;
offset += (pos.z >> pc.block_depth) * pc.slice_size;
offset += (pos.z & pc.block_depth_mask) << (GOB_SIZE_SHIFT + pc.block_height);
offset += (block_y >> pc.block_height) * pc.block_size;
offset += (block_y & pc.block_height_mask) << GOB_SIZE_SHIFT;
offset += (pos.x >> GOB_SIZE_X_SHIFT) << pc.x_shift;
offset += swizzle;
uvec4 texel = ReadTexel(offset);
uint words = 1u << (pc.bytes_per_block_log2 - 2u);
uint linear_index = coord.x + coord.y * pc.dim.x + coord.z * pc.dim.x * pc.dim.y;
uint out_idx = linear_index * words;
out_u32[out_idx] = texel.x;
if (words > 1u) {
out_u32[out_idx + 1u] = texel.y;
}
if (words > 2u) {
out_u32[out_idx + 2u] = texel.z;
out_u32[out_idx + 3u] = texel.w;
}
}
@@ -3,7 +3,14 @@
#version 450 core
layout(binding = 0) uniform sampler2DMS msaa_in;
#ifndef SAMPLER_TYPE
#define SAMPLER_TYPE sampler2DMS
#endif
#ifndef TEXEL_TYPE
#define TEXEL_TYPE vec4
#endif
layout(binding = 0) uniform SAMPLER_TYPE msaa_in;
layout(push_constant) uniform PushConstants {
ivec2 dst_offset;
@@ -11,7 +18,7 @@ layout(push_constant) uniform PushConstants {
ivec2 scale;
};
layout(location = 0) out vec4 frag_color;
layout(location = 0) out TEXEL_TYPE frag_color;
void main() {
const ivec2 coord = ivec2(gl_FragCoord.xy) - dst_offset + src_offset;
@@ -0,0 +1,20 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 450 core
layout(binding = 0) uniform sampler2DMS msaa_in;
layout(push_constant) uniform PushConstants {
ivec2 dst_offset;
ivec2 src_offset;
ivec2 scale;
};
void main() {
const ivec2 coord = ivec2(gl_FragCoord.xy) - dst_offset + src_offset;
const ivec2 msaa_coord = coord / scale;
const ivec2 sample_offset = coord % scale;
const int sample_id = sample_offset.x + scale.x * sample_offset.y;
gl_FragDepth = texelFetch(msaa_in, msaa_coord, sample_id).r;
}
@@ -0,0 +1,23 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 450 core
#extension GL_ARB_shader_stencil_export : require
layout(binding = 0) uniform sampler2DMS depth_tex;
layout(binding = 1) uniform usampler2DMS stencil_tex;
layout(push_constant) uniform PushConstants {
ivec2 dst_offset;
ivec2 src_offset;
ivec2 scale;
};
void main() {
const ivec2 coord = ivec2(gl_FragCoord.xy) - dst_offset + src_offset;
const ivec2 msaa_coord = coord / scale;
const ivec2 sample_offset = coord % scale;
const int sample_id = sample_offset.x + scale.x * sample_offset.y;
gl_FragDepth = texelFetch(depth_tex, msaa_coord, sample_id).r;
gl_FragStencilRefARB = int(texelFetch(stencil_tex, msaa_coord, sample_id).r);
}
@@ -3,7 +3,14 @@
#version 450 core
layout(binding = 0) uniform sampler2D img_in;
#ifndef SAMPLER_TYPE
#define SAMPLER_TYPE sampler2D
#endif
#ifndef TEXEL_TYPE
#define TEXEL_TYPE vec4
#endif
layout(binding = 0) uniform SAMPLER_TYPE img_in;
layout(push_constant) uniform PushConstants {
ivec2 dst_offset;
@@ -11,7 +18,7 @@ layout(push_constant) uniform PushConstants {
ivec2 scale;
};
layout(location = 0) out vec4 frag_color;
layout(location = 0) out TEXEL_TYPE frag_color;
void main() {
const ivec2 msaa_coord = ivec2(gl_FragCoord.xy) - dst_offset;
@@ -0,0 +1,19 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 450 core
layout(binding = 0) uniform sampler2D img_in;
layout(push_constant) uniform PushConstants {
ivec2 dst_offset;
ivec2 src_offset;
ivec2 scale;
};
void main() {
const ivec2 msaa_coord = ivec2(gl_FragCoord.xy) - dst_offset;
const ivec2 sample_offset = ivec2(gl_SampleID % scale.x, gl_SampleID / scale.x);
const ivec2 coord = msaa_coord * scale + sample_offset + src_offset;
gl_FragDepth = texelFetch(img_in, coord, 0).r;
}
@@ -0,0 +1,22 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 450 core
#extension GL_ARB_shader_stencil_export : require
layout(binding = 0) uniform sampler2D depth_tex;
layout(binding = 1) uniform usampler2D stencil_tex;
layout(push_constant) uniform PushConstants {
ivec2 dst_offset;
ivec2 src_offset;
ivec2 scale;
};
void main() {
const ivec2 msaa_coord = ivec2(gl_FragCoord.xy) - dst_offset;
const ivec2 sample_offset = ivec2(gl_SampleID % scale.x, gl_SampleID / scale.x);
const ivec2 coord = msaa_coord * scale + sample_offset + src_offset;
gl_FragDepth = texelFetch(depth_tex, coord, 0).r;
gl_FragStencilRefARB = int(texelFetch(stencil_tex, coord, 0).r);
}
@@ -7,6 +7,7 @@ layout(push_constant) uniform constants {
vec2 scale;
vec2 size;
vec2 resize_factor;
vec2 crop_offset;
float edge_sharpness;
};
layout(location = 0) out highp vec2 texcoord;
@@ -15,5 +16,5 @@ void main() {
float x = float((gl_VertexIndex & 1) << 2);
float y = float((gl_VertexIndex & 2) << 1);
gl_Position = vec4(x - 1.0f, y - 1.0f, 0.0, 1.0f) * vec4(sign(resize_factor), 1.f, 1.f);
texcoord = vec2(x, y) * abs(resize_factor) * 0.5;
texcoord = crop_offset + vec2(x, y) * abs(resize_factor) * 0.5;
}
@@ -6,34 +6,52 @@
precision highp float;
precision highp int;
// Operation modes: RGBA -> 1, RGBY -> 3, LERP -> 4
#define OPERATION_MODE 1
#define EDGE_THRESHOLD (8.0 / 255.0)
#define DIRECTION_EPSILON 6.5e-05
#define DEVIATION_FLOOR 6.0e-02
layout(push_constant) uniform constants {
vec2 scale;
vec2 size;
vec2 resize_factor;
vec2 crop_offset;
float edge_sharpness;
};
layout(set = 0, binding = 0) uniform sampler2D sampler0;
layout(location=0) in vec2 texcoord;
layout(location=0) out vec4 frag_color;
vec4 weightY(vec4 dx, vec4 dy, vec4 std) {
vec4 x = ((dx * dx) + (dy * dy)) * 0.55f + std;
return (x - 1.f) * (x - 4.f) * 3.8125f; // approx. of (x - 1) * (x - 4)^3
mediump vec4 fastLanczos2(mediump vec4 x) {
mediump vec4 wA = x - 4.0f;
mediump vec4 wB = x * wA - wA;
wA *= wA;
return wB * wA;
}
mediump vec2 edgeDirection(mediump vec4 left, mediump vec4 right) {
mediump float RxLz = right.x - left.z;
mediump float RwLy = right.w - left.y;
mediump vec2 delta = vec2(RxLz + RwLy, RxLz - RwLy);
mediump float length_inv =
inversesqrt((delta.x * delta.x + DIRECTION_EPSILON) + delta.y * delta.y);
return delta * length_inv;
}
mediump vec4 weightY(mediump vec4 dx, mediump vec4 dy, mediump vec4 c, mediump float std,
mediump vec2 dir) {
mediump vec4 edge_dis = dx * dir.y + dy * dir.x;
mediump vec4 x = (dx * dx + dy * dy) +
(edge_dis * edge_dis) * (clamp((c * c) * std, 0.0f, 1.0f) * 0.7f - 1.0f);
return fastLanczos2(x);
}
void main() {
vec4 color = textureLod(sampler0, texcoord.xy, 0.0f);
// image coord
vec2 icoord = (texcoord * size + vec2(-0.5f, 0.5f));
vec2 icoord_pixel = floor(icoord);
vec2 coord = icoord_pixel * scale;
vec2 pl = icoord - icoord_pixel;
// left: 0, right: 1, upDown: 2
mat3x4 dg = mat3x4(
mediump vec4 color = textureLod(sampler0, texcoord.xy, 0.0f);
highp vec2 icoord = (texcoord * size + vec2(-0.5f, 0.5f));
highp vec2 icoord_pixel = floor(icoord);
highp vec2 coord = icoord_pixel * scale;
mediump vec2 pl = icoord - icoord_pixel;
mediump mat3x4 dg = mat3x4(
textureGather(sampler0, coord, 1),
textureGather(sampler0, coord + vec2(2.f * scale.x, 0.0f), 1),
vec4(
@@ -41,42 +59,40 @@ void main() {
textureGather(sampler0, coord + vec2(scale.x, +scale.y), 1).yx
)
);
float edgeVote = abs(dg[0].z - dg[0].y) + abs(color.y - dg[0].y) + abs(color.y - dg[0].z);
mediump float edgeVote =
abs(dg[0].z - dg[0].y) + abs(color.y - dg[0].y) + abs(color.y - dg[0].z);
if (edgeVote > EDGE_THRESHOLD) {
float mean = (dg[0].y + dg[0].z + dg[1].x + dg[1].w) * 0.25f;
mediump float mean = (dg[0].y + dg[0].z + dg[1].x + dg[1].w) * 0.25f;
dg = dg - mean;
vec4 sum = abs(dg[0]) + abs(dg[1]) + abs(dg[2]);
float std = 2.181818f / (sum.x + sum.y + sum.z + sum.w);
mat2x4 w = mat2x4(
weightY(
pl.xxxx + vec4(+1.0f, +0.0f, +0.0f, +1.0f),
pl.yyyy + vec4(-1.0f, -1.0f, +0.0f, +0.0f),
clamp(abs(dg[0]) * std, 0.0f, 1.0f)
) + weightY(
pl.xxxx + vec4(-1.0f, -2.0f, -2.0f, -1.0f),
pl.yyyy + vec4(-1.0f, -1.0f, +0.0f, +0.0f),
clamp(abs(dg[1]) * std, 0.0f, 1.0f)
) + weightY(
pl.xxxx + vec4(+0.0f, -1.0f, -1.0f, +0.0f),
pl.yyyy + vec4(+1.0f, +1.0f, -2.0f, -2.0f),
clamp(abs(dg[2]) * std, 0.0f, 1.0f)
),
dg[0] + dg[1] + dg[2]
mediump float sum = dot(abs(dg[0]) + abs(dg[1]) + abs(dg[2]), vec4(1.0f));
mediump float sum_mean = 1.014185e+01f / max(sum, DEVIATION_FLOOR);
mediump float std = sum_mean * sum_mean;
mediump vec2 dir = edgeDirection(dg[0], dg[1]);
mediump vec4 w0 = weightY(
pl.xxxx + vec4(+1.0f, +0.0f, +0.0f, +1.0f),
pl.yyyy + vec4(-1.0f, -1.0f, +0.0f, +0.0f),
dg[0], std, dir
);
// compute final y with bounds
vec2 yb = vec2(
min(min(dg[0].y, dg[0].z), min(dg[1].x, dg[1].w)), // min
max(max(dg[0].y, dg[0].z), max(dg[1].x, dg[1].w)) // max
mediump vec4 w1 = weightY(
pl.xxxx + vec4(-1.0f, -2.0f, -2.0f, -1.0f),
pl.yyyy + vec4(-1.0f, -1.0f, +0.0f, +0.0f),
dg[1], std, dir
);
vec2 fvy = vec2(
w[0].x + w[0].y + w[0].z + w[0].w,
w[1].x + w[1].y + w[1].z + w[1].w
mediump vec4 w2 = weightY(
pl.xxxx + vec4(+0.0f, -1.0f, -1.0f, +0.0f),
pl.yyyy + vec4(+1.0f, +1.0f, -2.0f, -2.0f),
dg[2], std, dir
);
float fy = clamp((fvy.y / fvy.x) * edge_sharpness, yb[0], yb[1]);
// Smooth high contrast input
float dy = clamp(fy - color.y + mean, -23.0f / 255.0f, 23.0f / 255.0f);
mediump float sum_w = dot(w0 + w1 + w2, vec4(1.0f));
mediump float sum_wc = dot(w0 * dg[0] + w1 * dg[1] + w2 * dg[2], vec4(1.0f));
mediump vec2 yb = vec2(
min(min(dg[0].y, dg[0].z), min(dg[1].x, dg[1].w)),
max(max(dg[0].y, dg[0].z), max(dg[1].x, dg[1].w))
);
mediump float fy = clamp((sum_wc / sum_w) * edge_sharpness, yb[0], yb[1]);
mediump float dy = clamp(fy - color.y + mean, -23.0f / 255.0f, 23.0f / 255.0f);
color = clamp(color + dy, 0.0f, 1.0f);
}
color.w = 1.0f; //assume alpha channel is not used
color.w = 1.0f;
frag_color.xyzw = color;
}
}
@@ -13,6 +13,7 @@
layout( push_constant ) uniform constants {
vec4 ViewportInfo[1];
vec2 ResizeFactor;
vec2 CropOffset;
float EdgeSharpness;
};
layout(set = 0, binding = 0) uniform sampler2D ps0;
@@ -261,6 +261,7 @@ struct BufferCacheParams {
// TODO: Investigate why OpenGL seems to perform worse with persistently mapped buffer uploads
static constexpr bool USE_MEMORY_MAPS_FOR_UPLOADS = false;
static constexpr bool USE_UNIFIED_MEMORY = false;
};
using BufferCache = VideoCommon::BufferCache<BufferCacheParams>;
@@ -231,12 +231,14 @@ ShaderCache::ShaderCache(Tegra::MaxwellDeviceMemoryManager& device_memory_,
.has_broken_unsigned_image_offsets = true,
.has_broken_signed_operations = true,
.has_broken_fp16_float_controls = false,
.has_broken_fp32_denorm_flush = false,
.has_gl_component_indexing_bug = device.HasComponentIndexingBug(),
.has_gl_precise_bug = device.HasPreciseBug(),
.has_gl_cbuf_ftou_bug = device.HasCbufFtouBug(),
.has_gl_bool_ref_bug = device.HasBoolRefBug(),
.ignore_nan_fp_comparisons = true,
.gl_max_compute_smem_size = device.GetMaxComputeSharedMemorySize(),
.max_shared_memory_size = device.GetMaxComputeSharedMemorySize(),
.min_ssbo_alignment = device.GetShaderStorageBufferAlignment(),
// Use the host limit, but never more than the guest can produce. Maxwell exposes 8 clip
// distances and the SPIR-V output array is sized for at most 8, so clamping here keeps a
@@ -75,6 +75,8 @@ public:
void Finish();
void FlushDeferredClear() {}
StagingBufferMap UploadStagingBuffer(size_t size, bool deferred = false);
StagingBufferMap DownloadStagingBuffer(size_t size, bool deferred = false);
@@ -370,6 +372,8 @@ struct TextureCacheParams {
static constexpr bool HAS_EMULATED_COPIES = true;
static constexpr bool HAS_DEVICE_MEMORY_INFO = true;
static constexpr bool IMPLEMENTS_ASYNC_DOWNLOADS = true;
static constexpr bool HAS_MSAA_DOWNLOADS = false;
static constexpr bool USE_UNIFIED_MEMORY = false;
using Runtime = OpenGL::TextureCacheRuntime;
using Image = OpenGL::Image;
+555 -237
View File
@@ -12,6 +12,7 @@
#include "common/settings.h"
#include "video_core/host_shaders/blit_color_float_frag_spv.h"
#include "video_core/host_shaders/blit_color_msaa_frag_spv.h"
#include "video_core/host_shaders/blit_depth_frag_spv.h"
#include "video_core/host_shaders/blit_depth_msaa_frag_spv.h"
#include "video_core/host_shaders/blit_depth_stencil_msaa_frag_spv.h"
#include "video_core/host_shaders/convert_abgr8_to_d24s8_frag_spv.h"
@@ -21,7 +22,15 @@
#include "video_core/host_shaders/convert_depth_to_float_frag_spv.h"
#include "video_core/host_shaders/convert_float_to_depth_frag_spv.h"
#include "video_core/host_shaders/convert_msaa_to_non_msaa_frag_spv.h"
#include "video_core/host_shaders/convert_msaa_to_non_msaa_depth_frag_spv.h"
#include "video_core/host_shaders/convert_msaa_to_non_msaa_depth_stencil_frag_spv.h"
#include "video_core/host_shaders/convert_msaa_to_non_msaa_sint_frag_spv.h"
#include "video_core/host_shaders/convert_msaa_to_non_msaa_uint_frag_spv.h"
#include "video_core/host_shaders/convert_non_msaa_to_msaa_frag_spv.h"
#include "video_core/host_shaders/convert_non_msaa_to_msaa_sint_frag_spv.h"
#include "video_core/host_shaders/convert_non_msaa_to_msaa_uint_frag_spv.h"
#include "video_core/host_shaders/convert_non_msaa_to_msaa_depth_frag_spv.h"
#include "video_core/host_shaders/convert_non_msaa_to_msaa_depth_stencil_frag_spv.h"
#include "video_core/host_shaders/convert_s8d24_to_abgr8_frag_spv.h"
#include "video_core/host_shaders/full_screen_triangle_vert_spv.h"
#include "video_core/host_shaders/vulkan_blit_depth_stencil_frag_spv.h"
@@ -518,8 +527,19 @@ void RecordShaderReadBarrier(Scheduler& scheduler, const ImageView& image_view)
}
}
[[nodiscard]] MSAACopyFormatClass FormatClass(VideoCore::Surface::PixelFormat format) {
if (!VideoCore::Surface::IsPixelFormatInteger(format)) {
return MSAACopyFormatClass::Float;
}
if (VideoCore::Surface::IsPixelFormatSignedInteger(format)) {
return MSAACopyFormatClass::SignedInteger;
}
return MSAACopyFormatClass::UnsignedInteger;
}
[[nodiscard]] vk::ImageView MakeMSAACopyView(const vk::Device& device, VkImage image,
VkFormat format, u32 base_level) {
VkFormat format, u32 base_level, u32 base_layer,
VkImageAspectFlags aspect_mask) {
return device.CreateImageView(VkImageViewCreateInfo{
.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO,
.pNext = nullptr,
@@ -534,10 +554,10 @@ void RecordShaderReadBarrier(Scheduler& scheduler, const ImageView& image_view)
.a = VK_COMPONENT_SWIZZLE_IDENTITY,
},
.subresourceRange{
.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT,
.aspectMask = aspect_mask,
.baseMipLevel = base_level,
.levelCount = 1,
.baseArrayLayer = 0,
.baseArrayLayer = base_layer,
.layerCount = 1,
},
});
@@ -586,12 +606,17 @@ BlitImageHelper::BlitImageHelper(const Device& device_, Scheduler& scheduler_,
msaa_copy_pipeline_layout(device.GetLogical().CreatePipelineLayout(PipelineLayoutCreateInfo(
one_texture_set_layout.address(),
PUSH_CONSTANT_RANGE<VK_SHADER_STAGE_FRAGMENT_BIT, sizeof(MSAACopyPushConstants)>))),
msaa_copy_depth_stencil_pipeline_layout(
device.GetLogical().CreatePipelineLayout(PipelineLayoutCreateInfo(
two_textures_set_layout.address(),
PUSH_CONSTANT_RANGE<VK_SHADER_STAGE_FRAGMENT_BIT, sizeof(MSAACopyPushConstants)>))),
full_screen_vert(BuildShader(device, FULL_SCREEN_TRIANGLE_VERT_SPV)),
blit_color_to_color_frag(BuildShader(device, BLIT_COLOR_FLOAT_FRAG_SPV)),
blit_color_msaa_frag(BuildShader(device, BLIT_COLOR_MSAA_FRAG_SPV)),
blit_depth_stencil_frag(device.IsExtShaderStencilExportSupported()
? BuildShader(device, VULKAN_BLIT_DEPTH_STENCIL_FRAG_SPV)
: vk::ShaderModule{}),
blit_depth_frag(BuildShader(device, BLIT_DEPTH_FRAG_SPV)),
blit_depth_msaa_frag(BuildShader(device, BLIT_DEPTH_MSAA_FRAG_SPV)),
blit_depth_stencil_msaa_frag(device.IsExtShaderStencilExportSupported()
? BuildShader(device, BLIT_DEPTH_STENCIL_MSAA_FRAG_SPV)
@@ -609,7 +634,25 @@ BlitImageHelper::BlitImageHelper(const Device& device_, Scheduler& scheduler_,
convert_d24s8_to_abgr8_frag(BuildShader(device, CONVERT_D24S8_TO_ABGR8_FRAG_SPV)),
convert_s8d24_to_abgr8_frag(BuildShader(device, CONVERT_S8D24_TO_ABGR8_FRAG_SPV)),
convert_msaa_to_non_msaa_frag(BuildShader(device, CONVERT_MSAA_TO_NON_MSAA_FRAG_SPV)),
convert_msaa_to_non_msaa_sint_frag(
BuildShader(device, CONVERT_MSAA_TO_NON_MSAA_SINT_FRAG_SPV)),
convert_msaa_to_non_msaa_uint_frag(
BuildShader(device, CONVERT_MSAA_TO_NON_MSAA_UINT_FRAG_SPV)),
convert_msaa_to_non_msaa_depth_frag(
BuildShader(device, CONVERT_MSAA_TO_NON_MSAA_DEPTH_FRAG_SPV)),
convert_msaa_to_non_msaa_depth_stencil_frag(
BuildShader(device, CONVERT_MSAA_TO_NON_MSAA_DEPTH_STENCIL_FRAG_SPV)),
convert_non_msaa_to_msaa_frag(BuildShader(device, CONVERT_NON_MSAA_TO_MSAA_FRAG_SPV)),
convert_non_msaa_to_msaa_sint_frag(
BuildShader(device, CONVERT_NON_MSAA_TO_MSAA_SINT_FRAG_SPV)),
convert_non_msaa_to_msaa_uint_frag(
BuildShader(device, CONVERT_NON_MSAA_TO_MSAA_UINT_FRAG_SPV)),
convert_non_msaa_to_msaa_depth_frag(
BuildShader(device, CONVERT_NON_MSAA_TO_MSAA_DEPTH_FRAG_SPV)),
convert_non_msaa_to_msaa_depth_stencil_frag(
device.IsExtShaderStencilExportSupported()
? BuildShader(device, CONVERT_NON_MSAA_TO_MSAA_DEPTH_STENCIL_FRAG_SPV)
: vk::ShaderModule{}),
linear_sampler(device.GetLogical().CreateSampler(SAMPLER_CREATE_INFO<VK_FILTER_LINEAR>)),
nearest_sampler(device.GetLogical().CreateSampler(SAMPLER_CREATE_INFO<VK_FILTER_NEAREST>)) {}
@@ -624,24 +667,13 @@ void BlitImageHelper::BlitColor(const Framebuffer* dst_framebuffer, const ImageV
.renderpass = dst_framebuffer->RenderPass(),
.operation = operation,
};
const VkPipelineLayout layout = *one_texture_pipeline_layout;
const VkSampler sampler = is_linear ? *linear_sampler : *nearest_sampler;
const VkPipeline pipeline = FindOrEmplaceColorPipeline(key);
const VkImageView src_view = src_image_view.Handle(Shader::TextureType::Color2D);
RecordShaderReadBarrier(scheduler, src_image_view);
scheduler.RequestRenderpass(dst_framebuffer);
scheduler.Record([this, dst_region, src_region, pipeline, layout, sampler,
src_view](vk::CommandBuffer cmdbuf) {
const VkDescriptorSet descriptor_set = one_texture_descriptor_allocator.Commit();
UpdateOneTextureDescriptorSet(device, descriptor_set, sampler, src_view);
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
BindBlitState(cmdbuf, layout, dst_region, src_region);
cmdbuf.Draw(3, 1, 0, 0);
});
scheduler.InvalidateState();
VkSampler sampler = *nearest_sampler;
if (is_linear) {
sampler = *linear_sampler;
}
BlitImpl(dst_framebuffer, src_image_view, dst_region, src_region,
FindOrEmplaceColorPipeline(key), sampler,
src_image_view.Handle(Shader::TextureType::Color2D), VK_NULL_HANDLE, false);
}
void BlitImageHelper::BlitColor(const Framebuffer* dst_framebuffer, VkImageView src_image_view,
@@ -670,6 +702,38 @@ void BlitImageHelper::BlitColor(const Framebuffer* dst_framebuffer, VkImageView
});
}
void BlitImageHelper::BlitImpl(const Framebuffer* dst_framebuffer,
const ImageView& src_image_view, const Region2D& dst_region,
const Region2D& src_region, VkPipeline pipeline, VkSampler sampler,
VkImageView src_view, VkImageView src_stencil_view,
bool blit_stencil) {
VkPipelineLayout layout = *one_texture_pipeline_layout;
if (blit_stencil) {
layout = *two_textures_pipeline_layout;
}
RecordShaderReadBarrier(scheduler, src_image_view);
scheduler.RequestRenderpass(dst_framebuffer);
scheduler.Record([this, dst_region, src_region, pipeline, layout, sampler, src_view,
src_stencil_view, blit_stencil](vk::CommandBuffer cmdbuf) {
VkDescriptorSet descriptor_set = VK_NULL_HANDLE;
if (blit_stencil) {
descriptor_set = two_textures_descriptor_allocator.Commit();
UpdateTwoTexturesDescriptorSet(device, descriptor_set, sampler, src_view,
src_stencil_view);
} else {
descriptor_set = one_texture_descriptor_allocator.Commit();
UpdateOneTextureDescriptorSet(device, descriptor_set, sampler, src_view);
}
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
BindBlitState(cmdbuf, layout, dst_region, src_region);
cmdbuf.Draw(3, 1, 0, 0);
});
scheduler.InvalidateState();
}
void BlitImageHelper::BlitColorMSAA(const Framebuffer* dst_framebuffer,
const ImageView& src_image_view, const Region2D& dst_region,
const Region2D& src_region) {
@@ -677,24 +741,34 @@ void BlitImageHelper::BlitColorMSAA(const Framebuffer* dst_framebuffer,
.renderpass = dst_framebuffer->RenderPass(),
.samples = dst_framebuffer->Samples(),
};
const VkPipelineLayout layout = *one_texture_pipeline_layout;
const VkSampler sampler = *nearest_sampler;
const VkPipeline pipeline = FindOrEmplaceBlitColorMSAAPipeline(key);
const VkImageView src_view = src_image_view.Handle(Shader::TextureType::Color2D);
BlitImpl(dst_framebuffer, src_image_view, dst_region, src_region,
FindOrEmplaceBlitColorMSAAPipeline(key), *nearest_sampler,
src_image_view.Handle(Shader::TextureType::Color2D), VK_NULL_HANDLE, false);
}
RecordShaderReadBarrier(scheduler, src_image_view);
scheduler.RequestRenderpass(dst_framebuffer);
scheduler.Record([this, dst_region, src_region, pipeline, layout, sampler,
src_view](vk::CommandBuffer cmdbuf) {
const VkDescriptorSet descriptor_set = one_texture_descriptor_allocator.Commit();
UpdateOneTextureDescriptorSet(device, descriptor_set, sampler, src_view);
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
BindBlitState(cmdbuf, layout, dst_region, src_region);
cmdbuf.Draw(3, 1, 0, 0);
});
scheduler.InvalidateState();
void BlitImageHelper::BlitDepthStencilMSAA(const Framebuffer* dst_framebuffer,
ImageView& src_image_view, const Region2D& dst_region,
const Region2D& src_region) {
const bool blit_stencil =
dst_framebuffer->HasAspectStencilBit() && device.IsExtShaderStencilExportSupported();
const BlitMSAAPipelineKey key{
.renderpass = dst_framebuffer->RenderPass(),
.samples = dst_framebuffer->Samples(),
};
VkImageView src_stencil_view = VK_NULL_HANDLE;
if (blit_stencil) {
src_stencil_view = src_image_view.StencilView();
}
BlitImpl(dst_framebuffer, src_image_view, dst_region, src_region,
FindOrEmplaceBlitDepthStencilMSAAPipeline(key, blit_stencil), *nearest_sampler,
src_image_view.DepthView(), src_stencil_view, blit_stencil);
}
void BlitImageHelper::BlitDepth(const Framebuffer* dst_framebuffer, ImageView& src_image_view,
const Region2D& dst_region, const Region2D& src_region) {
BlitImpl(dst_framebuffer, src_image_view, dst_region, src_region,
FindOrEmplaceBlitDepthPipeline(dst_framebuffer->RenderPass()), *nearest_sampler,
src_image_view.DepthView(), VK_NULL_HANDLE, false);
}
void BlitImageHelper::ResolveDepthStencil(const Framebuffer* dst_framebuffer,
@@ -702,36 +776,14 @@ void BlitImageHelper::ResolveDepthStencil(const Framebuffer* dst_framebuffer,
const Region2D& src_region) {
const bool resolve_stencil =
dst_framebuffer->HasAspectStencilBit() && device.IsExtShaderStencilExportSupported();
const VkPipeline pipeline =
FindOrEmplaceResolveDepthStencilPipeline(dst_framebuffer->RenderPass(), resolve_stencil);
const VkPipelineLayout layout =
resolve_stencil ? *two_textures_pipeline_layout : *one_texture_pipeline_layout;
const VkSampler sampler = *nearest_sampler;
const VkImageView src_depth_view = src_image_view.DepthView();
const VkImageView src_stencil_view =
resolve_stencil ? src_image_view.StencilView() : VK_NULL_HANDLE;
RecordShaderReadBarrier(scheduler, src_image_view);
scheduler.RequestRenderpass(dst_framebuffer);
scheduler.Record([this, dst_region, src_region, pipeline, layout, sampler, src_depth_view,
src_stencil_view, resolve_stencil](vk::CommandBuffer cmdbuf) {
if (resolve_stencil) {
const VkDescriptorSet descriptor_set = two_textures_descriptor_allocator.Commit();
UpdateTwoTexturesDescriptorSet(device, descriptor_set, sampler, src_depth_view,
src_stencil_view);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
} else {
const VkDescriptorSet descriptor_set = one_texture_descriptor_allocator.Commit();
UpdateOneTextureDescriptorSet(device, descriptor_set, sampler, src_depth_view);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
}
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
BindBlitState(cmdbuf, layout, dst_region, src_region);
cmdbuf.Draw(3, 1, 0, 0);
});
scheduler.InvalidateState();
VkImageView src_stencil_view = VK_NULL_HANDLE;
if (resolve_stencil) {
src_stencil_view = src_image_view.StencilView();
}
BlitImpl(dst_framebuffer, src_image_view, dst_region, src_region,
FindOrEmplaceResolveDepthStencilPipeline(dst_framebuffer->RenderPass(),
resolve_stencil),
*nearest_sampler, src_image_view.DepthView(), src_stencil_view, resolve_stencil);
}
void BlitImageHelper::BlitDepthStencil(const Framebuffer* dst_framebuffer,
@@ -739,35 +791,23 @@ void BlitImageHelper::BlitDepthStencil(const Framebuffer* dst_framebuffer,
const Region2D& dst_region, const Region2D& src_region,
Tegra::Engines::Fermi2D::Filter filter,
Tegra::Engines::Fermi2D::Operation operation) {
if (!device.IsExtShaderStencilExportSupported()) {
return;
}
ASSERT(filter == Tegra::Engines::Fermi2D::Filter::Point);
ASSERT(operation == Tegra::Engines::Fermi2D::Operation::SrcCopy);
const bool blit_stencil = device.IsExtShaderStencilExportSupported();
const BlitImagePipelineKey key{
.renderpass = dst_framebuffer->RenderPass(),
.operation = operation,
};
const VkPipelineLayout layout = *two_textures_pipeline_layout;
const VkSampler sampler = *nearest_sampler;
const VkPipeline pipeline = FindOrEmplaceDepthStencilPipeline(key);
const VkImageView src_depth_view = src_image_view.DepthView();
const VkImageView src_stencil_view = src_image_view.StencilView();
RecordShaderReadBarrier(scheduler, src_image_view);
scheduler.RequestRenderpass(dst_framebuffer);
scheduler.Record([dst_region, src_region, pipeline, layout, sampler, src_depth_view,
src_stencil_view, this](vk::CommandBuffer cmdbuf) {
const VkDescriptorSet descriptor_set = two_textures_descriptor_allocator.Commit();
UpdateTwoTexturesDescriptorSet(device, descriptor_set, sampler, src_depth_view,
src_stencil_view);
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
BindBlitState(cmdbuf, layout, dst_region, src_region);
cmdbuf.Draw(3, 1, 0, 0);
});
scheduler.InvalidateState();
VkPipeline pipeline{};
VkImageView src_stencil_view = VK_NULL_HANDLE;
if (blit_stencil) {
pipeline = FindOrEmplaceDepthStencilPipeline(key);
src_stencil_view = src_image_view.StencilView();
} else {
pipeline = FindOrEmplaceBlitDepthPipeline(key.renderpass);
}
BlitImpl(dst_framebuffer, src_image_view, dst_region, src_region, pipeline, *nearest_sampler,
src_image_view.DepthView(), src_stencil_view, blit_stencil);
}
void BlitImageHelper::ConvertD32ToR32(const Framebuffer* dst_framebuffer,
@@ -882,19 +922,183 @@ void BlitImageHelper::ClearDepthStencil(const Framebuffer* dst_framebuffer, bool
scheduler.InvalidateState();
}
void BlitImageHelper::CopyMSAAImpl(VkRenderPass renderpass, VkPipeline pipeline,
VkPipelineLayout layout, VkImage dst_image,
VkFormat dst_vk_format, VkImage src_image,
VkFormat src_vk_format, s32 scale_x, s32 scale_y,
std::span<const VideoCommon::ImageCopy> copies,
const MSAACopyAspectInfo& aspect_info, bool copy_stencil) {
while (!msaa_copy_resources.empty() && scheduler.IsFree(msaa_copy_resources.front().tick)) {
msaa_copy_resources.pop_front();
}
const VkSampler sampler = *nearest_sampler;
for (const VideoCommon::ImageCopy& copy : copies) {
const s32 num_layers = (std::min)(copy.src_subresource.num_layers,
copy.dst_subresource.num_layers);
for (s32 layer = 0; layer < num_layers; ++layer) {
const u32 src_level = static_cast<u32>(copy.src_subresource.base_level);
const u32 src_layer = static_cast<u32>(copy.src_subresource.base_layer + layer);
vk::ImageView src_view =
MakeMSAACopyView(device.GetLogical(), src_image, src_vk_format, src_level,
src_layer, aspect_info.src_view_aspect);
vk::ImageView src_stencil_view;
if (copy_stencil) {
src_stencil_view =
MakeMSAACopyView(device.GetLogical(), src_image, src_vk_format, src_level,
src_layer, VK_IMAGE_ASPECT_STENCIL_BIT);
}
vk::ImageView dst_view =
MakeMSAACopyView(device.GetLogical(), dst_image, dst_vk_format,
static_cast<u32>(copy.dst_subresource.base_level),
static_cast<u32>(copy.dst_subresource.base_layer + layer),
aspect_info.attachment_aspect);
const VkOffset2D dst_offset{copy.dst_offset.x, copy.dst_offset.y};
const VkExtent2D dst_extent{copy.extent.width, copy.extent.height};
const VkRect2D render_area{
.offset = dst_offset,
.extent = dst_extent,
};
vk::Framebuffer framebuffer = device.GetLogical().CreateFramebuffer(VkFramebufferCreateInfo{
.sType = VK_STRUCTURE_TYPE_FRAMEBUFFER_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.renderPass = renderpass,
.attachmentCount = 1,
.pAttachments = dst_view.address(),
.width = static_cast<u32>(dst_offset.x) + dst_extent.width,
.height = static_cast<u32>(dst_offset.y) + dst_extent.height,
.layers = 1,
});
const MSAACopyPushConstants push_constants{
.dst_offset = {dst_offset.x, dst_offset.y},
.src_offset = {copy.src_offset.x, copy.src_offset.y},
.scale = {scale_x, scale_y},
};
VkImageView src_stencil_handle = VK_NULL_HANDLE;
if (copy_stencil) {
src_stencil_handle = *src_stencil_view;
}
scheduler.RequestOutsideRenderPassOperationContext();
scheduler.Record([this, pipeline, layout, sampler, renderpass,
framebuffer_handle = *framebuffer, src_view_handle = *src_view,
src_stencil_handle, src = src_image, dst = dst_image, render_area,
aspect_info, push_constants](vk::CommandBuffer cmdbuf) {
const VkImageSubresourceRange barrier_range{
.aspectMask = aspect_info.barrier_aspect,
.baseMipLevel = 0,
.levelCount = VK_REMAINING_MIP_LEVELS,
.baseArrayLayer = 0,
.layerCount = VK_REMAINING_ARRAY_LAYERS,
};
const std::array pre_barriers{
VkImageMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = aspect_info.pre_src_access,
.dstAccessMask = aspect_info.pre_src_dst_access,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = src,
.subresourceRange = barrier_range,
},
VkImageMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = aspect_info.pre_src_access,
.dstAccessMask = aspect_info.pre_dst_dst_access,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst,
.subresourceRange = barrier_range,
},
};
cmdbuf.PipelineBarrier(aspect_info.pre_src_stages, aspect_info.pre_dst_stages, 0,
nullptr, nullptr, pre_barriers);
const VkRenderPassBeginInfo renderpass_bi{
.sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO,
.pNext = nullptr,
.renderPass = renderpass,
.framebuffer = framebuffer_handle,
.renderArea = render_area,
.clearValueCount = 0,
.pClearValues = nullptr,
};
cmdbuf.BeginRenderPass(renderpass_bi, VK_SUBPASS_CONTENTS_INLINE);
VkDescriptorSet descriptor_set = VK_NULL_HANDLE;
if (src_stencil_handle != VK_NULL_HANDLE) {
descriptor_set = two_textures_descriptor_allocator.Commit();
UpdateTwoTexturesDescriptorSet(device, descriptor_set, sampler, src_view_handle,
src_stencil_handle);
} else {
descriptor_set = one_texture_descriptor_allocator.Commit();
UpdateOneTextureDescriptorSet(device, descriptor_set, sampler, src_view_handle);
}
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
const VkViewport viewport{
.x = static_cast<float>(render_area.offset.x),
.y = static_cast<float>(render_area.offset.y),
.width = static_cast<float>(render_area.extent.width),
.height = static_cast<float>(render_area.extent.height),
.minDepth = 0.0f,
.maxDepth = 1.0f,
};
cmdbuf.SetViewport(0, viewport);
cmdbuf.SetScissor(0, render_area);
cmdbuf.PushConstants(layout, VK_SHADER_STAGE_FRAGMENT_BIT, push_constants);
cmdbuf.Draw(3, 1, 0, 0);
cmdbuf.EndRenderPass();
const VkImageMemoryBarrier post_barrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = aspect_info.post_src_access,
.dstAccessMask = aspect_info.post_dst_access,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst,
.subresourceRange = barrier_range,
};
cmdbuf.PipelineBarrier(aspect_info.post_src_stages, aspect_info.post_dst_stages, 0,
post_barrier);
});
msaa_copy_resources.push_back(MSAACopyResources{
.tick = scheduler.CurrentTick(),
.src_view = std::move(src_view),
.dst_view = std::move(dst_view),
.framebuffer = std::move(framebuffer),
});
if (copy_stencil) {
msaa_copy_resources.push_back(MSAACopyResources{
.tick = scheduler.CurrentTick(),
.src_view = std::move(src_stencil_view),
.dst_view = vk::ImageView{},
.framebuffer = vk::Framebuffer{},
});
}
}
}
scheduler.InvalidateState();
}
void BlitImageHelper::CopyMSAA(RenderPassCache& render_pass_cache, VkImage dst_image,
VideoCore::Surface::PixelFormat dst_format, VkImage src_image,
VideoCore::Surface::PixelFormat src_format, u32 num_samples,
std::span<const VideoCommon::ImageCopy> copies,
bool msaa_to_non_msaa) {
while (!msaa_copy_resources.empty() && scheduler.IsFree(msaa_copy_resources.front().tick)) {
msaa_copy_resources.pop_front();
}
const auto [samples_x, samples_y] = VideoCommon::SamplesLog2(static_cast<int>(num_samples));
const s32 scale_x = 1 << samples_x;
const s32 scale_y = 1 << samples_y;
const VkSampleCountFlagBits samples =
msaa_to_non_msaa ? VK_SAMPLE_COUNT_1_BIT : SampleCountFlag(num_samples);
VkSampleCountFlagBits samples = SampleCountFlag(num_samples);
if (msaa_to_non_msaa) {
samples = VK_SAMPLE_COUNT_1_BIT;
}
RenderPassKey renderpass_key{};
renderpass_key.color_formats.fill(VideoCore::Surface::PixelFormat::Invalid);
renderpass_key.color_formats[0] = dst_format;
@@ -905,149 +1109,35 @@ void BlitImageHelper::CopyMSAA(RenderPassCache& render_pass_cache, VkImage dst_i
.renderpass = renderpass,
.samples = samples,
.msaa_to_non_msaa = msaa_to_non_msaa,
.format_class = FormatClass(dst_format),
};
const MSAACopyAspectInfo aspect_info{
.src_view_aspect = VK_IMAGE_ASPECT_COLOR_BIT,
.attachment_aspect = VK_IMAGE_ASPECT_COLOR_BIT,
.barrier_aspect = VK_IMAGE_ASPECT_COLOR_BIT,
.pre_src_access = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT | VK_ACCESS_SHADER_WRITE_BIT |
VK_ACCESS_TRANSFER_WRITE_BIT,
.pre_src_dst_access = VK_ACCESS_SHADER_READ_BIT,
.pre_dst_dst_access =
VK_ACCESS_COLOR_ATTACHMENT_READ_BIT | VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
.pre_src_stages = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT |
VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT |
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT | VK_PIPELINE_STAGE_TRANSFER_BIT,
.pre_dst_stages =
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT | VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
.post_src_access = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
.post_dst_access = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_TRANSFER_READ_BIT,
.post_src_stages = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
.post_dst_stages = VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT |
VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT | VK_PIPELINE_STAGE_TRANSFER_BIT,
};
const VkPipeline pipeline = FindOrEmplaceMSAACopyPipeline(key);
const VkPipelineLayout layout = *msaa_copy_pipeline_layout;
const VkSampler sampler = *nearest_sampler;
const VkFormat src_vk_format =
MaxwellToVK::SurfaceFormat(device, FormatType::Optimal, true, src_format).format;
const VkFormat dst_vk_format =
MaxwellToVK::SurfaceFormat(device, FormatType::Optimal, true, dst_format).format;
for (const VideoCommon::ImageCopy& copy : copies) {
ASSERT(copy.src_subresource.base_layer == 0);
ASSERT(copy.src_subresource.num_layers == 1);
ASSERT(copy.dst_subresource.base_layer == 0);
ASSERT(copy.dst_subresource.num_layers == 1);
vk::ImageView src_view =
MakeMSAACopyView(device.GetLogical(), src_image, src_vk_format,
static_cast<u32>(copy.src_subresource.base_level));
vk::ImageView dst_view =
MakeMSAACopyView(device.GetLogical(), dst_image, dst_vk_format,
static_cast<u32>(copy.dst_subresource.base_level));
const VkOffset2D dst_offset{copy.dst_offset.x, copy.dst_offset.y};
const VkExtent2D dst_extent{copy.extent.width, copy.extent.height};
const VkRect2D render_area{
.offset = dst_offset,
.extent = dst_extent,
};
vk::Framebuffer framebuffer = device.GetLogical().CreateFramebuffer(VkFramebufferCreateInfo{
.sType = VK_STRUCTURE_TYPE_FRAMEBUFFER_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.renderPass = renderpass,
.attachmentCount = 1,
.pAttachments = dst_view.address(),
.width = static_cast<u32>(dst_offset.x) + dst_extent.width,
.height = static_cast<u32>(dst_offset.y) + dst_extent.height,
.layers = 1,
});
const MSAACopyPushConstants push_constants{
.dst_offset = {dst_offset.x, dst_offset.y},
.src_offset = {copy.src_offset.x, copy.src_offset.y},
.scale = {scale_x, scale_y},
};
scheduler.RequestOutsideRenderPassOperationContext();
scheduler.Record([this, pipeline, layout, sampler, renderpass,
framebuffer_handle = *framebuffer, src_view_handle = *src_view,
src = src_image, dst = dst_image, render_area,
push_constants](vk::CommandBuffer cmdbuf) {
constexpr VkImageSubresourceRange color_range{
.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT,
.baseMipLevel = 0,
.levelCount = VK_REMAINING_MIP_LEVELS,
.baseArrayLayer = 0,
.layerCount = VK_REMAINING_ARRAY_LAYERS,
};
const std::array pre_barriers{
VkImageMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT |
VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_SHADER_READ_BIT,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = src,
.subresourceRange = color_range,
},
VkImageMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT |
VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_COLOR_ATTACHMENT_READ_BIT |
VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst,
.subresourceRange = color_range,
},
};
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT |
VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT |
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT |
VK_PIPELINE_STAGE_TRANSFER_BIT,
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT |
VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
0, nullptr, nullptr, pre_barriers);
const VkRenderPassBeginInfo renderpass_bi{
.sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO,
.pNext = nullptr,
.renderPass = renderpass,
.framebuffer = framebuffer_handle,
.renderArea = render_area,
.clearValueCount = 0,
.pClearValues = nullptr,
};
cmdbuf.BeginRenderPass(renderpass_bi, VK_SUBPASS_CONTENTS_INLINE);
const VkDescriptorSet descriptor_set = one_texture_descriptor_allocator.Commit();
UpdateOneTextureDescriptorSet(device, descriptor_set, sampler, src_view_handle);
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
const VkViewport viewport{
.x = static_cast<float>(render_area.offset.x),
.y = static_cast<float>(render_area.offset.y),
.width = static_cast<float>(render_area.extent.width),
.height = static_cast<float>(render_area.extent.height),
.minDepth = 0.0f,
.maxDepth = 1.0f,
};
cmdbuf.SetViewport(0, viewport);
cmdbuf.SetScissor(0, render_area);
cmdbuf.PushConstants(layout, VK_SHADER_STAGE_FRAGMENT_BIT, push_constants);
cmdbuf.Draw(3, 1, 0, 0);
cmdbuf.EndRenderPass();
const VkImageMemoryBarrier post_barrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
.dstAccessMask = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_TRANSFER_READ_BIT,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst,
.subresourceRange = color_range,
};
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT |
VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT |
VK_PIPELINE_STAGE_TRANSFER_BIT,
0, post_barrier);
});
msaa_copy_resources.push_back(MSAACopyResources{
.tick = scheduler.CurrentTick(),
.src_view = std::move(src_view),
.dst_view = std::move(dst_view),
.framebuffer = std::move(framebuffer),
});
}
scheduler.InvalidateState();
CopyMSAAImpl(renderpass, FindOrEmplaceMSAACopyPipeline(key), *msaa_copy_pipeline_layout,
dst_image, dst_vk_format, src_image, src_vk_format, scale_x, scale_y, copies,
aspect_info, false);
}
void BlitImageHelper::Convert(VkPipeline pipeline, const Framebuffer* dst_framebuffer,
@@ -1379,6 +1469,87 @@ VkPipeline BlitImageHelper::FindOrEmplaceBlitColorMSAAPipeline(const BlitMSAAPip
return *blit_msaa_color_pipelines.back();
}
VkPipeline BlitImageHelper::FindOrEmplaceBlitDepthStencilMSAAPipeline(
const BlitMSAAPipelineKey& key, bool blit_stencil) {
auto& keys = blit_stencil ? blit_msaa_depth_stencil_keys : blit_msaa_depth_keys;
auto& pipelines = blit_stencil ? blit_msaa_depth_stencil_pipelines : blit_msaa_depth_pipelines;
const auto it = std::ranges::find(keys, key);
if (it != keys.end()) {
return *pipelines[std::distance(keys.begin(), it)];
}
keys.push_back(key);
const std::array stages =
MakeStages(*full_screen_vert,
blit_stencil ? *blit_depth_stencil_msaa_frag : *blit_depth_msaa_frag);
const VkPipelineMultisampleStateCreateInfo multisample_ci{
.sType = VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.rasterizationSamples = key.samples,
.sampleShadingEnable = VK_TRUE,
.minSampleShading = 1.0f,
.pSampleMask = nullptr,
.alphaToCoverageEnable = VK_FALSE,
.alphaToOneEnable = VK_FALSE,
};
const VkPipelineInputAssemblyStateCreateInfo input_assembly_ci = GetPipelineInputAssemblyStateCreateInfo(device);
pipelines.push_back(device.GetLogical().CreateGraphicsPipeline({
.sType = VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.stageCount = static_cast<u32>(stages.size()),
.pStages = stages.data(),
.pVertexInputState = &PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO,
.pInputAssemblyState = &input_assembly_ci,
.pTessellationState = nullptr,
.pViewportState = &PIPELINE_VIEWPORT_STATE_CREATE_INFO,
.pRasterizationState = &PIPELINE_RASTERIZATION_STATE_CREATE_INFO,
.pMultisampleState = &multisample_ci,
.pDepthStencilState = blit_stencil ? &PIPELINE_DEPTH_STENCIL_STATE_CREATE_INFO
: &PIPELINE_DEPTH_ONLY_STATE_CREATE_INFO,
.pColorBlendState = &PIPELINE_COLOR_BLEND_STATE_EMPTY_CREATE_INFO,
.pDynamicState = &PIPELINE_DYNAMIC_STATE_CREATE_INFO,
.layout = blit_stencil ? *two_textures_pipeline_layout : *one_texture_pipeline_layout,
.renderPass = key.renderpass,
.subpass = 0,
.basePipelineHandle = VK_NULL_HANDLE,
.basePipelineIndex = 0,
}));
return *pipelines.back();
}
VkPipeline BlitImageHelper::FindOrEmplaceBlitDepthPipeline(VkRenderPass renderpass) {
const auto it = std::ranges::find(blit_depth_keys, renderpass);
if (it != blit_depth_keys.end()) {
return *blit_depth_pipelines[std::distance(blit_depth_keys.begin(), it)];
}
blit_depth_keys.push_back(renderpass);
const std::array stages = MakeStages(*full_screen_vert, *blit_depth_frag);
const VkPipelineInputAssemblyStateCreateInfo input_assembly_ci = GetPipelineInputAssemblyStateCreateInfo(device);
blit_depth_pipelines.push_back(device.GetLogical().CreateGraphicsPipeline({
.sType = VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.stageCount = static_cast<u32>(stages.size()),
.pStages = stages.data(),
.pVertexInputState = &PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO,
.pInputAssemblyState = &input_assembly_ci,
.pTessellationState = nullptr,
.pViewportState = &PIPELINE_VIEWPORT_STATE_CREATE_INFO,
.pRasterizationState = &PIPELINE_RASTERIZATION_STATE_CREATE_INFO,
.pMultisampleState = &PIPELINE_MULTISAMPLE_STATE_CREATE_INFO,
.pDepthStencilState = &PIPELINE_DEPTH_ONLY_STATE_CREATE_INFO,
.pColorBlendState = &PIPELINE_COLOR_BLEND_STATE_EMPTY_CREATE_INFO,
.pDynamicState = &PIPELINE_DYNAMIC_STATE_CREATE_INFO,
.layout = *one_texture_pipeline_layout,
.renderPass = renderpass,
.subpass = 0,
.basePipelineHandle = VK_NULL_HANDLE,
.basePipelineIndex = 0,
}));
return *blit_depth_pipelines.back();
}
VkPipeline BlitImageHelper::FindOrEmplaceResolveDepthStencilPipeline(VkRenderPass renderpass,
bool resolve_stencil) {
auto& keys = resolve_stencil ? resolve_depth_stencil_keys : resolve_depth_keys;
@@ -1417,15 +1588,83 @@ VkPipeline BlitImageHelper::FindOrEmplaceResolveDepthStencilPipeline(VkRenderPas
return *pipelines.back();
}
void BlitImageHelper::CopyMSAADepth(RenderPassCache& render_pass_cache, VkImage dst_image,
VideoCore::Surface::PixelFormat dst_format, VkImage src_image,
VideoCore::Surface::PixelFormat src_format, u32 num_samples,
std::span<const VideoCommon::ImageCopy> copies,
bool copy_stencil, bool msaa_to_non_msaa) {
const auto [samples_x, samples_y] = VideoCommon::SamplesLog2(static_cast<int>(num_samples));
const s32 scale_x = 1 << samples_x;
const s32 scale_y = 1 << samples_y;
VkSampleCountFlagBits samples = SampleCountFlag(num_samples);
if (msaa_to_non_msaa) {
samples = VK_SAMPLE_COUNT_1_BIT;
}
RenderPassKey renderpass_key{};
renderpass_key.color_formats.fill(VideoCore::Surface::PixelFormat::Invalid);
renderpass_key.depth_format = dst_format;
renderpass_key.samples = samples;
const VkRenderPass renderpass = render_pass_cache.Get(renderpass_key);
const MSAACopyPipelineKey key{
.renderpass = renderpass,
.samples = samples,
.msaa_to_non_msaa = msaa_to_non_msaa,
.format_class = MSAACopyFormatClass::Float,
};
VkImageAspectFlags attachment_aspect = VK_IMAGE_ASPECT_DEPTH_BIT;
if (VideoCore::Surface::GetFormatType(dst_format) ==
VideoCore::Surface::SurfaceType::DepthStencil) {
attachment_aspect |= VK_IMAGE_ASPECT_STENCIL_BIT;
}
VkPipelineLayout layout = *msaa_copy_pipeline_layout;
if (copy_stencil) {
layout = *msaa_copy_depth_stencil_pipeline_layout;
}
const MSAACopyAspectInfo aspect_info{
.src_view_aspect = VK_IMAGE_ASPECT_DEPTH_BIT,
.attachment_aspect = attachment_aspect,
.barrier_aspect = attachment_aspect,
.pre_src_access =
VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT | VK_ACCESS_TRANSFER_WRITE_BIT,
.pre_src_dst_access = VK_ACCESS_SHADER_READ_BIT,
.pre_dst_dst_access = VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT |
VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT,
.pre_src_stages = VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT |
VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT |
VK_PIPELINE_STAGE_TRANSFER_BIT,
.pre_dst_stages =
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT | VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT,
.post_src_access = VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT,
.post_dst_access = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_TRANSFER_READ_BIT |
VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT,
.post_src_stages = VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT,
.post_dst_stages = vk::PIPELINE_STAGE_GRAPHICS_COMPUTE_TRANSFER,
};
const VkFormat src_vk_format =
MaxwellToVK::SurfaceFormat(device, FormatType::Optimal, true, src_format).format;
const VkFormat dst_vk_format =
MaxwellToVK::SurfaceFormat(device, FormatType::Optimal, true, dst_format).format;
CopyMSAAImpl(renderpass, FindOrEmplaceMSAACopyDepthPipeline(key, copy_stencil), layout,
dst_image, dst_vk_format, src_image, src_vk_format, scale_x, scale_y, copies,
aspect_info, copy_stencil);
}
VkPipeline BlitImageHelper::FindOrEmplaceMSAACopyPipeline(const MSAACopyPipelineKey& key) {
const auto it = std::ranges::find(msaa_copy_keys, key);
if (it != msaa_copy_keys.end()) {
return *msaa_copy_pipelines[std::distance(msaa_copy_keys.begin(), it)];
}
msaa_copy_keys.push_back(key);
const std::array stages = MakeStages(*clear_color_vert, key.msaa_to_non_msaa
? *convert_msaa_to_non_msaa_frag
: *convert_non_msaa_to_msaa_frag);
VkShaderModule frag_module = key.msaa_to_non_msaa ? *convert_msaa_to_non_msaa_frag
: *convert_non_msaa_to_msaa_frag;
if (key.format_class == MSAACopyFormatClass::SignedInteger) {
frag_module = key.msaa_to_non_msaa ? *convert_msaa_to_non_msaa_sint_frag
: *convert_non_msaa_to_msaa_sint_frag;
} else if (key.format_class == MSAACopyFormatClass::UnsignedInteger) {
frag_module = key.msaa_to_non_msaa ? *convert_msaa_to_non_msaa_uint_frag
: *convert_non_msaa_to_msaa_uint_frag;
}
const std::array stages = MakeStages(*clear_color_vert, frag_module);
const VkPipelineMultisampleStateCreateInfo multisample_ci{
.sType = VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO,
.pNext = nullptr,
@@ -1462,6 +1701,85 @@ VkPipeline BlitImageHelper::FindOrEmplaceMSAACopyPipeline(const MSAACopyPipeline
return *msaa_copy_pipelines.back();
}
VkPipeline BlitImageHelper::FindOrEmplaceMSAACopyDepthPipeline(const MSAACopyPipelineKey& key,
bool copy_stencil) {
auto& keys = copy_stencil ? msaa_copy_depth_stencil_keys : msaa_copy_depth_keys;
auto& pipelines = copy_stencil ? msaa_copy_depth_stencil_pipelines : msaa_copy_depth_pipelines;
const auto it = std::ranges::find(keys, key);
if (it != keys.end()) {
return *pipelines[std::distance(keys.begin(), it)];
}
keys.push_back(key);
VkShaderModule frag_module;
if (key.msaa_to_non_msaa) {
frag_module = copy_stencil ? *convert_msaa_to_non_msaa_depth_stencil_frag
: *convert_msaa_to_non_msaa_depth_frag;
} else {
frag_module = copy_stencil ? *convert_non_msaa_to_msaa_depth_stencil_frag
: *convert_non_msaa_to_msaa_depth_frag;
}
const std::array stages = MakeStages(*clear_color_vert, frag_module);
const VkPipelineMultisampleStateCreateInfo multisample_ci{
.sType = VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.rasterizationSamples = key.samples,
.sampleShadingEnable = key.msaa_to_non_msaa ? VK_FALSE : VK_TRUE,
.minSampleShading = key.msaa_to_non_msaa ? 0.0f : 1.0f,
.pSampleMask = nullptr,
.alphaToCoverageEnable = VK_FALSE,
.alphaToOneEnable = VK_FALSE,
};
static constexpr VkStencilOpState REPLACE_STENCIL_OP{
.failOp = VK_STENCIL_OP_REPLACE,
.passOp = VK_STENCIL_OP_REPLACE,
.depthFailOp = VK_STENCIL_OP_REPLACE,
.compareOp = VK_COMPARE_OP_ALWAYS,
.compareMask = 0xFF,
.writeMask = 0xFF,
.reference = 0,
};
const VkPipelineDepthStencilStateCreateInfo depth_stencil_ci{
.sType = VK_STRUCTURE_TYPE_PIPELINE_DEPTH_STENCIL_STATE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.depthTestEnable = VK_TRUE,
.depthWriteEnable = VK_TRUE,
.depthCompareOp = VK_COMPARE_OP_ALWAYS,
.depthBoundsTestEnable = VK_FALSE,
.stencilTestEnable = copy_stencil ? VK_TRUE : VK_FALSE,
.front = copy_stencil ? REPLACE_STENCIL_OP : VkStencilOpState{},
.back = copy_stencil ? REPLACE_STENCIL_OP : VkStencilOpState{},
.minDepthBounds = 0.0f,
.maxDepthBounds = 0.0f,
};
const VkPipelineInputAssemblyStateCreateInfo input_assembly_ci =
GetPipelineInputAssemblyStateCreateInfo(device);
pipelines.push_back(device.GetLogical().CreateGraphicsPipeline({
.sType = VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.stageCount = static_cast<u32>(stages.size()),
.pStages = stages.data(),
.pVertexInputState = &PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO,
.pInputAssemblyState = &input_assembly_ci,
.pTessellationState = nullptr,
.pViewportState = &PIPELINE_VIEWPORT_STATE_CREATE_INFO,
.pRasterizationState = &PIPELINE_RASTERIZATION_STATE_CREATE_INFO,
.pMultisampleState = &multisample_ci,
.pDepthStencilState = &depth_stencil_ci,
.pColorBlendState = &PIPELINE_COLOR_BLEND_STATE_EMPTY_CREATE_INFO,
.pDynamicState = &PIPELINE_DYNAMIC_STATE_CREATE_INFO,
.layout = copy_stencil ? *msaa_copy_depth_stencil_pipeline_layout
: *msaa_copy_pipeline_layout,
.renderPass = key.renderpass,
.subpass = 0,
.basePipelineHandle = VK_NULL_HANDLE,
.basePipelineIndex = 0,
}));
return *pipelines.back();
}
void BlitImageHelper::ConvertDepthToColorPipeline(vk::Pipeline& pipeline, VkRenderPass renderpass) {
ConvertPipeline(pipeline, renderpass, false);
}
@@ -45,12 +45,19 @@ struct BlitDepthStencilPipelineKey {
u32 stencil_ref;
};
enum class MSAACopyFormatClass : u32 {
Float,
SignedInteger,
UnsignedInteger,
};
struct MSAACopyPipelineKey {
constexpr auto operator<=>(const MSAACopyPipelineKey&) const noexcept = default;
VkRenderPass renderpass;
VkSampleCountFlagBits samples;
bool msaa_to_non_msaa;
MSAACopyFormatClass format_class;
};
struct BlitMSAAPipelineKey {
@@ -78,6 +85,12 @@ public:
void BlitColorMSAA(const Framebuffer* dst_framebuffer, const ImageView& src_image_view,
const Region2D& dst_region, const Region2D& src_region);
void BlitDepthStencilMSAA(const Framebuffer* dst_framebuffer, ImageView& src_image_view,
const Region2D& dst_region, const Region2D& src_region);
void BlitDepth(const Framebuffer* dst_framebuffer, ImageView& src_image_view,
const Region2D& dst_region, const Region2D& src_region);
void ResolveDepthStencil(const Framebuffer* dst_framebuffer, ImageView& src_image_view,
const Region2D& dst_region, const Region2D& src_region);
@@ -116,7 +129,39 @@ public:
VideoCore::Surface::PixelFormat src_format, u32 num_samples,
std::span<const VideoCommon::ImageCopy> copies, bool msaa_to_non_msaa);
void CopyMSAADepth(RenderPassCache& render_pass_cache, VkImage dst_image,
VideoCore::Surface::PixelFormat dst_format, VkImage src_image,
VideoCore::Surface::PixelFormat src_format, u32 num_samples,
std::span<const VideoCommon::ImageCopy> copies, bool copy_stencil,
bool msaa_to_non_msaa);
private:
struct MSAACopyAspectInfo {
VkImageAspectFlags src_view_aspect;
VkImageAspectFlags attachment_aspect;
VkImageAspectFlags barrier_aspect;
VkAccessFlags pre_src_access;
VkAccessFlags pre_src_dst_access;
VkAccessFlags pre_dst_dst_access;
VkPipelineStageFlags pre_src_stages;
VkPipelineStageFlags pre_dst_stages;
VkAccessFlags post_src_access;
VkAccessFlags post_dst_access;
VkPipelineStageFlags post_src_stages;
VkPipelineStageFlags post_dst_stages;
};
void BlitImpl(const Framebuffer* dst_framebuffer, const ImageView& src_image_view,
const Region2D& dst_region, const Region2D& src_region, VkPipeline pipeline,
VkSampler sampler, VkImageView src_view, VkImageView src_stencil_view,
bool blit_stencil);
void CopyMSAAImpl(VkRenderPass renderpass, VkPipeline pipeline, VkPipelineLayout layout,
VkImage dst_image, VkFormat dst_vk_format, VkImage src_image,
VkFormat src_vk_format, s32 scale_x, s32 scale_y,
std::span<const VideoCommon::ImageCopy> copies,
const MSAACopyAspectInfo& aspect_info, bool copy_stencil);
void Convert(VkPipeline pipeline, const Framebuffer* dst_framebuffer,
const ImageView& src_image_view);
@@ -131,7 +176,13 @@ private:
[[nodiscard]] VkPipeline FindOrEmplaceClearStencilPipeline(
const BlitDepthStencilPipelineKey& key);
[[nodiscard]] VkPipeline FindOrEmplaceMSAACopyPipeline(const MSAACopyPipelineKey& key);
[[nodiscard]] VkPipeline FindOrEmplaceMSAACopyDepthPipeline(const MSAACopyPipelineKey& key,
bool copy_stencil);
[[nodiscard]] VkPipeline FindOrEmplaceBlitColorMSAAPipeline(const BlitMSAAPipelineKey& key);
[[nodiscard]] VkPipeline FindOrEmplaceBlitDepthStencilMSAAPipeline(
const BlitMSAAPipelineKey& key, bool blit_stencil);
[[nodiscard]] VkPipeline FindOrEmplaceBlitDepthPipeline(VkRenderPass renderpass);
[[nodiscard]] VkPipeline FindOrEmplaceResolveDepthStencilPipeline(VkRenderPass renderpass,
bool resolve_stencil);
@@ -162,10 +213,12 @@ private:
vk::PipelineLayout two_textures_pipeline_layout;
vk::PipelineLayout clear_color_pipeline_layout;
vk::PipelineLayout msaa_copy_pipeline_layout;
vk::PipelineLayout msaa_copy_depth_stencil_pipeline_layout;
vk::ShaderModule full_screen_vert;
vk::ShaderModule blit_color_to_color_frag;
vk::ShaderModule blit_color_msaa_frag;
vk::ShaderModule blit_depth_stencil_frag;
vk::ShaderModule blit_depth_frag;
vk::ShaderModule blit_depth_msaa_frag;
vk::ShaderModule blit_depth_stencil_msaa_frag;
vk::ShaderModule clear_color_vert;
@@ -179,7 +232,15 @@ private:
vk::ShaderModule convert_d24s8_to_abgr8_frag;
vk::ShaderModule convert_s8d24_to_abgr8_frag;
vk::ShaderModule convert_msaa_to_non_msaa_frag;
vk::ShaderModule convert_msaa_to_non_msaa_sint_frag;
vk::ShaderModule convert_msaa_to_non_msaa_uint_frag;
vk::ShaderModule convert_msaa_to_non_msaa_depth_frag;
vk::ShaderModule convert_msaa_to_non_msaa_depth_stencil_frag;
vk::ShaderModule convert_non_msaa_to_msaa_frag;
vk::ShaderModule convert_non_msaa_to_msaa_sint_frag;
vk::ShaderModule convert_non_msaa_to_msaa_uint_frag;
vk::ShaderModule convert_non_msaa_to_msaa_depth_frag;
vk::ShaderModule convert_non_msaa_to_msaa_depth_stencil_frag;
vk::Sampler linear_sampler;
vk::Sampler nearest_sampler;
@@ -193,8 +254,18 @@ private:
std::vector<vk::Pipeline> clear_stencil_pipelines;
std::vector<MSAACopyPipelineKey> msaa_copy_keys;
std::vector<vk::Pipeline> msaa_copy_pipelines;
std::vector<MSAACopyPipelineKey> msaa_copy_depth_keys;
std::vector<vk::Pipeline> msaa_copy_depth_pipelines;
std::vector<MSAACopyPipelineKey> msaa_copy_depth_stencil_keys;
std::vector<vk::Pipeline> msaa_copy_depth_stencil_pipelines;
std::vector<BlitMSAAPipelineKey> blit_msaa_color_keys;
std::vector<vk::Pipeline> blit_msaa_color_pipelines;
std::vector<VkRenderPass> blit_depth_keys;
std::vector<vk::Pipeline> blit_depth_pipelines;
std::vector<BlitMSAAPipelineKey> blit_msaa_depth_keys;
std::vector<vk::Pipeline> blit_msaa_depth_pipelines;
std::vector<BlitMSAAPipelineKey> blit_msaa_depth_stencil_keys;
std::vector<vk::Pipeline> blit_msaa_depth_stencil_pipelines;
std::vector<VkRenderPass> resolve_depth_keys;
std::vector<vk::Pipeline> resolve_depth_pipelines;
std::vector<VkRenderPass> resolve_depth_stencil_keys;
@@ -370,20 +370,9 @@ inline void PushImageDescriptors(TextureCache& texture_cache,
const VkImageView null_image_view{texture_cache.GetImageView(VideoCommon::NULL_IMAGE_VIEW_ID).Handle(desc.type)};
if (null_image_view != VK_NULL_HANDLE) vk_image_view = null_image_view;
}
const Sampler& sampler{texture_cache.GetSampler(sampler_id)};
const bool use_fallback_sampler{sampler.HasAddedAnisotropy() &&
!image_view.SupportsAnisotropy()};
VkSampler vk_sampler{use_fallback_sampler ? sampler.HandleWithDefaultAnisotropy()
: sampler.Handle()};
if (sampler.HasLinearFiltering() &&
VideoCore::Surface::IsPixelFormatInteger(image_view.format)) {
vk_sampler = sampler.HandleWithNearestFilter();
}
if (desc.is_depth && sampler.HasDepthComparison() &&
!image_view.SupportsDepthComparison()) {
vk_sampler = sampler.HandleWithoutDepthComparison();
}
guest_descriptor_queue.AddSampledImage(vk_image_view, vk_sampler);
Sampler& sampler{texture_cache.GetSampler(sampler_id)};
guest_descriptor_queue.AddSampledImage(vk_image_view,
sampler.HandleFor(image_view, desc.is_depth));
const bool element_rescaled{texture_cache.IsRescaling(image_view)};
is_rescaled |= element_rescaled;
}
+18 -14
View File
@@ -1,6 +1,8 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#include <algorithm>
#include "common/common_types.h"
#include "common/div_ceil.h"
#include "common/settings.h"
@@ -17,7 +19,7 @@
namespace Vulkan {
using PushConstants = std::array<u32, 4 + 2 + 1>;
using PushConstants = std::array<u32, 4 + 2 + 2 + 1>;
SGSR::SGSR(const Device& device, MemoryAllocator& memory_allocator, size_t image_count, VkExtent2D extent, bool edge_dir)
: m_memory_allocator{memory_allocator}
@@ -100,26 +102,28 @@ VkImageView SGSR::Draw(const Device& device, Scheduler& scheduler, size_t image_
const f32 input_image_width = f32(input_image_extent.width);
const f32 input_image_height = f32(input_image_extent.height);
const f32 viewport_width = (crop_rect.right - crop_rect.left) * input_image_width;
const f32 viewport_height = (crop_rect.bottom - crop_rect.top) * input_image_height;
// expected [0, 2]
const f32 sharpening = f32(Settings::values.fsr_sharpening_slider.GetValue()) / 100.0f;
const f32 crop_width = (crop_rect.right - crop_rect.left) * input_image_width;
const f32 crop_height = (crop_rect.bottom - crop_rect.top) * input_image_height;
static constexpr f32 EDGE_SHARPNESS_MAX = 2.0f;
const f32 edge_sharpness =
EDGE_SHARPNESS_MAX - f32(Settings::values.fsr_sharpening_slider.GetValue()) / 200.0f;
// p = (tex * viewport) / input = [0,n] (normalized texcoords)
// p * input = [0,1024], [0,768]
// layout( push_constant ) uniform constants {
// highp vec4 ViewportInfo[1];
// highp vec2 ResizeFactor;
// highp vec2 CropOffset;
// highp float EdgeSharpness;
// };
PushConstants viewport_con{};
viewport_con[0] = std::bit_cast<u32>(std::abs(1.f / viewport_width));
viewport_con[1] = std::bit_cast<u32>(std::abs(1.f / viewport_height));
viewport_con[2] = std::bit_cast<u32>(std::abs(viewport_width));
viewport_con[3] = std::bit_cast<u32>(std::abs(viewport_height));
viewport_con[4] = std::bit_cast<u32>(viewport_width / input_image_width);
viewport_con[5] = std::bit_cast<u32>(viewport_height / input_image_height);
viewport_con[6] = std::bit_cast<u32>(sharpening);
viewport_con[0] = std::bit_cast<u32>(1.f / input_image_width);
viewport_con[1] = std::bit_cast<u32>(1.f / input_image_height);
viewport_con[2] = std::bit_cast<u32>(input_image_width);
viewport_con[3] = std::bit_cast<u32>(input_image_height);
viewport_con[4] = std::bit_cast<u32>(crop_width / input_image_width);
viewport_con[5] = std::bit_cast<u32>(crop_height / input_image_height);
viewport_con[6] = std::bit_cast<u32>((std::min)(crop_rect.left, crop_rect.right));
viewport_con[7] = std::bit_cast<u32>((std::min)(crop_rect.top, crop_rect.bottom));
viewport_con[8] = std::bit_cast<u32>(edge_sharpness);
UploadImages(device, scheduler);
UpdateDescriptorSets(device, source_image_view, image_index);
@@ -7,7 +7,9 @@
#include <algorithm>
#include <array>
#include <cstring>
#include <limits>
#include <span>
#include <utility>
#include <vector>
#include "video_core/buffer_cache/buffer_cache_base.h"
@@ -32,6 +34,32 @@ VkBufferCopy MakeBufferCopy(const VideoCommon::BufferCopy& copy) {
};
}
constexpr size_t MAX_WINDOW_BARRIER_RANGES = 8;
using WindowRange = std::pair<VkDeviceSize, VkDeviceSize>;
using WindowRanges = boost::container::small_vector<WindowRange, MAX_WINDOW_BARRIER_RANGES>;
void CoalesceWindowRanges(WindowRanges& ranges) {
if (ranges.size() < 2) {
return;
}
std::sort(ranges.begin(), ranges.end());
size_t merged = 0;
for (size_t index = 1; index < ranges.size(); ++index) {
if (ranges[index].first <= ranges[merged].second) {
ranges[merged].second = (std::max)(ranges[merged].second, ranges[index].second);
} else {
ranges[++merged] = ranges[index];
}
}
ranges.resize(merged + 1);
if (ranges.size() > MAX_WINDOW_BARRIER_RANGES) {
const WindowRange bounding{ranges.front().first, ranges.back().second};
ranges.clear();
ranges.push_back(bounding);
}
}
VkIndexType IndexTypeFromNumElements(const Device& device, u32 num_elements) {
if (num_elements <= 0xff && device.IsExtIndexTypeUint8Supported()) {
return VK_INDEX_TYPE_UINT8_EXT;
@@ -42,6 +70,16 @@ VkIndexType IndexTypeFromNumElements(const Device& device, u32 num_elements) {
return VK_INDEX_TYPE_UINT32;
}
u32 GrowIndexCount(u32 current, u32 requested) {
constexpr u32 MinimumIndices = 4096;
constexpr u32 GrowthLimit = (std::numeric_limits<u32>::max)() / 2;
u32 grown = (std::max)(requested, MinimumIndices);
if (current <= GrowthLimit) {
grown = (std::max)(grown, current * 2);
}
return grown;
}
size_t BytesPerIndex(VkIndexType index_type) {
switch (index_type) {
case VK_INDEX_TYPE_UINT8_EXT:
@@ -158,13 +196,12 @@ public:
virtual ~QuadIndexBuffer() = default;
void UpdateBuffer(u32 num_indices_) {
ReleaseRetiredBuffers();
if (num_indices_ <= num_indices) {
return;
}
scheduler.Finish();
num_indices = num_indices_;
num_indices = GrowIndexCount(num_indices, num_indices_);
index_type = IndexTypeFromNumElements(device, num_indices);
const u32 num_quads = GetQuadsNum(num_indices);
@@ -182,6 +219,12 @@ public:
.queueFamilyIndexCount = 0,
.pQueueFamilyIndices = nullptr,
};
if (buffer) {
retired_buffers.push_back(RetiredBuffer{
.buffer = std::move(buffer),
.tick = scheduler.CurrentTick(),
});
}
buffer = memory_allocator.CreateBuffer(buffer_ci, MemoryUsage::DeviceLocal);
if (device.HasDebuggingToolAttached()) {
buffer.SetObjectNameEXT("Quad LUT");
@@ -249,6 +292,17 @@ protected:
virtual void MakeAndUpdateIndices(u8* staging_data, size_t quad_size, u32 quad, u32 first) = 0;
struct RetiredBuffer {
vk::Buffer buffer;
u64 tick;
};
void ReleaseRetiredBuffers() {
std::erase_if(retired_buffers, [this](const RetiredBuffer& entry) {
return scheduler.IsFree(entry.tick);
});
}
const Device& device;
MemoryAllocator& memory_allocator;
Scheduler& scheduler;
@@ -256,6 +310,7 @@ protected:
vk::Buffer buffer{};
MemoryCommit memory_commit{};
std::vector<RetiredBuffer> retired_buffers;
VkIndexType index_type{};
u32 num_indices = 0;
};
@@ -365,6 +420,138 @@ BufferCacheRuntime::BufferCacheRuntime(const Device& device_, MemoryAllocator& m
scheduler_, staging_pool_);
}
void BufferCacheRuntime::TryEnableUnifiedMemory(void* base, size_t size,
std::span<AHardwareBuffer* const> hardware_buffers,
size_t hardware_buffer_window,
size_t hardware_buffer_base) {
unified_memory = memory_allocator.CreateHostMemoryImport(
base, size, hardware_buffers, hardware_buffer_window, hardware_buffer_base);
}
void BufferCacheRuntime::CopyToUnifiedMemory(
size_t window_index, VkBuffer src_buffer,
std::span<const VideoCommon::BufferCopy> copies) {
if (!unified_memory || src_buffer == VK_NULL_HANDLE || copies.empty() ||
window_index >= unified_memory->GetWindowCount() ||
unified_memory->GetWindowBuffer(window_index) == VK_NULL_HANDLE) {
return;
}
PendingUnifiedCopy& pending = pending_unified_copies.emplace_back();
pending.window = window_index;
pending.buffer = src_buffer;
pending.copies.resize(copies.size());
std::ranges::transform(copies, pending.copies.begin(), MakeBufferCopy);
}
void BufferCacheRuntime::FlushUnifiedMemoryCopies() {
if (pending_unified_copies.empty()) {
return;
}
struct UnifiedCopyCommand {
VkBuffer buffer;
boost::container::small_vector<VkBufferCopy, 8> copies;
};
std::stable_sort(pending_unified_copies.begin(), pending_unified_copies.end(),
[](const PendingUnifiedCopy& lhs, const PendingUnifiedCopy& rhs) {
return lhs.window < rhs.window;
});
const bool foreign = unified_memory->NeedsForeignOwnershipTransfer();
const u32 queue_family = device.GetGraphicsFamily();
size_t group_begin = 0;
while (group_begin < pending_unified_copies.size()) {
const size_t window = pending_unified_copies[group_begin].window;
size_t group_end = group_begin;
while (group_end < pending_unified_copies.size() &&
pending_unified_copies[group_end].window == window) {
++group_end;
}
const VkBuffer window_buffer = unified_memory->GetWindowBuffer(window);
WindowRanges ranges;
for (size_t index = group_begin; index < group_end; ++index) {
for (const VkBufferCopy& copy : pending_unified_copies[index].copies) {
ranges.emplace_back(copy.dstOffset, copy.dstOffset + copy.size);
}
}
CoalesceWindowRanges(ranges);
boost::container::small_vector<VkBufferMemoryBarrier, MAX_WINDOW_BARRIER_RANGES> acquire;
boost::container::small_vector<VkBufferMemoryBarrier, MAX_WINDOW_BARRIER_RANGES> release;
if (foreign) {
for (const WindowRange& range : ranges) {
acquire.push_back(VkBufferMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = 0,
.dstAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_FOREIGN_EXT,
.dstQueueFamilyIndex = queue_family,
.buffer = window_buffer,
.offset = range.first,
.size = range.second - range.first,
});
release.push_back(VkBufferMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = 0,
.srcQueueFamilyIndex = queue_family,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_FOREIGN_EXT,
.buffer = window_buffer,
.offset = range.first,
.size = range.second - range.first,
});
}
}
boost::container::small_vector<UnifiedCopyCommand, 4> commands;
commands.reserve(group_end - group_begin);
for (size_t index = group_begin; index < group_end; ++index) {
PendingUnifiedCopy& pending = pending_unified_copies[index];
commands.push_back(UnifiedCopyCommand{pending.buffer, std::move(pending.copies)});
}
scheduler.RequestOutsideRenderPassOperationContext();
scheduler.Record([window_buffer, acquire = std::move(acquire), release = std::move(release),
commands = std::move(commands)](vk::CommandBuffer cmdbuf) {
if (!acquire.empty()) {
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT,
VK_PIPELINE_STAGE_TRANSFER_BIT, 0, {},
VideoCommon::FixSmallVectorADL(acquire), {});
}
for (const UnifiedCopyCommand& command : commands) {
cmdbuf.CopyBuffer(command.buffer, window_buffer,
VideoCommon::FixSmallVectorADL(command.copies));
}
if (!release.empty()) {
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_TRANSFER_BIT,
VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, 0, {},
VideoCommon::FixSmallVectorADL(release), {});
}
});
group_begin = group_end;
}
pending_unified_copies.clear();
}
void BufferCacheRuntime::UnifiedMemoryHostBarrier() {
static constexpr VkMemoryBarrier HOST_BARRIER{
.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_HOST_READ_BIT,
};
scheduler.RequestOutsideRenderPassOperationContext();
scheduler.Record([](vk::CommandBuffer cmdbuf) {
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_HOST_BIT, 0,
HOST_BARRIER);
});
}
StagingBufferRef BufferCacheRuntime::UploadStagingBuffer(size_t size) {
return staging_pool.Request(size, MemoryUsage::Upload);
}
@@ -402,6 +589,7 @@ u32 BufferCacheRuntime::GetStorageBufferAlignment() const {
}
void BufferCacheRuntime::TickFrame(Common::SlotVector<Buffer>& slot_buffers) noexcept {
FlushUnifiedMemoryCopies();
for (auto it = slot_buffers.begin(); it != slot_buffers.end(); it++) {
if (scheduler.IsFree(it->LastUsageTick())) {
it->ResetUsageTracking();
@@ -7,6 +7,10 @@
#pragma once
#include <limits>
#include <memory>
#include <span>
#include <boost/container/small_vector.hpp>
#include "video_core/buffer_cache/buffer_cache_base.h"
#include "video_core/buffer_cache/memory_tracker_base.h"
@@ -97,6 +101,33 @@ public:
void TickFrame(Common::SlotVector<Buffer>& slot_buffers) noexcept;
void TryEnableUnifiedMemory(void* base, size_t size,
std::span<AHardwareBuffer* const> hardware_buffers,
size_t hardware_buffer_window, size_t hardware_buffer_base);
[[nodiscard]] bool HasUnifiedMemory() const noexcept {
return unified_memory != nullptr && unified_memory->IsValid();
}
[[nodiscard]] u64 UnifiedMemorySize() const noexcept {
return unified_memory ? unified_memory->GetSize() : 0;
}
[[nodiscard]] u64 UnifiedMemoryBase() const noexcept {
return unified_memory ? unified_memory->GetBaseOffset() : 0;
}
[[nodiscard]] u64 UnifiedMemoryWindowSize() const noexcept {
return unified_memory ? unified_memory->GetWindowSize() : 0;
}
void CopyToUnifiedMemory(size_t window_index, VkBuffer src_buffer,
std::span<const VideoCommon::BufferCopy> copies);
void FlushUnifiedMemoryCopies();
void UnifiedMemoryHostBarrier();
u64 CurrentTick();
u64 KnownGpuTick();
@@ -164,6 +195,29 @@ public:
BindBuffer(buffer, offset, size);
}
void BindStorageBuffer(VkBuffer buffer, VkDeviceAddress address, u32 offset, u32 size,
[[maybe_unused]] bool is_written) {
guest_descriptor_queue.AddBuffer(buffer, address, offset, size);
}
[[nodiscard]] bool IsUnifiedMemoryBindable() const noexcept {
return unified_memory != nullptr && unified_memory->IsValid() &&
unified_memory->IsBindable();
}
[[nodiscard]] VkBuffer UnifiedWindowBuffer(size_t index) const noexcept {
return unified_memory->GetWindowBuffer(index);
}
[[nodiscard]] VkDeviceAddress UnifiedWindowAddress(size_t index) const noexcept {
return unified_memory->GetWindowAddress(index);
}
[[nodiscard]] bool IsUnifiedStorageRange(u32 size, u64 offset) const {
return size <= device.GetMaxStorageBufferRange() &&
(offset % device.GetStorageBufferAlignment()) == 0;
}
void BindTextureBuffer(Buffer& buffer, u32 offset, u32 size,
VideoCore::Surface::PixelFormat format) {
guest_descriptor_queue.AddTexelBuffer(buffer.View(offset, size, format),
@@ -180,6 +234,12 @@ public:
}
private:
struct PendingUnifiedCopy {
size_t window;
VkBuffer buffer;
boost::container::small_vector<VkBufferCopy, 8> copies;
};
void BindBuffer(const Buffer& buffer, u32 offset, u32 size) {
const VkBuffer handle = buffer.Handle();
if (handle == VK_NULL_HANDLE) {
@@ -204,6 +264,8 @@ private:
std::shared_ptr<QuadStripIndexBuffer> quad_strip_index_buffer;
vk::Buffer null_buffer;
HostMemoryImport* unified_memory{};
boost::container::small_vector<PendingUnifiedCopy, 8> pending_unified_copies;
std::unique_ptr<Uint8Pass> uint8_pass;
QuadIndexedPass quad_index_pass;
@@ -226,6 +288,7 @@ struct BufferCacheParams {
static constexpr bool USE_MEMORY_MAPS = true;
static constexpr bool SEPARATE_IMAGE_BUFFER_BINDINGS = false;
static constexpr bool USE_MEMORY_MAPS_FOR_UPLOADS = true;
static constexpr bool USE_UNIFIED_MEMORY = true;
};
using BufferCache = VideoCommon::BufferCache<BufferCacheParams>;
@@ -4,6 +4,7 @@
// SPDX-FileCopyrightText: Copyright 2019 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <algorithm>
#include <array>
#include <memory>
#include <numeric>
@@ -22,7 +23,9 @@
#include "video_core/host_shaders/resolve_conditional_render_comp_spv.h"
#include "video_core/host_shaders/vulkan_quad_indexed_comp_spv.h"
#include "video_core/host_shaders/vulkan_uint8_comp_spv.h"
#include "video_core/host_shaders/block_linear_unswizzle_2d_buffer_comp_spv.h"
#include "video_core/host_shaders/block_linear_unswizzle_3d_bcn_comp_spv.h"
#include "video_core/host_shaders/block_linear_unswizzle_3d_buffer_comp_spv.h"
#include "video_core/renderer_vulkan/vk_compute_pass.h"
#include "video_core/surface.h"
#include "video_core/renderer_vulkan/vk_descriptor_pool.h"
@@ -872,4 +875,516 @@ void BlockLinearUnswizzle3DPass::UnswizzleChunk(
});
}
namespace {
constexpr u32 BL2D_BINDING_INPUT_BUFFER = 0;
constexpr u32 BL2D_BINDING_OUTPUT_BUFFER = 1;
struct alignas(16) BlockLinearUnswizzle2DPushConstants {
std::array<u32, 3> dim;
u32 bytes_per_block_log2;
std::array<u32, 3> origin;
u32 layer_stride;
u32 block_size;
u32 x_shift;
u32 block_height;
u32 block_height_mask;
};
static_assert(sizeof(BlockLinearUnswizzle2DPushConstants) <= 128);
constexpr std::array<VkDescriptorSetLayoutBinding, 2> BL2D_BINDINGS{{
{
.binding = BL2D_BINDING_INPUT_BUFFER,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.descriptorCount = 1,
.stageFlags = VK_SHADER_STAGE_COMPUTE_BIT,
.pImmutableSamplers = nullptr,
},
{
.binding = BL2D_BINDING_OUTPUT_BUFFER,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.descriptorCount = 1,
.stageFlags = VK_SHADER_STAGE_COMPUTE_BIT,
.pImmutableSamplers = nullptr,
},
}};
constexpr std::array<VkDescriptorUpdateTemplateEntry, 2> BL2D_TEMPLATE{{
{
.dstBinding = BL2D_BINDING_INPUT_BUFFER,
.dstArrayElement = 0,
.descriptorCount = 1,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.offset = BL2D_BINDING_INPUT_BUFFER * sizeof(DescriptorUpdateEntry),
.stride = sizeof(DescriptorUpdateEntry),
},
{
.dstBinding = BL2D_BINDING_OUTPUT_BUFFER,
.dstArrayElement = 0,
.descriptorCount = 1,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.offset = BL2D_BINDING_OUTPUT_BUFFER * sizeof(DescriptorUpdateEntry),
.stride = sizeof(DescriptorUpdateEntry),
},
}};
constexpr DescriptorBankInfo BL2D_BANK_INFO{
.uniform_buffers = 0,
.storage_buffers = 2,
.texture_buffers = 0,
.image_buffers = 0,
.textures = 0,
.images = 0,
.score = 2,
};
} // Anonymous namespace
BlockLinearUnswizzle2DPass::BlockLinearUnswizzle2DPass(
const Device& device_, Scheduler& scheduler_, DescriptorPool& descriptor_pool_,
StagingBufferPool& staging_buffer_pool_,
ComputePassDescriptorQueue& compute_pass_descriptor_queue_)
: ComputePass(device_, scheduler_, descriptor_pool_, BL2D_BINDINGS, BL2D_TEMPLATE,
BL2D_BANK_INFO,
COMPUTE_PUSH_CONSTANT_RANGE<sizeof(BlockLinearUnswizzle2DPushConstants)>,
BLOCK_LINEAR_UNSWIZZLE_2D_BUFFER_COMP_SPV),
scheduler{scheduler_}, staging_buffer_pool{staging_buffer_pool_},
compute_pass_descriptor_queue{compute_pass_descriptor_queue_} {}
BlockLinearUnswizzle2DPass::~BlockLinearUnswizzle2DPass() = default;
bool BlockLinearUnswizzle2DPass::IsSupported(const Device& device,
const VideoCommon::ImageInfo& info) {
if (info.type != VideoCommon::ImageType::e2D) {
return false;
}
if (info.num_samples > 1) {
return false;
}
if (device.GetStorageBufferAlignment() > Tegra::Texture::GOB_SIZE) {
return false;
}
if (VideoCore::Surface::GetFormatType(info.format) !=
VideoCore::Surface::SurfaceType::ColorTexture) {
return false;
}
if (VideoCore::Surface::IsPixelFormatASTC(info.format) && !device.IsOptimalAstcSupported()) {
return false;
}
if (VideoCore::Surface::IsPixelFormatBCn(info.format) && !device.IsOptimalBcnSupported()) {
return false;
}
const u32 bytes_per_block = VideoCore::Surface::BytesPerBlock(info.format);
return bytes_per_block == 4 || bytes_per_block == 8 || bytes_per_block == 16;
}
void BlockLinearUnswizzle2DPass::Unswizzle(
Image& image, const StagingBufferRef& swizzled,
std::span<const VideoCommon::SwizzleParameters> swizzles) {
UnswizzleFrom(image, swizzled.buffer, swizzled.offset, swizzles);
}
void BlockLinearUnswizzle2DPass::UnswizzleFrom(
Image& image, VkBuffer source_buffer, VkDeviceSize source_offset,
std::span<const VideoCommon::SwizzleParameters> swizzles) {
const VkImage dst_image = image.Handle();
if (swizzles.empty() || source_buffer == VK_NULL_HANDLE || dst_image == VK_NULL_HANDLE) {
return;
}
const u32 layers = image.info.resources.layers;
const VkImageAspectFlags aspect = image.AspectMask();
scheduler.RequestOutsideRenderPassOperationContext();
VkAccessFlags pre_access = VK_ACCESS_NONE;
VkImageLayout pre_layout = VK_IMAGE_LAYOUT_UNDEFINED;
VkPipelineStageFlags pre_stage = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT;
if (image.ExchangeInitialization()) {
pre_access = VK_ACCESS_SHADER_READ_BIT;
pre_layout = VK_IMAGE_LAYOUT_GENERAL;
pre_stage = vk::PIPELINE_STAGE_GRAPHICS_COMPUTE;
}
scheduler.Record([dst_image, aspect, pre_access, pre_layout,
pre_stage](vk::CommandBuffer cmdbuf) {
const VkImageMemoryBarrier barrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = pre_access,
.dstAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.oldLayout = pre_layout,
.newLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst_image,
.subresourceRange{
.aspectMask = aspect,
.baseMipLevel = 0,
.levelCount = VK_REMAINING_MIP_LEVELS,
.baseArrayLayer = 0,
.layerCount = VK_REMAINING_ARRAY_LAYERS,
},
};
cmdbuf.PipelineBarrier(pre_stage, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, {}, {}, barrier);
});
for (const VideoCommon::SwizzleParameters& sw : swizzles) {
const auto params =
VideoCommon::Accelerated::MakeBlockLinearSwizzle2DParams(sw, image.info);
const u32 width = sw.num_tiles.width;
const u32 height = sw.num_tiles.height;
const u32 bytes_per_block = 1u << params.bytes_per_block_log2;
const VkDeviceSize output_size =
static_cast<VkDeviceSize>(width) * height * layers * bytes_per_block;
if (output_size == 0) {
continue;
}
const u32 level = static_cast<u32>(sw.level);
const u32 texel_width = (std::max)(1u, image.info.size.width >> level);
const u32 texel_height = (std::max)(1u, image.info.size.height >> level);
const StagingBufferRef output =
staging_buffer_pool.Request(static_cast<size_t>(output_size), MemoryUsage::DeviceLocal);
BlockLinearUnswizzle2DPushConstants pc{};
pc.dim = {width, height, layers};
pc.bytes_per_block_log2 = params.bytes_per_block_log2;
pc.origin = params.origin;
pc.layer_stride = params.layer_stride;
pc.block_size = params.block_size;
pc.x_shift = params.x_shift;
pc.block_height = params.block_height;
pc.block_height_mask = params.block_height_mask;
compute_pass_descriptor_queue.Acquire(scheduler, 2);
compute_pass_descriptor_queue.AddBuffer(source_buffer, sw.buffer_offset + source_offset,
image.guest_size_bytes - sw.buffer_offset);
compute_pass_descriptor_queue.AddBuffer(output.buffer, output.offset, output_size);
const void* descriptor_data = compute_pass_descriptor_queue.UpdateData();
const VkDescriptorSet set = descriptor_allocator.Commit();
const u32 gx = Common::DivCeil(width, 16u);
const u32 gy = Common::DivCeil(height, 8u);
const VkBuffer out_buffer = output.buffer;
const VkDeviceSize out_offset = output.offset;
scheduler.Record([this, set, descriptor_data, pc, gx, gy, layers, output_size, out_buffer,
out_offset, dst_image, aspect, texel_width, texel_height,
level](vk::CommandBuffer cmdbuf) {
if (out_buffer == VK_NULL_HANDLE) {
return;
}
device.GetLogical().UpdateDescriptorSet(set, *descriptor_template, descriptor_data);
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_COMPUTE, *pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_COMPUTE, *layout, 0, set, {});
cmdbuf.PushConstants(*layout, VK_SHADER_STAGE_COMPUTE_BIT, 0, sizeof(pc), &pc);
cmdbuf.Dispatch(gx, gy, layers);
const VkBufferMemoryBarrier buffer_barrier{
.sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_SHADER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_TRANSFER_READ_BIT,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.buffer = out_buffer,
.offset = out_offset,
.size = output_size,
};
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT,
VK_PIPELINE_STAGE_TRANSFER_BIT, 0, {}, buffer_barrier, {});
const VkBufferImageCopy copy{
.bufferOffset = out_offset,
.bufferRowLength = 0,
.bufferImageHeight = 0,
.imageSubresource{
.aspectMask = aspect,
.mipLevel = level,
.baseArrayLayer = 0,
.layerCount = layers,
},
.imageOffset = {0, 0, 0},
.imageExtent = {texel_width, texel_height, 1},
};
cmdbuf.CopyBufferToImage(out_buffer, dst_image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
copy);
});
}
scheduler.Record([dst_image, aspect](vk::CommandBuffer cmdbuf) {
const VkImageMemoryBarrier barrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_SHADER_READ_BIT,
.oldLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst_image,
.subresourceRange{
.aspectMask = aspect,
.baseMipLevel = 0,
.levelCount = VK_REMAINING_MIP_LEVELS,
.baseArrayLayer = 0,
.layerCount = VK_REMAINING_ARRAY_LAYERS,
},
};
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_TRANSFER_BIT,
vk::PIPELINE_STAGE_GRAPHICS_COMPUTE, 0, {}, {}, barrier);
});
}
namespace {
constexpr u32 BL3DB_BINDING_INPUT_BUFFER = 0;
constexpr u32 BL3DB_BINDING_OUTPUT_BUFFER = 1;
struct alignas(16) BlockLinearUnswizzle3DBufferPushConstants {
std::array<u32, 3> dim;
u32 bytes_per_block_log2;
std::array<u32, 3> origin;
u32 slice_size;
u32 block_size;
u32 x_shift;
u32 block_height;
u32 block_height_mask;
u32 block_depth;
u32 block_depth_mask;
};
static_assert(sizeof(BlockLinearUnswizzle3DBufferPushConstants) <= 128);
constexpr std::array<VkDescriptorSetLayoutBinding, 2> BL3DB_BINDINGS{{
{
.binding = BL3DB_BINDING_INPUT_BUFFER,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.descriptorCount = 1,
.stageFlags = VK_SHADER_STAGE_COMPUTE_BIT,
.pImmutableSamplers = nullptr,
},
{
.binding = BL3DB_BINDING_OUTPUT_BUFFER,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.descriptorCount = 1,
.stageFlags = VK_SHADER_STAGE_COMPUTE_BIT,
.pImmutableSamplers = nullptr,
},
}};
constexpr std::array<VkDescriptorUpdateTemplateEntry, 2> BL3DB_TEMPLATE{{
{
.dstBinding = BL3DB_BINDING_INPUT_BUFFER,
.dstArrayElement = 0,
.descriptorCount = 1,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.offset = BL3DB_BINDING_INPUT_BUFFER * sizeof(DescriptorUpdateEntry),
.stride = sizeof(DescriptorUpdateEntry),
},
{
.dstBinding = BL3DB_BINDING_OUTPUT_BUFFER,
.dstArrayElement = 0,
.descriptorCount = 1,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.offset = BL3DB_BINDING_OUTPUT_BUFFER * sizeof(DescriptorUpdateEntry),
.stride = sizeof(DescriptorUpdateEntry),
},
}};
constexpr DescriptorBankInfo BL3DB_BANK_INFO{
.uniform_buffers = 0,
.storage_buffers = 2,
.texture_buffers = 0,
.image_buffers = 0,
.textures = 0,
.images = 0,
.score = 2,
};
} // Anonymous namespace
BlockLinearUnswizzle3DBufferPass::BlockLinearUnswizzle3DBufferPass(
const Device& device_, Scheduler& scheduler_, DescriptorPool& descriptor_pool_,
StagingBufferPool& staging_buffer_pool_,
ComputePassDescriptorQueue& compute_pass_descriptor_queue_)
: ComputePass(device_, scheduler_, descriptor_pool_, BL3DB_BINDINGS, BL3DB_TEMPLATE,
BL3DB_BANK_INFO,
COMPUTE_PUSH_CONSTANT_RANGE<sizeof(BlockLinearUnswizzle3DBufferPushConstants)>,
BLOCK_LINEAR_UNSWIZZLE_3D_BUFFER_COMP_SPV),
scheduler{scheduler_}, staging_buffer_pool{staging_buffer_pool_},
compute_pass_descriptor_queue{compute_pass_descriptor_queue_} {}
BlockLinearUnswizzle3DBufferPass::~BlockLinearUnswizzle3DBufferPass() = default;
bool BlockLinearUnswizzle3DBufferPass::IsSupported(const Device& device,
const VideoCommon::ImageInfo& info) {
if (info.type != VideoCommon::ImageType::e3D) {
return false;
}
if (info.resources.levels != 1 || info.resources.layers != 1) {
return false;
}
if (info.num_samples > 1) {
return false;
}
if (info.size.depth <= 1) {
return false;
}
if (VideoCore::Surface::GetFormatType(info.format) !=
VideoCore::Surface::SurfaceType::ColorTexture) {
return false;
}
if (VideoCore::Surface::IsPixelFormatASTC(info.format)) {
return false;
}
if (VideoCore::Surface::IsPixelFormatBCn(info.format) && !device.IsOptimalBcnSupported()) {
return false;
}
const u32 bytes_per_block = VideoCore::Surface::BytesPerBlock(info.format);
return bytes_per_block == 4 || bytes_per_block == 8 || bytes_per_block == 16;
}
void BlockLinearUnswizzle3DBufferPass::Unswizzle(
Image& image, const StagingBufferRef& swizzled,
std::span<const VideoCommon::SwizzleParameters> swizzles) {
if (swizzles.empty()) {
return;
}
const VideoCommon::SwizzleParameters& sw = swizzles.front();
const auto params = VideoCommon::Accelerated::MakeBlockLinearSwizzle3DParams(sw, image.info);
const u32 blocks_x = sw.num_tiles.width;
const u32 blocks_y = sw.num_tiles.height;
const u32 blocks_z = sw.num_tiles.depth;
const u32 bytes_per_block = 1u << params.bytes_per_block_log2;
const VkDeviceSize output_size =
static_cast<VkDeviceSize>(blocks_x) * blocks_y * blocks_z * bytes_per_block;
const StagingBufferRef output =
staging_buffer_pool.Request(static_cast<size_t>(output_size), MemoryUsage::DeviceLocal);
BlockLinearUnswizzle3DBufferPushConstants pc{};
pc.dim = {blocks_x, blocks_y, blocks_z};
pc.bytes_per_block_log2 = params.bytes_per_block_log2;
pc.origin = params.origin;
pc.slice_size = params.slice_size;
pc.block_size = params.block_size;
pc.x_shift = params.x_shift;
pc.block_height = params.block_height;
pc.block_height_mask = params.block_height_mask;
pc.block_depth = params.block_depth;
pc.block_depth_mask = params.block_depth_mask;
scheduler.RequestOutsideRenderPassOperationContext();
compute_pass_descriptor_queue.Acquire(scheduler, 2);
compute_pass_descriptor_queue.AddBuffer(swizzled.buffer, sw.buffer_offset + swizzled.offset,
image.guest_size_bytes - sw.buffer_offset);
compute_pass_descriptor_queue.AddBuffer(output.buffer, output.offset, output_size);
const void* descriptor_data = compute_pass_descriptor_queue.UpdateData();
const VkDescriptorSet set = descriptor_allocator.Commit();
const u32 gx = Common::DivCeil(blocks_x, 8u);
const u32 gy = Common::DivCeil(blocks_y, 8u);
const u32 gz = Common::DivCeil(blocks_z, 4u);
const bool is_initialized = image.ExchangeInitialization();
const VkBuffer out_buffer = output.buffer;
const VkDeviceSize out_offset = output.offset;
const VkImage dst_image = image.Handle();
const VkImageAspectFlags aspect = image.AspectMask();
const VkExtent3D extent{
.width = image.info.size.width,
.height = image.info.size.height,
.depth = image.info.size.depth,
};
scheduler.Record([this, set, descriptor_data, pc, gx, gy, gz, output_size, out_buffer,
out_offset, dst_image, aspect, extent,
is_initialized](vk::CommandBuffer cmdbuf) {
if (dst_image == VK_NULL_HANDLE || out_buffer == VK_NULL_HANDLE) {
return;
}
device.GetLogical().UpdateDescriptorSet(set, *descriptor_template, descriptor_data);
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_COMPUTE, *pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_COMPUTE, *layout, 0, set, {});
cmdbuf.PushConstants(*layout, VK_SHADER_STAGE_COMPUTE_BIT, 0, sizeof(pc), &pc);
cmdbuf.Dispatch(gx, gy, gz);
const VkBufferMemoryBarrier buffer_barrier{
.sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_SHADER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_TRANSFER_READ_BIT,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.buffer = out_buffer,
.offset = out_offset,
.size = output_size,
};
VkAccessFlags pre_copy_access = VK_ACCESS_NONE;
VkImageLayout pre_copy_layout = VK_IMAGE_LAYOUT_UNDEFINED;
VkPipelineStageFlags pre_copy_stage = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT;
if (is_initialized) {
pre_copy_access = VK_ACCESS_SHADER_READ_BIT;
pre_copy_layout = VK_IMAGE_LAYOUT_GENERAL;
pre_copy_stage = vk::PIPELINE_STAGE_GRAPHICS_COMPUTE;
}
const VkImageMemoryBarrier pre_copy{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = pre_copy_access,
.dstAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.oldLayout = pre_copy_layout,
.newLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst_image,
.subresourceRange{
.aspectMask = aspect,
.baseMipLevel = 0,
.levelCount = VK_REMAINING_MIP_LEVELS,
.baseArrayLayer = 0,
.layerCount = VK_REMAINING_ARRAY_LAYERS,
},
};
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT | pre_copy_stage,
VK_PIPELINE_STAGE_TRANSFER_BIT, 0, {}, buffer_barrier, pre_copy);
const VkBufferImageCopy copy{
.bufferOffset = out_offset,
.bufferRowLength = 0,
.bufferImageHeight = 0,
.imageSubresource{
.aspectMask = aspect,
.mipLevel = 0,
.baseArrayLayer = 0,
.layerCount = 1,
},
.imageOffset = {0, 0, 0},
.imageExtent = extent,
};
cmdbuf.CopyBufferToImage(out_buffer, dst_image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, copy);
const VkImageMemoryBarrier post_copy{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_SHADER_READ_BIT,
.oldLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst_image,
.subresourceRange{
.aspectMask = aspect,
.baseMipLevel = 0,
.levelCount = VK_REMAINING_MIP_LEVELS,
.baseArrayLayer = 0,
.layerCount = VK_REMAINING_ARRAY_LAYERS,
},
};
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_TRANSFER_BIT, vk::PIPELINE_STAGE_GRAPHICS_COMPUTE,
0, {}, {}, post_copy);
});
}
} // namespace Vulkan
@@ -164,4 +164,46 @@ private:
ComputePassDescriptorQueue& compute_pass_descriptor_queue;
};
class BlockLinearUnswizzle2DPass final : public ComputePass {
public:
explicit BlockLinearUnswizzle2DPass(const Device& device_, Scheduler& scheduler_,
DescriptorPool& descriptor_pool_,
StagingBufferPool& staging_buffer_pool_,
ComputePassDescriptorQueue& compute_pass_descriptor_queue_);
~BlockLinearUnswizzle2DPass();
[[nodiscard]] static bool IsSupported(const Device& device,
const VideoCommon::ImageInfo& info);
void Unswizzle(Image& image, const StagingBufferRef& swizzled,
std::span<const VideoCommon::SwizzleParameters> swizzles);
void UnswizzleFrom(Image& image, VkBuffer source_buffer, VkDeviceSize source_offset,
std::span<const VideoCommon::SwizzleParameters> swizzles);
private:
Scheduler& scheduler;
StagingBufferPool& staging_buffer_pool;
ComputePassDescriptorQueue& compute_pass_descriptor_queue;
};
class BlockLinearUnswizzle3DBufferPass final : public ComputePass {
public:
explicit BlockLinearUnswizzle3DBufferPass(
const Device& device_, Scheduler& scheduler_, DescriptorPool& descriptor_pool_,
StagingBufferPool& staging_buffer_pool_,
ComputePassDescriptorQueue& compute_pass_descriptor_queue_);
~BlockLinearUnswizzle3DBufferPass();
[[nodiscard]] static bool IsSupported(const Device& device, const VideoCommon::ImageInfo& info);
void Unswizzle(Image& image, const StagingBufferRef& swizzled,
std::span<const VideoCommon::SwizzleParameters> swizzles);
private:
Scheduler& scheduler;
StagingBufferPool& staging_buffer_pool;
ComputePassDescriptorQueue& compute_pass_descriptor_queue;
};
} // namespace Vulkan
@@ -444,8 +444,10 @@ PipelineCache::PipelineCache(Tegra::MaxwellDeviceMemoryManager& device_memory_,
.has_broken_unsigned_image_offsets = false,
.has_broken_signed_operations = false,
.has_broken_fp16_float_controls = driver_id == VK_DRIVER_ID_NVIDIA_PROPRIETARY,
.has_broken_fp32_denorm_flush = driver_id == VK_DRIVER_ID_QUALCOMM_PROPRIETARY,
.ignore_nan_fp_comparisons = false,
.has_broken_spirv_subgroup_mask_vector_extract_dynamic = false,
.max_shared_memory_size = device.GetMaxComputeSharedMemorySize(),
.has_broken_robust =
device.IsNvidia() && device.GetNvidiaArch() <= NvidiaArchitecture::Arch_Pascal,
.min_ssbo_alignment = device.GetStorageBufferAlignment(),
@@ -921,19 +923,6 @@ std::unique_ptr<ComputePipeline> PipelineCache::CreateComputePipeline(
}
auto program{TranslateProgram(pools.inst, pools.block, env, cfg, host_info)};
const VkDriverIdKHR driver_id = device.GetDriverID();
const bool needs_shared_mem_clamp =
driver_id == VK_DRIVER_ID_QUALCOMM_PROPRIETARY ||
driver_id == VK_DRIVER_ID_ARM_PROPRIETARY;
const u32 max_shared_memory = device.GetMaxComputeSharedMemorySize();
if (needs_shared_mem_clamp && program.shared_memory_size > max_shared_memory) {
LOG_WARNING(Render_Vulkan,
"Compute shader {:#016x} requests {}KB shared memory but device max is {}KB - clamping",
key.unique_hash,
program.shared_memory_size / 1024,
max_shared_memory / 1024);
program.shared_memory_size = max_shared_memory;
}
const std::vector<u32> code{EmitSPIRV(profile, program)};
device.SaveShader(code);
vk::ShaderModule spv_module{BuildShader(device, code)};
@@ -225,6 +225,13 @@ RasterizerVulkan::RasterizerVulkan(Core::Frontend::EmuWindow& emu_window_, Tegra
fence_manager(*this, gpu, texture_cache, buffer_cache, query_cache, device, scheduler),
wfi_event(device.GetLogical().CreateEvent()) {
scheduler.SetQueryCache(query_cache);
if (Settings::values.use_unified_memory.GetValue() && device_memory.IsBackingShared()) {
buffer_cache_runtime.TryEnableUnifiedMemory(
device_memory.GetPhysicalBase(), device_memory.GetPhysicalSize(),
device_memory.GetBackingHardwareBuffers(),
device_memory.GetBackingHardwareBufferWindowSize(),
device_memory.GetBackingHardwareBufferBase());
}
}
RasterizerVulkan::~RasterizerVulkan() {
@@ -424,24 +431,12 @@ void RasterizerVulkan::Clear(u32 layer_count) {
const bool ds_deferrable =
!ds_used || ((!framebuffer->HasAspectDepthBit() || use_depth) &&
(!framebuffer->HasAspectStencilBit() || use_stencil) && !stencil_partial);
const bool can_defer_clear = ENABLE_DEFERRED_CLEAR && !regs.clear_control.use_scissor &&
regs.clear_surface.layer == 0 &&
!scheduler.IsRenderPassActive() &&
(!use_color || color_full_channels) && ds_deferrable;
if (!can_defer_clear) {
scheduler.RequestRenderpass(framebuffer);
}
query_cache.NotifySegment(true);
query_cache.CounterEnable(VideoCommon::QueryType::ZPassPixelCount64, maxwell3d->regs.zpass_pixel_count_enable);
u32 up_scale = 1;
u32 down_shift = 0;
if (texture_cache.IsRescaling()) {
up_scale = Settings::values.resolution_info.up_scale;
down_shift = Settings::values.resolution_info.down_shift;
}
UpdateViewportsState(regs);
VkRect2D default_scissor{};
default_scissor.offset.x = 0;
default_scissor.offset.y = 0;
@@ -497,6 +492,22 @@ void RasterizerVulkan::Clear(u32 layer_count) {
return;
}
const bool clear_covers_render_area =
clear_rect.rect.offset.x == 0 && clear_rect.rect.offset.y == 0 &&
clear_rect.rect.extent.width >= render_area.width &&
clear_rect.rect.extent.height >= render_area.height;
const bool can_defer_clear = ENABLE_DEFERRED_CLEAR && (!regs.clear_control.use_scissor || clear_covers_render_area) &&
regs.clear_surface.layer == 0 &&
!scheduler.IsRenderPassActive() &&
(!use_color || color_full_channels) && ds_deferrable;
if (!can_defer_clear) {
scheduler.RequestRenderpass(framebuffer);
}
query_cache.NotifySegment(true);
query_cache.CounterEnable(VideoCommon::QueryType::ZPassPixelCount64, maxwell3d->regs.zpass_pixel_count_enable);
UpdateViewportsState(regs);
const u32 color_attachment = regs.clear_surface.RT;
if (use_color && framebuffer->HasAspectColorBit(color_attachment)) {
const auto format = VideoCore::Surface::PixelFormatFromRenderTargetFormat(regs.rt[color_attachment].format);
@@ -65,8 +65,72 @@ using VideoCore::Surface::SurfaceType;
.finalLayout = VK_IMAGE_LAYOUT_GENERAL,
};
}
struct ResolveAspects {
bool depth;
bool stencil;
};
struct ResolveModes {
VkResolveModeFlagBits depth;
VkResolveModeFlagBits stencil;
};
constexpr ResolveAspects GetResolveAspects(PixelFormat format) {
const SurfaceType surface_type = GetSurfaceType(format);
return ResolveAspects{
.depth = surface_type == SurfaceType::Depth ||
surface_type == SurfaceType::DepthStencil,
.stencil = surface_type == SurfaceType::Stencil ||
surface_type == SurfaceType::DepthStencil,
};
}
ResolveModes PickResolveModes(const Device& device, PixelFormat format) {
constexpr VkResolveModeFlagBits mode = VK_RESOLVE_MODE_SAMPLE_ZERO_BIT;
const ResolveAspects aspects = GetResolveAspects(format);
const bool depth_mode_supported = (device.GetDepthResolveModes() & mode) != 0;
const bool stencil_mode_supported = (device.GetStencilResolveModes() & mode) != 0;
ResolveModes modes{
.depth = VK_RESOLVE_MODE_NONE,
.stencil = VK_RESOLVE_MODE_NONE,
};
if (aspects.depth && depth_mode_supported) {
modes.depth = mode;
}
if (aspects.stencil && stencil_mode_supported) {
modes.stencil = mode;
}
if (modes.depth == modes.stencil || device.SupportsIndependentResolveNone()) {
return modes;
}
if (modes.depth != VK_RESOLVE_MODE_NONE && stencil_mode_supported) {
modes.stencil = mode;
} else if (modes.stencil != VK_RESOLVE_MODE_NONE && depth_mode_supported) {
modes.depth = mode;
}
return modes;
}
} // Anonymous namespace
bool SupportsDepthStencilResolve(const Device& device, PixelFormat depth_format) {
if (depth_format == PixelFormat::Invalid || !device.IsKhrDepthStencilResolveSupported()) {
return false;
}
const ResolveAspects aspects = GetResolveAspects(depth_format);
if (!aspects.depth && !aspects.stencil) {
return false;
}
const ResolveModes modes = PickResolveModes(device, depth_format);
if ((aspects.depth && modes.depth == VK_RESOLVE_MODE_NONE) ||
(aspects.stencil && modes.stencil == VK_RESOLVE_MODE_NONE)) {
return false;
}
return modes.depth == modes.stencil || device.SupportsIndependentResolveNone();
}
RenderPassCache::RenderPassCache(const Device& device_) : device{&device_} {}
VkRenderPass RenderPassCache::Get(const RenderPassKey& key) {
@@ -75,7 +139,9 @@ VkRenderPass RenderPassCache::Get(const RenderPassKey& key) {
if (!is_new) {
return *pair->second;
}
boost::container::static_vector<VkAttachmentDescription, 9> descriptions;
static constexpr size_t MAX_ATTACHMENTS =
2 * std::tuple_size_v<decltype(RenderPassKey::color_formats)> + 2;
boost::container::static_vector<VkAttachmentDescription, MAX_ATTACHMENTS> descriptions;
std::array<VkAttachmentReference, 8> references{};
u32 num_attachments{};
u32 num_colors{};
@@ -109,8 +175,11 @@ VkRenderPass RenderPassCache::Get(const RenderPassKey& key) {
const VkAttachmentLoadOp depth_load_op = key.depth_stencil_clear
? VK_ATTACHMENT_LOAD_OP_CLEAR
: VK_ATTACHMENT_LOAD_OP_LOAD;
const VkAttachmentStoreOp depth_store_op = key.depth_stencil_discard
? VK_ATTACHMENT_STORE_OP_DONT_CARE
: VK_ATTACHMENT_STORE_OP_STORE;
descriptions.push_back(AttachmentDescription(*device, key.depth_format, key.samples,
depth_load_op, VK_ATTACHMENT_STORE_OP_STORE));
depth_load_op, depth_store_op));
}
std::array<VkAttachmentReference, 8> resolve_references{};
const bool do_resolve_color =
@@ -133,6 +202,21 @@ VkRenderPass RenderPassCache::Get(const RenderPassKey& key) {
}
}
}
const bool do_resolve_depth_stencil = key.resolve_depth_stencil && has_depth &&
key.samples != VK_SAMPLE_COUNT_1_BIT &&
SupportsDepthStencilResolve(*device, key.depth_format);
VkAttachmentReference depth_resolve_reference{};
if (do_resolve_depth_stencil) {
depth_resolve_reference = VkAttachmentReference{
.attachment = static_cast<u32>(descriptions.size()),
.layout = VK_IMAGE_LAYOUT_GENERAL,
};
VkAttachmentDescription resolve_desc =
AttachmentDescription(*device, key.depth_format, VK_SAMPLE_COUNT_1_BIT,
VK_ATTACHMENT_LOAD_OP_DONT_CARE, VK_ATTACHMENT_STORE_OP_STORE);
resolve_desc.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
descriptions.push_back(resolve_desc);
}
const VkSubpassDescription subpass{
.flags = 0,
.pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS,
@@ -157,6 +241,92 @@ VkRenderPass RenderPassCache::Get(const RenderPassKey& key) {
.dstAccessMask = VK_ACCESS_SHADER_READ_BIT,
.dependencyFlags = VK_DEPENDENCY_BY_REGION_BIT
};
if (device->IsKhrCreateRenderPass2Supported()) {
boost::container::static_vector<VkAttachmentDescription2, MAX_ATTACHMENTS> descriptions2;
for (const VkAttachmentDescription& description : descriptions) {
descriptions2.push_back(VkAttachmentDescription2{
.sType = VK_STRUCTURE_TYPE_ATTACHMENT_DESCRIPTION_2,
.pNext = nullptr,
.flags = description.flags,
.format = description.format,
.samples = description.samples,
.loadOp = description.loadOp,
.storeOp = description.storeOp,
.stencilLoadOp = description.stencilLoadOp,
.stencilStoreOp = description.stencilStoreOp,
.initialLayout = description.initialLayout,
.finalLayout = description.finalLayout,
});
}
const auto promote = [](const VkAttachmentReference& reference) {
return VkAttachmentReference2{
.sType = VK_STRUCTURE_TYPE_ATTACHMENT_REFERENCE_2,
.pNext = nullptr,
.attachment = reference.attachment,
.layout = reference.layout,
.aspectMask = 0,
};
};
std::array<VkAttachmentReference2, 8> references2{};
std::array<VkAttachmentReference2, 8> resolve_references2{};
for (size_t index = 0; index < references.size(); ++index) {
references2[index] = promote(references[index]);
resolve_references2[index] = promote(resolve_references[index]);
}
const VkAttachmentReference2 depth_reference2 = promote(depth_reference);
const VkAttachmentReference2 depth_resolve_reference2 = promote(depth_resolve_reference);
const ResolveModes resolve_modes = PickResolveModes(*device, key.depth_format);
const VkSubpassDescriptionDepthStencilResolve depth_stencil_resolve{
.sType = VK_STRUCTURE_TYPE_SUBPASS_DESCRIPTION_DEPTH_STENCIL_RESOLVE,
.pNext = nullptr,
.depthResolveMode = resolve_modes.depth,
.stencilResolveMode = resolve_modes.stencil,
.pDepthStencilResolveAttachment = &depth_resolve_reference2,
};
const VkSubpassDescription2 subpass2{
.sType = VK_STRUCTURE_TYPE_SUBPASS_DESCRIPTION_2,
.pNext = do_resolve_depth_stencil ? &depth_stencil_resolve : nullptr,
.flags = 0,
.pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS,
.viewMask = 0,
.inputAttachmentCount = 0,
.pInputAttachments = nullptr,
.colorAttachmentCount = num_attachments,
.pColorAttachments = references2.data(),
.pResolveAttachments = do_resolve_color ? resolve_references2.data() : nullptr,
.pDepthStencilAttachment = has_depth ? &depth_reference2 : nullptr,
.preserveAttachmentCount = 0,
.pPreserveAttachments = nullptr,
};
const VkSubpassDependency2 dependency2{
.sType = VK_STRUCTURE_TYPE_SUBPASS_DEPENDENCY_2,
.pNext = nullptr,
.srcSubpass = dependency.srcSubpass,
.dstSubpass = dependency.dstSubpass,
.srcStageMask = dependency.srcStageMask,
.dstStageMask = dependency.dstStageMask,
.srcAccessMask = dependency.srcAccessMask,
.dstAccessMask = dependency.dstAccessMask,
.dependencyFlags = dependency.dependencyFlags,
.viewOffset = 0,
};
pair->second = device->GetLogical().CreateRenderPass2({
.sType = VK_STRUCTURE_TYPE_RENDER_PASS_CREATE_INFO_2,
.pNext = nullptr,
.flags = 0,
.attachmentCount = static_cast<u32>(descriptions2.size()),
.pAttachments = descriptions2.empty() ? nullptr : descriptions2.data(),
.subpassCount = 1,
.pSubpasses = &subpass2,
.dependencyCount = 1,
.pDependencies = &dependency2,
.correlatedViewMaskCount = 0,
.pCorrelatedViewMasks = nullptr,
});
return *pair->second;
}
pair->second = device->GetLogical().CreateRenderPass({
.sType = VK_STRUCTURE_TYPE_RENDER_PASS_CREATE_INFO,
.pNext = nullptr,
@@ -9,6 +9,7 @@
#include <mutex>
#include <ankerl/unordered_dense.h>
#include "common/container_hash.h"
#include "video_core/surface.h"
#include "video_core/vulkan_common/vulkan_wrapper.h"
@@ -21,9 +22,11 @@ struct RenderPassKey {
VideoCore::Surface::PixelFormat depth_format;
VkSampleCountFlagBits samples;
bool resolve_color;
bool resolve_depth_stencil;
u32 color_clear_mask;
bool depth_stencil_clear;
u32 color_discard_mask;
bool depth_stencil_discard;
};
} // namespace Vulkan
@@ -31,17 +34,28 @@ struct RenderPassKey {
namespace std {
template <>
struct hash<Vulkan::RenderPassKey> {
static_assert(std::tuple_size_v<decltype(Vulkan::RenderPassKey::color_formats)> <= 8);
static_assert(static_cast<u32>(VideoCore::Surface::PixelFormat::Invalid) <= 0xFF);
static_assert(static_cast<u32>(VideoCore::Surface::PixelFormat::Max) <= 0xFF);
static_assert(VK_SAMPLE_COUNT_64_BIT <= 0xFF);
[[nodiscard]] size_t operator()(const Vulkan::RenderPassKey& key) const noexcept {
size_t value = static_cast<size_t>(key.depth_format) << 48;
value ^= static_cast<size_t>(key.samples) << 52;
value ^= static_cast<size_t>(key.resolve_color) << 63;
value ^= static_cast<size_t>(key.color_clear_mask) << 54;
value ^= static_cast<size_t>(key.depth_stencil_clear) << 62;
value ^= static_cast<size_t>(key.color_discard_mask) << 24;
for (size_t i = 0; i < key.color_formats.size(); ++i) {
value ^= static_cast<size_t>(key.color_formats[i]) << (i * 6);
u64 formats = 0;
for (size_t index = 0; index < key.color_formats.size(); ++index) {
formats |= static_cast<u64>(key.color_formats[index]) << (index * 8);
}
return value;
const u64 state = static_cast<u64>(key.depth_format) |
(static_cast<u64>(key.samples) << 8) |
(static_cast<u64>(key.color_clear_mask) << 16) |
(static_cast<u64>(key.color_discard_mask) << 24) |
(static_cast<u64>(key.resolve_color) << 32) |
(static_cast<u64>(key.depth_stencil_clear) << 33) |
(static_cast<u64>(key.resolve_depth_stencil) << 34) |
(static_cast<u64>(key.depth_stencil_discard) << 35);
size_t seed = 0;
Common::HashCombine(seed, formats);
Common::HashCombine(seed, state);
return seed;
}
};
} // namespace std
@@ -50,6 +64,9 @@ namespace Vulkan {
class Device;
[[nodiscard]] bool SupportsDepthStencilResolve(const Device& device,
VideoCore::Surface::PixelFormat depth_format);
class RenderPassCache {
public:
explicit RenderPassCache(const Device& device_);
@@ -132,6 +132,7 @@ void Scheduler::BeginRenderPassImpl(const Framebuffer* framebuffer, VkRenderPass
num_renderpass_images = framebuffer->NumImages();
renderpass_images = framebuffer->Images();
renderpass_image_ranges = framebuffer->ImageRanges();
framebuffer->MarkResolveShadowsUpToDate();
}
void Scheduler::RealizeDeferredClear() {
@@ -155,8 +156,10 @@ void Scheduler::RealizeDeferredClear() {
}
const u32 color_discard_mask =
dc.framebuffer->DiscardsMsaaColor() ? dc.color_clear_mask : 0u;
const bool depth_stencil_discard =
dc.depth_stencil && dc.framebuffer->DiscardsMsaaDepthStencil();
const VkRenderPass renderpass = dc.framebuffer->RenderPassVariant(
dc.color_clear_mask, dc.depth_stencil, color_discard_mask);
dc.color_clear_mask, dc.depth_stencil, color_discard_mask, depth_stencil_discard);
EndRenderPass();
BeginRenderPassImpl(dc.framebuffer, renderpass, clear_values.data(), count);
}
@@ -190,6 +193,14 @@ bool Scheduler::DeferDepthStencilClear(const Framebuffer* framebuffer, const VkC
return true;
}
void Scheduler::FlushDeferredClear() {
if (deferred_clear.framebuffer == nullptr) {
return;
}
RealizeDeferredClear();
EndRenderPass();
}
void Scheduler::RequestRenderpass(const Framebuffer* framebuffer) {
if (deferred_clear.framebuffer == framebuffer) {
RealizeDeferredClear();
@@ -65,6 +65,9 @@ public:
/// Defers a full depth/stencil clear so it becomes the next render pass.
bool DeferDepthStencilClear(const Framebuffer* framebuffer, const VkClearValue& value);
/// Realizes any pending deferred clear before its framebuffer can be moved or freed.
void FlushDeferredClear();
/// Requests the current execution context to be able to execute operations only allowed outside
/// of a renderpass.
void RequestOutsideRenderPassOperationContext();
@@ -33,10 +33,10 @@ constexpr VkDeviceSize MAX_ALIGNMENT = 256;
// Windows ones however, can intake bigger buffers and generally do not OOM.
// - GTX 960 on Windows will not OOM with 256mib
// - GT 1030 on ^NIX will OOM with 256mib
#if defined(_WIN32) || defined(__ANDROID__)
constexpr VkDeviceSize MAX_STREAM_BUFFER_SIZE = 256_MiB;
#else
#if defined(__FreeBSD__)
constexpr VkDeviceSize MAX_STREAM_BUFFER_SIZE = 128_MiB;
#else
constexpr VkDeviceSize MAX_STREAM_BUFFER_SIZE = 256_MiB;
#endif
size_t GetStreamBufferSize(const Device& device) {
File diff suppressed because it is too large Load Diff
+160 -42
View File
@@ -60,13 +60,19 @@ public:
void TickFrame();
void FlushDeferredClear();
u64 GetDeviceLocalMemory() const;
u64 GetDeviceMemoryUsage() const;
bool CanReportMemoryUsage() const;
std::optional<size_t> GetSamplerHeapBudget() const;
bool CanDownloadMsaa(const VideoCommon::ImageInfo& info) const;
[[nodiscard]] VkImage AcquireMsaaScratchImage(const VkImageCreateInfo& image_ci);
void ReleaseMsaaScratchImage(VkImage image);
void BlitImage(Framebuffer* dst_framebuffer, ImageView& dst, ImageView& src,
const Region2D& dst_region, const Region2D& src_region,
@@ -95,6 +101,21 @@ public:
std::span<const VideoCommon::SwizzleParameters>,
u32 z_start, u32 z_count);
[[nodiscard]] bool IsUnifiedMemoryBindable() const noexcept;
[[nodiscard]] u64 UnifiedMemoryBase() const noexcept;
[[nodiscard]] u64 UnifiedMemorySize() const noexcept;
[[nodiscard]] u64 UnifiedMemoryWindowSize() const noexcept;
[[nodiscard]] bool CanUploadImageDirectly(const VideoCommon::ImageInfo& info) const;
bool UploadImageDirectly(Image& image, size_t window_index, u64 window_offset,
std::span<const VideoCommon::SwizzleParameters> swizzles);
[[nodiscard]] u64 CurrentTick() const noexcept;
void InsertUploadMemoryBarrier() {}
void TransitionImageLayout(Image& image);
@@ -117,16 +138,20 @@ public:
VkFormat format = VK_FORMAT_UNDEFINED;
VkExtent2D extent{};
u32 layers = 0;
VkImageAspectFlags aspect_mask = VK_IMAGE_ASPECT_COLOR_BIT;
bool up_to_date = false;
};
[[nodiscard]] VkImageView GetOrCreateResolveShadow(VkImage msaa_image, VkFormat format,
VkExtent2D extent, u32 layers);
VkExtent2D extent, u32 layers,
VkImageAspectFlags aspect_mask);
[[nodiscard]] const ResolveShadow* GetValidResolveShadow(VkImage msaa_image) const;
void InvalidateResolveShadow(VkImage msaa_image);
void MarkResolveShadowUpToDate(VkImage msaa_image);
void EraseResolveShadow(VkImage msaa_image);
std::span<const VkFormat> ViewFormats(PixelFormat format) {
@@ -149,13 +174,37 @@ public:
std::optional<ASTCDecoderPass> astc_decoder_pass;
std::optional<BlockLinearUnswizzle3DPass> bl3d_unswizzle_pass;
std::optional<BlockLinearUnswizzle2DPass> bl2d_unswizzle_pass;
std::optional<BlockLinearUnswizzle3DBufferPass> bl3db_unswizzle_pass;
const Settings::ResolutionScalingInfo& resolution;
std::array<std::vector<VkFormat>, VideoCore::Surface::MaxPixelFormat> view_formats;
static constexpr size_t indexing_slots = 8 * sizeof(size_t);
std::array<vk::Buffer, indexing_slots> buffers{};
std::vector<std::pair<u64, vk::Image>> pending_msaa_images;
struct MsaaScratchKey {
VkFormat format;
VkImageType type;
u32 width;
u32 height;
u32 depth;
u32 levels;
u32 layers;
VkImageUsageFlags usage;
VkImageCreateFlags flags;
bool operator==(const MsaaScratchKey&) const noexcept = default;
};
struct MsaaScratchImage {
MsaaScratchKey key;
vk::Image image;
u64 tick;
u32 unused_frames;
};
std::vector<MsaaScratchImage> msaa_scratch_images;
ankerl::unordered_dense::map<VkImage, ResolveShadow> resolve_shadows;
std::vector<std::pair<u64, ResolveShadow>> pending_resolve_shadows;
};
class Framebuffer {
@@ -191,7 +240,8 @@ public:
}
[[nodiscard]] VkRenderPass RenderPassVariant(u32 color_clear_mask, bool depth_stencil_clear,
u32 color_discard_mask) const;
u32 color_discard_mask,
bool depth_stencil_discard) const;
[[nodiscard]] VkExtent2D RenderArea() const noexcept {
return render_area;
@@ -233,19 +283,21 @@ public:
return is_rescaled;
}
[[nodiscard]] bool HasResolveColor() const noexcept {
return !resolve_images.empty();
}
[[nodiscard]] VkImage ResolveColorImage(size_t index) const noexcept {
return index < resolve_images.size() ? *resolve_images[index] : VK_NULL_HANDLE;
}
[[nodiscard]] bool DiscardsMsaaColor() const noexcept {
return discard_msaa_color;
}
[[nodiscard]] bool DiscardsMsaaDepthStencil() const noexcept {
return discard_msaa_depth_stencil;
}
/// Records that a render pass has begun, so its resolve attachments will hold valid contents
/// once it ends.
void MarkResolveShadowsUpToDate() const;
private:
static constexpr size_t NUM_MEMOIZED_RENDER_PASS_VARIANTS = 8;
vk::Framebuffer framebuffer;
VkRenderPass renderpass{};
VkExtent2D render_area{};
@@ -258,11 +310,16 @@ private:
bool has_depth{};
bool has_stencil{};
bool is_rescaled{};
std::vector<vk::Image> resolve_images;
std::vector<vk::ImageView> resolve_image_views;
std::array<VkImage, 9> resolve_shadow_images{};
u32 num_resolve_shadows = 0;
TextureCacheRuntime* runtime_ptr{nullptr};
RenderPassKey render_pass_key{};
RenderPassCache* render_pass_cache{nullptr};
bool discard_msaa_color{};
bool discard_msaa_depth_stencil{};
mutable std::array<u32, NUM_MEMOIZED_RENDER_PASS_VARIANTS> variant_keys{};
mutable std::array<VkRenderPass, NUM_MEMOIZED_RENDER_PASS_VARIANTS> variant_render_passes{};
mutable u32 num_memoized_variants{};
};
class Image : public VideoCommon::ImageBase {
@@ -404,6 +461,22 @@ public:
return supports_depth_comparison;
}
[[nodiscard]] bool RequiresBorderColorFormat() const noexcept {
return requires_border_color_format;
}
[[nodiscard]] bool SupportsMinmaxFilter() const noexcept {
return supports_minmax_filter;
}
[[nodiscard]] const VkComponentMapping& Swizzle() const noexcept {
return swizzle_mapping;
}
[[nodiscard]] bool HasIdentitySwizzle() const noexcept {
return has_identity_swizzle;
}
[[nodiscard]] GPUVAddr GpuAddr() const noexcept {
return gpu_addr;
}
@@ -436,48 +509,91 @@ private:
VkSampleCountFlagBits samples = VK_SAMPLE_COUNT_1_BIT;
u32 buffer_size = 0;
VkComponentMapping swizzle_mapping{};
bool supports_depth_comparison = false;
bool requires_border_color_format = false;
bool supports_minmax_filter = false;
bool has_identity_swizzle = true;
};
class ImageAlloc : public VideoCommon::ImageAllocBase {};
class CustomBorderColorBudget {
public:
CustomBorderColorBudget() = default;
~CustomBorderColorBudget();
CustomBorderColorBudget(const CustomBorderColorBudget&) = delete;
CustomBorderColorBudget& operator=(const CustomBorderColorBudget&) = delete;
CustomBorderColorBudget(CustomBorderColorBudget&& rhs) noexcept;
CustomBorderColorBudget& operator=(CustomBorderColorBudget&& rhs) noexcept;
bool TryAcquire(const Device& device, size_t count);
private:
void Release() noexcept;
const Device* device_ptr = nullptr;
size_t held = 0;
};
class Sampler {
public:
explicit Sampler(TextureCacheRuntime&, const Tegra::Texture::TSCEntry&);
[[nodiscard]] VkSampler Handle() const noexcept {
return *sampler;
return *variants.front().sampler;
}
[[nodiscard]] VkSampler HandleWithDefaultAnisotropy() const noexcept {
return *sampler_default_anisotropy;
}
[[nodiscard]] bool HasAddedAnisotropy() const noexcept {
return static_cast<bool>(sampler_default_anisotropy);
}
[[nodiscard]] VkSampler HandleWithNearestFilter() const noexcept {
return *sampler_nearest;
}
[[nodiscard]] bool HasLinearFiltering() const noexcept {
return static_cast<bool>(sampler_nearest);
}
[[nodiscard]] VkSampler HandleWithoutDepthComparison() const noexcept {
return *sampler_noncompare;
}
[[nodiscard]] bool HasDepthComparison() const noexcept {
return static_cast<bool>(sampler_noncompare);
}
[[nodiscard]] VkSampler HandleFor(const ImageView& image_view, bool is_depth);
private:
vk::Sampler sampler;
vk::Sampler sampler_default_anisotropy;
vk::Sampler sampler_nearest;
vk::Sampler sampler_noncompare;
struct VariantKey {
bool reduce_anisotropy;
bool force_nearest;
bool drop_depth_comparison;
bool drop_reduction;
bool drop_custom_border;
bool srgb_border;
std::array<VkComponentSwizzle, 4> swizzle;
bool operator==(const VariantKey&) const noexcept = default;
[[nodiscard]] bool HasSwizzle() const noexcept {
return swizzle != std::array<VkComponentSwizzle, 4>{};
}
};
struct Variant {
VariantKey key;
vk::Sampler sampler;
};
static constexpr size_t MAX_VARIANTS = 32;
[[nodiscard]] VariantKey MakeKey(const ImageView& image_view, bool is_depth) const noexcept;
[[nodiscard]] VkSampler Find(const VariantKey& key) const noexcept;
VkSampler Emplace(VariantKey key);
CustomBorderColorBudget custom_border_color_budget;
std::vector<Variant> variants;
const Device* device_ptr{nullptr};
VkSamplerCreateInfo base_ci{};
VkSamplerReductionModeEXT reduction_mode{VK_SAMPLER_REDUCTION_MODE_WEIGHTED_AVERAGE_EXT};
std::array<float, 4> border_color{};
std::array<float, 4> srgb_border_color{};
f32 default_anisotropy{1.0f};
bool has_added_anisotropy{};
bool has_linear_filtering{};
bool has_depth_comparison{};
bool has_minmax_reduction{};
bool has_custom_border_colors{};
bool has_srgb_border_color{};
bool needs_swizzle_mapping{};
};
struct TextureCacheParams {
@@ -486,6 +602,8 @@ struct TextureCacheParams {
static constexpr bool HAS_EMULATED_COPIES = false;
static constexpr bool HAS_DEVICE_MEMORY_INFO = true;
static constexpr bool IMPLEMENTS_ASYNC_DOWNLOADS = true;
static constexpr bool HAS_MSAA_DOWNLOADS = true;
static constexpr bool USE_UNIFIED_MEMORY = true;
using Runtime = Vulkan::TextureCacheRuntime;
using Image = Vulkan::Image;
+2 -6
View File
@@ -1,4 +1,4 @@
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
@@ -121,7 +121,7 @@ void ImageBase::InsertView(const ImageViewInfo& view_info, ImageViewId image_vie
image_view_ids.push_back(image_view_id);
}
bool ImageBase::IsSafeDownload() const noexcept {
bool ImageBase::IsSafeGpuCopy() const noexcept {
// Skip images that were not modified from the GPU
if (False(flags & ImageFlagBits::GpuModified)) {
return false;
@@ -131,10 +131,6 @@ bool ImageBase::IsSafeDownload() const noexcept {
if (True(flags & ImageFlagBits::CpuModified)) {
return false;
}
if (info.num_samples > 1) {
LOG_WARNING(HW_GPU, "MSAA image downloads are not implemented");
return false;
}
return true;
}
+6 -1
View File
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
@@ -64,7 +67,7 @@ struct ImageBase {
void InsertView(const ImageViewInfo& view_info, ImageViewId image_view_id);
[[nodiscard]] bool IsSafeDownload() const noexcept;
[[nodiscard]] bool IsSafeGpuCopy() const noexcept;
[[nodiscard]] bool Overlaps(VAddr overlap_cpu_addr, size_t overlap_size) const noexcept {
const VAddr overlap_end = overlap_cpu_addr + overlap_size;
@@ -92,6 +95,8 @@ struct ImageBase {
u32 scale_rating = 0;
u64 scale_tick = 0;
bool has_scaled = false;
bool direct_upload_used = false;
bool direct_upload_blocked = false;
size_t channel = 0;
+79 -64
View File
@@ -134,7 +134,7 @@ void TextureCache<P>::RunGarbageCollector() {
if (True(image.flags & ImageFlagBits::IsDecoding)) {
return false;
}
const bool must_download = image.IsSafeDownload() && False(image.flags & ImageFlagBits::BadOverlap);
const bool must_download = IsDownloadable(image) && False(image.flags & ImageFlagBits::BadOverlap);
if ((!aggressive_mode && True(image.flags & ImageFlagBits::CostlyLoad)) || (!high_priority_mode && must_download)) {
return false;
}
@@ -577,6 +577,7 @@ FramebufferId TextureCache<P>::GetFramebufferId(const RenderTargets& key) {
return id ? &slot_image_views[id] : nullptr;
});
ImageView* const depth_buffer = key.depth_buffer_id ? &slot_image_views[key.depth_buffer_id] : nullptr;
runtime.FlushDeferredClear();
framebuffer_id = slot_framebuffers.insert(runtime, color_buffers, depth_buffer, key);
return framebuffer_id;
}
@@ -584,6 +585,10 @@ FramebufferId TextureCache<P>::GetFramebufferId(const RenderTargets& key) {
template <class P>
void TextureCache<P>::WriteMemory(DAddr cpu_addr, size_t size) {
ForEachImageInRegion(cpu_addr, size, [this](ImageId image_id, Image& image) {
if (image.direct_upload_used) {
image.direct_upload_used = false;
image.direct_upload_blocked = true;
}
if (True(image.flags & ImageFlagBits::CpuModified)) {
return;
}
@@ -594,11 +599,26 @@ void TextureCache<P>::WriteMemory(DAddr cpu_addr, size_t size) {
});
}
template <class P>
bool TextureCache<P>::IsDownloadable(const ImageBase& image) const noexcept {
if (!image.IsSafeGpuCopy()) {
return false;
}
if (image.info.num_samples == 1) {
return true;
}
if constexpr (P::HAS_MSAA_DOWNLOADS) {
return runtime.CanDownloadMsaa(image.info);
} else {
return false;
}
}
template <class P>
void TextureCache<P>::DownloadMemory(DAddr cpu_addr, size_t size) {
boost::container::small_vector<ImageId, 16> images;
ForEachImageInRegion(cpu_addr, size, [&images](ImageId image_id, ImageBase& image) {
if (!image.IsSafeDownload()) {
ForEachImageInRegion(cpu_addr, size, [this, &images](ImageId image_id, ImageBase& image) {
if (!IsDownloadable(image)) {
return;
}
image.flags &= ~ImageFlagBits::GpuModified;
@@ -1129,11 +1149,59 @@ void TextureCache<P>::RefreshContents(Image& image, ImageId image_id) {
QueueAsyncUnswizzle(image, image_id);
return;
}
if (True(image.flags & ImageFlagBits::AcceleratedUpload) &&
TryUploadFromUnifiedMemory(image)) {
runtime.InsertUploadMemoryBarrier();
return;
}
auto staging = runtime.UploadStagingBuffer(MapSizeBytes(image));
UploadImageContents(image, staging);
runtime.InsertUploadMemoryBarrier();
}
template <class P>
bool TextureCache<P>::TryUploadFromUnifiedMemory([[maybe_unused]] Image& image) {
if constexpr (USE_UNIFIED_MEMORY) {
if (image.direct_upload_blocked || image.guest_size_bytes == 0) {
return false;
}
if (!runtime.IsUnifiedMemoryBindable() || !runtime.CanUploadImageDirectly(image.info)) {
return false;
}
const u64 window_size = runtime.UnifiedMemoryWindowSize();
if (window_size == 0) {
return false;
}
const u8* const first = gpu_memory->GetSpan(image.gpu_addr, image.guest_size_bytes);
if (first == nullptr) {
return false;
}
const u64 phys_offset = static_cast<u64>(first - device_memory.GetPhysicalBase());
const u64 unified_base = runtime.UnifiedMemoryBase();
if (phys_offset < unified_base) {
return false;
}
const u64 relative = phys_offset - unified_base;
const u64 unified_size = runtime.UnifiedMemorySize();
if (relative >= unified_size || unified_size - relative < image.guest_size_bytes) {
return false;
}
const u64 local_offset = relative % window_size;
if (window_size - local_offset < image.guest_size_bytes) {
return false;
}
const auto swizzles = FullUploadSwizzles(image.info);
if (!runtime.UploadImageDirectly(image, static_cast<size_t>(relative / window_size),
local_offset, FixSmallVectorADL(swizzles))) {
return false;
}
image.direct_upload_used = true;
return true;
} else {
return false;
}
}
template <class P>
template <typename StagingBuffer>
void TextureCache<P>::UploadImageContents(Image& image, StagingBuffer& staging) {
@@ -1476,12 +1544,12 @@ template <class P>
bool TextureCache<P>::ScaleUp(Image& image) {
const bool has_copy = image.HasScaled();
const bool rescaled = image.ScaleUp();
if (!has_copy && image.HasScaled()) {
total_used_memory += GetScaledImageSizeBytes(image);
}
if (!rescaled) {
return false;
}
if (!has_copy) {
total_used_memory += GetScaledImageSizeBytes(image);
}
InvalidateScale(image);
return true;
}
@@ -1691,7 +1759,10 @@ ImageId TextureCache<P>::JoinImages(const ImageInfo& info, GPUVAddr gpu_addr, DA
for (const auto& copy_object : join_copies_to_do) {
Image& overlap = slot_images[copy_object.id];
if (copy_object.is_alias) {
if (!overlap.IsSafeDownload()) {
if (!overlap.IsSafeGpuCopy()) {
continue;
}
if (overlap.info.num_samples != new_image.info.num_samples) {
continue;
}
const auto alias_pointer = join_alias_indices.find(copy_object.id);
@@ -1877,67 +1948,10 @@ SamplerId TextureCache<P>::FindSampler(const TSCEntry& config, bool compute) {
const auto [pair, is_new] = channel_state->samplers.try_emplace(config);
if (is_new) {
pair->second = slot_samplers.insert(runtime, config);
EnforceSamplerBudget();
}
return pair->second;
}
template <class P>
std::optional<size_t> TextureCache<P>::QuerySamplerBudget() const {
if constexpr (requires { runtime.GetSamplerHeapBudget(); }) {
return runtime.GetSamplerHeapBudget();
} else {
return std::nullopt;
}
}
template <class P>
void TextureCache<P>::EnforceSamplerBudget() {
if (auto const budget = QuerySamplerBudget(); budget) {
if (slot_samplers.size() < *budget) {
return;
}
if (!channel_state) {
return;
}
if (last_sampler_gc_frame == frame_tick) {
return;
}
last_sampler_gc_frame = frame_tick;
TrimInactiveSamplers(*budget);
}
}
template <class P>
void TextureCache<P>::TrimInactiveSamplers(size_t budget) {
if (channel_state->samplers.size() > 0) {
constexpr size_t SAMPLER_GC_SLACK = 1024;
ankerl::unordered_dense::set<SamplerId> active_sampler_ids;
for (auto const& e : channel_state->sampler_ids)
active_sampler_ids.insert(e.second);
// Elements in the map must be necesarily valid
size_t removed = 0;
for (auto it = channel_state->samplers.begin(); it != channel_state->samplers.end();) {
const SamplerId sampler_id = it->second;
if (!sampler_id || sampler_id == CORRUPT_ID) {
it = channel_state->samplers.erase(it);
} else if (std::ranges::find(active_sampler_ids, sampler_id) != active_sampler_ids.end()) {
++it;
} else {
slot_samplers.erase(sampler_id);
it = channel_state->samplers.erase(it);
++removed;
if (slot_samplers.size() + SAMPLER_GC_SLACK <= budget) {
break;
}
}
}
if (removed != 0) {
LOG_WARNING(HW_GPU, "Sampler cache exceeded {} entries on this driver; reclaimed {} inactive samplers", budget, removed);
}
}
}
template <class P>
ImageViewId TextureCache<P>::FindColorBuffer(size_t index) {
const auto& regs = maxwell3d->regs;
@@ -2461,6 +2475,7 @@ void TextureCache<P>::RemoveImageViewReferences(std::span<const ImageViewId> rem
template <class P>
void TextureCache<P>::RemoveFramebuffers(std::span<const ImageViewId> removed_views) {
runtime.FlushDeferredClear();
auto it = framebuffers.begin();
while (it != framebuffers.end()) {
if (it->first.Contains(removed_views)) {
@@ -108,6 +108,7 @@ class TextureCache : public VideoCommon::ChannelSetupCaches<TextureCacheChannelI
static constexpr bool HAS_DEVICE_MEMORY_INFO = P::HAS_DEVICE_MEMORY_INFO;
/// True when the API can do asynchronous texture downloads.
static constexpr bool IMPLEMENTS_ASYNC_DOWNLOADS = P::IMPLEMENTS_ASYNC_DOWNLOADS;
static constexpr bool USE_UNIFIED_MEMORY = P::USE_UNIFIED_MEMORY;
static constexpr size_t UNSET_CHANNEL{(std::numeric_limits<size_t>::max)()};
@@ -303,8 +304,12 @@ private:
FramebufferId GetFramebufferId(const RenderTargets& key);
/// Refresh the contents (pixel data) of an image
[[nodiscard]] bool IsDownloadable(const ImageBase& image) const noexcept;
void RefreshContents(Image& image, ImageId image_id);
bool TryUploadFromUnifiedMemory(Image& image);
/// Upload data from guest to an image
template <typename StagingBuffer>
void UploadImageContents(Image& image, StagingBuffer& staging_buffer);
@@ -418,9 +423,6 @@ private:
void QueueAsyncDecode(Image& image, ImageId image_id);
void TickAsyncDecode();
void EnforceSamplerBudget();
void TrimInactiveSamplers(size_t budget);
std::optional<size_t> QuerySamplerBudget() const;
void QueueAsyncUnswizzle(Image& image, ImageId image_id);
void TickAsyncUnswizzle();
@@ -507,7 +509,6 @@ private:
u64 modification_tick = 0;
u64 frame_tick = 0;
u64 last_sampler_gc_frame = (std::numeric_limits<u64>::max)();
Common::ThreadWorker texture_decode_worker{1, "TextureDecoder", {},
Common::ThreadPlacement::Efficiency};
+22 -49
View File
@@ -1,10 +1,11 @@
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <array>
#include <cmath>
#include "common/cityhash.h"
#include "common/settings.h"
@@ -17,53 +18,25 @@ namespace Tegra::Texture {
namespace {
[[maybe_unused]] constexpr std::array<float, 256> SRGB_CONVERSION_LUT = {
0.000000f, 0.000000f, 0.000000f, 0.000012f, 0.000021f, 0.000033f, 0.000046f, 0.000062f,
0.000081f, 0.000102f, 0.000125f, 0.000151f, 0.000181f, 0.000214f, 0.000251f, 0.000293f,
0.000338f, 0.000388f, 0.000443f, 0.000503f, 0.000568f, 0.000639f, 0.000715f, 0.000798f,
0.000887f, 0.000983f, 0.001085f, 0.001195f, 0.001312f, 0.001437f, 0.001569f, 0.001710f,
0.001860f, 0.002019f, 0.002186f, 0.002364f, 0.002551f, 0.002748f, 0.002955f, 0.003174f,
0.003403f, 0.003643f, 0.003896f, 0.004160f, 0.004436f, 0.004725f, 0.005028f, 0.005343f,
0.005672f, 0.006015f, 0.006372f, 0.006744f, 0.007130f, 0.007533f, 0.007950f, 0.008384f,
0.008834f, 0.009301f, 0.009785f, 0.010286f, 0.010805f, 0.011342f, 0.011898f, 0.012472f,
0.013066f, 0.013680f, 0.014313f, 0.014967f, 0.015641f, 0.016337f, 0.017054f, 0.017793f,
0.018554f, 0.019337f, 0.020144f, 0.020974f, 0.021828f, 0.022706f, 0.023609f, 0.024536f,
0.025489f, 0.026468f, 0.027473f, 0.028504f, 0.029563f, 0.030649f, 0.031762f, 0.032904f,
0.034074f, 0.035274f, 0.036503f, 0.037762f, 0.039050f, 0.040370f, 0.041721f, 0.043103f,
0.044518f, 0.045964f, 0.047444f, 0.048956f, 0.050503f, 0.052083f, 0.053699f, 0.055349f,
0.057034f, 0.058755f, 0.060513f, 0.062307f, 0.064139f, 0.066008f, 0.067915f, 0.069861f,
0.071845f, 0.073869f, 0.075933f, 0.078037f, 0.080182f, 0.082369f, 0.084597f, 0.086867f,
0.089180f, 0.091535f, 0.093935f, 0.096378f, 0.098866f, 0.101398f, 0.103977f, 0.106601f,
0.109271f, 0.111988f, 0.114753f, 0.117565f, 0.120426f, 0.123335f, 0.126293f, 0.129301f,
0.132360f, 0.135469f, 0.138629f, 0.141841f, 0.145105f, 0.148421f, 0.151791f, 0.155214f,
0.158691f, 0.162224f, 0.165810f, 0.169453f, 0.173152f, 0.176907f, 0.180720f, 0.184589f,
0.188517f, 0.192504f, 0.196549f, 0.200655f, 0.204820f, 0.209046f, 0.213334f, 0.217682f,
0.222093f, 0.226567f, 0.231104f, 0.235704f, 0.240369f, 0.245099f, 0.249894f, 0.254754f,
0.259681f, 0.264674f, 0.269736f, 0.274864f, 0.280062f, 0.285328f, 0.290664f, 0.296070f,
0.301546f, 0.307094f, 0.312713f, 0.318404f, 0.324168f, 0.330006f, 0.335916f, 0.341902f,
0.347962f, 0.354097f, 0.360309f, 0.366597f, 0.372961f, 0.379403f, 0.385924f, 0.392524f,
0.399202f, 0.405960f, 0.412798f, 0.419718f, 0.426719f, 0.433802f, 0.440967f, 0.448216f,
0.455548f, 0.462965f, 0.470465f, 0.478052f, 0.485725f, 0.493484f, 0.501329f, 0.509263f,
0.517285f, 0.525396f, 0.533595f, 0.541885f, 0.550265f, 0.558736f, 0.567299f, 0.575954f,
0.584702f, 0.593542f, 0.602477f, 0.611507f, 0.620632f, 0.629852f, 0.639168f, 0.648581f,
0.658092f, 0.667700f, 0.677408f, 0.687214f, 0.697120f, 0.707127f, 0.717234f, 0.727443f,
0.737753f, 0.748167f, 0.758685f, 0.769305f, 0.780031f, 0.790861f, 0.801798f, 0.812839f,
0.823989f, 0.835246f, 0.846611f, 0.858085f, 0.869668f, 0.881360f, 0.893164f, 0.905078f,
0.917104f, 0.929242f, 0.941493f, 0.953859f, 0.966338f, 1.000000f, 1.000000f, 1.000000f,
};
float SrgbToLinear(u32 value) {
const float encoded = static_cast<float>(value) / 255.0f;
if (encoded <= 0.04045f) {
return encoded / 12.92f;
}
return std::pow((encoded + 0.055f) / 1.055f, 2.4f);
}
} // Anonymous namespace
std::array<float, 4> TSCEntry::BorderColor() const noexcept {
// TODO: Handle SRGB correctly. Using this breaks shadows in some games (Xenoblade).
// if (!srgb_conversion) {
// return border_color;
//}
// return {SRGB_CONVERSION_LUT[srgb_border_color_r], SRGB_CONVERSION_LUT[srgb_border_color_g],
// SRGB_CONVERSION_LUT[srgb_border_color_b], border_color[3]};
return border_color;
}
std::array<float, 4> TSCEntry::SrgbBorderColor() const noexcept {
return {SrgbToLinear(srgb_border_color_r), SrgbToLinear(srgb_border_color_g),
SrgbToLinear(srgb_border_color_b), border_color[3]};
}
float TSCEntry::MaxAnisotropy() const noexcept {
const bool is_suitable_mipmap_filter = mipmap_filter != TextureMipmapFilter::None;
const bool has_regular_lods = min_lod_clamp == 0 && max_lod_clamp >= 256;
@@ -80,16 +53,16 @@ float TSCEntry::MaxAnisotropy() const noexcept {
case Settings::AnisotropyMode::X4:
case Settings::AnisotropyMode::X8:
case Settings::AnisotropyMode::X16:
case Settings::AnisotropyMode::X32:
case Settings::AnisotropyMode::X64:
added_anisotropic = u32(anisotropic_settings) - 1U;
added_anisotropic = s32(anisotropic_settings) - 1;
break;
case Settings::AnisotropyMode::Automatic:
added_anisotropic = Settings::values.resolution_info.up_scale >> Settings::values.resolution_info.down_shift;
added_anisotropic = (std::max)(added_anisotropic - 1U, 0U);
case Settings::AnisotropyMode::Automatic: {
const u32 resolution_scale = Settings::values.resolution_info.up_scale >>
Settings::values.resolution_info.down_shift;
if (resolution_scale > 1U) {
added_anisotropic = static_cast<s32>(resolution_scale - 1U);
}
break;
case Settings::AnisotropyMode::None:
return 1.0f; //No use of anisotropy
}
}
return float(1U << (max_anisotropy + added_anisotropic));
}
+5
View File
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2018 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
@@ -378,6 +381,8 @@ struct TSCEntry {
std::array<float, 4> BorderColor() const noexcept;
std::array<float, 4> SrgbBorderColor() const noexcept;
float MaxAnisotropy() const noexcept;
float MinLod() const {
+82 -37
View File
@@ -16,6 +16,7 @@
#include <fmt/format.h>
#include "common/assert.h"
#include "common/host_memory.h"
#include "common/literals.h"
#include <ranges>
#include "common/settings.h"
@@ -508,17 +509,9 @@ Device::Device(VkInstance instance_, vk::PhysicalDevice physical_, VkSurfaceKHR
LOG_WARNING(Render_Vulkan, "Qualcomm drivers require scaled vertex format emulation.");
has_broken_descriptor_aliasing = true;
LOG_WARNING(Render_Vulkan, "Qualcomm drivers have broken descriptor aliasing.");
LOG_WARNING(Render_Vulkan, "Qualcomm drivers have broken custom border color.");
RemoveExtensionFeature(extensions.custom_border_color, features.custom_border_color,
VK_EXT_CUSTOM_BORDER_COLOR_EXTENSION_NAME);
LOG_WARNING(Render_Vulkan, "Qualcomm drivers have broken border color swizzle.");
RemoveExtensionFeature(extensions.border_color_swizzle, features.border_color_swizzle,
VK_EXT_BORDER_COLOR_SWIZZLE_EXTENSION_NAME);
LOG_WARNING(Render_Vulkan, "Qualcomm drivers have broken color write enable.");
RemoveExtensionFeature(extensions.color_write_enable, features.color_write_enable,
VK_EXT_COLOR_WRITE_ENABLE_EXTENSION_NAME);
LOG_WARNING(Render_Vulkan, "Qualcomm drivers have broken shader float controls.");
RemoveExtension(extensions.shader_float_controls, VK_KHR_SHADER_FLOAT_CONTROLS_EXTENSION_NAME);
LOG_WARNING(Render_Vulkan, "Qualcomm drivers have broken shader atomic int64.");
RemoveExtensionFeature(extensions.shader_atomic_int64, features.shader_atomic_int64,
VK_KHR_SHADER_ATOMIC_INT64_EXTENSION_NAME);
@@ -619,21 +612,6 @@ Device::Device(VkInstance instance_, vk::PhysicalDevice physical_, VkSurfaceKHR
}
}
if (is_qualcomm) {
const size_t sampler_limit = properties.properties.limits.maxSamplerAllocationCount;
if (sampler_limit > 0) {
constexpr size_t MIN_SAMPLER_BUDGET = 1024U;
const size_t reserved = sampler_limit / 4U;
const size_t derived_budget =
(std::max)(MIN_SAMPLER_BUDGET, sampler_limit - reserved);
sampler_heap_budget = derived_budget;
LOG_WARNING(Render_Vulkan,
"Qualcomm driver reports max {} samplers; reserving {} (25%) and "
"allowing Eden to use {} (75%) to avoid heap exhaustion",
sampler_limit, reserved, sampler_heap_budget);
}
}
if (extensions.sampler_filter_minmax && is_amd) {
// Disable ext_sampler_filter_minmax on AMD GCN4 and lower as it is broken.
if (!features.shader_float16_int8.shaderFloat16) {
@@ -650,13 +628,6 @@ Device::Device(VkInstance instance_, vk::PhysicalDevice physical_, VkSurfaceKHR
features.shader_float16_int8.shaderFloat16 = false;
}
if (is_intel_windows) {
LOG_WARNING(Render_Vulkan,
"Intel proprietary drivers do not support MSAA->MSAA image blits. "
"MSAA scaling will use 3D helpers. MSAA resolves work normally.");
cant_blit_msaa = true;
}
has_broken_compute =
CheckBrokenCompute(properties.driver.driverID, properties.properties.driverVersion) &&
!Settings::values.enable_compute_pipelines.GetValue();
@@ -974,6 +945,13 @@ bool Device::GetSuitability(bool requires_swapchain) {
FOR_EACH_VK_FEATURE_EXT(FEATURE_EXTENSION);
FOR_EACH_VK_EXTENSION(EXTENSION);
FOR_EACH_VK_PLATFORM_EXTENSION(EXTENSION);
extensions.depth_stencil_resolve =
extensions.depth_stencil_resolve &&
(instance_version >= VK_API_VERSION_1_2 || extensions.create_renderpass2);
RemoveExtensionIfUnsuitable(extensions.depth_stencil_resolve,
VK_KHR_DEPTH_STENCIL_RESOLVE_EXTENSION_NAME);
if (supported_extensions.contains(VK_KHR_ROBUSTNESS_2_EXTENSION_NAME)) {
loaded_extensions.erase(VK_EXT_ROBUSTNESS_2_EXTENSION_NAME);
@@ -986,6 +964,13 @@ bool Device::GetSuitability(bool requires_swapchain) {
extensions.robustness_2 = false;
}
#ifdef __ANDROID__
if (extensions.external_memory_ahb && !extensions.queue_family_foreign) {
loaded_extensions.erase(VK_ANDROID_EXTERNAL_MEMORY_ANDROID_HARDWARE_BUFFER_EXTENSION_NAME);
extensions.external_memory_ahb = false;
}
#endif
#undef FEATURE_EXTENSION
#undef EXTENSION
@@ -1122,6 +1107,11 @@ bool Device::GetSuitability(bool requires_swapchain) {
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PUSH_DESCRIPTOR_PROPERTIES_KHR;
SetNext(next, properties.push_descriptor);
}
if (extensions.depth_stencil_resolve || instance_version >= VK_API_VERSION_1_2) {
properties.depth_stencil_resolve.sType =
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_DEPTH_STENCIL_RESOLVE_PROPERTIES;
SetNext(next, properties.depth_stencil_resolve);
}
if (extensions.descriptor_buffer) {
properties.descriptor_buffer.sType =
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_DESCRIPTOR_BUFFER_PROPERTIES_EXT;
@@ -1142,6 +1132,26 @@ bool Device::GetSuitability(bool requires_swapchain) {
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MAINTENANCE_5_PROPERTIES_KHR;
SetNext(next, properties.maintenance5);
}
if (extensions.custom_border_color) {
properties.custom_border_color.sType =
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_CUSTOM_BORDER_COLOR_PROPERTIES_EXT;
SetNext(next, properties.custom_border_color);
}
if (extensions.external_memory_host) {
properties.external_memory_host.sType =
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_EXTERNAL_MEMORY_HOST_PROPERTIES_EXT;
SetNext(next, properties.external_memory_host);
}
if (extensions.maintenance3 || instance_version >= VK_API_VERSION_1_1) {
properties.maintenance3.sType =
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MAINTENANCE_3_PROPERTIES;
SetNext(next, properties.maintenance3);
}
if (extensions.maintenance4 || features.maintenance4.maintenance4) {
properties.maintenance4.sType =
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MAINTENANCE_4_PROPERTIES;
SetNext(next, properties.maintenance4);
}
// Perform the property fetch.
physical.GetProperties2(properties2);
@@ -1238,9 +1248,7 @@ void Device::RemoveUnsuitableExtensions() {
// VK_EXT_border_color_swizzle
if (extensions.border_color_swizzle) {
extensions.border_color_swizzle =
extensions.custom_border_color &&
features.border_color_swizzle.borderColorSwizzle &&
features.border_color_swizzle.borderColorSwizzleFromImage;
extensions.custom_border_color && features.border_color_swizzle.borderColorSwizzle;
}
RemoveExtensionFeatureIfUnsuitable(extensions.border_color_swizzle,
features.border_color_swizzle,
@@ -1486,11 +1494,26 @@ void Device::SetupFamilies(VkSurfaceKHR surface) {
}
}
std::optional<size_t> Device::GetSamplerHeapBudget() const {
if (sampler_heap_budget == 0) {
return std::nullopt;
bool Device::TryReserveCustomBorderColorSamplers(size_t count) const {
const size_t limit = properties.custom_border_color.maxCustomBorderColorSamplers;
if (limit == 0) {
return true;
}
return sampler_heap_budget;
size_t used = custom_border_color_samplers_used.load(std::memory_order_relaxed);
while (used + count <= limit) {
if (custom_border_color_samplers_used.compare_exchange_weak(
used, used + count, std::memory_order_relaxed, std::memory_order_relaxed)) {
return true;
}
}
return false;
}
void Device::ReleaseCustomBorderColorSamplers(size_t count) const {
if (count == 0) {
return;
}
custom_border_color_samplers_used.fetch_sub(count, std::memory_order_relaxed);
}
u64 Device::GetDeviceMemoryUsage() const {
@@ -1516,12 +1539,27 @@ void Device::CollectPhysicalMemoryInfo() {
device_access_memory = 0;
u64 device_initial_usage = 0;
u64 local_memory = 0;
const auto heap_has_usable_type = [&mem_properties](size_t heap) {
for (u32 index = 0; index < mem_properties.memoryTypeCount; ++index) {
if (mem_properties.memoryTypes[index].heapIndex != heap) {
continue;
}
if ((mem_properties.memoryTypes[index].propertyFlags &
VK_MEMORY_PROPERTY_PROTECTED_BIT) == 0) {
return true;
}
}
return false;
};
for (size_t element = 0; element < num_properties; ++element) {
const bool is_heap_local =
(mem_properties.memoryHeaps[element].flags & VK_MEMORY_HEAP_DEVICE_LOCAL_BIT) != 0;
if (!is_integrated && !is_heap_local) {
continue;
}
if (!heap_has_usable_type(element)) {
continue;
}
valid_heap_memory.push_back(element);
if (is_heap_local) {
local_memory += mem_properties.memoryHeaps[element].size;
@@ -1533,6 +1571,13 @@ void Device::CollectPhysicalMemoryInfo() {
}
device_access_memory += mem_properties.memoryHeaps[element].size;
}
const u64 committed_backing = Common::GetCommittedBackingSize();
if (committed_backing != 0) {
LOG_INFO(Render_Vulkan, "Discounting {} MiB of guest memory committed by the host",
committed_backing >> 20);
local_memory -= (std::min)(local_memory, committed_backing);
device_access_memory -= (std::min)(device_access_memory, committed_backing);
}
if (is_integrated) {
const s64 available_memory = static_cast<s64>(device_access_memory - device_initial_usage);
const u64 memory_size = Settings::values.vram_usage_mode.GetValue() == Settings::VramUsageMode::Aggressive ? 6_GiB : 4_GiB;
+84 -19
View File
@@ -6,6 +6,7 @@
#pragma once
#include <atomic>
#include <optional>
#include <set>
#include <span>
@@ -84,6 +85,7 @@ VK_DEFINE_HANDLE(VmaAllocator)
EXTENSION(EXT, CONDITIONAL_RENDERING, conditional_rendering) \
EXTENSION(EXT, CONSERVATIVE_RASTERIZATION, conservative_rasterization) \
EXTENSION(EXT, DEPTH_RANGE_UNRESTRICTED, depth_range_unrestricted) \
EXTENSION(EXT, EXTERNAL_MEMORY_HOST, external_memory_host) \
EXTENSION(EXT, MEMORY_BUDGET, memory_budget) \
EXTENSION(EXT, ROBUSTNESS_2, robustness_2) \
EXTENSION(EXT, SAMPLER_FILTER_MINMAX, sampler_filter_minmax) \
@@ -91,6 +93,8 @@ VK_DEFINE_HANDLE(VmaAllocator)
EXTENSION(EXT, SHADER_VIEWPORT_INDEX_LAYER, shader_viewport_index_layer) \
EXTENSION(EXT, TOOLING_INFO, tooling_info) \
EXTENSION(EXT, VERTEX_ATTRIBUTE_DIVISOR, vertex_attribute_divisor) \
EXTENSION(KHR, CREATE_RENDERPASS_2, create_renderpass2) \
EXTENSION(KHR, DEPTH_STENCIL_RESOLVE, depth_stencil_resolve) \
EXTENSION(KHR, DRAW_INDIRECT_COUNT, draw_indirect_count) \
EXTENSION(KHR, DRIVER_PROPERTIES, driver_properties) \
EXTENSION(KHR, PUSH_DESCRIPTOR, push_descriptor) \
@@ -113,6 +117,14 @@ VK_DEFINE_HANDLE(VmaAllocator)
EXTENSION(IMG, FILTER_CUBIC, filter_cubic_img) \
EXTENSION(QCOM, FILTER_CUBIC_WEIGHTS, filter_cubic_weights)
#ifdef __ANDROID__
#define FOR_EACH_VK_PLATFORM_EXTENSION(EXTENSION) \
EXTENSION(EXT, QUEUE_FAMILY_FOREIGN, queue_family_foreign) \
EXTENSION(ANDROID, EXTERNAL_MEMORY_ANDROID_HARDWARE_BUFFER, external_memory_ahb)
#else
#define FOR_EACH_VK_PLATFORM_EXTENSION(EXTENSION)
#endif
// Define extensions which must be supported.
#define FOR_EACH_VK_MANDATORY_EXTENSION(EXTENSION_NAME) \
EXTENSION_NAME(VK_EXT_VERTEX_ATTRIBUTE_DIVISOR_EXTENSION_NAME) \
@@ -613,6 +625,32 @@ FN_MAX_LIMIT_LIST
return extensions.shader_stencil_export;
}
/// Returns true if the device supports VK_KHR_create_renderpass2.
bool IsKhrCreateRenderPass2Supported() const {
return extensions.create_renderpass2 || instance_version >= VK_API_VERSION_1_2;
}
/// Returns true if the device supports VK_KHR_depth_stencil_resolve.
bool IsKhrDepthStencilResolveSupported() const {
return (extensions.depth_stencil_resolve || instance_version >= VK_API_VERSION_1_2) &&
IsKhrCreateRenderPass2Supported();
}
/// Returns the supported resolve modes for the depth aspect.
VkResolveModeFlags GetDepthResolveModes() const {
return properties.depth_stencil_resolve.supportedDepthResolveModes;
}
/// Returns the supported resolve modes for the stencil aspect.
VkResolveModeFlags GetStencilResolveModes() const {
return properties.depth_stencil_resolve.supportedStencilResolveModes;
}
/// Returns true if only one of the depth and stencil aspects may be resolved.
bool SupportsIndependentResolveNone() const {
return properties.depth_stencil_resolve.independentResolveNone == VK_TRUE;
}
/// Returns true if depth/stencil operations can be performed efficiently.
/// Either through shader export or hardware blits.
bool CanPerformDepthStencilOperations() const {
@@ -671,20 +709,18 @@ FN_MAX_LIMIT_LIST
return features.transform_feedback.geometryStreams;
}
/// Returns true if the device supports VK_EXT_custom_border_color.
bool IsExtCustomBorderColorSupported() const {
return extensions.custom_border_color;
/// Returns true if custom border colors can be created without a format.
bool IsCustomBorderColorUsable() const {
return extensions.custom_border_color &&
features.custom_border_color.customBorderColors &&
features.custom_border_color.customBorderColorWithoutFormat;
}
/// Returns true if customBorderColors feature is available.
bool IsCustomBorderColorsSupported() const {
return features.custom_border_color.customBorderColors;
}
/// Takes budget for samplers carrying a custom border color, false when exhausted.
bool TryReserveCustomBorderColorSamplers(size_t count) const;
/// Returns true if customBorderColorWithoutFormat feature is available.
bool IsCustomBorderColorWithoutFormatSupported() const {
return features.custom_border_color.customBorderColorWithoutFormat;
}
/// Gives back budget taken by TryReserveCustomBorderColorSamplers.
void ReleaseCustomBorderColorSamplers(size_t count) const;
/// Returns true if the device supports VK_EXT_color_write_enable.
bool IsExtColorWriteEnableSupported() const {
@@ -696,6 +732,12 @@ FN_MAX_LIMIT_LIST
return extensions.border_color_swizzle;
}
/// Returns true if samplers must be carried with border color swizzle mapping.
bool NeedsBorderColorSwizzleMapping() const {
return extensions.border_color_swizzle &&
!features.border_color_swizzle.borderColorSwizzleFromImage;
}
/// Returns true if borderColorSwizzleFromImage is available.
bool IsBorderColorSwizzleFromImageSupported() const {
return features.border_color_swizzle.borderColorSwizzleFromImage;
@@ -844,6 +886,30 @@ FN_MAX_LIMIT_LIST
return extensions.conditional_rendering;
}
bool IsExtExternalMemoryHostSupported() const {
return extensions.external_memory_host;
}
bool IsExtExternalMemoryAhbSupported() const {
#ifdef __ANDROID__
return extensions.external_memory_ahb && extensions.queue_family_foreign;
#else
return false;
#endif
}
u64 GetMinImportedHostPointerAlignment() const {
return properties.external_memory_host.minImportedHostPointerAlignment;
}
u64 GetMaxBufferSize() const {
return properties.maintenance4.maxBufferSize;
}
u64 GetMaxMemoryAllocationSize() const {
return properties.maintenance3.maxMemoryAllocationSize;
}
bool IsExtAstcDecodeModeSupported() const {
return extensions.astc_decode_mode;
}
@@ -891,8 +957,6 @@ FN_MAX_LIMIT_LIST
return has_broken_parallel_compiling;
}
std::optional<size_t> GetSamplerHeapBudget() const;
/// Returns the vendor name reported from Vulkan.
std::string_view GetVendorName() const {
return properties.driver.driverName;
@@ -921,10 +985,6 @@ FN_MAX_LIMIT_LIST
return supports_d24_depth;
}
bool CantBlitMSAA() const {
return cant_blit_msaa;
}
bool MustEmulateScaledFormats() const {
return must_emulate_scaled_formats;
}
@@ -1116,6 +1176,7 @@ private:
FOR_EACH_VK_FEATURE_1_4(FEATURE);
FOR_EACH_VK_FEATURE_EXT(FEATURE);
FOR_EACH_VK_EXTENSION(EXTENSION);
FOR_EACH_VK_PLATFORM_EXTENSION(EXTENSION);
#undef EXTENSION
#undef FEATURE
@@ -1147,7 +1208,12 @@ private:
VkPhysicalDeviceDescriptorBufferPropertiesEXT descriptor_buffer{};
VkPhysicalDeviceSubgroupSizeControlProperties subgroup_size_control{};
VkPhysicalDeviceTransformFeedbackPropertiesEXT transform_feedback{};
VkPhysicalDeviceMaintenance3Properties maintenance3{};
VkPhysicalDeviceMaintenance4Properties maintenance4{};
VkPhysicalDeviceMaintenance5PropertiesKHR maintenance5{};
VkPhysicalDeviceDepthStencilResolveProperties depth_stencil_resolve{};
VkPhysicalDeviceCustomBorderColorPropertiesEXT custom_border_color{};
VkPhysicalDeviceExternalMemoryHostPropertiesEXT external_memory_host{};
VkPhysicalDeviceProperties properties{};
};
@@ -1175,7 +1241,6 @@ private:
bool has_nsight_graphics{}; ///< Has Nsight Graphics attached
bool has_radeon_gpu_profiler{}; ///< Has Radeon GPU Profiler attached.
bool supports_d24_depth{}; ///< Supports D24 depth buffers.
bool cant_blit_msaa{}; ///< Does not support MSAA<->MSAA blitting.
bool must_emulate_scaled_formats{}; ///< Requires scaled vertex format emulation
bool dynamic_state3_blending{}; ///< Has blending features of dynamic_state3.
bool dynamic_state3_enables{}; ///< Has at least one enable feature of dynamic_state3.
@@ -1187,7 +1252,7 @@ private:
bool dynamic_state3_alpha_to_coverage{};
bool dynamic_state3_alpha_to_one{};
bool supports_conditional_barriers{}; ///< Allows barriers in conditional control flow.
size_t sampler_heap_budget{}; ///< Sampler budget for buggy drivers (0 = unlimited).
mutable std::atomic<size_t> custom_border_color_samplers_used{};
u64 device_access_memory{}; ///< Total size of device local memory in bytes.
u32 sets_per_pool{}; ///< Sets per Description Pool
NvidiaArchitecture nvidia_arch{NvidiaArchitecture::Arch_AmpereOrNewer};
@@ -25,9 +25,34 @@
#include "video_core/gpu_logging/gpu_logging.h"
#include "common/settings.h"
#ifdef __ANDROID__
#include <android/hardware_buffer.h>
#endif
namespace Vulkan {
namespace {
[[nodiscard]] std::optional<u32> FindImportMemoryType(
const VkPhysicalDeviceMemoryProperties &props, u32 type_mask) {
const auto find = [&](VkMemoryPropertyFlags wanted) -> std::optional<u32> {
for (u32 i = 0; i < props.memoryTypeCount; ++i) {
if (((type_mask >> i) & 1u) != 0 &&
(props.memoryTypes[i].propertyFlags & wanted) == wanted) {
return i;
}
}
return std::nullopt;
};
auto type_index = find(VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT |
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT |
VK_MEMORY_PROPERTY_HOST_CACHED_BIT);
if (!type_index) {
type_index = find(VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT |
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT);
}
return type_index;
}
// Helpers translating MemoryUsage to flags/usage
[[maybe_unused]] VkMemoryPropertyFlags MemoryUsagePropertyFlags(MemoryUsage usage) {
@@ -200,6 +225,300 @@ namespace Vulkan {
size = 0;
}
HostMemoryImport::HostMemoryImport(const Device &device_, void *base, size_t size,
std::span<AHardwareBuffer *const> hardware_buffers,
size_t hardware_buffer_window, size_t hardware_buffer_base)
: device{device_} {
if (ImportHardwareBuffers(hardware_buffers, hardware_buffer_window, hardware_buffer_base,
size)) {
return;
}
if (device.IsTiler()) {
return;
}
if (!hardware_buffers.empty()) {
return;
}
if (ImportHostPointer(base, size)) {
return;
}
LOG_INFO(Render_Vulkan, "Unified memory disabled, no host memory import path");
}
bool HostMemoryImport::ImportHostPointer(void *base, size_t size) {
if (!device.IsExtExternalMemoryHostSupported()) {
return false;
}
const u64 alignment = device.GetMinImportedHostPointerAlignment();
if (alignment == 0 || !Common::IsAligned(reinterpret_cast<uintptr_t>(base), alignment) ||
!Common::IsAligned(size, alignment)) {
return false;
}
using namespace Common::Literals;
constexpr VkDeviceSize DesktopWindowSize = 4_GiB;
VkDeviceSize candidate_window = DesktopWindowSize;
const u64 max_buffer_size = device.GetMaxBufferSize();
if (max_buffer_size != 0 && max_buffer_size < candidate_window) {
candidate_window = max_buffer_size;
}
const u64 max_allocation_size = device.GetMaxMemoryAllocationSize();
if (max_allocation_size != 0 && max_allocation_size < candidate_window) {
candidate_window = max_allocation_size;
}
candidate_window = Common::AlignDown(candidate_window, alignment);
if (candidate_window == 0) {
return false;
}
window_size = candidate_window;
const auto &logical = device.GetLogical();
const auto memory_props = device.GetPhysical().GetMemoryProperties().memoryProperties;
for (size_t offset = 0; offset < size; offset += window_size) {
u8 *const window_base = static_cast<u8 *>(base) + offset;
const VkDeviceSize window_len =
(std::min)(static_cast<VkDeviceSize>(size - offset), window_size);
VkMemoryHostPointerPropertiesEXT host_props{
.sType = VK_STRUCTURE_TYPE_MEMORY_HOST_POINTER_PROPERTIES_EXT,
.pNext = nullptr,
.memoryTypeBits = 0,
};
if (logical.GetMemoryHostPointerPropertiesEXT(
VK_EXTERNAL_MEMORY_HANDLE_TYPE_HOST_ALLOCATION_BIT_EXT, window_base,
&host_props) != VK_SUCCESS ||
host_props.memoryTypeBits == 0) {
break;
}
const VkExternalMemoryBufferCreateInfo external_info{
.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_BUFFER_CREATE_INFO,
.pNext = nullptr,
.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_HOST_ALLOCATION_BIT_EXT,
};
const VkBufferCreateInfo buffer_ci{
.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO,
.pNext = &external_info,
.flags = 0,
.size = window_len,
.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT,
.sharingMode = VK_SHARING_MODE_EXCLUSIVE,
.queueFamilyIndexCount = 0,
.pQueueFamilyIndices = nullptr,
};
VkBuffer new_buffer{};
if (logical.CreateBufferRaw(buffer_ci, &new_buffer) != VK_SUCCESS) {
break;
}
const VkMemoryRequirements requirements =
logical.GetBufferMemoryRequirements(new_buffer);
const u32 type_mask = requirements.memoryTypeBits & host_props.memoryTypeBits;
if (type_mask == 0 || requirements.size > window_len) {
logical.DestroyBufferRaw(new_buffer);
break;
}
const auto type_index = FindImportMemoryType(memory_props, type_mask);
if (!type_index) {
logical.DestroyBufferRaw(new_buffer);
break;
}
const u32 heap_index = memory_props.memoryTypes[*type_index].heapIndex;
const VkDeviceSize heap_size = memory_props.memoryHeaps[heap_index].size;
if (imported_size + window_len > heap_size / 2) {
logical.DestroyBufferRaw(new_buffer);
break;
}
const VkImportMemoryHostPointerInfoEXT import_info{
.sType = VK_STRUCTURE_TYPE_IMPORT_MEMORY_HOST_POINTER_INFO_EXT,
.pNext = nullptr,
.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_HOST_ALLOCATION_BIT_EXT,
.pHostPointer = window_base,
};
const VkMemoryAllocateInfo alloc_info{
.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO,
.pNext = &import_info,
.allocationSize = window_len,
.memoryTypeIndex = *type_index,
};
vk::DeviceMemory memory = logical.TryAllocateMemory(alloc_info);
if (!memory) {
logical.DestroyBufferRaw(new_buffer);
break;
}
if (logical.BindBufferMemory(new_buffer, *memory, 0) != VK_SUCCESS) {
logical.DestroyBufferRaw(new_buffer);
break;
}
windows.push_back(Window{
.memory = std::move(memory),
.buffer = new_buffer,
});
imported_size += static_cast<size_t>(window_len);
}
if (windows.empty()) {
return false;
}
return true;
}
bool HostMemoryImport::ImportHardwareBuffers(
[[maybe_unused]] std::span<AHardwareBuffer *const> hardware_buffers,
[[maybe_unused]] size_t hardware_buffer_window,
[[maybe_unused]] size_t hardware_buffer_base, [[maybe_unused]] size_t size) {
#ifdef __ANDROID__
if (hardware_buffers.empty() || hardware_buffer_window == 0 ||
!device.IsExtExternalMemoryAhbSupported()) {
return false;
}
const u64 max_allocation_size = device.GetMaxMemoryAllocationSize();
if (max_allocation_size != 0 && hardware_buffer_window > max_allocation_size) {
return false;
}
if (hardware_buffer_base >= size) {
return false;
}
const auto &logical = device.GetLogical();
const auto memory_props = device.GetPhysical().GetMemoryProperties().memoryProperties;
window_size = hardware_buffer_window;
base_offset = hardware_buffer_base;
const auto import_all = [&](VkBufferUsageFlags usage, bool want_address) {
for (size_t i = 0; i < hardware_buffers.size(); ++i) {
const size_t offset = hardware_buffer_base + i * hardware_buffer_window;
if (offset >= size) {
break;
}
const VkDeviceSize window_len = (std::min)(
static_cast<VkDeviceSize>(size - offset),
static_cast<VkDeviceSize>(hardware_buffer_window));
VkAndroidHardwareBufferPropertiesANDROID ahb_props{
.sType = VK_STRUCTURE_TYPE_ANDROID_HARDWARE_BUFFER_PROPERTIES_ANDROID,
.pNext = nullptr,
.allocationSize = 0,
.memoryTypeBits = 0,
};
if (logical.GetAndroidHardwareBufferPropertiesANDROID(hardware_buffers[i],
&ahb_props) != VK_SUCCESS ||
ahb_props.memoryTypeBits == 0 || ahb_props.allocationSize < window_len) {
break;
}
const VkExternalMemoryBufferCreateInfo external_info{
.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_BUFFER_CREATE_INFO,
.pNext = nullptr,
.handleTypes =
VK_EXTERNAL_MEMORY_HANDLE_TYPE_ANDROID_HARDWARE_BUFFER_BIT_ANDROID,
};
const VkBufferCreateInfo buffer_ci{
.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO,
.pNext = &external_info,
.flags = 0,
.size = window_len,
.usage = usage,
.sharingMode = VK_SHARING_MODE_EXCLUSIVE,
.queueFamilyIndexCount = 0,
.pQueueFamilyIndices = nullptr,
};
VkBuffer new_buffer{};
if (logical.CreateBufferRaw(buffer_ci, &new_buffer) != VK_SUCCESS) {
break;
}
const VkMemoryRequirements requirements =
logical.GetBufferMemoryRequirements(new_buffer);
const u32 type_mask = requirements.memoryTypeBits & ahb_props.memoryTypeBits;
if (type_mask == 0 || requirements.size > ahb_props.allocationSize) {
logical.DestroyBufferRaw(new_buffer);
break;
}
const auto type_index = FindImportMemoryType(memory_props, type_mask);
if (!type_index) {
logical.DestroyBufferRaw(new_buffer);
break;
}
const VkImportAndroidHardwareBufferInfoANDROID import_info{
.sType = VK_STRUCTURE_TYPE_IMPORT_ANDROID_HARDWARE_BUFFER_INFO_ANDROID,
.pNext = nullptr,
.buffer = hardware_buffers[i],
};
const VkMemoryDedicatedAllocateInfo dedicated_info{
.sType = VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO,
.pNext = &import_info,
.image = VK_NULL_HANDLE,
.buffer = new_buffer,
};
const VkMemoryAllocateFlagsInfo flags_info{
.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_FLAGS_INFO,
.pNext = &dedicated_info,
.flags = VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT,
.deviceMask = 0,
};
const void *alloc_next = &dedicated_info;
if (want_address) {
alloc_next = &flags_info;
}
const VkMemoryAllocateInfo alloc_info{
.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO,
.pNext = alloc_next,
.allocationSize = ahb_props.allocationSize,
.memoryTypeIndex = *type_index,
};
vk::DeviceMemory memory = logical.TryAllocateMemory(alloc_info);
if (!memory) {
logical.DestroyBufferRaw(new_buffer);
break;
}
if (logical.BindBufferMemory(new_buffer, *memory, 0) != VK_SUCCESS) {
logical.DestroyBufferRaw(new_buffer);
break;
}
VkDeviceAddress address = 0;
if (want_address) {
address = logical.GetBufferDeviceAddress(new_buffer);
}
windows.push_back(Window{
.memory = std::move(memory),
.buffer = new_buffer,
.address = address,
});
imported_size += static_cast<size_t>(window_len);
}
return !windows.empty();
};
constexpr VkBufferUsageFlags TransferUsage =
VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;
VkBufferUsageFlags shader_usage = TransferUsage |
VK_BUFFER_USAGE_UNIFORM_TEXEL_BUFFER_BIT |
VK_BUFFER_USAGE_STORAGE_TEXEL_BUFFER_BIT | VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT |
VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_INDEX_BUFFER_BIT |
VK_BUFFER_USAGE_VERTEX_BUFFER_BIT | VK_BUFFER_USAGE_INDIRECT_BUFFER_BIT;
const bool want_address = device.IsBufferDeviceAddressSupported();
if (want_address) {
shader_usage |= VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT;
}
bindable = import_all(shader_usage, want_address);
if (!bindable) {
imported_size = 0;
import_all(TransferUsage, false);
}
if (windows.empty()) {
window_size = 0;
base_offset = 0;
return false;
}
foreign_ownership = true;
return true;
#else
return false;
#endif
}
HostMemoryImport::~HostMemoryImport() {
for (Window &window : windows) {
if (window.buffer != VK_NULL_HANDLE) {
device.GetLogical().DestroyBufferRaw(window.buffer);
}
}
}
MemoryAllocator::MemoryAllocator(const Device &device_)
: device{device_}, allocator{device.GetAllocator()},
properties{device_.GetPhysical().GetMemoryProperties().memoryProperties},
@@ -332,6 +651,17 @@ namespace Vulkan {
return MemoryCommit(allocator, a, info);
}
HostMemoryImport *MemoryAllocator::CreateHostMemoryImport(
void *base, size_t size, std::span<AHardwareBuffer *const> hardware_buffers,
size_t hardware_buffer_window, size_t hardware_buffer_base) {
unified_memory = std::make_unique<HostMemoryImport>(
device, base, size, hardware_buffers, hardware_buffer_window, hardware_buffer_base);
if (!unified_memory->IsValid()) {
unified_memory.reset();
}
return unified_memory.get();
}
MemoryCommit MemoryAllocator::Commit(const vk::Buffer &buffer, MemoryUsage usage) {
// Allocate memory appropriate for this buffer automatically
const auto vma_usage = MemoryUsageVma(usage);
@@ -15,6 +15,8 @@
#include "video_core/vulkan_common/vulkan_wrapper.h"
#include "video_core/vulkan_common/vma.h"
struct AHardwareBuffer;
namespace Vulkan {
class Device;
@@ -84,6 +86,76 @@ namespace Vulkan {
void *mapped_ptr{}; ///< Optional persistent mapped pointer
};
class HostMemoryImport {
public:
explicit HostMemoryImport(const Device &device_, void *base, size_t size,
std::span<AHardwareBuffer *const> hardware_buffers,
size_t hardware_buffer_window, size_t hardware_buffer_base);
~HostMemoryImport();
HostMemoryImport(const HostMemoryImport &) = delete;
HostMemoryImport &operator=(const HostMemoryImport &) = delete;
[[nodiscard]] bool IsValid() const noexcept {
return !windows.empty();
}
[[nodiscard]] size_t GetSize() const noexcept {
return imported_size;
}
[[nodiscard]] size_t GetBaseOffset() const noexcept {
return base_offset;
}
[[nodiscard]] bool NeedsForeignOwnershipTransfer() const noexcept {
return foreign_ownership;
}
[[nodiscard]] VkDeviceSize GetWindowSize() const noexcept {
return window_size;
}
[[nodiscard]] VkBuffer GetWindowBuffer(size_t index) const noexcept {
return windows[index].buffer;
}
[[nodiscard]] VkDeviceAddress GetWindowAddress(size_t index) const noexcept {
return windows[index].address;
}
[[nodiscard]] size_t GetWindowCount() const noexcept {
return windows.size();
}
[[nodiscard]] bool IsBindable() const noexcept {
return bindable;
}
private:
struct Window {
vk::DeviceMemory memory;
VkBuffer buffer{};
VkDeviceAddress address{};
};
bool ImportHostPointer(void *base, size_t size);
bool ImportHardwareBuffers(std::span<AHardwareBuffer *const> hardware_buffers,
size_t hardware_buffer_window, size_t hardware_buffer_base,
size_t size);
const Device &device;
std::vector<Window> windows;
VkDeviceSize window_size{};
size_t imported_size{};
size_t base_offset{};
bool foreign_ownership{};
bool bindable{};
};
/// Memory allocator container.
/// Allocates and releases memory allocations on demand.
class MemoryAllocator {
@@ -120,6 +192,15 @@ namespace Vulkan {
/// Commits memory required by the buffer and binds it (for buffers created outside VMA).
MemoryCommit Commit(const vk::Buffer &buffer, MemoryUsage usage);
HostMemoryImport *CreateHostMemoryImport(void *base, size_t size,
std::span<AHardwareBuffer *const> hardware_buffers,
size_t hardware_buffer_window,
size_t hardware_buffer_base);
[[nodiscard]] HostMemoryImport *GetHostMemoryImport() const noexcept {
return unified_memory.get();
}
private:
static bool IsAutoUsage(VmaMemoryUsage u) noexcept {
switch (u) {
@@ -137,6 +218,7 @@ namespace Vulkan {
const VkPhysicalDeviceMemoryProperties properties; ///< Physical device memory properties.
VkDeviceSize buffer_image_granularity; ///< Adjacent buffer/image granularity
u32 valid_memory_types{~0u};
std::unique_ptr<HostMemoryImport> unified_memory;
};
} // namespace Vulkan
@@ -184,6 +184,7 @@ void Load(VkDevice device, DeviceDispatch& dld) noexcept {
X(vkCreatePipelineLayout);
X(vkCreateQueryPool);
X(vkCreateRenderPass);
X(vkCreateRenderPass2);
X(vkCreateSampler);
X(vkCreateSemaphore);
X(vkCreateShaderModule);
@@ -216,12 +217,16 @@ void Load(VkDevice device, DeviceDispatch& dld) noexcept {
X(vkGetBufferMemoryRequirements2);
X(vkGetDeviceQueue);
X(vkGetEventStatus);
X(vkGetMemoryHostPointerPropertiesEXT);
X(vkGetFenceStatus);
X(vkGetImageMemoryRequirements);
X(vkGetPipelineCacheData);
X(vkGetMemoryFdKHR);
#ifdef _WIN32
X(vkGetMemoryWin32HandleKHR);
#endif
#ifdef __ANDROID__
X(vkGetAndroidHardwareBufferPropertiesANDROID);
#endif
X(vkGetQueryPoolResults);
X(vkGetPipelineExecutablePropertiesKHR);
@@ -270,6 +275,10 @@ void Load(VkDevice device, DeviceDispatch& dld) noexcept {
if (!dld.vkQueueSubmit2) {
Proc(dld.vkQueueSubmit2, dld, "vkQueueSubmit2KHR", device);
}
if (!dld.vkCreateRenderPass2) {
Proc(dld.vkCreateRenderPass2, dld, "vkCreateRenderPass2KHR", device);
}
#undef X
}
@@ -725,6 +734,12 @@ RenderPass Device::CreateRenderPass(const VkRenderPassCreateInfo& ci) const {
return RenderPass(object, handle, *dld);
}
RenderPass Device::CreateRenderPass2(const VkRenderPassCreateInfo2& ci) const {
VkRenderPass object;
Check(dld->vkCreateRenderPass2(handle, &ci, nullptr, &object));
return RenderPass(object, handle, *dld);
}
DescriptorSetLayout Device::CreateDescriptorSetLayout(
const VkDescriptorSetLayoutCreateInfo& ci) const {
VkDescriptorSetLayout object;
@@ -300,6 +300,7 @@ struct DeviceDispatch : InstanceDispatch {
PFN_vkCreatePipelineLayout vkCreatePipelineLayout{};
PFN_vkCreateQueryPool vkCreateQueryPool{};
PFN_vkCreateRenderPass vkCreateRenderPass{};
PFN_vkCreateRenderPass2 vkCreateRenderPass2{};
PFN_vkCreateSampler vkCreateSampler{};
PFN_vkCreateSemaphore vkCreateSemaphore{};
PFN_vkCreateShaderModule vkCreateShaderModule{};
@@ -332,12 +333,16 @@ struct DeviceDispatch : InstanceDispatch {
PFN_vkGetBufferMemoryRequirements2 vkGetBufferMemoryRequirements2{};
PFN_vkGetDeviceQueue vkGetDeviceQueue{};
PFN_vkGetEventStatus vkGetEventStatus{};
PFN_vkGetMemoryHostPointerPropertiesEXT vkGetMemoryHostPointerPropertiesEXT{};
PFN_vkGetFenceStatus vkGetFenceStatus{};
PFN_vkGetImageMemoryRequirements vkGetImageMemoryRequirements{};
PFN_vkGetPipelineCacheData vkGetPipelineCacheData{};
PFN_vkGetMemoryFdKHR vkGetMemoryFdKHR{};
#ifdef _WIN32
PFN_vkGetMemoryWin32HandleKHR vkGetMemoryWin32HandleKHR{};
#endif
#ifdef __ANDROID__
PFN_vkGetAndroidHardwareBufferPropertiesANDROID vkGetAndroidHardwareBufferPropertiesANDROID{};
#endif
PFN_vkGetPipelineExecutablePropertiesKHR vkGetPipelineExecutablePropertiesKHR{};
PFN_vkGetPipelineExecutableStatisticsKHR vkGetPipelineExecutableStatisticsKHR{};
@@ -1045,6 +1050,8 @@ public:
[[nodiscard]] RenderPass CreateRenderPass(const VkRenderPassCreateInfo& ci) const;
[[nodiscard]] RenderPass CreateRenderPass2(const VkRenderPassCreateInfo2& ci) const;
[[nodiscard]] DescriptorSetLayout CreateDescriptorSetLayout(
const VkDescriptorSetLayoutCreateInfo& ci) const;
@@ -1082,6 +1089,34 @@ public:
VkMemoryRequirements GetBufferMemoryRequirements(VkBuffer buffer,
void* pnext = nullptr) const noexcept;
VkResult GetMemoryHostPointerPropertiesEXT(
VkExternalMemoryHandleTypeFlagBits handle_type, const void* host_pointer,
VkMemoryHostPointerPropertiesEXT* out_properties) const noexcept {
return dld->vkGetMemoryHostPointerPropertiesEXT(handle, handle_type, host_pointer,
out_properties);
}
#ifdef __ANDROID__
VkResult GetAndroidHardwareBufferPropertiesANDROID(
const struct AHardwareBuffer* buffer,
VkAndroidHardwareBufferPropertiesANDROID* out_properties) const noexcept {
return dld->vkGetAndroidHardwareBufferPropertiesANDROID(handle, buffer, out_properties);
}
#endif
VkResult CreateBufferRaw(const VkBufferCreateInfo& ci, VkBuffer* out_buffer) const noexcept {
return dld->vkCreateBuffer(handle, &ci, nullptr, out_buffer);
}
void DestroyBufferRaw(VkBuffer buffer) const noexcept {
dld->vkDestroyBuffer(handle, buffer, nullptr);
}
VkResult BindBufferMemory(VkBuffer buffer, VkDeviceMemory memory,
VkDeviceSize offset) const noexcept {
return dld->vkBindBufferMemory(handle, buffer, memory, offset);
}
VkMemoryRequirements GetImageMemoryRequirements(VkImage image) const noexcept;
std::vector<VkPipelineExecutablePropertiesKHR> GetPipelineExecutablePropertiesKHR(