Compare commits

..

12 Commits

Author SHA1 Message Date
lizzie eded728f91 got it ok 2026-08-05 17:36:17 +00:00
lizzie c8799e42c6 fix2 2026-08-05 17:19:42 +00:00
lizzie d87ffba82a A 2026-08-05 16:53:47 +00:00
lizzie 09c6b57c99 [audio_core] Fix jamboree crash due to OOB access
Signed-off-by: lizzie <lizzie@eden-emu.dev>
2026-08-05 16:52:40 +00:00
CamilleLaVey 49a0ca6d5d [vulkan] Initial implementation on Bindless Buffer/Descriptors (#4251)
Adds a VK_EXT_descriptor_buffer, VK_KHR_device_buffer_address path alongside push descriptors and descriptor sets. The ring is chunked and sized from the device limits, so drivers with a small sampler range (QCOM reports 128 KiB) get more, chunks per frame instead of losing the feature. A pipeline uses one path or the other, never both, since some drivers do not support bufferless push descriptors. The chunk binding is only re-emitted when it changes, and a draw that repeats the previous payload reuses its allocation instead of rewriting it. This whole change will help to reduce the constant use of descriptors on the slower path, improves the "slow implementation" on QCOM drivers (A7xx and older) for push descriptors, improves latency when compiling pipelines. The implementation on indexing descriptors was also refined.

_Special Thanks_

1.- Meowly The Bindless Smol (@Gidoly)

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4251
2026-08-05 16:58:43 +02:00
CamilleLaVey 5e1d5e82dc [vulkan] Re-adjusting ASTC resolve (#4250)
This is an adjustment made due to problems caused by my ASTC changes some pr's earlier; added sanity to decoded blocks, removes the inconditional storage view for ASTC formats (verified against MaxwellToVk logic), added support to VK_EXT_astc_decode_mode which will ensure a better way to resolve RGFA16/RGFA32 on tilers, simplified ASTC HDR decoding into LDR valid formats.

This fixes some issues with SteamDeck, Fire Emblem: Engage washed colors and other problems related to ASTC.

_Special Thanks_

1.- Meowly The Smol (@Gidoly)
2.- @simply0001 for the actual idea on the redundant storage views on the transcoded ASTC path.

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4250
2026-08-05 09:24:35 +02:00
lizzie ba9130fbf9 Revert "[externals] remove SPIRV-Headers and SPIRV-Tools (#3989)" (#4247)
This reverts commit ee197e6222.

- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4247
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-04 02:03:18 +02:00
lizzie 7d5f390ffb [android] fix crash when loader for invalid file is found (#4003)
Signed-off-by: lizzie <lizzie@eden-emu.dev>
Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4003
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-04 01:52:32 +02:00
lizzie 8fe1e6efa2 [core/file_sys] implement IApplicationFunctions::GetPseudoDeviceId; common PatchManager::GetNACP for base|updates (#4159)
a bit of refactoring so there's less code duplication...
probably a good idea to make it a lambda anyways
maybe this is why QLauncher didn't have proper DLCs and updates?

either way "GetPseudoDeviceId" is now implemented (still stubbed) with a proper hash instead of just 0

Signed-off-by: lizzie <lizzie@eden-emu.dev>

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4159
Reviewed-by: Maufeat <sahyno1996@gmail.com>
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-04 01:50:54 +02:00
lizzie ee197e6222 [externals] remove SPIRV-Headers and SPIRV-Tools (#3989)
only dep that uses it is sirit, but sirit already has it's own bundled SPIRV-Headers... so

Signed-off-by: lizzie <lizzie@eden-emu.dev>

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/3989
Reviewed-by: Maufeat <sahyno1996@gmail.com>
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-03 00:00:55 +02:00
Maufeat 612409c7ba [hid] Add Quaternion to ReloadInput (#4240)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

Adds Quaternion to ReloadInput. What does it fix? Displays correct space in VR (only test on SSBU) not tested any further.

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4240
Reviewed-by: Lizzie <lizzie@eden-emu.dev>
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-07-31 19:45:38 +02:00
simply0001 54046ac60e [video_core/macro] check HLE hashes before compiling (#4236)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

Known HLE macros are identified by a hash, but MacroEngine compiled them first and and afterwards it threw the compiled program away when the hash matched. This fix makes it so it checks the hash first and caches the HLE implementation directly, so it only compiles when the hash is unknown or if HLE is disabled.

Cached macros were also constantly checking the hash again and walking through each `std::get_if` until their variant matched. So I dispatched them through `std::visit` instead, and keep one resolved code span for hashing, compiling, and dumping so mid-method uploads use the right range.

Continues the macro hot path work from [#4067](https://git.eden-emu.dev/eden-emu/eden/pulls/4067)

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4236
Reviewed-by: Shinmegumi <shinmegumi@eden-emu.dev>
Reviewed-by: Lizzie <lizzie@eden-emu.dev>
2026-07-30 06:25:43 +02:00
74 changed files with 1877 additions and 1747 deletions
+1 -1
View File
@@ -65,7 +65,7 @@ android {
defaultConfig {
applicationId = "dev.eden.eden_emulator"
minSdk = 33
minSdk = 24
targetSdk = 36
versionName = getGitVersion()
versionCode = autoVersion
@@ -594,7 +594,7 @@ abstract class SettingsItem(
IntSetting.ANDROID_PIPELINE_WORKERS,
titleId = R.string.pipeline_worker_cores,
descriptionId = R.string.pipeline_worker_cores_description,
min = 2,
min = 4,
max = 8,
units = "cores"
)
@@ -1,4 +1,4 @@
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: 2024 yuzu Emulator Project
@@ -169,7 +169,7 @@ class InputDialogFragment : DialogFragment() {
NativeInput.onGamePadButtonEvent(
controllerData.getGUID(),
controllerData.getPort(),
event.keyCode,
InputHandler.getButtonIdFromEvent(event),
action
)
onInputReceived(event.device)
@@ -49,6 +49,12 @@ object InputHandler {
MotionEvent.AXIS_RTRIGGER
)
// Currently, Android doesn't support Joy-Con D-pad buttons. We fall back to the scan code
private const val LINUX_BUTTON_DPAD_UP = 0x220
private const val LINUX_BUTTON_DPAD_DOWN = 0x221
private const val LINUX_BUTTON_DPAD_LEFT = 0x222
private const val LINUX_BUTTON_DPAD_RIGHT = 0x223
fun isPhysicalGameController(device: InputDevice?): Boolean {
device ?: return false
@@ -87,12 +93,25 @@ object InputHandler {
NativeInput.onGamePadButtonEvent(
controllerData.getGUID(),
controllerData.getPort(),
event.keyCode,
getButtonIdFromEvent(event),
action
)
return true
}
fun getButtonIdFromEvent(event: KeyEvent): Int {
if (event.keyCode == 0) {
return when (event.scanCode) {
LINUX_BUTTON_DPAD_UP -> KeyEvent.KEYCODE_DPAD_UP
LINUX_BUTTON_DPAD_DOWN -> KeyEvent.KEYCODE_DPAD_DOWN
LINUX_BUTTON_DPAD_LEFT -> KeyEvent.KEYCODE_DPAD_LEFT
LINUX_BUTTON_DPAD_RIGHT -> KeyEvent.KEYCODE_DPAD_RIGHT
else -> return 0
}
}
return event.keyCode
}
fun dispatchGenericMotionEvent(event: MotionEvent): Boolean {
val controllerData =
androidControllers[event.device.controllerNumber] ?: return false
+61 -97
View File
@@ -20,132 +20,96 @@ struct RomMetadata {
std::vector<u8> icon;
bool isHomebrew;
};
static ankerl::unordered_dense::map<std::string, RomMetadata> m_rom_metadata_cache;
ankerl::unordered_dense::map<std::string, RomMetadata> m_rom_metadata_cache;
static RomMetadata CacheRomMetadata(const std::string& path) {
auto& instance = EmulationSession::GetInstance();
const auto file = Core::GetGameFileFromPath(instance.System().GetFilesystem(), path);
if (auto loader = Loader::GetLoader(instance.System(), file, 0, 0); loader) {
RomMetadata entry;
loader->ReadTitle(entry.title);
loader->ReadProgramId(entry.programId);
loader->ReadIcon(entry.icon);
RomMetadata CacheRomMetadata(const std::string& path) {
const auto file =
Core::GetGameFileFromPath(EmulationSession::GetInstance().System().GetFilesystem(), path);
auto loader = Loader::GetLoader(EmulationSession::GetInstance().System(), file, 0, 0);
const FileSys::PatchManager pm{
entry.programId,
instance.System().GetFileSystemController(),
instance.System().GetContentProvider()
};
const auto control = pm.GetControlMetadata();
RomMetadata entry;
loader->ReadTitle(entry.title);
loader->ReadProgramId(entry.programId);
loader->ReadIcon(entry.icon);
const FileSys::PatchManager pm{
entry.programId, EmulationSession::GetInstance().System().GetFileSystemController(),
EmulationSession::GetInstance().System().GetContentProvider()};
const auto control = pm.GetControlMetadata();
if (control.first != nullptr) {
entry.developer = control.first->GetDeveloperName();
entry.version = control.first->GetVersionString();
} else {
FileSys::NACP nacp;
if (loader->ReadControlData(nacp) == Loader::ResultStatus::Success) {
entry.developer = nacp.GetDeveloperName();
if (control.first != nullptr) {
entry.developer = control.first->GetDeveloperName();
entry.version = control.first->GetVersionString();
} else {
entry.developer = "";
FileSys::NACP nacp{};
entry.developer = loader->ReadControlData(nacp) == Loader::ResultStatus::Success
? nacp.GetDeveloperName()
: "";
entry.version = "1.0.0";
}
entry.version = "1.0.0";
if (loader->GetFileType() == Loader::FileType::NRO) {
auto loader_nro = reinterpret_cast<Loader::AppLoader_NRO*>(loader.get());
entry.isHomebrew = loader_nro->IsHomebrew();
} else {
entry.isHomebrew = false;
}
m_rom_metadata_cache[path] = entry;
return entry;
}
if (loader->GetFileType() == Loader::FileType::NRO) {
auto loader_nro = reinterpret_cast<Loader::AppLoader_NRO*>(loader.get());
entry.isHomebrew = loader_nro->IsHomebrew();
} else {
entry.isHomebrew = false;
}
m_rom_metadata_cache[path] = entry;
return entry;
return {};
}
RomMetadata GetRomMetadata(const std::string& path, bool reload = false) {
if (reload) {
static RomMetadata GetRomMetadata(const std::string& path, bool reload = false) {
if (reload)
return CacheRomMetadata(path);
}
if (auto search = m_rom_metadata_cache.find(path); search != m_rom_metadata_cache.end()) {
return search->second;
}
if (auto it = m_rom_metadata_cache.find(path); it != m_rom_metadata_cache.end())
return it->second;
return CacheRomMetadata(path);
}
extern "C" {
jboolean Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getIsValid(JNIEnv* env, jobject obj,
jstring jpath) {
const auto file = EmulationSession::GetInstance().System().GetFilesystem()->OpenFile(
Common::Android::GetJString(env, jpath), FileSys::OpenMode::Read);
if (!file) {
return false;
jboolean Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getIsValid(JNIEnv* env, jobject obj, jstring jpath) {
if (auto const file = EmulationSession::GetInstance().System().GetFilesystem()->OpenFile(Common::Android::GetJString(env, jpath), FileSys::OpenMode::Read); file) {
if (auto loader = Loader::GetLoader(EmulationSession::GetInstance().System(), file); loader) {
auto const file_type = loader->GetFileType();
if (file_type == Loader::FileType::Unknown || file_type == Loader::FileType::Error)
return false;
if ((file_type == Loader::FileType::NSP || file_type == Loader::FileType::XCI) && !Loader::IsBootableGameContainer(file, file_type))
return false;
u64 program_id = 0;
return loader->ReadProgramId(program_id) == Loader::ResultStatus::Success;
}
}
auto loader = Loader::GetLoader(EmulationSession::GetInstance().System(), file);
if (!loader) {
return false;
}
const auto file_type = loader->GetFileType();
if (file_type == Loader::FileType::Unknown || file_type == Loader::FileType::Error) {
return false;
}
if ((file_type == Loader::FileType::NSP || file_type == Loader::FileType::XCI) &&
!Loader::IsBootableGameContainer(file, file_type)) {
return false;
}
u64 program_id = 0;
Loader::ResultStatus res = loader->ReadProgramId(program_id);
if (res != Loader::ResultStatus::Success) {
return false;
}
return true;
return false;
}
jstring Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getTitle(JNIEnv* env, jobject obj,
jstring jpath) {
return Common::Android::ToJString(
env, GetRomMetadata(Common::Android::GetJString(env, jpath)).title);
jstring Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getTitle(JNIEnv* env, jobject obj, jstring jpath) {
return Common::Android::ToJString(env, GetRomMetadata(Common::Android::GetJString(env, jpath)).title);
}
jstring Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getProgramId(JNIEnv* env, jobject obj,
jstring jpath) {
return Common::Android::ToJString(
env, std::to_string(GetRomMetadata(Common::Android::GetJString(env, jpath)).programId));
jstring Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getProgramId(JNIEnv* env, jobject obj, jstring jpath) {
return Common::Android::ToJString(env, std::to_string(GetRomMetadata(Common::Android::GetJString(env, jpath)).programId));
}
jstring Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getDeveloper(JNIEnv* env, jobject obj,
jstring jpath) {
return Common::Android::ToJString(
env, GetRomMetadata(Common::Android::GetJString(env, jpath)).developer);
jstring Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getDeveloper(JNIEnv* env, jobject obj, jstring jpath) {
return Common::Android::ToJString(env, GetRomMetadata(Common::Android::GetJString(env, jpath)).developer);
}
jstring Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getVersion(JNIEnv* env, jobject obj,
jstring jpath, jboolean jreload) {
return Common::Android::ToJString(
env, GetRomMetadata(Common::Android::GetJString(env, jpath), jreload).version);
jstring Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getVersion(JNIEnv* env, jobject obj, jstring jpath, jboolean jreload) {
return Common::Android::ToJString(env, GetRomMetadata(Common::Android::GetJString(env, jpath), jreload).version);
}
jbyteArray Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getIcon(JNIEnv* env, jobject obj,
jstring jpath) {
jbyteArray Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getIcon(JNIEnv* env, jobject obj, jstring jpath) {
auto icon_data = GetRomMetadata(Common::Android::GetJString(env, jpath)).icon;
jbyteArray icon = env->NewByteArray(static_cast<jsize>(icon_data.size()));
env->SetByteArrayRegion(icon, 0, env->GetArrayLength(icon),
reinterpret_cast<jbyte*>(icon_data.data()));
jbyteArray icon = env->NewByteArray(jsize(icon_data.size()));
env->SetByteArrayRegion(icon, 0, env->GetArrayLength(icon), reinterpret_cast<jbyte*>(icon_data.data()));
return icon;
}
jboolean Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getIsHomebrew(JNIEnv* env, jobject obj,
jstring jpath) {
return static_cast<jboolean>(
GetRomMetadata(Common::Android::GetJString(env, jpath)).isHomebrew);
jboolean Java_org_yuzu_yuzu_1emu_utils_GameMetadata_getIsHomebrew(JNIEnv* env, jobject obj, jstring jpath) {
return jboolean(GetRomMetadata(Common::Android::GetJString(env, jpath)).isHomebrew);
}
void Java_org_yuzu_yuzu_1emu_utils_GameMetadata_resetMetadata(JNIEnv* env, jobject obj) {
@@ -18,20 +18,15 @@
namespace AudioCore::Renderer {
constexpr u32 TempBufferSize = 0x3F00;
constexpr std::array<u8, 3> PitchBySrcQuality = {4, 8, 4};
/**
* Decode PCM data. Only s16 or f32 is supported.
*
* @tparam T - Type to decode. Only s16 and f32 are supported.
* @param memory - Core memory for reading samples.
* @param out_buffer - Output mix buffer to receive the samples.
* @param req - Information for how to decode.
* @return Number of samples decoded.
*/
/// @brief Decode PCM data. Only s16 or f32 is supported.
/// @tparam T - Type to decode. Only s16 and f32 are supported.
/// @param memory - Core memory for reading samples.
/// @param out_buffer - Output mix buffer to receive the samples.
/// @param req - Information for how to decode.
/// @return Number of samples decoded.
template <typename T>
static u32 DecodePcm(Core::Memory::Memory& memory, std::span<s16> out_buffer,
const DecodeArg& req) {
static u32 DecodePcm(Core::Memory::Memory& memory, std::span<s16> out_buffer, const DecodeArg& req) {
constexpr s32 min{(std::numeric_limits<s16>::min)()};
constexpr s32 max{(std::numeric_limits<s16>::max)()};
@@ -94,16 +89,12 @@ static u32 DecodePcm(Core::Memory::Memory& memory, std::span<s16> out_buffer,
return samples_to_decode;
}
/**
* Decode ADPCM data.
*
* @param memory - Core memory for reading samples.
* @param out_buffer - Output mix buffer to receive the samples.
* @param req - Information for how to decode.
* @return Number of samples decoded.
*/
static u32 DecodeAdpcm(Core::Memory::Memory& memory, std::span<s16> out_buffer,
const DecodeArg& req) {
/// @brief Decode ADPCM data.
/// @param memory - Core memory for reading samples.
/// @param out_buffer - Output mix buffer to receive the samples.
/// @param req - Information for how to decode.
/// @return Number of samples decoded.
static u32 DecodeAdpcm(Core::Memory::Memory& memory, std::span<s16> out_buffer, const DecodeArg& req) {
constexpr u32 SamplesPerFrame{14};
constexpr u32 NibblesPerFrame{16};
@@ -115,8 +106,7 @@ static u32 DecodeAdpcm(Core::Memory::Memory& memory, std::span<s16> out_buffer,
return 0;
}
auto end{(req.end_offset % SamplesPerFrame) +
NibblesPerFrame * (req.end_offset / SamplesPerFrame)};
auto end{(req.end_offset % SamplesPerFrame) + NibblesPerFrame * (req.end_offset / SamplesPerFrame)};
if (req.end_offset % SamplesPerFrame) {
end += 3;
} else {
@@ -133,52 +123,49 @@ static u32 DecodeAdpcm(Core::Memory::Memory& memory, std::span<s16> out_buffer,
return 0;
}
auto samples_to_read{samples_to_process};
auto samples_remaining_in_frame{start_pos % SamplesPerFrame};
auto position_in_frame{(start_pos / SamplesPerFrame) * NibblesPerFrame +
samples_remaining_in_frame};
auto samples_to_read = samples_to_process;
auto samples_remaining_in_frame = start_pos % SamplesPerFrame;
auto position_in_frame = (start_pos / SamplesPerFrame) * NibblesPerFrame + samples_remaining_in_frame;
if (samples_remaining_in_frame) {
position_in_frame += 2;
}
const auto size{(std::max)((samples_to_process / 8U) * SamplesPerFrame, 8U)};
Core::Memory::CpuGuestMemory<u8, Core::Memory::GuestMemoryFlags::UnsafeRead> wavebuffer(
memory, req.buffer + position_in_frame / 2, size);
Core::Memory::CpuGuestMemory<u8, Core::Memory::GuestMemoryFlags::UnsafeRead> wavebuffer(memory, req.buffer + position_in_frame / 2, size);
auto context{req.adpcm_context};
auto header{context->header};
u8 coeff_index{static_cast<u8>((header >> 4U) & 0xFU)};
u8 scale{static_cast<u8>(header & 0xFU)};
s32 coeff0{req.coefficients[coeff_index * 2 + 0]};
s32 coeff1{req.coefficients[coeff_index * 2 + 1]};
auto context = req.adpcm_context;
auto header = context->header;
u8 scale = u8(header & 0xfU);
u8 coeff_index = u8((header >> 4U) & 0x7u);
s32 coeff0 = req.coefficients[coeff_index * 2 + 0];
s32 coeff1 = req.coefficients[coeff_index * 2 + 1];
auto yn0{context->yn0};
auto yn1{context->yn1};
static constexpr std::array<s32, 16> Steps{
0, 1, 2, 3, 4, 5, 6, 7, -8, -7, -6, -5, -4, -3, -2, -1,
auto yn0 = context->yn0;
auto yn1 = context->yn1;
auto const get_step = [](u32 index) {
// Emulates the following table
// 0, 1, 2, 3, 4, 5, 6, 7, -8, -7, -6, -5, -4, -3, -2, -1,
constexpr u64 steps_table = 0x1234567876543210ull;
constexpr u64 steps_sign = 0b1111111100000000ull;
auto const r = s32((steps_table >> (index * 4)) & 0xf);
return ((steps_sign >> index) & 1) == 0 ? -r : r;
};
const auto decode_sample = [&](const s32 code) -> s16 {
auto const decode_sample = [&](const s32 code) -> s16 {
const auto xn = code * (1 << scale);
const auto prediction = coeff0 * yn0 + coeff1 * yn1;
const auto sample = ((xn << 11) + 0x400 + prediction) >> 11;
const auto saturated = std::clamp<s32>(sample, -0x8000, 0x7FFF);
yn1 = yn0;
yn0 = static_cast<s16>(saturated);
return yn0;
return yn0 = s16(saturated);
};
u32 read_index{0};
u32 write_index{0};
while (samples_to_read > 0) {
u32 read_index = 0;
for (u32 write_index = 0; samples_to_read > 0 && write_index < out_buffer.size(); ) {
// Are we at a new frame?
if ((position_in_frame % NibblesPerFrame) == 0) {
header = wavebuffer[read_index++];
coeff_index = (header >> 4) & 0xF;
scale = header & 0xF;
scale = header & 0xFu;
coeff_index = (header >> 4) & 0x7u;
coeff0 = req.coefficients[coeff_index * 2 + 0];
coeff1 = req.coefficients[coeff_index * 2 + 1];
position_in_frame += 2;
@@ -187,14 +174,12 @@ static u32 DecodeAdpcm(Core::Memory::Memory& memory, std::span<s16> out_buffer,
if (samples_to_read >= SamplesPerFrame) {
// Can grab all samples until the next header
for (u32 i = 0; i < SamplesPerFrame / 2; i++) {
auto code0{Steps[(wavebuffer[read_index] >> 4) & 0xF]};
auto code1{Steps[wavebuffer[read_index] & 0xF]};
read_index++;
auto code0 = get_step((wavebuffer[read_index + i] >> 4) & 0xF);
auto code1 = get_step(wavebuffer[read_index + i] & 0xF);
out_buffer[write_index++] = decode_sample(code0);
out_buffer[write_index++] = decode_sample(code1);
}
read_index += SamplesPerFrame / 2;
position_in_frame += SamplesPerFrame;
samples_to_read -= SamplesPerFrame;
continue;
@@ -202,15 +187,14 @@ static u32 DecodeAdpcm(Core::Memory::Memory& memory, std::span<s16> out_buffer,
}
// Decode a single sample
auto code{wavebuffer[read_index]};
auto code = wavebuffer[read_index];
if (position_in_frame & 1) {
code &= 0xF;
read_index++;
} else {
code >>= 4;
}
out_buffer[write_index++] = decode_sample(Steps[code]);
out_buffer[write_index++] = decode_sample(get_step(code));
position_in_frame++;
samples_to_read--;
@@ -219,27 +203,21 @@ static u32 DecodeAdpcm(Core::Memory::Memory& memory, std::span<s16> out_buffer,
context->header = header;
context->yn0 = yn0;
context->yn1 = yn1;
return samples_to_process;
}
/**
* Decode implementation.
* Decode wavebuffers according to the given args.
*
* @param memory - Core memory to read data from.
* @param args - The wavebuffer data, and information for how to decode it.
*/
/// @brief Decode implementation.
/// Decode wavebuffers according to the given args.
///
/// @param memory - Core memory to read data from.
/// @param args - The wavebuffer data, and information for how to decode it.
void DecodeFromWaveBuffers(Core::Memory::Memory& memory, const DecodeFromWaveBuffersArgs& args) {
static constexpr auto EndWaveBuffer = [](auto& voice_state, auto& wavebuffer, auto& index,
auto& played_samples, auto& consumed) -> void {
constexpr auto EndWaveBuffer = [](auto& voice_state, auto& wavebuffer, auto& index, auto& played_samples, auto& consumed) -> void {
voice_state.wave_buffer_valid[index] = false;
voice_state.loop_count = 0;
if (wavebuffer.stream_ended) {
played_samples = 0;
}
index = (index + 1) % MaxWaveBuffers;
consumed++;
};
@@ -255,8 +233,9 @@ void DecodeFromWaveBuffers(Core::Memory::Memory& memory, const DecodeFromWaveBuf
return;
}
auto pitch{PitchBySrcQuality[static_cast<u32>(args.src_quality)]};
if (static_cast<u32>(pitch + size_required.to_int_floor()) > TempBufferSize) {
// 0 -> 4, 1 -> 8, 2 -> 4
auto pitch = u32((0x040804ul >> (u32(args.src_quality) * 8)) & 0xfful);
if (u32(pitch + size_required.to_int_floor()) > TempBufferSize) {
return;
}
@@ -272,7 +251,7 @@ void DecodeFromWaveBuffers(Core::Memory::Memory& memory, const DecodeFromWaveBuf
bool is_buffer_starved{false};
u32 offset{voice_state.offset};
auto output_buffer{args.output};
auto output_buffer = args.output;
std::array<s16, TempBufferSize> temp_buffer{};
while (remaining_sample_count > 0) {
@@ -294,8 +273,8 @@ void DecodeFromWaveBuffers(Core::Memory::Memory& memory, const DecodeFromWaveBuf
if (wavebuffer_index >= MaxWaveBuffers) {
LOG_ERROR(Service_Audio, "Invalid wavebuffer index! {}", wavebuffer_index);
wavebuffer_index = 0;
voice_state.wave_buffer_valid.fill(false);
wavebuffers_consumed = MaxWaveBuffers;
voice_state.wave_buffer_valid.fill(false);
}
if (!voice_state.wave_buffer_valid[wavebuffer_index]) {
@@ -303,12 +282,9 @@ void DecodeFromWaveBuffers(Core::Memory::Memory& memory, const DecodeFromWaveBuf
break;
}
auto& wavebuffer{args.wave_buffers[wavebuffer_index]};
if (offset == 0 && args.sample_format == SampleFormat::Adpcm &&
wavebuffer.context != 0) {
memory.ReadBlockUnsafe(wavebuffer.context, &voice_state.adpcm_context,
wavebuffer.context_size);
auto& wavebuffer = args.wave_buffers[wavebuffer_index];
if (offset == 0 && args.sample_format == SampleFormat::Adpcm && wavebuffer.context != 0) {
memory.ReadBlockUnsafe(wavebuffer.context, &voice_state.adpcm_context, wavebuffer.context_size);
}
auto start_offset{wavebuffer.start_offset};
@@ -351,9 +327,7 @@ void DecodeFromWaveBuffers(Core::Memory::Memory& memory, const DecodeFromWaveBuf
case SampleFormat::Adpcm: {
decode_arg.adpcm_context = &voice_state.adpcm_context;
memory.ReadBlockUnsafe(args.data_address, &decode_arg.coefficients, args.data_size);
samples_decoded = DecodeAdpcm(
memory, {&temp_buffer[temp_buffer_pos], TempBufferSize - temp_buffer_pos},
decode_arg);
samples_decoded = DecodeAdpcm( memory, {&temp_buffer[temp_buffer_pos], TempBufferSize - temp_buffer_pos}, decode_arg);
} break;
default:
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2022 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
@@ -5,21 +8,13 @@
namespace AudioCore::Renderer {
static void ResampleLowQuality(std::span<s32> output, std::span<const s16> input,
const Common::FixedPoint<49, 15>& sample_rate_ratio,
Common::FixedPoint<49, 15>& fraction, const u32 samples_to_write) {
if (sample_rate_ratio == 1.0f) {
for (u32 i = 0; i < samples_to_write; i++) {
output[i] = input[i];
}
} else {
u32 read_index{0};
for (u32 i = 0; i < samples_to_write; i++) {
output[i] = input[read_index + (fraction >= 0.5f)];
fraction += sample_rate_ratio;
read_index += static_cast<u32>(fraction.to_int_floor());
fraction.clear_int();
}
static void ResampleLowQuality(std::span<s32> output, std::span<const s16> input, const Common::FixedPoint<49, 15>& sample_rate_ratio, Common::FixedPoint<49, 15>& fraction, const u32 samples_to_write) {
u32 read_index{0};
for (u32 i = 0; i < samples_to_write; i++) {
output[i] = input[read_index + (fraction >= 0.5f)];
fraction += sample_rate_ratio;
read_index += u32(fraction.to_int_floor());
fraction.clear_int();
}
}
@@ -295,7 +290,7 @@ static void ResampleNormalQuality(std::span<s32> output, std::span<const s16> in
auto lut{get_lut()};
u32 read_index{0};
for (u32 i = 0; i < samples_to_write; i++) {
const auto lut_index{(fraction.get_frac() >> 8) * 4};
const auto lut_index = ((fraction.get_frac() >> 8) << 2) & 511;
const Common::FixedPoint<56, 8> sample0{input[read_index + 0] * lut[lut_index + 0]};
const Common::FixedPoint<56, 8> sample1{input[read_index + 1] * lut[lut_index + 1]};
const Common::FixedPoint<56, 8> sample2{input[read_index + 2] * lut[lut_index + 2]};
@@ -845,7 +840,7 @@ static void ResampleHighQuality(std::span<s32> output, std::span<const s16> inpu
auto lut{get_lut()};
u32 read_index{0};
for (u32 i = 0; i < samples_to_write; i++) {
const auto lut_index{(fraction.get_frac() >> 8) * 8};
const auto lut_index = ((fraction.get_frac() >> 8) << 3) & 1023;
const Common::FixedPoint<56, 8> sample0{input[read_index + 0] * lut[lut_index + 0]};
const Common::FixedPoint<56, 8> sample1{input[read_index + 1] * lut[lut_index + 1]};
const Common::FixedPoint<56, 8> sample2{input[read_index + 2] * lut[lut_index + 2]};
+20 -139
View File
@@ -1,6 +1,5 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: 2013 Dolphin Emulator Project
// SPDX-FileCopyrightText: 2014 Citra Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
@@ -40,110 +39,6 @@
#include <unistd.h>
#endif
#ifdef __ANDROID__
#include <sys/resource.h>
#include <algorithm>
#include <fstream>
#include <utility>
#include <vector>
namespace {
[[maybe_unused]] constexpr int ANDROID_THREAD_PRIORITY_URGENT_AUDIO = -19;
[[maybe_unused]] constexpr int ANDROID_THREAD_PRIORITY_AUDIO = -16;
[[maybe_unused]] constexpr int ANDROID_THREAD_PRIORITY_URGENT_DISPLAY = -8;
[[maybe_unused]] constexpr int ANDROID_THREAD_PRIORITY_DISPLAY = -4;
[[maybe_unused]] constexpr int ANDROID_THREAD_PRIORITY_FOREGROUND = -2;
[[maybe_unused]] constexpr int ANDROID_THREAD_PRIORITY_MORE_FAVORABLE = -1;
[[maybe_unused]] constexpr int ANDROID_THREAD_PRIORITY_DEFAULT = 0;
[[maybe_unused]] constexpr int ANDROID_THREAD_PRIORITY_LESS_FAVORABLE = 1;
[[maybe_unused]] constexpr int ANDROID_THREAD_PRIORITY_BACKGROUND = 10;
[[maybe_unused]] constexpr int ANDROID_THREAD_PRIORITY_LOWEST = 19;
constexpr size_t ANDROID_MINIMUM_PERFORMANCE_CORES = 4;
cpu_set_t ComputePerformanceCoreMask() {
cpu_set_t mask;
CPU_ZERO(&mask);
cpu_set_t allowed;
CPU_ZERO(&allowed);
if (sched_getaffinity(gettid(), sizeof(allowed), &allowed) != 0) {
return mask;
}
std::vector<std::pair<long, int>> cores;
const int total = static_cast<int>(std::thread::hardware_concurrency());
for (int cpu = 0; cpu < total; ++cpu) {
if (!CPU_ISSET(cpu, &allowed)) {
continue;
}
long max_frequency = 0;
std::ifstream file("/sys/devices/system/cpu/cpu" + std::to_string(cpu) +
"/cpufreq/cpuinfo_max_freq");
if (!file || !(file >> max_frequency) || max_frequency <= 0) {
CPU_ZERO(&mask);
return mask;
}
cores.emplace_back(max_frequency, cpu);
}
if (cores.empty()) {
return mask;
}
std::sort(cores.begin(), cores.end(),
[](const auto& lhs, const auto& rhs) { return lhs.first > rhs.first; });
size_t taken = 0;
long cluster_frequency = cores.front().first;
for (const auto& [frequency, cpu] : cores) {
if (frequency != cluster_frequency) {
if (taken >= ANDROID_MINIMUM_PERFORMANCE_CORES) {
break;
}
cluster_frequency = frequency;
}
CPU_SET(cpu, &mask);
++taken;
}
return mask;
}
const cpu_set_t& PerformanceCoreMask() {
static const cpu_set_t mask = ComputePerformanceCoreMask();
return mask;
}
cpu_set_t ComputeEfficiencyCoreMask() {
cpu_set_t mask;
CPU_ZERO(&mask);
const cpu_set_t& performance = PerformanceCoreMask();
if (CPU_COUNT(&performance) == 0) {
return mask;
}
cpu_set_t allowed;
CPU_ZERO(&allowed);
if (sched_getaffinity(gettid(), sizeof(allowed), &allowed) != 0) {
return mask;
}
const int total = static_cast<int>(std::thread::hardware_concurrency());
for (int cpu = 0; cpu < total; ++cpu) {
if (CPU_ISSET(cpu, &allowed) && !CPU_ISSET(cpu, &performance)) {
CPU_SET(cpu, &mask);
}
}
return mask;
}
const cpu_set_t& EfficiencyCoreMask() {
static const cpu_set_t mask = ComputeEfficiencyCoreMask();
return mask;
}
} // Anonymous namespace
#endif
#include "common/cpu_features.h"
#ifdef ARCHITECTURE_x86_64
#ifdef _MSC_VER
@@ -183,21 +78,6 @@ void SetCurrentThreadPriority(ThreadPriority new_priority) {
}
}();
set_thread_priority(find_thread(NULL), priority);
#elif defined(__ANDROID__)
const int nice_value = [&]() {
switch (new_priority) {
case ThreadPriority::Low: return ANDROID_THREAD_PRIORITY_BACKGROUND;
case ThreadPriority::Normal: return ANDROID_THREAD_PRIORITY_DEFAULT;
case ThreadPriority::High: return ANDROID_THREAD_PRIORITY_DISPLAY;
case ThreadPriority::VeryHigh: return ANDROID_THREAD_PRIORITY_URGENT_DISPLAY;
case ThreadPriority::Critical: return ANDROID_THREAD_PRIORITY_AUDIO;
default: return ANDROID_THREAD_PRIORITY_DEFAULT;
}
}();
if (setpriority(PRIO_PROCESS, static_cast<id_t>(gettid()), nice_value) != 0) {
LOG_DEBUG(Common, "Could not set thread nice value to {}: {}", nice_value,
GetLastErrorMsg());
}
#else
pthread_t this_thread = pthread_self();
const auto scheduling_type = SCHED_OTHER;
@@ -252,28 +132,29 @@ void SetCurrentThreadName(const char* name) {
#endif
}
void SetCurrentThreadToPerformanceCores() {
void PinCurrentThreadToPerformanceCore(size_t core_id) {
ASSERT(core_id < 4);
// If we set a flag for a CPU that doesn't exist, the thread may not be allowed to
// run in ANY processor!
auto const total_cores = std::thread::hardware_concurrency();
if (core_id < total_cores) {
#if defined(__ANDROID__)
const cpu_set_t& mask = PerformanceCoreMask();
if (CPU_COUNT(&mask) == 0) {
return;
}
if (sched_setaffinity(gettid(), sizeof(mask), &mask) != 0) {
LOG_DEBUG(Common, "Could not restrict thread to performance cores: {}", GetLastErrorMsg());
}
cpu_set_t set;
CPU_ZERO(&set);
CPU_SET(core_id, &set);
sched_setaffinity(pthread_self(), sizeof(set), &set);
#elif defined(__linux__) || defined(__FreeBSD__)
cpu_set_t set;
CPU_ZERO(&set);
CPU_SET(core_id, &set);
pthread_setaffinity_np(pthread_self(), sizeof(set), &set);
#elif defined(_WIN32)
DWORD set = 1UL << core_id;
SetThreadAffinityMask(GetCurrentThread(), set);
#else
// No pin functionality implemented
#endif
}
void SetCurrentThreadToEfficiencyCores() {
#if defined(__ANDROID__)
const cpu_set_t& mask = EfficiencyCoreMask();
if (CPU_COUNT(&mask) == 0) {
return;
}
if (sched_setaffinity(gettid(), sizeof(mask), &mask) != 0) {
LOG_DEBUG(Common, "Could not restrict thread to efficiency cores: {}", GetLastErrorMsg());
}
#endif
}
#ifdef ARCHITECTURE_x86_64
+1 -7
View File
@@ -99,14 +99,8 @@ enum class ThreadPriority : u32 {
Critical = 4,
};
enum class ThreadPlacement : u32 {
Default = 0,
Background = 1,
};
void SetCurrentThreadPriority(ThreadPriority new_priority);
void SetCurrentThreadName(const char* name);
void SetCurrentThreadToPerformanceCores();
void SetCurrentThreadToEfficiencyCores();
void PinCurrentThreadToPerformanceCore(size_t core_id);
} // namespace Common
+3 -8
View File
@@ -1,4 +1,4 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
@@ -37,15 +37,10 @@ class StatefulThreadWorker {
using StateMaker = std::conditional_t<with_state, std::function<StateType()>, DummyCallable>;
public:
explicit StatefulThreadWorker(size_t num_workers, std::string name, StateMaker func = {},
ThreadPlacement placement = ThreadPlacement::Default)
explicit StatefulThreadWorker(size_t num_workers, std::string name, StateMaker func = {})
: workers_queued{num_workers}, thread_name{std::move(name)} {
const auto lambda = [this, func, placement](std::stop_token stop_token) {
const auto lambda = [this, func](std::stop_token stop_token) {
Common::SetCurrentThreadName(thread_name.c_str());
if (placement == ThreadPlacement::Background) {
Common::SetCurrentThreadPriority(ThreadPriority::Low);
Common::SetCurrentThreadToEfficiencyCores();
}
{
[[maybe_unused]] std::conditional_t<with_state, StateType, int> state{func()};
while (!stop_token.stop_requested()) {
+8
View File
@@ -208,4 +208,12 @@ UUID UUID::MakeRandomRFC4122V4() {
return uuid;
}
UUID UUID::MakeRFC4122V5(std::span<u8, 20> sha1) {
UUID uuid{};
std::memcpy(&uuid.uuid, sha1.data(), sizeof(UUID));
uuid.uuid[8] = 0x80 | (uuid.uuid[8] & 0x3F);
uuid.uuid[6] = 0x50 | (uuid.uuid[6] & 0xF);
return uuid;
}
} // namespace Common
+16 -20
View File
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2022 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
@@ -5,6 +8,7 @@
#include <array>
#include <functional>
#include <span>
#include <string>
#include "common/common_types.h"
@@ -86,28 +90,20 @@ struct UUID {
};
}
/**
* Creates a random UUID.
*
* @returns A random UUID.
*/
static UUID MakeRandom();
/// @brief Creates a random UUID.
/// @returns A random UUID.
[[nodiscard]] static UUID MakeRandom();
/**
* Creates a random UUID with a seed.
*
* @param seed A seed to initialize the Mersenne-Twister RNG
*
* @returns A random UUID.
*/
static UUID MakeRandomWithSeed(u32 seed);
/// @brief Creates a random UUID with a seed.
/// @param seed A seed to initialize the Mersenne-Twister RNG
/// @returns A random UUID.
[[nodiscard]] static UUID MakeRandomWithSeed(u32 seed);
/**
* Creates a random UUID. The generated UUID is RFC 4122 Version 4 compliant.
*
* @returns A random UUID that is RFC 4122 Version 4 compliant.
*/
static UUID MakeRandomRFC4122V4();
/// @brief Creates a random UUID. The generated UUID is RFC 4122 Version 4 compliant.
/// @returns A random UUID that is RFC 4122 Version 4 compliant.
[[nodiscard]] static UUID MakeRandomRFC4122V4();
[[nodiscard]] static UUID MakeRFC4122V5(std::span<u8, 20> sha1);
friend constexpr bool operator==(const UUID& lhs, const UUID& rhs) = default;
};
+6 -1
View File
@@ -174,7 +174,12 @@ void CpuManager::RunThread(std::stop_token token, std::size_t core) {
std::string name = is_multicore ? ("CPUCore_" + std::to_string(core)) : std::string{"CPUThread"};
Common::SetCurrentThreadName(name.c_str());
Common::SetCurrentThreadPriority(Common::ThreadPriority::Critical);
Common::SetCurrentThreadToPerformanceCores();
#ifdef __ANDROID__
// Aimed specifically for Snapdragon 8 Elite devices
// This kills performance on desktop, but boosts perf for UMA devices
// like the S8E. Mediatek and Mali likely won't suffer.
Common::PinCurrentThreadToPerformanceCore(core);
#endif
auto& data = core_data[core];
data.host_context = Common::Fiber::ThreadToFiber();
+15 -16
View File
@@ -274,23 +274,22 @@ public:
explicit NACP(VirtualFile file);
~NACP();
const LanguageEntry& GetLanguageEntry() const;
std::string GetApplicationName() const;
std::string GetDeveloperName() const;
u64 GetTitleId() const;
u64 GetDLCBaseTitleId() const;
std::string GetVersionString() const;
u64 GetDefaultNormalSaveSize() const;
u64 GetDefaultJournalSaveSize() const;
u32 GetSupportedLanguages() const;
std::vector<std::string> GetApplicationNames() const;
std::vector<u8> GetRawBytes() const;
bool GetUserAccountSwitchLock() const;
u64 GetDeviceSaveDataSize() const;
u32 GetParentalControlFlag() const;
const std::array<u8, 0x20>& GetRatingAge() const;
[[nodiscard]] const LanguageEntry& GetLanguageEntry() const;
[[nodiscard]] std::string GetApplicationName() const;
[[nodiscard]] std::string GetDeveloperName() const;
[[nodiscard]] u64 GetTitleId() const;
[[nodiscard]] u64 GetDLCBaseTitleId() const;
[[nodiscard]] std::string GetVersionString() const;
[[nodiscard]] u64 GetDefaultNormalSaveSize() const;
[[nodiscard]] u64 GetDefaultJournalSaveSize() const;
[[nodiscard]] u32 GetSupportedLanguages() const;
[[nodiscard]] std::vector<std::string> GetApplicationNames() const;
[[nodiscard]] std::vector<u8> GetRawBytes() const;
[[nodiscard]] bool GetUserAccountSwitchLock() const;
[[nodiscard]] u64 GetDeviceSaveDataSize() const;
[[nodiscard]] u32 GetParentalControlFlag() const;
[[nodiscard]] const std::array<u8, 0x20>& GetRatingAge() const;
private:
RawNACP raw{};
std::vector<LanguageEntry> language_entries;
};
+10
View File
@@ -1156,4 +1156,14 @@ PatchManager::Metadata PatchManager::ParseControlNCA(const NCA& nca) const {
return {std::move(nacp), icon_file};
}
[[nodiscard]] PatchManager::Metadata PatchManager::GetMetadataFromBaseOrUpdate(Core::System& system, u64 application_id) noexcept {
const FileSys::PatchManager pm{application_id, system.GetFileSystemController(), system.GetContentProvider()};
auto metadata = pm.GetControlMetadata();
if (metadata.first != nullptr)
return metadata;
const FileSys::PatchManager pm_update{FileSys::GetUpdateTitleID(application_id), system.GetFileSystemController(), system.GetContentProvider()};
return pm_update.GetControlMetadata();
}
} // namespace FileSys
+3
View File
@@ -105,6 +105,9 @@ public:
// Version of GetControlMetadata that takes an arbitrary NCA
[[nodiscard]] Metadata ParseControlNCA(const NCA& nca) const;
/// @brief Gets NACP metadata (accounting for any patches or updates)
[[nodiscard]] static PatchManager::Metadata GetMetadataFromBaseOrUpdate(Core::System& system, u64 application_id) noexcept;
private:
[[nodiscard]] std::vector<VirtualFile> CollectPatches(const std::vector<VirtualDir>& patch_dirs,
const std::string& build_id) const;
@@ -4,6 +4,8 @@
// SPDX-FileCopyrightText: Copyright 2024 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <openssl/evp.h>
#include "common/settings.h"
#include "common/uuid.h"
#include "core/file_sys/control_metadata.h"
@@ -154,20 +156,7 @@ Result IApplicationFunctions::GetDesiredLanguage(Out<u64> out_language_code) {
// Default to 0 (all languages supported)
u32 supported_languages = 0;
const auto res = [this] {
const FileSys::PatchManager pm{m_applet->program_id, system.GetFileSystemController(),
system.GetContentProvider()};
auto metadata = pm.GetControlMetadata();
if (metadata.first != nullptr) {
return metadata;
}
const FileSys::PatchManager pm_update{FileSys::GetUpdateTitleID(m_applet->program_id),
system.GetFileSystemController(),
system.GetContentProvider()};
return pm_update.GetControlMetadata();
}();
const auto res = FileSys::PatchManager::GetMetadataFromBaseOrUpdate(system, m_applet->program_id);
if (res.first != nullptr) {
supported_languages = res.first->GetSupportedLanguages();
}
@@ -205,20 +194,7 @@ Result IApplicationFunctions::SetTerminateResult(Result terminate_result) {
Result IApplicationFunctions::GetDisplayVersion(Out<DisplayVersion> out_display_version) {
LOG_DEBUG(Service_AM, "called");
const auto res = [this] {
const FileSys::PatchManager pm{m_applet->program_id, system.GetFileSystemController(),
system.GetContentProvider()};
auto metadata = pm.GetControlMetadata();
if (metadata.first != nullptr) {
return metadata;
}
const FileSys::PatchManager pm_update{FileSys::GetUpdateTitleID(m_applet->program_id),
system.GetFileSystemController(),
system.GetContentProvider()};
return pm_update.GetControlMetadata();
}();
const auto res = FileSys::PatchManager::GetMetadataFromBaseOrUpdate(system, m_applet->program_id);
if (res.first != nullptr) {
const auto& version = res.first->GetVersionString();
std::memcpy(out_display_version->string.data(), version.data(),
@@ -347,8 +323,21 @@ Result IApplicationFunctions::NotifyRunning(Out<bool> out_became_running) {
}
Result IApplicationFunctions::GetPseudoDeviceId(Out<Common::UUID> out_pseudo_device_id) {
LOG_WARNING(Service_AM, "(STUBBED) called");
*out_pseudo_device_id = {};
LOG_WARNING(Service_AM, "(stubbed)");
// This should be hashed with the device specific hash
// for now this will do
const auto res = FileSys::PatchManager::GetMetadataFromBaseOrUpdate(system, m_applet->program_id);
u8 hash[EVP_MAX_MD_SIZE];
unsigned int hash_len = 0;
auto const seed = res.first->raw.seed_for_pseudo_device_id;
EVP_MD_CTX *ctx = EVP_MD_CTX_new();
auto const algorithm = EVP_sha1();
EVP_DigestInit_ex(ctx, algorithm, nullptr);
EVP_DigestUpdate(ctx, &seed, sizeof(seed));
EVP_DigestFinal_ex(ctx, hash, &hash_len);
EVP_MD_CTX_free(ctx);
*out_pseudo_device_id = Common::UUID::MakeRFC4122V5(std::span<u8, 20>{hash, std::size(hash)});
R_SUCCEED();
}
@@ -252,19 +252,7 @@ Result ILibraryAppletSelfAccessor::GetMainAppletApplicationDesiredLanguage(
// Default to 0 (all languages supported)
u32 supported_languages = 0;
const auto res = [this, identity] {
const FileSys::PatchManager pm{identity.application_id, system.GetFileSystemController(),
system.GetContentProvider()};
auto metadata = pm.GetControlMetadata();
if (metadata.first != nullptr) {
return metadata;
}
const FileSys::PatchManager pm_update{FileSys::GetUpdateTitleID(identity.application_id),
system.GetFileSystemController(),
system.GetContentProvider()};
return pm_update.GetControlMetadata();
}();
const auto res = FileSys::PatchManager::GetMetadataFromBaseOrUpdate(system, identity.application_id);
if (res.first != nullptr) {
supported_languages = res.first->GetSupportedLanguages();
@@ -100,6 +100,7 @@ void EmulatedConsole::ReloadInput() {
motion.gyro = emulated_motion.GetGyroscope();
motion.rotation = emulated_motion.GetRotations();
motion.orientation = emulated_motion.GetOrientation();
motion.quaternion = emulated_motion.GetQuaternion();
motion.is_at_rest = !emulated_motion.IsMoving(motion_sensitivity);
// Unique index for identifying touch device source
+3 -1
View File
@@ -33,7 +33,7 @@ add_library(video_core STATIC
control/channel_state_cache.h
control/scheduler.cpp
control/scheduler.h
deferred_destruction_queue.h
delayed_destruction_ring.h
dirty_flags.cpp
dirty_flags.h
dma_pusher.cpp
@@ -158,6 +158,8 @@ add_library(video_core STATIC
renderer_vulkan/vk_compute_pass.h
renderer_vulkan/vk_compute_pipeline.cpp
renderer_vulkan/vk_compute_pipeline.h
renderer_vulkan/vk_descriptor_buffer.cpp
renderer_vulkan/vk_descriptor_buffer.h
renderer_vulkan/vk_descriptor_pool.cpp
renderer_vulkan/vk_descriptor_pool.h
renderer_vulkan/vk_fence_manager.cpp
+32 -62
View File
@@ -31,77 +31,44 @@ BufferCache<P>::BufferCache(Tegra::MaxwellDeviceMemoryManager& device_memory_, R
immediately_free = (Settings::values.vram_usage_mode.GetValue() == Settings::VramUsageMode::Aggressive);
#endif
if (!runtime.CanReportMemoryUsage()) {
memory_budget = FALLBACK_MEMORY_BUDGET;
minimum_memory = DEFAULT_EXPECTED_MEMORY;
critical_memory = DEFAULT_CRITICAL_MEMORY;
return;
}
memory_budget = runtime.GetDeviceLocalMemory();
const s64 device_local_memory = static_cast<s64>(runtime.GetDeviceLocalMemory());
const s64 min_spacing_expected = device_local_memory - 1_GiB;
const s64 min_spacing_critical = device_local_memory - 512_MiB;
const s64 mem_threshold = (std::min)(device_local_memory, TARGET_THRESHOLD);
const s64 min_vacancy_expected = (6 * mem_threshold) / 10;
const s64 min_vacancy_critical = (2 * mem_threshold) / 10;
minimum_memory = static_cast<u64>(
(std::max)((std::min)(device_local_memory - min_vacancy_expected, min_spacing_expected),
DEFAULT_EXPECTED_MEMORY));
critical_memory = static_cast<u64>(
(std::max)((std::min)(device_local_memory - min_vacancy_critical, min_spacing_critical),
DEFAULT_CRITICAL_MEMORY));
}
template <class P>
BufferCache<P>::~BufferCache() = default;
template <class P>
u64 BufferCache<P>::DeviceUsage(bool force_refresh) {
if (!runtime.CanReportAllocationUsage()) {
return total_used_memory;
}
if (force_refresh || usage_refresh_countdown == 0) {
cached_device_usage = runtime.GetDeviceAllocationUsage();
usage_refresh_countdown = USAGE_REFRESH_INTERVAL;
} else {
--usage_refresh_countdown;
}
return cached_device_usage;
}
template <class P>
u64 BufferCache<P>::ReclaimMemory(u64 target_bytes, bool allow_download) {
if (target_bytes == 0 || in_reclaim) {
return 0;
}
in_reclaim = true;
sentenced_buffers.Reclaim(runtime.CompletedSyncPoint());
u64 freed = 0;
const auto clean_up = [&](BufferId buffer_id) {
if (freed >= target_bytes) {
void BufferCache<P>::RunGarbageCollector() {
const bool aggressive_gc = total_used_memory >= critical_memory;
const u64 ticks_to_destroy = aggressive_gc ? 60 : 120;
int num_iterations = aggressive_gc ? 64 : 32;
const auto clean_up = [this, &num_iterations](BufferId buffer_id) {
if (num_iterations == 0) {
return true;
}
--num_iterations;
auto& buffer = slot_buffers[buffer_id];
if (!allow_download && IsRegionGpuModified(buffer.CpuAddr(), buffer.SizeBytes())) {
return false;
}
const u64 buffer_bytes = Common::AlignUp(buffer.SizeBytes(), 1024);
DownloadBufferMemory(buffer);
DeleteBuffer(buffer_id);
freed += buffer_bytes;
return false;
};
const u64 cold_tick =
frame_tick > RECLAIM_GUARD_FRAMES ? frame_tick - RECLAIM_GUARD_FRAMES : 0;
lru_cache.ForEachItemBelow(cold_tick, clean_up);
if (freed < target_bytes) {
lru_cache.ForEachItemBelow(frame_tick > 0 ? frame_tick - 1 : 0, clean_up);
}
sentenced_buffers.Reclaim(runtime.CompletedSyncPoint());
in_reclaim = false;
usage_refresh_countdown = 0;
reclaim_stalled = freed == 0;
return freed;
}
template <class P>
void BufferCache<P>::EnsureHeadroom(bool allow_download) {
if (reclaim_stalled) {
return;
}
const u64 limit = memory_budget > RECLAIM_HEADROOM ? memory_budget - RECLAIM_HEADROOM : 0;
const u64 usage = DeviceUsage(false);
if (usage <= limit) {
return;
}
const u64 target = (limit / 100) * RECLAIM_TARGET_PERCENT;
ReclaimMemory((std::min)(usage - target, total_used_memory), allow_download);
lru_cache.ForEachItemBelow(frame_tick - ticks_to_destroy, clean_up);
}
template <class P>
@@ -129,11 +96,15 @@ void BufferCache<P>::TickFrame() {
const bool skip_preferred = hits * 256 < shots * 251;
channel_state->uniform_buffer_skip_cache_size = skip_preferred ? DEFAULT_SKIP_CACHE_SIZE : 0;
usage_refresh_countdown = 0;
reclaim_stalled = false;
EnsureHeadroom(true);
// If we can obtain the memory info, use it instead of the estimate.
if (runtime.CanReportMemoryUsage()) {
total_used_memory = runtime.GetDeviceMemoryUsage();
}
if (total_used_memory >= minimum_memory) {
RunGarbageCollector();
}
++frame_tick;
sentenced_buffers.Reclaim(runtime.CompletedSyncPoint());
delayed_destruction_ring.Tick();
for (auto& buffer : async_buffers_death_ring) {
runtime.FreeDeferredStagingBuffer(buffer);
@@ -1605,7 +1576,6 @@ void BufferCache<P>::JoinOverlap(BufferId new_buffer_id, BufferId overlap_id,
template <class P>
BufferId BufferCache<P>::CreateBuffer(DAddr device_addr, u32 wanted_size) {
EnsureHeadroom(false);
DAddr device_addr_end = Common::AlignUp(device_addr + wanted_size, CACHING_PAGESIZE);
device_addr = Common::AlignDown(device_addr, CACHING_PAGESIZE);
wanted_size = static_cast<u32>(device_addr_end - device_addr);
@@ -1643,7 +1613,7 @@ void BufferCache<P>::ChangeRegister(BufferId buffer_id) {
total_used_memory += Common::AlignUp(size, 1024);
buffer.setLRUID(lru_cache.Insert(buffer_id, frame_tick));
} else {
total_used_memory -= std::min<u64>(total_used_memory, Common::AlignUp(size, 1024));
total_used_memory -= Common::AlignUp(size, 1024);
lru_cache.Free(buffer.getLRUID());
}
const DAddr device_addr_begin = buffer.CpuAddr();
@@ -1902,7 +1872,7 @@ void BufferCache<P>::DeleteBuffer(BufferId buffer_id, bool do_not_mark) {
#ifdef YUZU_LEGACY
if (!do_not_mark || !immediately_free)
#endif
sentenced_buffers.Push(std::move(slot_buffers[buffer_id]), runtime.CurrentSyncPoint());
delayed_destruction_ring.Push(std::move(slot_buffers[buffer_id]));
slot_buffers.erase(buffer_id);
+14 -19
View File
@@ -9,7 +9,6 @@
#include <algorithm>
#include <array>
#include <bit>
#include <deque>
#include <functional>
#include <memory>
#include <mutex>
@@ -31,7 +30,7 @@
#include "common/slot_vector.h"
#include "video_core/buffer_cache/buffer_base.h"
#include "video_core/control/channel_state_cache.h"
#include "video_core/deferred_destruction_queue.h"
#include "video_core/delayed_destruction_ring.h"
#include "video_core/dirty_flags.h"
#include "video_core/engines/maxwell_3d.h"
#include "video_core/engines/kepler_compute.h"
@@ -183,15 +182,13 @@ class BufferCache : public VideoCommon::ChannelSetupCaches<BufferCacheChannelInf
static constexpr bool USE_MEMORY_MAPS_FOR_UPLOADS = P::USE_MEMORY_MAPS_FOR_UPLOADS;
#ifdef YUZU_LEGACY
static constexpr u64 RECLAIM_HEADROOM = 384_MiB;
static constexpr s64 TARGET_THRESHOLD = 3_GiB;
#else
static constexpr u64 RECLAIM_HEADROOM = 512_MiB;
static constexpr s64 TARGET_THRESHOLD = 4_GiB;
#endif
static constexpr u64 FALLBACK_MEMORY_BUDGET = 2_GiB;
static constexpr u32 USAGE_REFRESH_INTERVAL = 16;
static constexpr u64 RECLAIM_GUARD_FRAMES = 8;
static constexpr u64 RECLAIM_TARGET_PERCENT = 88;
static constexpr s64 DEFAULT_EXPECTED_MEMORY = 512_MiB;
static constexpr s64 DEFAULT_CRITICAL_MEMORY = 1_GiB;
// Debug Flags.
@@ -218,8 +215,6 @@ public:
void TickFrame();
u64 ReclaimMemory(u64 target_bytes, bool allow_download);
void WriteMemory(DAddr device_addr, u64 size);
void CachedWriteMemory(DAddr device_addr, u64 size);
@@ -363,9 +358,7 @@ private:
((device_addr + size) & ~Core::DEVICE_PAGEMASK);
}
u64 DeviceUsage(bool force_refresh);
void EnsureHeadroom(bool allow_download);
void RunGarbageCollector();
void BindHostIndexBuffer();
@@ -482,7 +475,12 @@ private:
Tegra::MaxwellDeviceMemoryManager& device_memory;
Common::SlotVector<Buffer> slot_buffers;
DeferredDestructionQueue<Buffer> sentenced_buffers;
#ifdef YUZU_LEGACY
static constexpr size_t TICKS_TO_DESTROY = 6;
#else
static constexpr size_t TICKS_TO_DESTROY = 8;
#endif
DelayedDestructionRing<Buffer, TICKS_TO_DESTROY> delayed_destruction_ring;
const Tegra::Engines::Maxwell3D::DrawManager::IndirectParams* current_draw_indirect{};
@@ -517,11 +515,8 @@ private:
Common::LeastRecentlyUsedCache<LRUItemParams> lru_cache;
u64 frame_tick = 0;
u64 total_used_memory = 0;
u64 memory_budget = 0;
u64 cached_device_usage = 0;
u32 usage_refresh_countdown = 0;
bool in_reclaim = false;
bool reclaim_stalled = false;
u64 minimum_memory = 0;
u64 critical_memory = 0;
BufferId inline_buffer_id;
#ifdef YUZU_LEGACY
bool immediately_free = false;
@@ -1,56 +0,0 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#pragma once
#include <cstddef>
#include <utility>
#include <boost/container/deque.hpp>
#include <boost/container/options.hpp>
#include "common/common_types.h"
namespace VideoCommon {
template <typename T>
class DeferredDestructionQueue {
public:
void Push(T&& object, u64 sync_point) {
entries.emplace_back(std::move(object), sync_point);
}
void Reclaim(u64 completed_sync_point) {
while (!entries.empty() && entries.front().sync_point <= completed_sync_point) {
entries.pop_front();
}
}
void Clear() {
entries.clear();
}
[[nodiscard]] size_t Size() const noexcept {
return entries.size();
}
[[nodiscard]] bool Empty() const noexcept {
return entries.empty();
}
private:
struct Entry {
Entry(T&& object_, u64 sync_point_) noexcept
: object{std::move(object_)}, sync_point{sync_point_} {}
T object;
u64 sync_point;
};
using EntryDequeOptions =
boost::container::deque_options<boost::container::block_size<8u>>::type;
boost::container::deque<Entry, void, EntryDequeOptions> entries;
};
} // namespace VideoCommon
+34
View File
@@ -0,0 +1,34 @@
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <array>
#include <cstddef>
#include <utility>
#include <vector>
namespace VideoCommon {
/// Container to push objects to be destroyed a few ticks in the future
template <typename T, size_t TICKS_TO_DESTROY>
class DelayedDestructionRing {
public:
void Tick() {
index = (index + 1) % TICKS_TO_DESTROY;
elements[index].clear();
}
void Push(T&& object) {
elements[index].push_back(std::move(object));
}
private:
size_t index = 0;
std::array<std::vector<T>, TICKS_TO_DESTROY> elements;
};
} // namespace VideoCommon
+5 -8
View File
@@ -18,7 +18,7 @@
#include "common/common_types.h"
#include "common/settings.h"
#include "common/thread.h"
#include "video_core/deferred_destruction_queue.h"
#include "video_core/delayed_destruction_ring.h"
#include "video_core/gpu.h"
#include "video_core/host1x/host1x.h"
#include "video_core/host1x/syncpoint_manager.h"
@@ -50,8 +50,7 @@ public:
/// Notify the fence manager about a new frame
void TickFrame() {
std::unique_lock lock(ring_guard);
++retire_tick;
sentenced_fences.Reclaim(retire_tick > RETIRE_DELAY ? retire_tick - RETIRE_DELAY : 0);
delayed_destruction_ring.Tick();
}
// Unlike other fences, this one doesn't
@@ -187,7 +186,7 @@ private:
}
{
std::unique_lock lock(ring_guard);
sentenced_fences.Push(std::move(current_fence), retire_tick);
delayed_destruction_ring.Push(std::move(current_fence));
}
fences.pop();
}
@@ -220,7 +219,7 @@ private:
}
{
std::unique_lock lock(ring_guard);
sentenced_fences.Push(std::move(current_fence), retire_tick);
delayed_destruction_ring.Push(std::move(current_fence));
}
}
}
@@ -265,9 +264,7 @@ private:
std::jthread fence_thread;
static constexpr u64 RETIRE_DELAY = 8;
u64 retire_tick = 1;
DeferredDestructionQueue<TFence> sentenced_fences;
DelayedDestructionRing<TFence, 8> delayed_destruction_ring;
};
} // namespace VideoCommon
-1
View File
@@ -30,7 +30,6 @@ void ThreadManager::StartThread(VideoCore::RendererBase& renderer, Core::Fronten
thread = std::jthread([&](std::stop_token stop_token) {
Common::SetCurrentThreadName("GPU");
Common::SetCurrentThreadPriority(Common::ThreadPriority::Critical);
Common::SetCurrentThreadToPerformanceCores();
system.RegisterHostThread();
auto current_context = context.Acquire();
@@ -32,7 +32,6 @@ set(SHADER_FILES
${CMAKE_CURRENT_SOURCE_DIR}/convert_msaa_to_non_msaa.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_non_msaa_to_msaa.comp
${CMAKE_CURRENT_SOURCE_DIR}/convert_non_msaa_to_msaa.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_non_msaa_to_msaa_depth.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_s8d24_to_abgr8.frag
${CMAKE_CURRENT_SOURCE_DIR}/full_screen_triangle.vert
${CMAKE_CURRENT_SOURCE_DIR}/fxaa.frag
+16 -12
View File
@@ -77,14 +77,8 @@ uvec4 local_buff;
uvec4 color_endpoint_data;
int color_bitsread = 0;
// Global "vector" to be pushed into when decoding
// At most will require BLOCK_WIDTH x BLOCK_HEIGHT in single plane mode
// At most will require BLOCK_WIDTH x BLOCK_HEIGHT x 2 in dual plane mode
// So the maximum would be 144 (12 x 12) elements, x 2 for two planes
#define DIVCEIL(number, divisor) (number + divisor - 1) / divisor
#define ARRAY_NUM_ELEMENTS 144
#define VECTOR_ARRAY_SIZE DIVCEIL(ARRAY_NUM_ELEMENTS * 2, 4)
uint result_vector[ARRAY_NUM_ELEMENTS * 2];
#define MAX_WEIGHT_VALUES 64
uint result_vector[MAX_WEIGHT_VALUES];
int result_index = 0;
uint result_vector_max_index;
@@ -492,7 +486,7 @@ void DecodeColorValues(uvec4 modes, uint num_partitions, uint color_data_bits, o
A = ReplicateBitTo9((bitval & 1));
switch (encoding) {
case JUST_BITS:
color_values[++out_index] = FastReplicateTo8(bitval, bitlen);
color_values[out_index++] = FastReplicateTo8(bitval, bitlen);
break;
case TRIT: {
D = QuintTritValue(val);
@@ -571,7 +565,7 @@ void DecodeColorValues(uvec4 modes, uint num_partitions, uint color_data_bits, o
uint T = (D * C) + B;
T ^= A;
T = (A & 0x80) | (T >> 2);
color_values[++out_index] = T;
color_values[out_index++] = T;
}
}
}
@@ -753,12 +747,12 @@ void ComputeEndpoints(out uvec4 ep1, out uvec4 ep2, uint color_endpoint_mode, ui
#define READ_UINT_VALUES(N) \
uvec4 V[2]; \
for (uint i = 0; i < N; i++) { \
V[i / 4][i % 4] = color_values[++colvals_index]; \
V[i / 4][i % 4] = color_values[colvals_index++]; \
}
#define READ_INT_VALUES(N) \
ivec4 V[2]; \
for (uint i = 0; i < N; i++) { \
V[i / 4][i % 4] = int(color_values[++colvals_index]); \
V[i / 4][i % 4] = int(color_values[colvals_index++]); \
}
switch (color_endpoint_mode) {
@@ -1225,6 +1219,10 @@ void DecompressBlock(ivec3 coord) {
FillError(coord);
return;
}
if (GetNumWeightValues(size_params, dual_plane) > MAX_WEIGHT_VALUES) {
FillError(coord);
return;
}
uint partition_index = 1;
uvec4 color_endpoint_mode = uvec4(0);
uint ced_pointer = 0;
@@ -1239,6 +1237,10 @@ void DecompressBlock(ivec3 coord) {
const uint base_mode = base_cem & 3;
const uint max_weight = DecodeMaxWeight(mode);
const uint weight_bits = GetPackedBitSize(size_params, dual_plane, max_weight);
if (weight_bits < 24 || weight_bits > 96) {
FillError(coord);
return;
}
uint remaining_bits = 128 - weight_bits - total_bitsread;
uint extra_cem_bits = 0;
if (base_mode > 0) {
@@ -1253,6 +1255,7 @@ void DecompressBlock(ivec3 coord) {
extra_cem_bits += 8;
break;
default:
FillError(coord);
return;
}
}
@@ -1262,6 +1265,7 @@ void DecompressBlock(ivec3 coord) {
if (remaining_bits > 128) {
// Bad data, more remaining bits than 4 bytes
// return early
FillError(coord);
return;
}
// Read color data...
@@ -1,19 +0,0 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 450 core
layout(binding = 0) uniform sampler2D img_in;
layout(push_constant) uniform PushConstants {
ivec2 dst_offset;
ivec2 src_offset;
ivec2 scale;
};
void main() {
const ivec2 msaa_coord = ivec2(gl_FragCoord.xy) - dst_offset;
const ivec2 sample_offset = ivec2(gl_SampleID % scale.x, gl_SampleID / scale.x);
const ivec2 coord = msaa_coord * scale + sample_offset + src_offset;
gl_FragDepth = texelFetch(img_in, coord, 0).r;
}
+83 -79
View File
@@ -417,14 +417,6 @@ void HLE_TransformFeedbackSetup::Execute(Core::System& system, Engines::Maxwell3
default: return std::monostate{};
}
}
[[nodiscard]] inline bool CanBeHLEProgram(u64 hash) noexcept {
switch (hash) {
#define HLE_MACRO_ELEM(HASH, TY, VAL) case HASH: return true;
HLE_MACRO_LIST
#undef HLE_MACRO_ELEM
default: return false;
}
}
void MacroInterpreterImpl::Execute(Core::System& system, Engines::Maxwell3D& maxwell3d, std::span<const u32> params, u32 method) {
Reset();
@@ -1345,80 +1337,92 @@ static void Dump(u64 hash, std::span<const u32> code, bool decompiled = false) {
macro_file.write(reinterpret_cast<const char*>(code.data()), code.size_bytes());
}
void MacroEngine::Execute(Core::System& system, Engines::Maxwell3D& maxwell3d, u32 method, std::span<const u32> parameters) {
auto const execute_variant = [&system, &maxwell3d, &parameters, method](AnyCachedMacro& acm) {
if (auto a = std::get_if<HLE_DrawArraysIndirect>(&acm))
return a->Execute(system, maxwell3d, parameters, method);
if (auto a = std::get_if<HLE_DrawIndexedIndirect>(&acm))
return a->Execute(system, maxwell3d, parameters, method);
if (auto a = std::get_if<HLE_MultiDrawIndexedIndirectCount>(&acm))
return a->Execute(system, maxwell3d, parameters, method);
if (auto a = std::get_if<HLE_MultiLayerClear>(&acm))
return a->Execute(system, maxwell3d, parameters, method);
if (auto a = std::get_if<HLE_C713C83D8F63CCF3>(&acm))
return a->Execute(system, maxwell3d, parameters, method);
if (auto a = std::get_if<HLE_D7333D26E0A93EDE>(&acm))
return a->Execute(system, maxwell3d, parameters, method);
if (auto a = std::get_if<HLE_BindShader>(&acm))
return a->Execute(system, maxwell3d, parameters, method);
if (auto a = std::get_if<HLE_SetRasterBoundingBox>(&acm))
return a->Execute(system, maxwell3d, parameters, method);
if (auto a = std::get_if<HLE_ClearConstBuffer>(&acm))
return a->Execute(system, maxwell3d, parameters, method);
if (auto a = std::get_if<HLE_ClearMemory>(&acm))
return a->Execute(system, maxwell3d, parameters, method);
if (auto a = std::get_if<HLE_TransformFeedbackSetup>(&acm))
return a->Execute(system, maxwell3d, parameters, method);
if (auto a = std::get_if<HLE_DrawIndirectByteCount>(&acm))
return a->Execute(system, maxwell3d, parameters, method);
if (auto a = std::get_if<MacroInterpreterImpl>(&acm))
return a->Execute(system, maxwell3d, parameters, method);
if (auto a = std::get_if<std::unique_ptr<DynamicCachedMacro>>(&acm))
return a->get()->Execute(system, maxwell3d, parameters, method);
};
if (auto const it = macro_cache.find(method); it != macro_cache.end()) {
auto& ci = it->second;
if (!CanBeHLEProgram(ci.hash) || Settings::values.disable_macro_hle)
maxwell3d.RefreshParameters(); //LLE must reload parameters
execute_variant(ci.program);
} else {
// Macro not compiled, check if it's uploaded and if so, compile it
std::optional<u32> mid_method;
const auto macro_code = uploaded_macro_code.find(method);
if (macro_code == uploaded_macro_code.end()) {
for (const auto& [method_base, code] : uploaded_macro_code) {
if (method >= method_base && (method - method_base) < code.size()) {
mid_method = method_base;
break;
}
}
if (!mid_method.has_value()) {
ASSERT_MSG(false, "Macro 0x{0:x} was not uploaded", method);
return;
}
}
auto& ci = macro_cache[method];
if (mid_method) {
const auto& macro_cached = uploaded_macro_code[mid_method.value()];
const auto rebased_method = method - mid_method.value();
auto& code = uploaded_macro_code[method];
code.resize(macro_cached.size() - rebased_method);
std::memcpy(code.data(), macro_cached.data() + rebased_method, code.size() * sizeof(u32));
ci.hash = Common::HashValue(code);
ci.program = Compile(system, maxwell3d, code);
} else {
ci.program = Compile(system, maxwell3d, macro_code->second);
ci.hash = Common::HashValue(macro_code->second);
}
if (CanBeHLEProgram(ci.hash) && !Settings::values.disable_macro_hle) {
ci.program = GetHLEProgram(ci.hash);
} else {
void MacroEngine::Execute(Core::System& system, Engines::Maxwell3D& maxwell3d, u32 method,
std::span<const u32> parameters) {
const auto execute_variant = [&system, &maxwell3d, &parameters,
method](AnyCachedMacro& cached) {
if (std::holds_alternative<MacroInterpreterImpl>(cached) ||
std::holds_alternative<std::unique_ptr<DynamicCachedMacro>>(cached) ||
Settings::values.disable_macro_hle) {
maxwell3d.RefreshParameters();
}
execute_variant(ci.program);
if (Settings::values.dump_macros) {
Dump(ci.hash, macro_code->second, !std::holds_alternative<std::monostate>(ci.program));
if (auto program = std::get_if<HLE_DrawArraysIndirect>(&cached))
return program->Execute(system, maxwell3d, parameters, method);
if (auto program = std::get_if<HLE_DrawIndexedIndirect>(&cached))
return program->Execute(system, maxwell3d, parameters, method);
if (auto program = std::get_if<HLE_MultiDrawIndexedIndirectCount>(&cached))
return program->Execute(system, maxwell3d, parameters, method);
if (auto program = std::get_if<HLE_MultiLayerClear>(&cached))
return program->Execute(system, maxwell3d, parameters, method);
if (auto program = std::get_if<HLE_C713C83D8F63CCF3>(&cached))
return program->Execute(system, maxwell3d, parameters, method);
if (auto program = std::get_if<HLE_D7333D26E0A93EDE>(&cached))
return program->Execute(system, maxwell3d, parameters, method);
if (auto program = std::get_if<HLE_BindShader>(&cached))
return program->Execute(system, maxwell3d, parameters, method);
if (auto program = std::get_if<HLE_SetRasterBoundingBox>(&cached))
return program->Execute(system, maxwell3d, parameters, method);
if (auto program = std::get_if<HLE_ClearConstBuffer>(&cached))
return program->Execute(system, maxwell3d, parameters, method);
if (auto program = std::get_if<HLE_ClearMemory>(&cached))
return program->Execute(system, maxwell3d, parameters, method);
if (auto program = std::get_if<HLE_TransformFeedbackSetup>(&cached))
return program->Execute(system, maxwell3d, parameters, method);
if (auto program = std::get_if<HLE_DrawIndirectByteCount>(&cached))
return program->Execute(system, maxwell3d, parameters, method);
if (auto program = std::get_if<MacroInterpreterImpl>(&cached))
return program->Execute(system, maxwell3d, parameters, method);
if (auto program = std::get_if<std::unique_ptr<DynamicCachedMacro>>(&cached))
return program->get()->Execute(system, maxwell3d, parameters, method);
UNREACHABLE();
};
if (auto const it = macro_cache.find(method); it != macro_cache.end()) {
execute_variant(it->second.program);
return;
}
// Macro not compiled, check if it's uploaded and if so, compile it
std::span<const u32> code;
auto macro_code = uploaded_macro_code.find(method);
if (macro_code == uploaded_macro_code.end()) {
std::optional<u32> mid_method;
for (const auto& [method_base, uploaded_code] : uploaded_macro_code) {
if (method >= method_base && (method - method_base) < uploaded_code.size()) {
mid_method = method_base;
break;
}
}
if (!mid_method) {
ASSERT_MSG(false, "Macro 0x{0:x} was not uploaded", method);
return;
}
const auto source = uploaded_macro_code.find(*mid_method);
ASSERT(source != uploaded_macro_code.end());
const auto rebased_method = method - *mid_method;
std::vector<u32> rebased_code(source->second.begin() + rebased_method,
source->second.end());
const auto [it, inserted] = uploaded_macro_code.emplace(method, std::move(rebased_code));
ASSERT(inserted);
code = it->second;
} else {
code = macro_code->second;
}
auto& ci = macro_cache[method];
ci.hash = Common::HashRange(code.begin(), code.end());
if (!Settings::values.disable_macro_hle) {
ci.program = GetHLEProgram(ci.hash);
}
if (std::holds_alternative<std::monostate>(ci.program)) {
ci.program = Compile(system, maxwell3d, code);
}
execute_variant(ci.program);
if (Settings::values.dump_macros) {
Dump(ci.hash, code, !std::holds_alternative<std::monostate>(ci.program));
}
}
@@ -93,17 +93,7 @@ public:
void PostCopyBarrier();
void Finish();
void TickFrame(Common::SlotVector<Buffer>&) noexcept {
++sync_point;
}
u64 CurrentSyncPoint() const noexcept {
return sync_point;
}
u64 CompletedSyncPoint() const noexcept {
return sync_point > SYNC_POINT_DELAY ? sync_point - SYNC_POINT_DELAY : 0;
}
void TickFrame(Common::SlotVector<Buffer>&) noexcept {}
void ClearBuffer(Buffer& dest_buffer, u32 offset, size_t size, u32 value);
@@ -138,14 +128,6 @@ public:
u64 GetDeviceMemoryUsage() const;
u64 GetDeviceAllocationUsage() const {
return GetDeviceMemoryUsage();
}
bool CanReportAllocationUsage() const {
return device.CanReportMemoryUsage();
}
void BindFastUniformBuffer(size_t stage, u32 binding_index, u32 size) {
const GLuint handle = fast_uniforms[stage][binding_index].handle;
const GLsizeiptr gl_size = static_cast<GLsizeiptr>(size);
@@ -231,13 +213,9 @@ private:
GL_FRAGMENT_PROGRAM_PARAMETER_BUFFER_NV,
};
static constexpr u64 SYNC_POINT_DELAY = 8;
const Device& device;
StagingBufferPool& staging_buffer_pool;
u64 sync_point = 1;
bool has_fast_buffer_sub_data = false;
bool use_assembly_shaders = false;
bool has_unified_vertex_buffers = false;
@@ -87,14 +87,6 @@ public:
u64 GetDeviceMemoryUsage() const;
u64 GetDeviceAllocationUsage() const {
return GetDeviceMemoryUsage();
}
bool CanReportAllocationUsage() const {
return device.CanReportMemoryUsage();
}
bool CanReportMemoryUsage() const {
return device.CanReportMemoryUsage();
}
@@ -147,19 +139,7 @@ public:
bool HasNativeASTC() const noexcept;
void TickFrame() {
++sync_point;
}
u64 CurrentSyncPoint() const noexcept {
return sync_point;
}
u64 CompletedSyncPoint() const noexcept {
return sync_point > SYNC_POINT_DELAY ? sync_point - SYNC_POINT_DELAY : 0;
}
void WaitSyncPoint(u64) {}
void TickFrame() {}
StateTracker& GetStateTracker() {
return state_tracker;
@@ -194,9 +174,6 @@ private:
std::array<OGLFramebuffer, 4> rescale_read_fbos;
const Settings::ResolutionScalingInfo& resolution;
u64 device_access_memory;
static constexpr u64 SYNC_POINT_DELAY = 8;
u64 sync_point = 1;
};
class Image : public VideoCommon::ImageBase {
@@ -393,7 +370,6 @@ struct TextureCacheParams {
static constexpr bool HAS_EMULATED_COPIES = true;
static constexpr bool HAS_DEVICE_MEMORY_INFO = true;
static constexpr bool IMPLEMENTS_ASYNC_DOWNLOADS = true;
static constexpr bool HAS_TIMELINE_SYNC_POINTS = false;
using Runtime = OpenGL::TextureCacheRuntime;
using Image = OpenGL::Image;
+27 -91
View File
@@ -21,7 +21,6 @@
#include "video_core/host_shaders/convert_depth_to_float_frag_spv.h"
#include "video_core/host_shaders/convert_float_to_depth_frag_spv.h"
#include "video_core/host_shaders/convert_msaa_to_non_msaa_frag_spv.h"
#include "video_core/host_shaders/convert_non_msaa_to_msaa_depth_frag_spv.h"
#include "video_core/host_shaders/convert_non_msaa_to_msaa_frag_spv.h"
#include "video_core/host_shaders/convert_s8d24_to_abgr8_frag_spv.h"
#include "video_core/host_shaders/full_screen_triangle_vert_spv.h"
@@ -520,8 +519,7 @@ void RecordShaderReadBarrier(Scheduler& scheduler, const ImageView& image_view)
}
[[nodiscard]] vk::ImageView MakeMSAACopyView(const vk::Device& device, VkImage image,
VkFormat format, u32 base_level,
VkImageAspectFlags aspect_mask) {
VkFormat format, u32 base_level) {
return device.CreateImageView(VkImageViewCreateInfo{
.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO,
.pNext = nullptr,
@@ -536,7 +534,7 @@ void RecordShaderReadBarrier(Scheduler& scheduler, const ImageView& image_view)
.a = VK_COMPONENT_SWIZZLE_IDENTITY,
},
.subresourceRange{
.aspectMask = aspect_mask,
.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT,
.baseMipLevel = base_level,
.levelCount = 1,
.baseArrayLayer = 0,
@@ -612,8 +610,6 @@ BlitImageHelper::BlitImageHelper(const Device& device_, Scheduler& scheduler_,
convert_s8d24_to_abgr8_frag(BuildShader(device, CONVERT_S8D24_TO_ABGR8_FRAG_SPV)),
convert_msaa_to_non_msaa_frag(BuildShader(device, CONVERT_MSAA_TO_NON_MSAA_FRAG_SPV)),
convert_non_msaa_to_msaa_frag(BuildShader(device, CONVERT_NON_MSAA_TO_MSAA_FRAG_SPV)),
convert_non_msaa_to_msaa_depth_frag(
BuildShader(device, CONVERT_NON_MSAA_TO_MSAA_DEPTH_FRAG_SPV)),
linear_sampler(device.GetLogical().CreateSampler(SAMPLER_CREATE_INFO<VK_FILTER_LINEAR>)),
nearest_sampler(device.GetLogical().CreateSampler(SAMPLER_CREATE_INFO<VK_FILTER_NEAREST>)) {}
@@ -899,34 +895,16 @@ void BlitImageHelper::CopyMSAA(RenderPassCache& render_pass_cache, VkImage dst_i
const s32 scale_y = 1 << samples_y;
const VkSampleCountFlagBits samples =
msaa_to_non_msaa ? VK_SAMPLE_COUNT_1_BIT : SampleCountFlag(num_samples);
const auto dst_surface_type = VideoCore::Surface::GetFormatType(dst_format);
const bool is_depth = dst_surface_type == VideoCore::Surface::SurfaceType::Depth ||
dst_surface_type == VideoCore::Surface::SurfaceType::DepthStencil;
const bool has_stencil = dst_surface_type == VideoCore::Surface::SurfaceType::DepthStencil;
const VkImageAspectFlags view_aspect =
is_depth ? VK_IMAGE_ASPECT_DEPTH_BIT : VK_IMAGE_ASPECT_COLOR_BIT;
VkImageAspectFlags barrier_aspect = VK_IMAGE_ASPECT_COLOR_BIT;
if (is_depth) {
barrier_aspect = VK_IMAGE_ASPECT_DEPTH_BIT;
if (has_stencil) {
barrier_aspect |= VK_IMAGE_ASPECT_STENCIL_BIT;
}
}
RenderPassKey renderpass_key{};
renderpass_key.color_formats.fill(VideoCore::Surface::PixelFormat::Invalid);
if (is_depth) {
renderpass_key.depth_format = dst_format;
} else {
renderpass_key.color_formats[0] = dst_format;
renderpass_key.depth_format = VideoCore::Surface::PixelFormat::Invalid;
}
renderpass_key.color_formats[0] = dst_format;
renderpass_key.depth_format = VideoCore::Surface::PixelFormat::Invalid;
renderpass_key.samples = samples;
const VkRenderPass renderpass = render_pass_cache.Get(renderpass_key);
const MSAACopyPipelineKey key{
.renderpass = renderpass,
.samples = samples,
.msaa_to_non_msaa = msaa_to_non_msaa,
.is_depth = is_depth,
};
const VkPipeline pipeline = FindOrEmplaceMSAACopyPipeline(key);
const VkPipelineLayout layout = *msaa_copy_pipeline_layout;
@@ -942,10 +920,10 @@ void BlitImageHelper::CopyMSAA(RenderPassCache& render_pass_cache, VkImage dst_i
ASSERT(copy.dst_subresource.num_layers == 1);
vk::ImageView src_view =
MakeMSAACopyView(device.GetLogical(), src_image, src_vk_format,
static_cast<u32>(copy.src_subresource.base_level), view_aspect);
static_cast<u32>(copy.src_subresource.base_level));
vk::ImageView dst_view =
MakeMSAACopyView(device.GetLogical(), dst_image, dst_vk_format,
static_cast<u32>(copy.dst_subresource.base_level), view_aspect);
static_cast<u32>(copy.dst_subresource.base_level));
const VkOffset2D dst_offset{copy.dst_offset.x, copy.dst_offset.y};
const VkExtent2D dst_extent{copy.extent.width, copy.extent.height};
const VkRect2D render_area{
@@ -971,64 +949,50 @@ void BlitImageHelper::CopyMSAA(RenderPassCache& render_pass_cache, VkImage dst_i
scheduler.RequestOutsideRenderPassOperationContext();
scheduler.Record([this, pipeline, layout, sampler, renderpass,
framebuffer_handle = *framebuffer, src_view_handle = *src_view,
src = src_image, dst = dst_image, render_area, is_depth, barrier_aspect,
src = src_image, dst = dst_image, render_area,
push_constants](vk::CommandBuffer cmdbuf) {
const VkImageSubresourceRange src_range{
.aspectMask = barrier_aspect,
constexpr VkImageSubresourceRange color_range{
.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT,
.baseMipLevel = 0,
.levelCount = VK_REMAINING_MIP_LEVELS,
.baseArrayLayer = 0,
.layerCount = VK_REMAINING_ARRAY_LAYERS,
};
const VkImageSubresourceRange dst_range = src_range;
const VkAccessFlags attachment_read =
is_depth ? VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT
: VK_ACCESS_COLOR_ATTACHMENT_READ_BIT;
const VkAccessFlags attachment_write =
is_depth ? VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT
: VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT;
const VkPipelineStageFlags depth_stage =
VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT |
VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT;
const VkPipelineStageFlags attachment_stage =
is_depth ? depth_stage
: static_cast<VkPipelineStageFlags>(
VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT);
const std::array pre_barriers{
VkImageMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = attachment_write | VK_ACCESS_SHADER_WRITE_BIT |
VK_ACCESS_TRANSFER_WRITE_BIT,
.srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT |
VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_SHADER_READ_BIT,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = src,
.subresourceRange = src_range,
.subresourceRange = color_range,
},
VkImageMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = attachment_write | VK_ACCESS_SHADER_WRITE_BIT |
VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = attachment_read | attachment_write,
.srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT |
VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_COLOR_ATTACHMENT_READ_BIT |
VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst,
.subresourceRange = dst_range,
.subresourceRange = color_range,
},
};
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT |
VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT |
VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT |
VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT |
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT |
VK_PIPELINE_STAGE_TRANSFER_BIT,
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT | attachment_stage,
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT |
VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
0, nullptr, nullptr, pre_barriers);
const VkRenderPassBeginInfo renderpass_bi{
.sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO,
@@ -1061,16 +1025,16 @@ void BlitImageHelper::CopyMSAA(RenderPassCache& render_pass_cache, VkImage dst_i
const VkImageMemoryBarrier post_barrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = attachment_write,
.srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
.dstAccessMask = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_TRANSFER_READ_BIT,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst,
.subresourceRange = dst_range,
.subresourceRange = color_range,
};
cmdbuf.PipelineBarrier(attachment_stage,
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT |
VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT |
VK_PIPELINE_STAGE_TRANSFER_BIT,
@@ -1459,36 +1423,9 @@ VkPipeline BlitImageHelper::FindOrEmplaceMSAACopyPipeline(const MSAACopyPipeline
return *msaa_copy_pipelines[std::distance(msaa_copy_keys.begin(), it)];
}
msaa_copy_keys.push_back(key);
const VkShaderModule frag_module =
key.msaa_to_non_msaa
? *convert_msaa_to_non_msaa_frag
: (key.is_depth ? *convert_non_msaa_to_msaa_depth_frag
: *convert_non_msaa_to_msaa_frag);
const std::array stages = MakeStages(*clear_color_vert, frag_module);
const VkPipelineDepthStencilStateCreateInfo depth_stencil_ci{
.sType = VK_STRUCTURE_TYPE_PIPELINE_DEPTH_STENCIL_STATE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.depthTestEnable = VK_TRUE,
.depthWriteEnable = VK_TRUE,
.depthCompareOp = VK_COMPARE_OP_ALWAYS,
.depthBoundsTestEnable = VK_FALSE,
.stencilTestEnable = VK_FALSE,
.front = {},
.back = {},
.minDepthBounds = 0.0f,
.maxDepthBounds = 0.0f,
};
static constexpr VkPipelineColorBlendStateCreateInfo no_color_blend_ci{
.sType = VK_STRUCTURE_TYPE_PIPELINE_COLOR_BLEND_STATE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.logicOpEnable = VK_FALSE,
.logicOp = VK_LOGIC_OP_CLEAR,
.attachmentCount = 0,
.pAttachments = nullptr,
.blendConstants = {0.0f, 0.0f, 0.0f, 0.0f},
};
const std::array stages = MakeStages(*clear_color_vert, key.msaa_to_non_msaa
? *convert_msaa_to_non_msaa_frag
: *convert_non_msaa_to_msaa_frag);
const VkPipelineMultisampleStateCreateInfo multisample_ci{
.sType = VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO,
.pNext = nullptr,
@@ -1513,9 +1450,8 @@ VkPipeline BlitImageHelper::FindOrEmplaceMSAACopyPipeline(const MSAACopyPipeline
.pViewportState = &PIPELINE_VIEWPORT_STATE_CREATE_INFO,
.pRasterizationState = &PIPELINE_RASTERIZATION_STATE_CREATE_INFO,
.pMultisampleState = &multisample_ci,
.pDepthStencilState = key.is_depth ? &depth_stencil_ci : nullptr,
.pColorBlendState = key.is_depth ? &no_color_blend_ci
: &PIPELINE_COLOR_BLEND_STATE_GENERIC_CREATE_INFO,
.pDepthStencilState = nullptr,
.pColorBlendState = &PIPELINE_COLOR_BLEND_STATE_GENERIC_CREATE_INFO,
.pDynamicState = &PIPELINE_DYNAMIC_STATE_CREATE_INFO,
.layout = *msaa_copy_pipeline_layout,
.renderPass = key.renderpass,
@@ -51,7 +51,6 @@ struct MSAACopyPipelineKey {
VkRenderPass renderpass;
VkSampleCountFlagBits samples;
bool msaa_to_non_msaa;
bool is_depth;
};
struct BlitMSAAPipelineKey {
@@ -181,7 +180,6 @@ private:
vk::ShaderModule convert_s8d24_to_abgr8_frag;
vk::ShaderModule convert_msaa_to_non_msaa_frag;
vk::ShaderModule convert_non_msaa_to_msaa_frag;
vk::ShaderModule convert_non_msaa_to_msaa_depth_frag;
vk::Sampler linear_sampler;
vk::Sampler nearest_sampler;
@@ -164,9 +164,7 @@ void FixedPipelineState::Refresh(Tegra::Engines::Maxwell3D& maxwell3d, DynamicFe
}
provoking_vertex_last.Assign(use_last_provoking_vertex ? 1 : 0);
if (!features.has_dynamic_state3_conservative_raster_mode) {
conservative_raster_enable.Assign(regs.conservative_raster_enable != 0 ? 1 : 0);
}
conservative_raster_enable.Assign(regs.conservative_raster_enable != 0 ? 1 : 0);
smooth_lines.Assign(regs.line_anti_alias_enable != 0 ? 1 : 0);
alpha_to_coverage_enabled.Assign(regs.anti_alias_alpha_control.alpha_to_coverage != 0 ? 1 : 0);
alpha_to_one_enabled.Assign(regs.anti_alias_alpha_control.alpha_to_one != 0 ? 1 : 0);
@@ -362,35 +360,18 @@ void FixedPipelineState::DynamicState::Refresh2(const Maxwell& regs,
depth_bias_enable.Assign(enabled_lut[POLYGON_OFFSET_ENABLE_LUT[topology_index]] != 0 ? 1 : 0);
}
bool IsDepthClipEnabled(const Maxwell& regs) {
const auto clip = regs.viewport_clip_control.geometry_clip.Value();
return clip == Maxwell::ViewportClipControl::GeometryClip::Passthrough ||
clip == Maxwell::ViewportClipControl::GeometryClip::FrustumXYZ ||
clip == Maxwell::ViewportClipControl::GeometryClip::FrustumZ;
}
bool IsDepthClampEnabled(const Maxwell& regs, bool has_depth_clip_enable) {
if (!IsDepthClipEnabled(regs)) {
return true;
}
if (!has_depth_clip_enable) {
return false;
}
return regs.viewport_clip_control.pixel_min_z.Value() != 0 ||
regs.viewport_clip_control.pixel_max_z.Value() != 0;
}
void FixedPipelineState::DynamicState::Refresh3(const Maxwell& regs,
const DynamicFeatures& features) {
if (!features.has_dynamic_state3_logic_op_enable) {
logic_op_enable.Assign(regs.logic_op.enable != 0 ? 1 : 0);
}
if (features.has_depth_clip_enable) {
depth_clip_disabled.Assign(IsDepthClipEnabled(regs) ? 0 : 1);
}
if (!features.has_dynamic_state3_depth_clamp_enable) {
depth_clamp_disabled.Assign(
IsDepthClampEnabled(regs, features.has_depth_clip_enable) ? 0 : 1);
depth_clamp_disabled.Assign(regs.viewport_clip_control.geometry_clip ==
Maxwell::ViewportClipControl::GeometryClip::Passthrough ||
regs.viewport_clip_control.geometry_clip ==
Maxwell::ViewportClipControl::GeometryClip::FrustumXYZ ||
regs.viewport_clip_control.geometry_clip ==
Maxwell::ViewportClipControl::GeometryClip::FrustumZ);
}
if (!features.has_dynamic_state3_line_stipple_enable) {
line_stipple_enable.Assign(regs.line_stipple_enable);
@@ -30,8 +30,6 @@ struct DynamicFeatures {
bool has_extended_dynamic_state_3_blend;
bool has_extended_dynamic_state_3_enables;
bool has_dynamic_state3_depth_clamp_enable;
bool has_dynamic_state3_conservative_raster_mode;
bool has_depth_clip_enable;
bool has_dynamic_state3_logic_op_enable;
bool has_dynamic_state3_line_stipple_enable;
bool has_dynamic_vertex_input;
@@ -167,7 +165,6 @@ struct FixedPipelineState {
BitField<10, 1, u32> logic_op_enable;
BitField<11, 1, u32> depth_clamp_disabled;
BitField<12, 1, u32> line_stipple_enable;
BitField<13, 1, u32> depth_clip_disabled;
};
union {
u32 raw2;
@@ -301,9 +298,6 @@ static_assert(std::has_unique_object_representations_v<FixedPipelineState>);
static_assert(std::is_trivially_copyable_v<FixedPipelineState>);
static_assert(std::is_trivially_constructible_v<FixedPipelineState>);
bool IsDepthClipEnabled(const Maxwell& regs);
bool IsDepthClampEnabled(const Maxwell& regs, bool has_depth_clip_enable);
} // namespace Vulkan
namespace std {
@@ -47,6 +47,93 @@ using Shader::Backend::SPIRV::NUM_TEXTURE_AND_IMAGE_SCALING_WORDS;
return std::nullopt;
}
[[nodiscard]] inline VkDeviceSize DescriptorSizeForType(const Device& device,
VkDescriptorType type) {
const auto& props = device.DescriptorBufferProperties();
const bool robust = device.IsRobustBufferAccessEnabled();
switch (type) {
case VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER:
return robust ? props.robustUniformBufferDescriptorSize : props.uniformBufferDescriptorSize;
case VK_DESCRIPTOR_TYPE_STORAGE_BUFFER:
return robust ? props.robustStorageBufferDescriptorSize : props.storageBufferDescriptorSize;
case VK_DESCRIPTOR_TYPE_UNIFORM_TEXEL_BUFFER:
return robust ? props.robustUniformTexelBufferDescriptorSize
: props.uniformTexelBufferDescriptorSize;
case VK_DESCRIPTOR_TYPE_STORAGE_TEXEL_BUFFER:
return robust ? props.robustStorageTexelBufferDescriptorSize
: props.storageTexelBufferDescriptorSize;
case VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER:
return props.combinedImageSamplerDescriptorSize;
case VK_DESCRIPTOR_TYPE_STORAGE_IMAGE:
return props.storageImageDescriptorSize;
default:
return 0;
}
}
struct DescriptorBufferBinding {
VkDescriptorType type;
u32 count;
VkDeviceSize offset;
VkDeviceSize stride;
};
struct DescriptorBufferLayout {
VkDeviceSize size{};
boost::container::small_vector<DescriptorBufferBinding, 32> bindings;
[[nodiscard]] bool Empty() const noexcept {
return bindings.empty();
}
};
inline void WriteDescriptorBuffer(const Device& device, const DescriptorBufferLayout& layout,
const DescriptorUpdateEntry* payload, u8* host) {
const vk::Device& dev = device.GetLogical();
for (const DescriptorBufferBinding& binding : layout.bindings) {
for (u32 index = 0; index < binding.count; ++index) {
const DescriptorUpdateEntry& entry = *(payload++);
const VkDescriptorAddressInfoEXT address_info{
.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_ADDRESS_INFO_EXT,
.pNext = nullptr,
.address = entry.address.address,
.range = entry.address.range,
.format = entry.address.format,
};
VkDescriptorGetInfoEXT get_info{
.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_GET_INFO_EXT,
.pNext = nullptr,
.type = binding.type,
.data{},
};
switch (binding.type) {
case VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER:
get_info.data.pUniformBuffer = &address_info;
break;
case VK_DESCRIPTOR_TYPE_STORAGE_BUFFER:
get_info.data.pStorageBuffer = &address_info;
break;
case VK_DESCRIPTOR_TYPE_UNIFORM_TEXEL_BUFFER:
get_info.data.pUniformTexelBuffer = &address_info;
break;
case VK_DESCRIPTOR_TYPE_STORAGE_TEXEL_BUFFER:
get_info.data.pStorageTexelBuffer = &address_info;
break;
case VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER:
get_info.data.pCombinedImageSampler = &entry.image;
break;
case VK_DESCRIPTOR_TYPE_STORAGE_IMAGE:
get_info.data.pStorageImage = &entry.image;
break;
default:
continue;
}
dev.GetDescriptorEXT(get_info, binding.stride,
host + binding.offset + index * binding.stride);
}
}
}
[[nodiscard]] inline u32 NumDescriptorEntries(const Shader::Info& info) {
return Shader::NumDescriptors(info.constant_buffer_descriptors) +
Shader::NumDescriptors(info.storage_buffers_descriptors) +
@@ -61,20 +148,70 @@ public:
DescriptorLayoutBuilder(const Device& device_) : device{&device_} {}
bool CanUsePushDescriptor() const noexcept {
return device->IsKhrPushDescriptorSupported() &&
num_descriptors <= device->MaxPushDescriptors();
if (!device->IsKhrPushDescriptorSupported() ||
num_descriptors > device->MaxPushDescriptors()) {
return false;
}
return !device->IsExtDescriptorBufferSupported() ||
device->DescriptorBufferProperties().bufferlessPushDescriptors;
}
// TODO(crueter): utilize layout binding flags
vk::DescriptorSetLayout CreateDescriptorSetLayout(bool use_push_descriptor) const {
bool CanUseDescriptorBuffer() const noexcept {
const auto& props = device->DescriptorBufferProperties();
if (!device->IsExtDescriptorBufferSupported() || bindings.empty() ||
!props.combinedImageSamplerDescriptorSingleArray) {
return false;
}
return !props.bufferlessPushDescriptors || !CanUsePushDescriptor();
}
DescriptorBufferLayout MakeDescriptorBufferLayout(VkDescriptorSetLayout layout) const {
DescriptorBufferLayout result;
if (!layout) {
return result;
}
const vk::Device& dev = device->GetLogical();
result.size = dev.GetDescriptorSetLayoutSizeEXT(layout);
result.bindings.reserve(bindings.size());
for (const VkDescriptorSetLayoutBinding& entry : bindings) {
result.bindings.push_back(DescriptorBufferBinding{
.type = entry.descriptorType,
.count = entry.descriptorCount,
.offset = dev.GetDescriptorSetLayoutBindingOffsetEXT(layout, entry.binding),
.stride = DescriptorSizeForType(*device, entry.descriptorType),
});
}
return result;
}
vk::DescriptorSetLayout CreateDescriptorSetLayout(bool use_push_descriptor,
bool use_descriptor_buffer = false) const {
if (bindings.empty()) {
return nullptr;
}
const VkDescriptorSetLayoutCreateFlags flags =
use_push_descriptor ? VK_DESCRIPTOR_SET_LAYOUT_CREATE_PUSH_DESCRIPTOR_BIT_KHR : 0;
VkDescriptorSetLayoutCreateFlags flags = 0;
if (use_push_descriptor) {
flags |= VK_DESCRIPTOR_SET_LAYOUT_CREATE_PUSH_DESCRIPTOR_BIT_KHR;
}
if (use_descriptor_buffer) {
flags |= VK_DESCRIPTOR_SET_LAYOUT_CREATE_DESCRIPTOR_BUFFER_BIT_EXT;
}
boost::container::small_vector<VkDescriptorBindingFlags, 32> binding_flags;
VkDescriptorSetLayoutBindingFlagsCreateInfo binding_flags_ci{};
const void* pnext = nullptr;
if (!use_push_descriptor && device->IsDescriptorBindingPartiallyBoundSupported()) {
binding_flags.assign(bindings.size(), VK_DESCRIPTOR_BINDING_PARTIALLY_BOUND_BIT);
binding_flags_ci = {
.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_BINDING_FLAGS_CREATE_INFO,
.pNext = nullptr,
.bindingCount = static_cast<u32>(binding_flags.size()),
.pBindingFlags = binding_flags.data(),
};
pnext = &binding_flags_ci;
}
return device->GetLogical().CreateDescriptorSetLayout({
.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO,
.pNext = nullptr,
.pNext = pnext,
.flags = flags,
.bindingCount = static_cast<u32>(bindings.size()),
.pBindings = bindings.data(),
@@ -69,6 +69,9 @@ vk::Buffer CreateBuffer(const Device& device, const MemoryAllocator& memory_allo
if (device.IsExtConditionalRendering()) {
flags |= VK_BUFFER_USAGE_CONDITIONAL_RENDERING_BIT_EXT;
}
if (device.IsBufferDeviceAddressSupported()) {
flags |= VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT;
}
const VkBufferCreateInfo buffer_ci = {
.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO,
.pNext = nullptr,
@@ -91,6 +94,9 @@ Buffer::Buffer(BufferCacheRuntime& runtime, VideoCommon::NullBufferParams null_p
device = &runtime.device;
buffer = runtime.CreateNullBuffer();
is_null = true;
if (device->IsBufferDeviceAddressSupported()) {
device_address = device->GetLogical().GetBufferDeviceAddress(*buffer);
}
}
Buffer::Buffer(BufferCacheRuntime& runtime, DAddr cpu_addr_, u64 size_bytes_)
@@ -100,6 +106,9 @@ Buffer::Buffer(BufferCacheRuntime& runtime, DAddr cpu_addr_, u64 size_bytes_)
if (runtime.device.HasDebuggingToolAttached()) {
buffer.SetObjectNameEXT(fmt::format("Buffer {:#x}", CpuAddr()).c_str());
}
if (device->IsBufferDeviceAddressSupported()) {
device_address = device->GetLogical().GetBufferDeviceAddress(*buffer);
}
}
void Buffer::MarkUsage(u64 offset, u64 size) noexcept {
@@ -246,6 +255,7 @@ protected:
StagingBufferPool& staging_pool;
vk::Buffer buffer{};
MemoryCommit memory_commit{};
VkIndexType index_type{};
u32 num_indices = 0;
};
@@ -363,6 +373,10 @@ StagingBufferRef BufferCacheRuntime::DownloadStagingBuffer(size_t size, bool def
return staging_pool.Request(size, MemoryUsage::Download, deferred);
}
VkFormat BufferCacheRuntime::TexelBufferFormat(VideoCore::Surface::PixelFormat format) const {
return MaxwellToVK::SurfaceFormat(device, FormatType::Buffer, false, format).format;
}
void BufferCacheRuntime::FreeDeferredStagingBuffer(StagingBufferRef& ref) {
staging_pool.FreeDeferred(ref);
}
@@ -375,10 +389,6 @@ u64 BufferCacheRuntime::GetDeviceMemoryUsage() const {
return device.GetDeviceMemoryUsage();
}
u64 BufferCacheRuntime::GetDeviceAllocationUsage() const {
return device.GetMemoryBudgetInfo().allocation_bytes;
}
bool BufferCacheRuntime::CanReportMemoryUsage() const {
return device.CanReportMemoryUsage();
}
@@ -407,16 +417,6 @@ u64 BufferCacheRuntime::KnownGpuTick() {
return scheduler.GetMasterSemaphore().KnownGpuTick();
}
u64 BufferCacheRuntime::CurrentSyncPoint() const noexcept {
return scheduler.GetMasterSemaphore().CurrentTick();
}
u64 BufferCacheRuntime::CompletedSyncPoint() const {
auto& master_semaphore = scheduler.GetMasterSemaphore();
master_semaphore.Refresh();
return master_semaphore.KnownGpuTick();
}
void BufferCacheRuntime::Wait(u64 buffer_tick) {
scheduler.Wait(buffer_tick);
}
@@ -654,7 +654,6 @@ void BufferCacheRuntime::BindTransformFeedbackBuffer(u32 index, VkBuffer buffer,
offset = 0;
size = 0;
}
scheduler.MarkTransformFeedbackUsed();
scheduler.Record([index, buffer, offset, size](vk::CommandBuffer cmdbuf) {
const VkDeviceSize vk_offset = offset;
const VkDeviceSize vk_size = size;
@@ -667,26 +666,19 @@ void BufferCacheRuntime::BindTransformFeedbackBuffers(VideoCommon::HostBindings<
// Already logged in the rasterizer
return;
}
const u32 count = std::min<u32>(static_cast<u32>(bindings.buffers.size()),
VideoCommon::NUM_TRANSFORM_FEEDBACK_BUFFERS);
std::array<VkBuffer, VideoCommon::NUM_TRANSFORM_FEEDBACK_BUFFERS> handles{};
std::array<VkDeviceSize, VideoCommon::NUM_TRANSFORM_FEEDBACK_BUFFERS> offsets{};
std::array<VkDeviceSize, VideoCommon::NUM_TRANSFORM_FEEDBACK_BUFFERS> sizes{};
for (u32 i = 0; i < count; ++i) {
boost::container::static_vector<VkBuffer, VideoCommon::NUM_VERTEX_BUFFERS> buffer_handles(bindings.buffers.size());
for (u32 i = 0; i < bindings.buffers.size(); ++i) {
auto handle = bindings.buffers[i]->Handle();
if (handle == VK_NULL_HANDLE) {
ReserveNullBuffer();
handle = *null_buffer;
} else {
offsets[i] = bindings.offsets[i];
sizes[i] = bindings.sizes[i];
bindings.offsets[i] = 0;
bindings.sizes[i] = 0;
}
handles[i] = handle;
buffer_handles[i] = handle;
}
scheduler.MarkTransformFeedbackUsed();
scheduler.Record([count, handles, offsets, sizes](vk::CommandBuffer cmdbuf) {
cmdbuf.BindTransformFeedbackBuffersEXT(0, count, handles.data(), offsets.data(),
sizes.data());
scheduler.Record([bindings_ = std::move(bindings), buffer_handles_ = std::move(buffer_handles)](vk::CommandBuffer cmdbuf) {
cmdbuf.BindTransformFeedbackBuffersEXT(0, u32(buffer_handles_.size()), buffer_handles_.data(), bindings_.offsets.data(), bindings_.sizes.data());
});
}
@@ -711,6 +703,9 @@ vk::Buffer BufferCacheRuntime::CreateNullBuffer() {
if (device.IsExtTransformFeedbackSupported()) {
create_info.usage |= VK_BUFFER_USAGE_TRANSFORM_FEEDBACK_BUFFER_BIT_EXT;
}
if (device.IsBufferDeviceAddressSupported()) {
create_info.usage |= VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT;
}
vk::Buffer ret = memory_allocator.CreateBuffer(create_info, MemoryUsage::DeviceLocal);
if (device.HasDebuggingToolAttached()) {
ret.SetObjectNameEXT("Null buffer");
@@ -39,6 +39,10 @@ public:
return *buffer;
}
[[nodiscard]] VkDeviceAddress DeviceAddress() const noexcept {
return device_address;
}
[[nodiscard]] bool IsRegionUsed(u64 offset, u64 size) const noexcept {
return tracker.IsUsed(offset, size);
}
@@ -70,6 +74,7 @@ private:
vk::Buffer buffer;
std::vector<BufferView> views;
VideoCommon::UsageTracker tracker;
VkDeviceAddress device_address{};
u64 last_usage_tick{};
bool is_null{};
};
@@ -100,20 +105,10 @@ public:
void Finish();
u64 CurrentSyncPoint() const noexcept;
u64 CompletedSyncPoint() const;
u64 GetDeviceLocalMemory() const;
u64 GetDeviceMemoryUsage() const;
u64 GetDeviceAllocationUsage() const;
bool CanReportAllocationUsage() const noexcept {
return true;
}
bool CanReportMemoryUsage() const;
u32 GetUniformBufferAlignment() const;
@@ -155,22 +150,25 @@ public:
[[maybe_unused]] u32 binding_index,
u32 size) {
const StagingBufferRef ref = staging_pool.Request(size, MemoryUsage::Upload);
BindBuffer(ref.buffer, static_cast<u32>(ref.offset), size);
guest_descriptor_queue.AddBuffer(ref.buffer, ref.device_address,
static_cast<u32>(ref.offset), size);
return ref.mapped_span;
}
void BindUniformBuffer(VkBuffer buffer, u32 offset, u32 size) {
void BindUniformBuffer(const Buffer& buffer, u32 offset, u32 size) {
BindBuffer(buffer, offset, size);
}
void BindStorageBuffer(VkBuffer buffer, u32 offset, u32 size,
void BindStorageBuffer(const Buffer& buffer, u32 offset, u32 size,
[[maybe_unused]] bool is_written) {
BindBuffer(buffer, offset, size);
}
void BindTextureBuffer(Buffer& buffer, u32 offset, u32 size,
VideoCore::Surface::PixelFormat format) {
guest_descriptor_queue.AddTexelBuffer(buffer.View(offset, size, format));
guest_descriptor_queue.AddTexelBuffer(buffer.View(offset, size, format),
buffer.DeviceAddress(), offset, size,
TexelBufferFormat(format));
}
bool ShouldLimitDynamicStorageBuffers() const {
@@ -182,14 +180,17 @@ public:
}
private:
void BindBuffer(VkBuffer buffer, u32 offset, u32 size) {
if (buffer == VK_NULL_HANDLE) {
guest_descriptor_queue.AddBuffer(buffer, 0, VK_WHOLE_SIZE);
void BindBuffer(const Buffer& buffer, u32 offset, u32 size) {
const VkBuffer handle = buffer.Handle();
if (handle == VK_NULL_HANDLE) {
guest_descriptor_queue.AddBuffer(handle, 0, 0, VK_WHOLE_SIZE);
} else {
guest_descriptor_queue.AddBuffer(buffer, offset, size);
guest_descriptor_queue.AddBuffer(handle, buffer.DeviceAddress(), offset, size);
}
}
VkFormat TexelBufferFormat(VideoCore::Surface::PixelFormat format) const;
void ReserveNullBuffer();
vk::Buffer CreateNullBuffer();
@@ -34,12 +34,14 @@ using Tegra::Texture::TexturePair;
ComputePipeline::ComputePipeline(const Device& device_, Scheduler& scheduler, vk::PipelineCache& pipeline_cache_,
DescriptorPool& descriptor_pool,
GuestDescriptorQueue& guest_descriptor_queue_,
DescriptorBufferRing& descriptor_buffer_ring_,
Common::ThreadWorker* thread_worker,
PipelineStatistics* pipeline_statistics,
VideoCore::ShaderNotify* shader_notify, const Shader::Info& info_,
vk::ShaderModule spv_module_, u64 shader_hash_)
: device{device_},
pipeline_cache(pipeline_cache_), guest_descriptor_queue{guest_descriptor_queue_}, info{info_},
pipeline_cache(pipeline_cache_), guest_descriptor_queue{guest_descriptor_queue_},
descriptor_buffer_ring{descriptor_buffer_ring_}, info{info_},
shader_hash{shader_hash_}, spv_module(std::move(spv_module_)) {
if (shader_notify) {
shader_notify->MarkShaderBuilding();
@@ -48,18 +50,36 @@ ComputePipeline::ComputePipeline(const Device& device_, Scheduler& scheduler, vk
uniform_buffer_sizes.begin());
num_descriptor_entries = NumDescriptorEntries(info);
auto func{[this, &scheduler, &descriptor_pool, shader_notify, pipeline_statistics] {
DescriptorLayoutBuilder builder{device};
builder.Add(info, VK_SHADER_STAGE_COMPUTE_BIT);
DescriptorLayoutBuilder builder{device};
builder.Add(info, VK_SHADER_STAGE_COMPUTE_BIT);
uses_push_descriptor = builder.CanUsePushDescriptor();
descriptor_set_layout = builder.CreateDescriptorSetLayout(uses_push_descriptor);
pipeline_layout = builder.CreatePipelineLayout(*descriptor_set_layout);
uses_push_descriptor = builder.CanUsePushDescriptor();
uses_descriptor_buffer = builder.CanUseDescriptorBuffer() && descriptor_buffer_ring.IsValid();
descriptor_set_layout =
builder.CreateDescriptorSetLayout(uses_push_descriptor, uses_descriptor_buffer);
if (uses_descriptor_buffer) {
descriptor_buffer_layout = builder.MakeDescriptorBufferLayout(*descriptor_set_layout);
if (!descriptor_buffer_ring.CanAllocate(descriptor_buffer_layout.size)) {
LOG_DEBUG(Render_Vulkan,
"Compute shader {:016X} needs {} descriptor bytes per dispatch, falling "
"back to sets",
shader_hash, descriptor_buffer_layout.size);
uses_descriptor_buffer = false;
descriptor_buffer_layout = {};
descriptor_set_layout = builder.CreateDescriptorSetLayout(false);
}
}
pipeline_layout = builder.CreatePipelineLayout(*descriptor_set_layout);
if (!uses_descriptor_buffer) {
descriptor_update_template =
builder.CreateTemplate(*descriptor_set_layout, *pipeline_layout, uses_push_descriptor);
if (!uses_push_descriptor) {
descriptor_allocator = descriptor_pool.Allocator(device, scheduler, *descriptor_set_layout, info);
descriptor_allocator =
descriptor_pool.Allocator(device, scheduler, *descriptor_set_layout, info);
}
}
auto func{[this, shader_notify, pipeline_statistics] {
const VkPipelineShaderStageRequiredSubgroupSizeCreateInfoEXT subgroup_size_ci{
.sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_REQUIRED_SUBGROUP_SIZE_CREATE_INFO_EXT,
.pNext = nullptr,
@@ -69,6 +89,9 @@ ComputePipeline::ComputePipeline(const Device& device_, Scheduler& scheduler, vk
if (device.IsKhrPipelineExecutablePropertiesEnabled() && Settings::values.renderer_debug.GetValue()) {
flags |= VK_PIPELINE_CREATE_CAPTURE_STATISTICS_BIT_KHR;
}
if (uses_descriptor_buffer) {
flags |= VK_PIPELINE_CREATE_DESCRIPTOR_BUFFER_BIT_EXT;
}
const VkComputePipelineCreateInfo compute_ci{
.sType = VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO,
.pNext = nullptr,
@@ -125,10 +148,10 @@ ComputePipeline::ComputePipeline(const Device& device_, Scheduler& scheduler, vk
}
}
void ComputePipeline::Configure(Tegra::Engines::KeplerCompute& kepler_compute,
bool ComputePipeline::Configure(Tegra::Engines::KeplerCompute& kepler_compute,
Tegra::MemoryManager& gpu_memory, Scheduler& scheduler,
BufferCache& buffer_cache, TextureCache& texture_cache) {
guest_descriptor_queue.Acquire(scheduler, num_descriptor_entries);
guest_descriptor_queue.Acquire(scheduler, num_descriptor_entries, uses_descriptor_buffer);
buffer_cache.SetComputeUniformBufferState(info.constant_buffer_mask, &uniform_buffer_sizes);
buffer_cache.UnbindComputeStorageBuffers();
@@ -249,10 +272,33 @@ void ComputePipeline::Configure(Tegra::Engines::KeplerCompute& kepler_compute,
GPU::Logging::GPULogger::GetInstance().LogPipelineBind(true, "compute pipeline");
}
const void* const descriptor_data{guest_descriptor_queue.UpdateData()};
const DescriptorUpdateEntry* const descriptor_data{guest_descriptor_queue.UpdateData()};
VkDeviceSize descriptor_buffer_offset{};
u32 descriptor_buffer_chunk{};
if (uses_descriptor_buffer) {
const DescriptorBufferRing::Allocation alloc{
descriptor_buffer_ring.Allocate(scheduler, descriptor_buffer_layout.size)};
if (!alloc.host) {
LOG_DEBUG(Render_Vulkan, "Failed to reserve descriptor memory, skipping dispatch");
return false;
}
WriteDescriptorBuffer(device, descriptor_buffer_layout, descriptor_data, alloc.host);
descriptor_buffer_offset = alloc.offset;
descriptor_buffer_chunk = alloc.chunk;
}
const bool bind_descriptor_buffer{
uses_descriptor_buffer && scheduler.UpdateDescriptorBufferChunk(descriptor_buffer_chunk)};
const bool is_rescaling = !info.texture_descriptors.empty() || !info.image_descriptors.empty();
scheduler.Record([this, descriptor_data, is_rescaling,
scheduler.Record([this, descriptor_data, is_rescaling, descriptor_buffer_offset,
descriptor_buffer_chunk, bind_descriptor_buffer,
rescaling_data = rescaling.Data()](vk::CommandBuffer cmdbuf) {
if (bind_descriptor_buffer) {
const VkDescriptorBufferBindingInfoEXT binding_info{
descriptor_buffer_ring.BindingInfo(descriptor_buffer_chunk)};
cmdbuf.BindDescriptorBuffersEXT(binding_info);
}
if (!pipeline) {
return;
}
@@ -265,7 +311,11 @@ void ComputePipeline::Configure(Tegra::Engines::KeplerCompute& kepler_compute,
RESCALING_LAYOUT_WORDS_OFFSET, sizeof(rescaling_data),
rescaling_data.data());
}
if (uses_push_descriptor) {
if (uses_descriptor_buffer) {
const u32 buffer_index{};
cmdbuf.SetDescriptorBufferOffsetsEXT(VK_PIPELINE_BIND_POINT_COMPUTE, *pipeline_layout,
0, buffer_index, descriptor_buffer_offset);
} else if (uses_push_descriptor) {
cmdbuf.PushDescriptorSetWithTemplateKHR(*descriptor_update_template, *pipeline_layout,
0, descriptor_data);
} else {
@@ -276,6 +326,7 @@ void ComputePipeline::Configure(Tegra::Engines::KeplerCompute& kepler_compute,
descriptor_set, nullptr);
}
});
return true;
}
} // namespace Vulkan
@@ -13,7 +13,9 @@
#include "common/common_types.h"
#include "common/thread_worker.h"
#include "shader_recompiler/shader_info.h"
#include "video_core/renderer_vulkan/pipeline_helper.h"
#include "video_core/renderer_vulkan/vk_buffer_cache.h"
#include "video_core/renderer_vulkan/vk_descriptor_buffer.h"
#include "video_core/renderer_vulkan/vk_descriptor_pool.h"
#include "video_core/renderer_vulkan/vk_texture_cache.h"
#include "video_core/renderer_vulkan/vk_update_descriptor.h"
@@ -34,6 +36,7 @@ public:
explicit ComputePipeline(const Device& device, Scheduler& scheduler, vk::PipelineCache& pipeline_cache,
DescriptorPool& descriptor_pool,
GuestDescriptorQueue& guest_descriptor_queue,
DescriptorBufferRing& descriptor_buffer_ring,
Common::ThreadWorker* thread_worker,
PipelineStatistics* pipeline_statistics,
VideoCore::ShaderNotify* shader_notify, const Shader::Info& info,
@@ -45,8 +48,9 @@ public:
ComputePipeline& operator=(const ComputePipeline&) = delete;
ComputePipeline(const ComputePipeline&) = delete;
void Configure(Tegra::Engines::KeplerCompute& kepler_compute, Tegra::MemoryManager& gpu_memory,
Scheduler& scheduler, BufferCache& buffer_cache, TextureCache& texture_cache);
[[nodiscard]] bool Configure(Tegra::Engines::KeplerCompute& kepler_compute,
Tegra::MemoryManager& gpu_memory, Scheduler& scheduler,
BufferCache& buffer_cache, TextureCache& texture_cache);
bool IsBound() const noexcept {
return static_cast<bool>(pipeline);
@@ -56,6 +60,7 @@ private:
const Device& device;
vk::PipelineCache& pipeline_cache;
GuestDescriptorQueue& guest_descriptor_queue;
DescriptorBufferRing& descriptor_buffer_ring;
Shader::Info info;
u64 shader_hash{};
u32 num_descriptor_entries{};
@@ -65,6 +70,8 @@ private:
vk::ShaderModule spv_module;
vk::DescriptorSetLayout descriptor_set_layout;
bool uses_push_descriptor{false};
bool uses_descriptor_buffer{false};
DescriptorBufferLayout descriptor_buffer_layout;
DescriptorAllocator descriptor_allocator;
vk::PipelineLayout pipeline_layout;
vk::DescriptorUpdateTemplate descriptor_update_template;
@@ -0,0 +1,141 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#include <algorithm>
#include "common/alignment.h"
#include "common/assert.h"
#include "common/logging.h"
#include "video_core/renderer_vulkan/vk_descriptor_buffer.h"
#include "video_core/renderer_vulkan/vk_scheduler.h"
#include "video_core/vulkan_common/vulkan_device.h"
namespace Vulkan {
DescriptorBufferRing::DescriptorBufferRing(const Device& device_,
MemoryAllocator& memory_allocator)
: device{device_} {
if (!device.IsExtDescriptorBufferSupported() || !device.IsBufferDeviceAddressSupported()) {
return;
}
const VkPhysicalDeviceDescriptorBufferPropertiesEXT& props{device.DescriptorBufferProperties()};
alignment = std::max<VkDeviceSize>(props.descriptorBufferOffsetAlignment, 1);
const VkDeviceSize max_bound{(std::min)({props.maxSamplerDescriptorBufferRange,
props.maxResourceDescriptorBufferRange,
props.samplerDescriptorBufferAddressSpaceSize,
props.resourceDescriptorBufferAddressSpaceSize,
props.descriptorBufferAddressSpaceSize})};
const VkDeviceSize frame_size{device.IsTiler() ? TILER_FRAME_SIZE : DESKTOP_FRAME_SIZE};
const VkDeviceSize chunk_size{
Common::AlignDown((std::min)(frame_size, max_bound), alignment)};
if (chunk_size <= alignment) {
LOG_DEBUG(Render_Vulkan, "Descriptor buffer binding limit of {} is unusable, disabling",
max_bound);
return;
}
chunk_capacity = chunk_size - alignment;
chunks_per_frame = static_cast<size_t>(frame_size / chunk_size);
const VkBufferCreateInfo buffer_ci{
.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.size = chunk_size,
.usage = VK_BUFFER_USAGE_RESOURCE_DESCRIPTOR_BUFFER_BIT_EXT |
VK_BUFFER_USAGE_SAMPLER_DESCRIPTOR_BUFFER_BIT_EXT |
VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT,
.sharingMode = VK_SHARING_MODE_EXCLUSIVE,
.queueFamilyIndexCount = 0,
.pQueueFamilyIndices = nullptr,
};
const size_t total_chunks{chunks_per_frame * FRAMES_IN_FLIGHT};
chunks.reserve(total_chunks);
chunk_addresses.reserve(total_chunks);
chunk_hosts.reserve(total_chunks);
for (size_t index = 0; index < total_chunks; ++index) {
vk::Buffer buffer{memory_allocator.CreateBuffer(buffer_ci, MemoryUsage::Upload)};
if (!buffer.IsHostVisible()) {
LOG_DEBUG(Render_Vulkan, "Descriptor buffer is not host visible, disabling");
chunks.clear();
return;
}
if (!buffer.IsHostCoherent()) {
LOG_DEBUG(Render_Vulkan, "Descriptor buffer is not host coherent, disabling");
chunks.clear();
return;
}
if (device.HasDebuggingToolAttached()) {
buffer.SetObjectNameEXT("Descriptor buffer");
}
const VkDeviceAddress raw_address{device.GetLogical().GetBufferDeviceAddress(*buffer)};
const VkDeviceAddress address{Common::AlignUp(raw_address, alignment)};
chunk_addresses.push_back(address);
chunk_hosts.push_back(buffer.Mapped().data() + (address - raw_address));
chunks.push_back(std::move(buffer));
}
}
DescriptorBufferRing::~DescriptorBufferRing() = default;
void DescriptorBufferRing::TickFrame() {
if (++frame_index >= FRAMES_IN_FLIGHT) {
frame_index = 0;
}
chunk_cursor = 0;
cursor = 0;
++generation;
frame_reused = true;
}
void DescriptorBufferRing::TouchFrame(Scheduler& scheduler) {
frame_ticks[frame_index] = scheduler.CurrentTick();
}
DescriptorBufferRing::Allocation DescriptorBufferRing::Allocate(Scheduler& scheduler,
VkDeviceSize size) {
ASSERT(!chunks.empty());
if (!CanAllocate(size)) {
LOG_DEBUG(Render_Vulkan, "Descriptor set of {} bytes exceeds chunk capacity {}", size,
chunk_capacity);
return Allocation{};
}
const VkDeviceSize needed{Common::AlignUp(size, alignment)};
if (frame_reused) {
frame_reused = false;
scheduler.Wait(frame_ticks[frame_index]);
}
if (cursor + needed > chunk_capacity) {
if (chunk_cursor + 1 < chunks_per_frame) {
++chunk_cursor;
} else {
LOG_DEBUG(Render_Vulkan, "Descriptor buffer frame exhausted, stalling on the GPU");
scheduler.Finish();
chunk_cursor = 0;
++generation;
}
cursor = 0;
}
const size_t chunk{frame_index * chunks_per_frame + chunk_cursor};
const VkDeviceSize offset{cursor};
cursor += needed;
frame_ticks[frame_index] = scheduler.CurrentTick();
return Allocation{
.host = chunk_hosts[chunk] + offset,
.offset = offset,
.chunk = static_cast<u32>(chunk),
.generation = generation,
};
}
VkDescriptorBufferBindingInfoEXT DescriptorBufferRing::BindingInfo(u32 chunk) const noexcept {
return VkDescriptorBufferBindingInfoEXT{
.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_BUFFER_BINDING_INFO_EXT,
.pNext = nullptr,
.address = chunk_addresses[chunk],
.usage = VK_BUFFER_USAGE_RESOURCE_DESCRIPTOR_BUFFER_BIT_EXT |
VK_BUFFER_USAGE_SAMPLER_DESCRIPTOR_BUFFER_BIT_EXT,
};
}
} // namespace Vulkan
@@ -0,0 +1,71 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#pragma once
#include <array>
#include <vector>
#include "common/alignment.h"
#include "common/common_types.h"
#include "video_core/vulkan_common/vulkan_memory_allocator.h"
#include "video_core/vulkan_common/vulkan_wrapper.h"
namespace Vulkan {
class Device;
class Scheduler;
class DescriptorBufferRing final {
static constexpr size_t FRAMES_IN_FLIGHT = 8;
static constexpr VkDeviceSize TILER_FRAME_SIZE = 2 * 1024 * 1024;
static constexpr VkDeviceSize DESKTOP_FRAME_SIZE = 4 * 1024 * 1024;
public:
explicit DescriptorBufferRing(const Device& device_, MemoryAllocator& memory_allocator);
~DescriptorBufferRing();
struct Allocation {
u8* host{};
VkDeviceSize offset{};
u32 chunk{};
u64 generation{};
};
[[nodiscard]] u64 CurrentGeneration() const noexcept {
return generation;
}
void TouchFrame(Scheduler& scheduler);
[[nodiscard]] bool CanAllocate(VkDeviceSize size) const noexcept {
return Common::AlignUp(size, alignment) <= chunk_capacity;
}
void TickFrame();
[[nodiscard]] Allocation Allocate(Scheduler& scheduler, VkDeviceSize size);
[[nodiscard]] VkDescriptorBufferBindingInfoEXT BindingInfo(u32 chunk) const noexcept;
[[nodiscard]] bool IsValid() const noexcept {
return !chunks.empty();
}
private:
const Device& device;
std::vector<vk::Buffer> chunks;
std::vector<VkDeviceAddress> chunk_addresses;
std::vector<u8*> chunk_hosts;
VkDeviceSize alignment{1};
VkDeviceSize chunk_capacity{};
size_t chunks_per_frame{};
size_t frame_index{};
size_t chunk_cursor{};
VkDeviceSize cursor{};
u64 generation{1};
std::array<u64, FRAMES_IN_FLIGHT> frame_ticks{};
bool frame_reused{};
};
} // namespace Vulkan
@@ -37,7 +37,7 @@ void InnerFence::Wait() {
if (is_stubbed) {
return;
}
scheduler.WaitSubmitted(wait_tick);
scheduler.Wait(wait_tick);
}
FenceManager::FenceManager(VideoCore::RasterizerInterface& rasterizer_, Tegra::GPU& gpu_,
@@ -5,6 +5,7 @@
// SPDX-License-Identifier: GPL-2.0-or-later
#include <algorithm>
#include <cstring>
#include <iostream>
#include <span>
@@ -250,13 +251,15 @@ GraphicsPipeline::GraphicsPipeline(
Scheduler& scheduler_, BufferCache& buffer_cache_, TextureCache& texture_cache_,
vk::PipelineCache& pipeline_cache_, VideoCore::ShaderNotify* shader_notify,
const Device& device_, DescriptorPool& descriptor_pool,
GuestDescriptorQueue& guest_descriptor_queue_, Common::ThreadWorker* worker_thread,
GuestDescriptorQueue& guest_descriptor_queue_, DescriptorBufferRing& descriptor_buffer_ring_,
Common::ThreadWorker* worker_thread,
PipelineStatistics* pipeline_statistics, RenderPassCache& render_pass_cache,
const GraphicsPipelineCacheKey& key_, std::array<vk::ShaderModule, NUM_STAGES> stages,
const std::array<const Shader::Info*, NUM_STAGES>& infos)
: key{key_}, device{device_}, texture_cache{texture_cache_}, buffer_cache{buffer_cache_},
pipeline_cache(pipeline_cache_), scheduler{scheduler_},
guest_descriptor_queue{guest_descriptor_queue_}, spv_modules{std::move(stages)} {
guest_descriptor_queue{guest_descriptor_queue_},
descriptor_buffer_ring{descriptor_buffer_ring_}, spv_modules{std::move(stages)} {
if (shader_notify) {
shader_notify->MarkShaderBuilding();
}
@@ -276,20 +279,37 @@ GraphicsPipeline::GraphicsPipeline(
num_descriptor_entries += NumDescriptorEntries(*info);
}
fragment_has_color0_output = stage_infos[NUM_STAGES - 1].stores_frag_color[0];
auto func{[this, shader_notify, &render_pass_cache, &descriptor_pool, pipeline_statistics] {
DescriptorLayoutBuilder builder{MakeBuilder(device, stage_infos)};
uses_push_descriptor = builder.CanUsePushDescriptor();
descriptor_set_layout = builder.CreateDescriptorSetLayout(uses_push_descriptor);
if (!uses_push_descriptor) {
descriptor_allocator = descriptor_pool.Allocator(device, scheduler, *descriptor_set_layout, stage_infos);
DescriptorLayoutBuilder builder{MakeBuilder(device, stage_infos)};
uses_push_descriptor = builder.CanUsePushDescriptor();
uses_descriptor_buffer = builder.CanUseDescriptorBuffer() && descriptor_buffer_ring.IsValid();
descriptor_set_layout =
builder.CreateDescriptorSetLayout(uses_push_descriptor, uses_descriptor_buffer);
if (uses_descriptor_buffer) {
descriptor_buffer_layout = builder.MakeDescriptorBufferLayout(*descriptor_set_layout);
if (!descriptor_buffer_ring.CanAllocate(descriptor_buffer_layout.size)) {
LOG_WARNING(Render_Vulkan,
"Graphics pipeline {:016X} needs {} descriptor bytes per draw, falling back "
"to sets",
key.Hash(), descriptor_buffer_layout.size);
uses_descriptor_buffer = false;
descriptor_buffer_layout = {};
descriptor_set_layout = builder.CreateDescriptorSetLayout(uses_push_descriptor);
}
}
const VkDescriptorSetLayout set_layout{*descriptor_set_layout};
pipeline_layout = builder.CreatePipelineLayout(set_layout);
const VkDescriptorSetLayout set_layout{*descriptor_set_layout};
pipeline_layout = builder.CreatePipelineLayout(set_layout);
if (!uses_descriptor_buffer) {
descriptor_update_template =
builder.CreateTemplate(set_layout, *pipeline_layout, uses_push_descriptor);
if (!uses_push_descriptor) {
descriptor_allocator =
descriptor_pool.Allocator(device, scheduler, set_layout, stage_infos);
}
}
auto func{[this, shader_notify, &render_pass_cache, pipeline_statistics] {
const VkRenderPass render_pass{render_pass_cache.Get(MakeRenderPassKey(key.state, device))};
Validate();
try {
@@ -496,7 +516,7 @@ bool GraphicsPipeline::ConfigureImpl(bool is_indexed) {
buffer_cache.UpdateGraphicsBuffers(is_indexed);
buffer_cache.BindHostGeometryBuffers(is_indexed);
guest_descriptor_queue.Acquire(scheduler, num_descriptor_entries);
guest_descriptor_queue.Acquire(scheduler, num_descriptor_entries, uses_descriptor_buffer);
RescalingPushConstant rescaling;
RenderAreaPushConstant render_area;
@@ -538,13 +558,43 @@ bool GraphicsPipeline::ConfigureImpl(bool is_indexed) {
if (IsBuilt() && !pipeline) {
return false;
}
ConfigureDraw(rescaling, render_area);
return true;
return ConfigureDraw(rescaling, render_area);
}
void GraphicsPipeline::ConfigureDraw(const RescalingPushConstant& rescaling,
bool GraphicsPipeline::ConfigureDraw(const RescalingPushConstant& rescaling,
const RenderAreaPushConstant& render_area) {
const void* const descriptor_data{guest_descriptor_queue.UpdateData()};
VkDeviceSize descriptor_buffer_offset{};
u32 descriptor_buffer_chunk{};
if (descriptor_set_layout && uses_descriptor_buffer) {
const auto* const entries = static_cast<const DescriptorUpdateEntry*>(descriptor_data);
const bool reuse_allocation =
last_descriptor_buffer_generation == descriptor_buffer_ring.CurrentGeneration() &&
last_descriptor_payload.size() == num_descriptor_entries &&
std::memcmp(last_descriptor_payload.data(), entries,
num_descriptor_entries * sizeof(DescriptorUpdateEntry)) == 0;
if (reuse_allocation) {
descriptor_buffer_offset = last_descriptor_buffer_offset;
descriptor_buffer_chunk = last_descriptor_buffer_chunk;
descriptor_buffer_ring.TouchFrame(scheduler);
} else {
const DescriptorBufferRing::Allocation alloc{
descriptor_buffer_ring.Allocate(scheduler, descriptor_buffer_layout.size)};
if (!alloc.host) {
LOG_DEBUG(Render_Vulkan, "Failed to reserve descriptor memory, skipping draw");
return false;
}
WriteDescriptorBuffer(device, descriptor_buffer_layout, entries, alloc.host);
descriptor_buffer_offset = alloc.offset;
descriptor_buffer_chunk = alloc.chunk;
last_descriptor_buffer_offset = alloc.offset;
last_descriptor_buffer_chunk = alloc.chunk;
last_descriptor_buffer_generation = alloc.generation;
last_descriptor_payload.assign(entries, entries + num_descriptor_entries);
}
}
scheduler.RequestRenderpass(texture_cache.GetFramebuffer());
if (!is_built.load(std::memory_order::relaxed)) {
// Wait for the pipeline to be built
@@ -556,6 +606,9 @@ void GraphicsPipeline::ConfigureDraw(const RescalingPushConstant& rescaling,
const bool is_rescaling{texture_cache.IsRescaling()};
const bool update_rescaling{scheduler.UpdateRescaling(is_rescaling)};
const bool bind_pipeline{scheduler.UpdateGraphicsPipeline(this)};
const bool bind_descriptor_buffer{
descriptor_set_layout && uses_descriptor_buffer &&
scheduler.UpdateDescriptorBufferChunk(descriptor_buffer_chunk)};
// Log graphics pipeline binding
if (bind_pipeline && GPU::Logging::IsActive() &&
@@ -564,11 +617,27 @@ void GraphicsPipeline::ConfigureDraw(const RescalingPushConstant& rescaling,
GPU::Logging::GPULogger::GetInstance().LogPipelineBind(false, pipeline_info);
}
const void* const descriptor_data{guest_descriptor_queue.UpdateData()};
scheduler.Record([this, descriptor_data, bind_pipeline, rescaling_data = rescaling.Data(),
is_rescaling, update_rescaling,
bool update_descriptors = true;
if (descriptor_set_layout && !uses_push_descriptor && !uses_descriptor_buffer) {
const auto* const entries = static_cast<const DescriptorUpdateEntry*>(descriptor_data);
update_descriptors =
bind_pipeline || last_descriptor_payload.size() != num_descriptor_entries ||
std::memcmp(last_descriptor_payload.data(), entries,
num_descriptor_entries * sizeof(DescriptorUpdateEntry)) != 0;
if (update_descriptors) {
last_descriptor_payload.assign(entries, entries + num_descriptor_entries);
}
}
scheduler.Record([this, descriptor_data, bind_pipeline, update_descriptors,
descriptor_buffer_offset, descriptor_buffer_chunk, bind_descriptor_buffer,
rescaling_data = rescaling.Data(), is_rescaling, update_rescaling,
uses_render_area = render_area.uses_render_area,
render_area_data = render_area.words](vk::CommandBuffer cmdbuf) {
if (bind_descriptor_buffer) {
const VkDescriptorBufferBindingInfoEXT binding_info{
descriptor_buffer_ring.BindingInfo(descriptor_buffer_chunk)};
cmdbuf.BindDescriptorBuffersEXT(binding_info);
}
if (bind_pipeline) {
if (!pipeline) {
return;
@@ -593,10 +662,14 @@ void GraphicsPipeline::ConfigureDraw(const RescalingPushConstant& rescaling,
if (!descriptor_set_layout) {
return;
}
if (uses_push_descriptor) {
if (uses_descriptor_buffer) {
const u32 buffer_index{};
cmdbuf.SetDescriptorBufferOffsetsEXT(VK_PIPELINE_BIND_POINT_GRAPHICS, *pipeline_layout,
0, buffer_index, descriptor_buffer_offset);
} else if (uses_push_descriptor) {
cmdbuf.PushDescriptorSetWithTemplateKHR(*descriptor_update_template, *pipeline_layout,
0, descriptor_data);
} else {
} else if (update_descriptors) {
const VkDescriptorSet descriptor_set{descriptor_allocator.Commit()};
const vk::Device& dev{device.GetLogical()};
dev.UpdateDescriptorSet(descriptor_set, *descriptor_update_template, descriptor_data);
@@ -604,6 +677,7 @@ void GraphicsPipeline::ConfigureDraw(const RescalingPushConstant& rescaling,
descriptor_set, nullptr);
}
});
return true;
}
void GraphicsPipeline::MakePipeline(VkRenderPass render_pass) {
@@ -757,13 +831,16 @@ void GraphicsPipeline::MakePipeline(VkRenderPass render_pass) {
.lineWidth = 1.0f,
// TODO(alekpop): Transfer from regs
};
const VkLineRasterizationModeEXT line_raster_mode =
device.GetLineRasterizationMode(key.state.smooth_lines != 0);
const bool stippled_lines_supported = device.SupportsStippleForMode(line_raster_mode);
const bool smooth_lines_supported =
device.IsExtLineRasterizationSupported() && device.SupportsSmoothLines();
const bool stippled_lines_supported =
device.IsExtLineRasterizationSupported() && device.SupportsStippledRectangularLines();
VkPipelineRasterizationLineStateCreateInfoEXT line_state{
.sType = VK_STRUCTURE_TYPE_PIPELINE_RASTERIZATION_LINE_STATE_CREATE_INFO_EXT,
.pNext = nullptr,
.lineRasterizationMode = line_raster_mode,
.lineRasterizationMode = key.state.smooth_lines != 0 && smooth_lines_supported
? VK_LINE_RASTERIZATION_MODE_RECTANGULAR_SMOOTH_EXT
: VK_LINE_RASTERIZATION_MODE_RECTANGULAR_EXT,
.stippledLineEnable =
(dynamic.line_stipple_enable && stippled_lines_supported) ? VK_TRUE : VK_FALSE,
.lineStippleFactor = key.state.line_stipple_factor,
@@ -802,16 +879,6 @@ void GraphicsPipeline::MakePipeline(VkRenderPass render_pass) {
if (device.IsExtProvokingVertexSupported()) {
provoking_vertex.pNext = std::exchange(rasterization_ci.pNext, &provoking_vertex);
}
VkPipelineRasterizationDepthClipStateCreateInfoEXT depth_clip_state{
.sType = VK_STRUCTURE_TYPE_PIPELINE_RASTERIZATION_DEPTH_CLIP_STATE_CREATE_INFO_EXT,
.pNext = nullptr,
.flags = 0,
.depthClipEnable = static_cast<VkBool32>(dynamic.depth_clip_disabled == 0 ? VK_TRUE
: VK_FALSE),
};
if (device.IsExtDepthClipEnableSupported()) {
depth_clip_state.pNext = std::exchange(rasterization_ci.pNext, &depth_clip_state);
}
const bool supports_alpha_output = fragment_has_color0_output;
const bool alpha_to_one_supported = device.SupportsAlphaToOne();
@@ -1002,6 +1069,9 @@ void GraphicsPipeline::MakePipeline(VkRenderPass render_pass) {
if (device.IsKhrPipelineExecutablePropertiesEnabled() && Settings::values.renderer_debug.GetValue()) {
flags |= VK_PIPELINE_CREATE_CAPTURE_STATISTICS_BIT_KHR;
}
if (uses_descriptor_buffer) {
flags |= VK_PIPELINE_CREATE_DESCRIPTOR_BUFFER_BIT_EXT;
}
pipeline = device.GetLogical().CreateGraphicsPipeline({
.sType = VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO,
@@ -12,14 +12,18 @@
#include <condition_variable>
#include <mutex>
#include <type_traits>
#include <vector>
#include "common/thread_worker.h"
#include "shader_recompiler/shader_info.h"
#include "video_core/engines/maxwell_3d.h"
#include "video_core/renderer_vulkan/fixed_pipeline_state.h"
#include "video_core/renderer_vulkan/pipeline_helper.h"
#include "video_core/renderer_vulkan/vk_buffer_cache.h"
#include "video_core/renderer_vulkan/vk_descriptor_buffer.h"
#include "video_core/renderer_vulkan/vk_descriptor_pool.h"
#include "video_core/renderer_vulkan/vk_texture_cache.h"
#include "video_core/renderer_vulkan/vk_update_descriptor.h"
#include "video_core/vulkan_common/vulkan_wrapper.h"
namespace VideoCore {
@@ -76,7 +80,8 @@ public:
Scheduler& scheduler, BufferCache& buffer_cache, TextureCache& texture_cache,
vk::PipelineCache& pipeline_cache, VideoCore::ShaderNotify* shader_notify,
const Device& device, DescriptorPool& descriptor_pool,
GuestDescriptorQueue& guest_descriptor_queue, Common::ThreadWorker* worker_thread,
GuestDescriptorQueue& guest_descriptor_queue,
DescriptorBufferRing& descriptor_buffer_ring, Common::ThreadWorker* worker_thread,
PipelineStatistics* pipeline_statistics, RenderPassCache& render_pass_cache,
const GraphicsPipelineCacheKey& key, std::array<vk::ShaderModule, NUM_STAGES> stages,
const std::array<const Shader::Info*, NUM_STAGES>& infos);
@@ -132,7 +137,7 @@ private:
template <typename Spec>
bool ConfigureImpl(bool is_indexed);
void ConfigureDraw(const RescalingPushConstant& rescaling,
bool ConfigureDraw(const RescalingPushConstant& rescaling,
const RenderAreaPushConstant& render_are);
void MakePipeline(VkRenderPass render_pass);
@@ -148,6 +153,7 @@ private:
vk::PipelineCache& pipeline_cache;
Scheduler& scheduler;
GuestDescriptorQueue& guest_descriptor_queue;
DescriptorBufferRing& descriptor_buffer_ring;
bool (*configure_func)(GraphicsPipeline*, bool){};
@@ -170,10 +176,17 @@ private:
vk::DescriptorUpdateTemplate descriptor_update_template;
vk::Pipeline pipeline;
DescriptorBufferLayout descriptor_buffer_layout;
std::vector<DescriptorUpdateEntry> last_descriptor_payload;
VkDeviceSize last_descriptor_buffer_offset{};
u32 last_descriptor_buffer_chunk{};
u64 last_descriptor_buffer_generation{};
std::condition_variable build_condvar;
std::mutex build_mutex;
std::atomic_bool is_built{false};
bool uses_push_descriptor{false};
bool uses_descriptor_buffer{false};
};
} // namespace Vulkan
@@ -305,7 +305,7 @@ size_t GetTotalPipelineWorkers() {
std::max<size_t>(static_cast<size_t>(std::thread::hardware_concurrency()), 2ULL) - 1ULL;
#ifdef __ANDROID__
const int configured = AndroidSettings::values.pipeline_worker_count.GetValue();
const int clamped = std::clamp(configured, 2, 8);
const int clamped = std::clamp(configured, 4, 8);
const size_t desired = static_cast<size_t>(clamped);
if (desired == 0) {
return 1ULL;
@@ -340,18 +340,19 @@ PipelineCache::PipelineCache(Tegra::MaxwellDeviceMemoryManager& device_memory_,
const Device& device_, Scheduler& scheduler_,
DescriptorPool& descriptor_pool_,
GuestDescriptorQueue& guest_descriptor_queue_,
DescriptorBufferRing& descriptor_buffer_ring_,
RenderPassCache& render_pass_cache_, BufferCache& buffer_cache_,
TextureCache& texture_cache_, VideoCore::ShaderNotify& shader_notify_)
: VideoCommon::ShaderCache{device_memory_}, device{device_}, scheduler{scheduler_},
descriptor_pool{descriptor_pool_}, guest_descriptor_queue{guest_descriptor_queue_},
descriptor_buffer_ring{descriptor_buffer_ring_},
render_pass_cache{render_pass_cache_}, buffer_cache{buffer_cache_},
texture_cache{texture_cache_}, shader_notify{shader_notify_},
use_asynchronous_shaders{Settings::values.use_asynchronous_shaders.GetValue()},
use_vulkan_pipeline_cache{Settings::values.use_vulkan_driver_pipeline_cache.GetValue()},
workers(device.HasBrokenParallelShaderCompiling() ? 1ULL : GetTotalPipelineWorkers(),
"VkPipelineBuilder", {}, Common::ThreadPlacement::Background),
serialization_thread(1, "VkPipelineSerialization", {},
Common::ThreadPlacement::Background) {
"VkPipelineBuilder"),
serialization_thread(1, "VkPipelineSerialization") {
const auto& float_control{device.FloatControlProperties()};
const VkDriverId driver_id{device.GetDriverID()};
const VkShaderStageFlags subgroup_stages{device.GetSubgroupSupportedStages()};
@@ -515,11 +516,6 @@ PipelineCache::PipelineCache(Tegra::MaxwellDeviceMemoryManager& device_memory_,
dynamic_features.has_dynamic_state3_depth_clamp_enable =
dynamic_features.has_extended_dynamic_state_3_enables &&
device.SupportsDynamicState3DepthClampEnable();
dynamic_features.has_dynamic_state3_conservative_raster_mode =
dynamic_features.has_extended_dynamic_state_3_enables &&
device.SupportsDynamicState3ConservativeRasterizationMode();
dynamic_features.has_depth_clip_enable =
device.IsExtDepthClipEnableSupported();
dynamic_features.has_dynamic_state3_logic_op_enable =
dynamic_features.has_extended_dynamic_state_3_enables &&
device.SupportsDynamicState3LogicOpEnable();
@@ -532,8 +528,7 @@ PipelineCache::PipelineCache(Tegra::MaxwellDeviceMemoryManager& device_memory_,
device.IsExtVertexInputDynamicStateSupported() &&
Settings::values.vertex_input_dynamic_state.GetValue();
dynamic_features.has_provoking_vertex =
device.IsExtProvokingVertexSupported();
dynamic_features.has_provoking_vertex = device.IsExtProvokingVertexSupported();
dynamic_features.has_provoking_vertex_first_mode =
device.SupportsProvokingVertexFirstMode();
dynamic_features.has_provoking_vertex_last_mode =
@@ -843,8 +838,8 @@ std::unique_ptr<GraphicsPipeline> PipelineCache::CreateGraphicsPipeline(
Common::ThreadWorker* const thread_worker{build_in_parallel ? &workers : nullptr};
return std::make_unique<GraphicsPipeline>(
scheduler, buffer_cache, texture_cache, vulkan_pipeline_cache, &shader_notify, device,
descriptor_pool, guest_descriptor_queue, thread_worker, statistics, render_pass_cache, key,
std::move(modules), infos);
descriptor_pool, guest_descriptor_queue, descriptor_buffer_ring, thread_worker, statistics,
render_pass_cache, key, std::move(modules), infos);
} catch (const Shader::Exception& exception) {
auto hash = key.Hash();
@@ -964,7 +959,8 @@ std::unique_ptr<ComputePipeline> PipelineCache::CreateComputePipeline(
}
Common::ThreadWorker* const thread_worker{build_in_parallel ? &workers : nullptr};
return std::make_unique<ComputePipeline>(device, scheduler, vulkan_pipeline_cache, descriptor_pool,
guest_descriptor_queue, thread_worker, statistics,
guest_descriptor_queue, descriptor_buffer_ring,
thread_worker, statistics,
&shader_notify, program.info, std::move(spv_module),
key.unique_hash);
@@ -105,6 +105,7 @@ public:
explicit PipelineCache(Tegra::MaxwellDeviceMemoryManager& device_memory_, const Device& device,
Scheduler& scheduler, DescriptorPool& descriptor_pool,
GuestDescriptorQueue& guest_descriptor_queue,
DescriptorBufferRing& descriptor_buffer_ring,
RenderPassCache& render_pass_cache, BufferCache& buffer_cache,
TextureCache& texture_cache, VideoCore::ShaderNotify& shader_notify_);
~PipelineCache();
@@ -147,6 +148,7 @@ private:
Scheduler& scheduler;
DescriptorPool& descriptor_pool;
GuestDescriptorQueue& guest_descriptor_queue;
DescriptorBufferRing& descriptor_buffer_ring;
RenderPassCache& render_pass_cache;
BufferCache& buffer_cache;
TextureCache& texture_cache;
@@ -296,6 +296,9 @@ void PresentManager::RecreateSwapchain(Frame* frame) {
}
void PresentManager::SetImageCount() {
// We cannot have more than 7 images in flight at any given time.
// FRAMES_IN_FLIGHT is 8, and the cache TICKS_TO_DESTROY is 8.
// Mali drivers will give us 6.
image_count = std::min<size_t>(swapchain.GetImageCount(), 7);
}
@@ -919,7 +919,7 @@ private:
return;
}
has_flushed_end_pending = true;
scheduler.MarkTransformFeedbackUsed();
// Refresh buffers state before beginning transform feedback so counters are up-to-date
UpdateBuffers();
if (!has_started || buffers_count == 0) {
// No counter buffers available: begin without counters
@@ -6,7 +6,6 @@
#include <algorithm>
#include <array>
#include <limits>
#include <memory>
#include <mutex>
@@ -204,8 +203,10 @@ RasterizerVulkan::RasterizerVulkan(Core::Frontend::EmuWindow& emu_window_, Tegra
: gpu{gpu_}, device_memory{device_memory_}, device{device_},
memory_allocator{memory_allocator_}, state_tracker{state_tracker_}, scheduler{scheduler_},
staging_pool(device, memory_allocator, scheduler), descriptor_pool(device, scheduler),
guest_descriptor_queue(device, UpdateDescriptorQueue::GUEST_FRAME_PAYLOAD_SIZE),
guest_descriptor_queue(device, UpdateDescriptorQueue::GUEST_FRAME_PAYLOAD_SIZE,
device.IsExtDescriptorBufferSupported()),
compute_pass_descriptor_queue(device, UpdateDescriptorQueue::COMPUTE_FRAME_PAYLOAD_SIZE),
descriptor_buffer_ring(device, memory_allocator),
blit_image(device, scheduler, state_tracker, descriptor_pool), render_pass_cache(device),
texture_cache_runtime{
device, scheduler, memory_allocator, staging_pool,
@@ -218,28 +219,15 @@ RasterizerVulkan::RasterizerVulkan(Core::Frontend::EmuWindow& emu_window_, Tegra
staging_pool, compute_pass_descriptor_queue, descriptor_pool, texture_cache),
query_cache(gpu, *this, device_memory, query_cache_runtime),
pipeline_cache(device_memory, device, scheduler, descriptor_pool, guest_descriptor_queue,
render_pass_cache, buffer_cache, texture_cache, gpu.ShaderNotify()),
descriptor_buffer_ring, render_pass_cache, buffer_cache, texture_cache,
gpu.ShaderNotify()),
accelerate_dma(buffer_cache, texture_cache, scheduler),
fence_manager(*this, gpu, texture_cache, buffer_cache, query_cache, device, scheduler),
wfi_event(device.GetLogical().CreateEvent()) {
scheduler.SetQueryCache(query_cache);
memory_allocator.SetReclaimCallback([this](u64 bytes) -> u64 {
u64 freed = staging_pool.ReclaimMemory(bytes);
if (freed < bytes) {
freed += texture_cache.ReclaimMemory(bytes - freed, false);
}
if (freed < bytes) {
freed += buffer_cache.ReclaimMemory(bytes - freed, false);
}
auto& master_semaphore = scheduler.GetMasterSemaphore();
master_semaphore.Refresh();
vk::TickDeletionQueue(master_semaphore.KnownGpuTick());
return freed;
});
}
RasterizerVulkan::~RasterizerVulkan() {
memory_allocator.SetReclaimCallback(nullptr);
scheduler.WaitWorker();
scheduler.Finish();
}
@@ -599,7 +587,10 @@ void RasterizerVulkan::DispatchCompute() {
return;
}
std::scoped_lock lock{texture_cache.mutex, buffer_cache.mutex};
pipeline->Configure(*kepler_compute, *gpu_memory, scheduler, buffer_cache, texture_cache);
if (!pipeline->Configure(*kepler_compute, *gpu_memory, scheduler, buffer_cache,
texture_cache)) {
return;
}
const auto& qmd{kepler_compute->launch_description};
auto indirect_address = kepler_compute->GetIndirectComputeAddress();
@@ -896,11 +887,9 @@ void RasterizerVulkan::FlushCommands() {
void RasterizerVulkan::TickFrame() {
draw_counter = 0;
auto& master_semaphore = scheduler.GetMasterSemaphore();
master_semaphore.Refresh();
vk::TickDeletionQueue(master_semaphore.KnownGpuTick());
guest_descriptor_queue.TickFrame();
compute_pass_descriptor_queue.TickFrame();
descriptor_buffer_ring.TickFrame();
fence_manager.TickFrame();
staging_pool.TickFrame();
{
@@ -1470,10 +1459,7 @@ void RasterizerVulkan::UpdateLineWidth(Tegra::Engines::Maxwell3D::Regs& regs) {
}
const float width =
regs.line_anti_alias_enable ? regs.line_width_smooth : regs.line_width_aliased;
const float clamped_width = device.ClampLineWidth(width);
scheduler.Record([clamped_width](vk::CommandBuffer cmdbuf) {
cmdbuf.SetLineWidth(clamped_width);
});
scheduler.Record([width](vk::CommandBuffer cmdbuf) { cmdbuf.SetLineWidth(width); });
}
void RasterizerVulkan::UpdateCullMode(Tegra::Engines::Maxwell3D::Regs& regs) {
@@ -1570,10 +1556,7 @@ void RasterizerVulkan::UpdateLineStippleEnable(Tegra::Engines::Maxwell3D::Regs&
return;
}
const VkLineRasterizationModeEXT mode =
device.GetLineRasterizationMode(regs.line_anti_alias_enable != 0);
const bool enable = regs.line_stipple_enable != 0 && device.SupportsStippleForMode(mode);
scheduler.Record([enable](vk::CommandBuffer cmdbuf) {
scheduler.Record([enable = regs.line_stipple_enable](vk::CommandBuffer cmdbuf) {
cmdbuf.SetLineStippleEnableEXT(enable);
});
}
@@ -1587,24 +1570,28 @@ void RasterizerVulkan::UpdateLineRasterizationMode(Tegra::Engines::Maxwell3D::Re
}
if (!device.SupportsDynamicState3LineRasterizationMode()) {
static std::once_flag warn_missing_dynamic_state;
std::call_once(warn_missing_dynamic_state, [] {
static std::once_flag warn_missing_rect;
std::call_once(warn_missing_rect, [] {
LOG_WARNING(Render_Vulkan,
"Driver lacks dynamic line rasterization mode; the pipeline static value "
"is used instead");
"Driver lacks rectangular line rasterization support; skipping dynamic "
"line state updates");
});
return;
}
const bool wants_smooth = regs.line_anti_alias_enable != 0;
const VkLineRasterizationModeEXT mode = device.GetLineRasterizationMode(wants_smooth);
if (wants_smooth && mode != VK_LINE_RASTERIZATION_MODE_RECTANGULAR_SMOOTH_EXT) {
static std::once_flag warn_missing_smooth;
std::call_once(warn_missing_smooth, [] {
LOG_WARNING(Render_Vulkan,
"Line anti-aliasing requested but smoothLines feature unavailable; "
"falling back to the closest supported mode");
});
VkLineRasterizationModeEXT mode = VK_LINE_RASTERIZATION_MODE_RECTANGULAR_EXT;
if (wants_smooth) {
if (device.SupportsSmoothLines()) {
mode = VK_LINE_RASTERIZATION_MODE_RECTANGULAR_SMOOTH_EXT;
} else {
static std::once_flag warn_missing_smooth;
std::call_once(warn_missing_smooth, [] {
LOG_WARNING(Render_Vulkan,
"Line anti-aliasing requested but smoothLines feature unavailable; "
"using rectangular rasterization");
});
}
}
scheduler.Record([mode](vk::CommandBuffer cmdbuf) {
cmdbuf.SetLineRasterizationModeEXT(mode);
@@ -1664,7 +1651,12 @@ void RasterizerVulkan::UpdateDepthClampEnable(Tegra::Engines::Maxwell3D::Regs& r
if (!device.SupportsDynamicState3DepthClampEnable()) {
return;
}
const bool is_enabled = IsDepthClampEnabled(regs, device.IsExtDepthClipEnableSupported());
bool is_enabled = !(regs.viewport_clip_control.geometry_clip ==
Maxwell::ViewportClipControl::GeometryClip::Passthrough ||
regs.viewport_clip_control.geometry_clip ==
Maxwell::ViewportClipControl::GeometryClip::FrustumXYZ ||
regs.viewport_clip_control.geometry_clip ==
Maxwell::ViewportClipControl::GeometryClip::FrustumZ);
scheduler.Record(
[is_enabled](vk::CommandBuffer cmdbuf) { cmdbuf.SetDepthClampEnableEXT(is_enabled); });
}
@@ -17,6 +17,7 @@
#include "video_core/rasterizer_interface.h"
#include "video_core/renderer_vulkan/blit_image.h"
#include "video_core/renderer_vulkan/vk_buffer_cache.h"
#include "video_core/renderer_vulkan/vk_descriptor_buffer.h"
#include "video_core/renderer_vulkan/vk_descriptor_pool.h"
#include "video_core/renderer_vulkan/vk_fence_manager.h"
#include "video_core/renderer_vulkan/vk_pipeline_cache.h"
@@ -179,6 +180,7 @@ private:
void UpdateRasterizerDiscardEnable(Tegra::Engines::Maxwell3D::Regs& regs);
void UpdateConservativeRasterizationMode(Tegra::Engines::Maxwell3D::Regs& regs);
void UpdateLineStippleEnable(Tegra::Engines::Maxwell3D::Regs& regs);
void UpdateLineStipple(Tegra::Engines::Maxwell3D::Regs& regs);
void UpdateLineRasterizationMode(Tegra::Engines::Maxwell3D::Regs& regs);
void UpdateDepthBiasEnable(Tegra::Engines::Maxwell3D::Regs& regs);
void UpdateLogicOpEnable(Tegra::Engines::Maxwell3D::Regs& regs);
@@ -206,6 +208,7 @@ private:
DescriptorPool descriptor_pool;
GuestDescriptorQueue guest_descriptor_queue;
ComputePassDescriptorQueue compute_pass_descriptor_queue;
DescriptorBufferRing descriptor_buffer_ring;
BlitImageHelper blit_image;
RenderPassCache render_pass_cache;
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
@@ -12,10 +15,7 @@ ResourcePool::ResourcePool(MasterSemaphore& master_semaphore_, size_t grow_step_
: master_semaphore{&master_semaphore_}, grow_step{grow_step_} {}
size_t ResourcePool::CommitResource() {
// Refresh semaphore to query updated results
master_semaphore->Refresh();
const u64 gpu_tick = master_semaphore->KnownGpuTick();
const auto search = [this, gpu_tick](size_t begin, size_t end) -> std::optional<size_t> {
const auto search = [this](size_t begin, size_t end, u64 gpu_tick) -> std::optional<size_t> {
for (size_t iterator = begin; iterator < end; ++iterator) {
if (gpu_tick >= ticks[iterator]) {
ticks[iterator] = master_semaphore->CurrentTick();
@@ -24,11 +24,17 @@ size_t ResourcePool::CommitResource() {
}
return std::nullopt;
};
// Try to find a free resource from the hinted position to the end.
std::optional<size_t> found = search(hint_iterator, ticks.size());
const auto find_free = [&](u64 gpu_tick) -> std::optional<size_t> {
std::optional<size_t> result = search(hint_iterator, ticks.size(), gpu_tick);
if (!result) {
result = search(0, hint_iterator, gpu_tick);
}
return result;
};
std::optional<size_t> found = find_free(master_semaphore->KnownGpuTick());
if (!found) {
// Search from beginning to the hinted position.
found = search(0, hint_iterator);
master_semaphore->Refresh();
found = find_free(master_semaphore->KnownGpuTick());
if (!found) {
// Both searches failed, the pool is full; handle it.
const size_t free_resource = ManageOverflow();
@@ -47,7 +47,6 @@ Scheduler::Scheduler(const Device& device_, StateTracker& state_tracker_)
master_semaphore{std::make_unique<MasterSemaphore>(device)},
command_pool{std::make_unique<CommandPool>(*master_semaphore, device)} {
vk::SetDeletionTimeline(master_semaphore->CurrentTick());
AcquireNewChunk();
AllocateWorkerCommandBuffer();
worker_thread = std::jthread([this](std::stop_token token) { WorkerThread(token); });
@@ -248,6 +247,15 @@ bool Scheduler::UpdateRescaling(bool is_rescaling) {
return true;
}
bool Scheduler::UpdateDescriptorBufferChunk(u32 descriptor_chunk) {
if (state.descriptor_buffer_bound && descriptor_chunk == state.descriptor_buffer_chunk) {
return false;
}
state.descriptor_buffer_bound = true;
state.descriptor_buffer_chunk = descriptor_chunk;
return true;
}
void Scheduler::WorkerThread(std::stop_token stop_token) {
Common::SetCurrentThreadName("VulkanWorker");
@@ -323,7 +331,6 @@ u64 Scheduler::SubmitExecution(VkSemaphore signal_semaphore, VkSemaphore wait_se
InvalidateState();
const u64 signal_value = master_semaphore->NextTick();
vk::SetDeletionTimeline(master_semaphore->CurrentTick());
RecordWithUploadBuffer([signal_semaphore, wait_semaphore, signal_value,
this](vk::CommandBuffer cmdbuf, vk::CommandBuffer upload_cmdbuf) {
static constexpr VkMemoryBarrier WRITE_BARRIER{
@@ -371,6 +378,7 @@ void Scheduler::AllocateNewContext() {
void Scheduler::InvalidateState() {
state.graphics_pipeline = nullptr;
state.rescaling_defined = false;
state.descriptor_buffer_bound = false;
state_tracker.InvalidateCommandBufferState();
}
@@ -400,7 +408,7 @@ void Scheduler::EndRenderPass()
Record([num_images = num_renderpass_images,
images = renderpass_images,
ranges = renderpass_image_ranges,
has_transform_feedback = state.uses_transform_feedback](
has_transform_feedback = device.IsExtTransformFeedbackSupported()](
vk::CommandBuffer cmdbuf) {
std::array<VkImageMemoryBarrier, 9> barriers;
for (size_t i = 0; i < num_images; ++i) {
@@ -455,7 +463,6 @@ void Scheduler::EndRenderPass()
});
state.renderpass = VkRenderPass{};
state.uses_transform_feedback = false;
num_renderpass_images = 0;
}
+29 -42
View File
@@ -74,17 +74,15 @@ public:
return state.renderpass != VK_NULL_HANDLE;
}
/// Flags that transform feedback writes have been recorded since the last render pass end.
void MarkTransformFeedbackUsed() noexcept {
state.uses_transform_feedback = true;
}
/// Update the pipeline to the current execution context.
bool UpdateGraphicsPipeline(GraphicsPipeline* pipeline);
/// Update the rescaling state. Returns true if the state has to be updated.
bool UpdateRescaling(bool is_rescaling);
/// Returns true when the descriptor buffer chunk has to be bound into the command buffer.
bool UpdateDescriptorBufferChunk(u32 descriptor_chunk);
/// Invalidates current command buffer state except for render passes
void InvalidateState();
@@ -136,16 +134,33 @@ public:
}
master_semaphore->Wait(tick);
}
ApplyFramePacing(target_fps);
}
void WaitSubmitted(u64 tick, double target_fps = 0.0) {
if (tick > 0 && tick < master_semaphore->CurrentTick()) {
master_semaphore->Wait(tick);
if (Settings::values.use_speed_limit.GetValue() && target_fps > 0.0) {
auto now = std::chrono::steady_clock::now();
if (last_target_fps != target_fps) {
frame_interval = std::chrono::duration_cast<std::chrono::steady_clock::duration>(std::chrono::duration<double>(1.0 / target_fps));
max_frame_count = static_cast<int>(0.1 * target_fps);
last_target_fps = target_fps;
frame_counter = 0;
start_time = now;
}
frame_counter++;
auto target_time = start_time + frame_interval * frame_counter;
if (target_time >= now) {
auto sleep_time = target_time - now;
if (sleep_time > std::chrono::milliseconds(15)) {
std::this_thread::sleep_for(sleep_time - std::chrono::milliseconds(1));
}
while (std::chrono::steady_clock::now() < target_time) {
std::this_thread::yield();
}
} else if (frame_counter > max_frame_count) {
frame_counter = 0;
start_time = now;
}
}
ApplyFramePacing(target_fps);
}
/// Returns the master timeline semaphore.
[[nodiscard]] MasterSemaphore& GetMasterSemaphore() const noexcept {
return *master_semaphore;
}
@@ -153,35 +168,6 @@ public:
std::mutex submit_mutex;
private:
void ApplyFramePacing(double target_fps) {
if (!Settings::values.use_speed_limit.GetValue() || target_fps <= 0.0) {
return;
}
auto now = std::chrono::steady_clock::now();
if (last_target_fps != target_fps) {
frame_interval = std::chrono::duration_cast<std::chrono::steady_clock::duration>(
std::chrono::duration<double>(1.0 / target_fps));
max_frame_count = static_cast<int>(0.1 * target_fps);
last_target_fps = target_fps;
frame_counter = 0;
start_time = now;
}
frame_counter++;
auto target_time = start_time + frame_interval * frame_counter;
if (target_time >= now) {
auto sleep_time = target_time - now;
if (sleep_time > std::chrono::milliseconds(15)) {
std::this_thread::sleep_for(sleep_time - std::chrono::milliseconds(1));
}
while (std::chrono::steady_clock::now() < target_time) {
std::this_thread::yield();
}
} else if (frame_counter > max_frame_count) {
frame_counter = 0;
start_time = now;
}
}
class Command {
public:
virtual ~Command() = default;
@@ -272,7 +258,8 @@ private:
bool is_rescaling = false;
bool rescaling_defined = false;
bool needs_state_enable_refresh = false;
bool uses_transform_feedback = false;
u32 descriptor_buffer_chunk = 0;
bool descriptor_buffer_bound = false;
};
struct DeferredClear {
@@ -84,10 +84,16 @@ StagingBufferPool::StagingBufferPool(const Device& device_, MemoryAllocator& mem
if (device.IsExtTransformFeedbackSupported()) {
stream_ci.usage |= VK_BUFFER_USAGE_TRANSFORM_FEEDBACK_BUFFER_BIT_EXT;
}
if (device.IsBufferDeviceAddressSupported()) {
stream_ci.usage |= VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT;
}
stream_buffer = memory_allocator.CreateBuffer(stream_ci, MemoryUsage::Stream);
if (device.HasDebuggingToolAttached()) {
stream_buffer.SetObjectNameEXT("Stream Buffer");
}
if (device.IsBufferDeviceAddressSupported()) {
stream_buffer_address = device.GetLogical().GetBufferDeviceAddress(*stream_buffer);
}
stream_pointer = stream_buffer.Mapped();
ASSERT_MSG(!stream_pointer.empty(), "Stream buffer must be host visible!");
}
@@ -149,6 +155,7 @@ StagingBufferRef StagingBufferPool::GetStreamBuffer(size_t size) {
iterator = Common::AlignUp(iterator + size, MAX_ALIGNMENT);
return StagingBufferRef{
.buffer = *stream_buffer,
.device_address = stream_buffer_address,
.offset = static_cast<VkDeviceSize>(offset),
.mapped_span = stream_pointer.subspan(offset, size),
.usage{},
@@ -212,14 +219,22 @@ StagingBufferRef StagingBufferPool::CreateStagingBuffer(size_t size, MemoryUsage
if (device.IsExtTransformFeedbackSupported()) {
buffer_ci.usage |= VK_BUFFER_USAGE_TRANSFORM_FEEDBACK_BUFFER_BIT_EXT;
}
if (device.IsBufferDeviceAddressSupported()) {
buffer_ci.usage |= VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT;
}
vk::Buffer buffer = memory_allocator.CreateBuffer(buffer_ci, usage);
if (device.HasDebuggingToolAttached()) {
++buffer_index;
buffer.SetObjectNameEXT(fmt::format("Staging Buffer {}", buffer_index).c_str());
}
const std::span<u8> mapped_span = buffer.Mapped();
const VkDeviceAddress buffer_address =
device.IsBufferDeviceAddressSupported()
? device.GetLogical().GetBufferDeviceAddress(*buffer)
: VkDeviceAddress{};
StagingBuffer& entry = GetCache(usage)[log2_size].entries.emplace_back(StagingBuffer{
.buffer = std::move(buffer),
.device_address = buffer_address,
.mapped_span = mapped_span,
.usage = usage,
.log2_level = log2_size,
@@ -252,62 +267,25 @@ void StagingBufferPool::ReleaseLevel(StagingBuffersCache& cache, size_t log2) {
constexpr size_t deletions_per_tick = 16;
auto& staging = cache[log2];
auto& entries = staging.entries;
if (entries.empty()) {
staging.delete_index = 0;
staging.iterate_index = 0;
return;
}
const size_t old_size = entries.size();
const auto is_deletable = [this](const StagingBuffer& entry) {
return scheduler.IsFree(entry.tick);
};
const size_t begin_offset = (std::min)(staging.delete_index, entries.size());
const size_t end_offset = (std::min)(begin_offset + deletions_per_tick, entries.size());
const size_t begin_offset = staging.delete_index;
const size_t end_offset = (std::min)(begin_offset + deletions_per_tick, old_size);
const auto begin = entries.begin() + begin_offset;
const auto end = entries.begin() + end_offset;
const auto surviving_end = std::remove_if(begin, end, is_deletable);
const size_t removed = static_cast<size_t>(std::distance(surviving_end, end));
entries.erase(surviving_end, end);
entries.erase(std::remove_if(begin, end, is_deletable), end);
staging.delete_index = end_offset - removed;
if (staging.delete_index >= entries.size()) {
const size_t new_size = entries.size();
staging.delete_index += deletions_per_tick;
if (staging.delete_index >= new_size) {
staging.delete_index = 0;
}
if (staging.iterate_index > entries.size()) {
if (staging.iterate_index > new_size) {
staging.iterate_index = 0;
}
}
u64 StagingBufferPool::ReclaimMemory(u64 target_bytes) {
u64 freed = 0;
const auto is_deletable = [this](const StagingBuffer& entry) {
return scheduler.IsFree(entry.tick);
};
const auto reclaim_cache = [&](StagingBuffersCache& cache) {
for (size_t level = NUM_LEVELS; level-- > 0 && freed < target_bytes;) {
auto& staging = cache[level];
auto& entries = staging.entries;
if (entries.empty()) {
continue;
}
const u64 entry_bytes = 1ULL << level;
auto it = entries.begin();
while (it != entries.end() && freed < target_bytes) {
if (is_deletable(*it)) {
it = entries.erase(it);
freed += entry_bytes;
} else {
++it;
}
}
staging.delete_index = 0;
staging.iterate_index = 0;
}
};
reclaim_cache(device_local_cache);
reclaim_cache(upload_cache);
reclaim_cache(download_cache);
return freed;
}
} // namespace Vulkan
@@ -21,6 +21,7 @@ class Scheduler;
struct StagingBufferRef {
VkBuffer buffer;
VkDeviceAddress device_address;
VkDeviceSize offset;
std::span<u8> mapped_span;
MemoryUsage usage;
@@ -45,8 +46,6 @@ public:
void TickFrame();
u64 ReclaimMemory(u64 target_bytes);
private:
struct StreamBufferCommit {
size_t upper_bound;
@@ -55,6 +54,7 @@ private:
struct StagingBuffer {
vk::Buffer buffer;
VkDeviceAddress device_address;
std::span<u8> mapped_span;
MemoryUsage usage;
u32 log2_level;
@@ -65,6 +65,7 @@ private:
StagingBufferRef Ref() const noexcept {
return {
.buffer = *buffer,
.device_address = device_address,
.offset = 0,
.mapped_span = mapped_span,
.usage = usage,
@@ -108,6 +109,7 @@ private:
Scheduler& scheduler;
vk::Buffer stream_buffer;
VkDeviceAddress stream_buffer_address{};
std::span<u8> stream_pointer;
VkDeviceSize stream_buffer_size;
VkDeviceSize region_size;
@@ -9,6 +9,10 @@
#include <limits>
#include <vector>
#ifdef __ANDROID__
#include <android/api-level.h>
#endif
#include "common/logging.h"
#include "common/settings.h"
#include "common/settings_enums.h"
@@ -172,26 +176,34 @@ bool Swapchain::AcquireNextImage() {
break;
}
#ifdef __ANDROID__
scheduler.WaitSubmitted(resource_ticks[image_index]);
#else
const auto wait_with_frame_pacing = [this] {
switch (Settings::values.frame_pacing_mode.GetValue()) {
case Settings::FramePacingMode::Target_Auto:
scheduler.WaitSubmitted(resource_ticks[image_index]);
scheduler.Wait(resource_ticks[image_index]);
break;
case Settings::FramePacingMode::Target_30:
scheduler.WaitSubmitted(resource_ticks[image_index], 30.0);
scheduler.Wait(resource_ticks[image_index], 30.0);
break;
case Settings::FramePacingMode::Target_60:
scheduler.WaitSubmitted(resource_ticks[image_index], 60.0);
scheduler.Wait(resource_ticks[image_index], 60.0);
break;
case Settings::FramePacingMode::Target_90:
scheduler.WaitSubmitted(resource_ticks[image_index], 90.0);
scheduler.Wait(resource_ticks[image_index], 90.0);
break;
case Settings::FramePacingMode::Target_120:
scheduler.WaitSubmitted(resource_ticks[image_index], 120.0);
scheduler.Wait(resource_ticks[image_index], 120.0);
break;
}
};
#ifdef __ANDROID__
if (android_get_device_api_level() >= 30) {
scheduler.Wait(resource_ticks[image_index]);
} else {
wait_with_frame_pacing();
}
#else
wait_with_frame_pacing();
#endif
resource_ticks[image_index] = scheduler.CurrentTick();
@@ -207,11 +207,7 @@ constexpr VkBorderColor ConvertBorderColor(const std::array<float, 4>& color) {
return device.IsFormatSupported(view_format, VK_FORMAT_FEATURE_STORAGE_IMAGE_BIT,
FormatType::Optimal);
});
const bool storage_allowed_for_samples =
image_ci.samples == VK_SAMPLE_COUNT_1_BIT ||
(device.GetStorageImageSampleCounts() &
static_cast<VkSampleCountFlags>(image_ci.samples)) != 0;
if (has_storage_compatible_view && storage_allowed_for_samples) {
if (has_storage_compatible_view) {
image_ci.usage |= VK_IMAGE_USAGE_STORAGE_BIT;
}
@@ -268,6 +264,10 @@ constexpr VkBorderColor ConvertBorderColor(const std::array<float, 4>& color) {
}
}
[[nodiscard]] bool IsLdrAstcFormat(VkFormat format) {
return format >= VK_FORMAT_ASTC_4x4_UNORM_BLOCK && format <= VK_FORMAT_ASTC_12x12_SRGB_BLOCK;
}
[[nodiscard]] VkImageAspectFlags ImageViewAspectMask(const VideoCommon::ImageViewInfo& info) {
if (info.IsRenderTarget()) {
return ImageAspectMask(info.format);
@@ -895,15 +895,6 @@ void BlitScale(Scheduler& scheduler, VkImage src_image, VkImage dst_image, const
0, nullptr, nullptr, write_barriers);
});
}
[[nodiscard]] bool CanBlitNatively(const Device& device, PixelFormat format) {
static constexpr auto OPTIMAL_FORMAT = FormatType::Optimal;
static constexpr VkFormatFeatureFlags BLIT_USAGE =
VK_FORMAT_FEATURE_BLIT_SRC_BIT | VK_FORMAT_FEATURE_BLIT_DST_BIT;
const VkFormat vk_format =
MaxwellToVK::SurfaceFormat(device, OPTIMAL_FORMAT, false, format).format;
return device.IsFormatSupported(vk_format, BLIT_USAGE, OPTIMAL_FORMAT);
}
} // Anonymous namespace
TextureCacheRuntime::TextureCacheRuntime(const Device& device_, Scheduler& scheduler_,
@@ -1236,19 +1227,27 @@ void TextureCacheRuntime::BlitImage(Framebuffer* dst_framebuffer, ImageView& dst
blit_image_helper.ResolveDepthStencil(dst_framebuffer, src, dst_region, src_region);
return;
}
static constexpr VkImageAspectFlags DEPTH_STENCIL_ASPECTS =
VK_IMAGE_ASPECT_DEPTH_BIT | VK_IMAGE_ASPECT_STENCIL_BIT;
if ((aspect_mask & DEPTH_STENCIL_ASPECTS) != 0 && !CanBlitNatively(device, src.format)) {
if (aspect_mask != DEPTH_STENCIL_ASPECTS) {
UNIMPLEMENTED_MSG("Host cannot blit format {} and no helper path exists for aspect "
"mask 0x{:x}",
src.format, aspect_mask);
if (aspect_mask == (VK_IMAGE_ASPECT_DEPTH_BIT | VK_IMAGE_ASPECT_STENCIL_BIT)) {
const auto format = src.format;
const auto can_blit_depth_stencil = [this, format] {
switch (format) {
case VideoCore::Surface::PixelFormat::D24_UNORM_S8_UINT:
case VideoCore::Surface::PixelFormat::S8_UINT_D24_UNORM:
return device.IsBlitDepth24Stencil8Supported();
case VideoCore::Surface::PixelFormat::D32_FLOAT_S8_UINT:
return device.IsBlitDepth32Stencil8Supported();
default:
UNREACHABLE();
}
}();
// Use shader-based depth/stencil blits if hardware doesn't support the format
// Note: MSAA resolves (MSAA->single) use vkCmdResolveImage which works fine
if (!can_blit_depth_stencil) {
UNIMPLEMENTED_IF(is_src_msaa || is_dst_msaa);
blit_image_helper.BlitDepthStencil(dst_framebuffer, src, dst_region, src_region,
filter, operation);
return;
}
UNIMPLEMENTED_IF(is_src_msaa || is_dst_msaa);
blit_image_helper.BlitDepthStencil(dst_framebuffer, src, dst_region, src_region, filter,
operation);
return;
}
ASSERT(!(is_dst_msaa && !is_src_msaa));
ASSERT(operation == Fermi2D::Operation::SrcCopy);
@@ -1643,14 +1642,7 @@ void TextureCacheRuntime::CopyImageMSAA(Image& dst, Image& src,
const u32 num_samples = msaa_to_non_msaa ? src.info.num_samples : dst.info.num_samples;
if (dst.AspectMask() != VK_IMAGE_ASPECT_COLOR_BIT ||
VideoCore::Surface::IsPixelFormatInteger(dst.info.format)) {
const u64 key{(static_cast<u64>(dst.AspectMask()) << 32) |
static_cast<u64>(dst.info.format)};
if (unsupported_msaa_resolves.insert(key).second) {
LOG_WARNING(Render_Vulkan,
"MSAA resolve unsupported: format={}, aspect={:#x}, samples {}->{}",
dst.info.format, dst.AspectMask(), src.info.num_samples,
dst.info.num_samples);
}
UNIMPLEMENTED_MSG("Copying images with different samples is not supported.");
return;
}
if (ENABLE_MSAA_RESOLVE_CONSUME && msaa_to_non_msaa && copies.size() == 1 &&
@@ -1760,20 +1752,6 @@ void TextureCacheRuntime::CopyImageMSAA(Image& dst, Image& src,
src.info.format, num_samples, copies, msaa_to_non_msaa);
}
u64 TextureCacheRuntime::CurrentSyncPoint() const noexcept {
return scheduler.CurrentTick();
}
u64 TextureCacheRuntime::CompletedSyncPoint() const {
auto& master_semaphore = scheduler.GetMasterSemaphore();
master_semaphore.Refresh();
return master_semaphore.KnownGpuTick();
}
void TextureCacheRuntime::WaitSyncPoint(u64 sync_point) {
scheduler.Wait(sync_point);
}
u64 TextureCacheRuntime::GetDeviceLocalMemory() const {
return device.GetDeviceLocalMemory();
}
@@ -1782,10 +1760,6 @@ u64 TextureCacheRuntime::GetDeviceMemoryUsage() const {
return device.GetDeviceMemoryUsage();
}
u64 TextureCacheRuntime::GetDeviceAllocationUsage() const {
return device.GetMemoryBudgetInfo().allocation_bytes;
}
bool TextureCacheRuntime::CanReportMemoryUsage() const {
return device.CanReportMemoryUsage();
}
@@ -1795,7 +1769,6 @@ std::optional<size_t> TextureCacheRuntime::GetSamplerHeapBudget() const {
}
void TextureCacheRuntime::TickFrame() {
device.TickAllocatorFrame();
std::erase_if(pending_msaa_images, [this](const auto& pending) {
return scheduler.IsFree(pending.first);
});
@@ -1833,9 +1806,7 @@ Image::Image(TextureCacheRuntime& runtime_, const ImageInfo& info_, GPUVAddr gpu
}
current_image = &Image::original_image;
storage_image_views.resize(info.resources.levels);
if (IsPixelFormatASTC(info.format) && !runtime->device.IsOptimalAstcSupported() &&
Settings::values.astc_recompression.GetValue() ==
Settings::AstcRecompression::Uncompressed) {
if (WillUseAcceleratedAstcDecode(runtime->device, info)) {
const auto& device = runtime->device.GetLogical();
const VkFormat storage_format = VK_FORMAT_A8B8G8R8_UNORM_PACK32;
for (s32 level = 0; level < info.resources.levels; ++level) {
@@ -1911,11 +1882,9 @@ void Image::UploadMemory(VkBuffer buffer, VkDeviceSize offset,
ScaleDown(true);
}
const bool is_color_upload = (aspect_mask & VK_IMAGE_ASPECT_COLOR_BIT) != 0
const bool wants_msaa_upload = info.num_samples > 1
&& (aspect_mask & VK_IMAGE_ASPECT_COLOR_BIT) != 0
&& !VideoCore::Surface::IsPixelFormatInteger(info.format);
const bool is_depth_upload = (aspect_mask & VK_IMAGE_ASPECT_DEPTH_BIT) != 0;
const bool wants_msaa_upload =
info.num_samples > 1 && (is_color_upload || is_depth_upload);
if (wants_msaa_upload) {
ImageInfo temp_info = info;
@@ -1954,10 +1923,10 @@ void Image::UploadMemory(VkBuffer buffer, VkDeviceSize offset,
image_copies.push_back(image_copy);
}
runtime->TransitionImageLayout(*this);
runtime->blit_image_helper.CopyMSAA(runtime->render_pass_cache, Handle(), info.format,
temp_vk_image, info.format, info.num_samples,
image_copies, false);
initialized = true;
runtime->pending_msaa_images.emplace_back(scheduler->CurrentTick(), std::move(temp_image));
if (is_rescaled) {
@@ -1968,9 +1937,6 @@ void Image::UploadMemory(VkBuffer buffer, VkDeviceSize offset,
if (info.num_samples > 1) {
LOG_WARNING(Render_Vulkan, "MSAA upload not implemented for format {}", info.format);
if (runtime != nullptr) {
runtime->TransitionImageLayout(*this);
}
if (is_rescaled) {
ScaleUp();
}
@@ -2422,9 +2388,18 @@ ImageView::ImageView(TextureCacheRuntime& runtime, const VideoCommon::ImageViewI
.pNext = nullptr,
.usage = clamped_view_usage,
};
const VkImageViewASTCDecodeModeEXT astc_decode_mode{
.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_ASTC_DECODE_MODE_EXT,
.pNext = &image_view_usage,
.decodeMode = VK_FORMAT_R8G8B8A8_UNORM,
};
const void* view_next = &image_view_usage;
if (device->IsExtAstcDecodeModeSupported() && IsLdrAstcFormat(format_info.format)) {
view_next = &astc_decode_mode;
}
const VkImageViewCreateInfo create_info{
.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO,
.pNext = &image_view_usage,
.pNext = view_next,
.flags = 0,
.image = image.Handle(),
.viewType = VkImageViewType{},
@@ -2547,12 +2522,11 @@ VkImageView ImageView::StorageView(Shader::TextureType texture_type,
Shader::ImageFormat image_format) {
if (image_handle) {
if (image_format == Shader::ImageFormat::Typeless) {
auto& view{typeless_storage_views[static_cast<size_t>(texture_type)]};
if (!view) {
if (!typeless_storage_view) {
auto info = MaxwellToVK::SurfaceFormat(*device, FormatType::Optimal, true, format);
view = MakeView(info.format, VK_IMAGE_ASPECT_COLOR_BIT, texture_type);
typeless_storage_view = MakeView(info.format, VK_IMAGE_ASPECT_COLOR_BIT, texture_type);
}
return *view;
return *typeless_storage_view;
}
const bool is_signed = image_format == Shader::ImageFormat::R8_SINT
|| image_format == Shader::ImageFormat::R16_SINT;
@@ -60,22 +60,10 @@ public:
void TickFrame();
u64 CurrentSyncPoint() const noexcept;
u64 CompletedSyncPoint() const;
void WaitSyncPoint(u64 sync_point);
u64 GetDeviceLocalMemory() const;
u64 GetDeviceMemoryUsage() const;
u64 GetDeviceAllocationUsage() const;
bool CanReportAllocationUsage() const noexcept {
return true;
}
bool CanReportMemoryUsage() const;
std::optional<size_t> GetSamplerHeapBudget() const;
@@ -168,7 +156,6 @@ public:
std::array<vk::Buffer, indexing_slots> buffers{};
std::vector<std::pair<u64, vk::Image>> pending_msaa_images;
ankerl::unordered_dense::map<VkImage, ResolveShadow> resolve_shadows;
ankerl::unordered_dense::set<u64> unsupported_msaa_resolves;
};
class Framebuffer {
@@ -439,7 +426,7 @@ private:
std::array<vk::ImageView, Shader::NUM_TEXTURE_TYPES> image_views;
std::optional<StorageViews> storage_views;
std::array<vk::ImageView, Shader::NUM_TEXTURE_TYPES> typeless_storage_views;
vk::ImageView typeless_storage_view;
vk::ImageView depth_view;
vk::ImageView stencil_view;
vk::ImageView color_view;
@@ -499,7 +486,6 @@ struct TextureCacheParams {
static constexpr bool HAS_EMULATED_COPIES = false;
static constexpr bool HAS_DEVICE_MEMORY_INFO = true;
static constexpr bool IMPLEMENTS_ASYNC_DOWNLOADS = true;
static constexpr bool HAS_TIMELINE_SYNC_POINTS = true;
using Runtime = Vulkan::TextureCacheRuntime;
using Image = Vulkan::Image;
@@ -16,8 +16,10 @@
namespace Vulkan {
UpdateDescriptorQueue::UpdateDescriptorQueue(const Device& device_, size_t frame_payload_size_)
UpdateDescriptorQueue::UpdateDescriptorQueue(const Device& device_, size_t frame_payload_size_,
bool supports_descriptor_buffer_)
: device{device_}, frame_payload_size{frame_payload_size_},
supports_descriptor_buffer{supports_descriptor_buffer_},
payload(frame_payload_size_ * FRAMES_IN_FLIGHT)
{
payload_start = payload.data();
@@ -34,7 +36,9 @@ void UpdateDescriptorQueue::TickFrame() {
payload_cursor = payload_start;
}
void UpdateDescriptorQueue::Acquire(Scheduler& scheduler, size_t required_entries) {
void UpdateDescriptorQueue::Acquire(Scheduler& scheduler, size_t required_entries,
bool use_descriptor_buffer_) {
use_descriptor_buffer = supports_descriptor_buffer && use_descriptor_buffer_;
static constexpr size_t DEFAULT_REQUIRED_ENTRIES = 0x400;
const size_t reserve = required_entries > 0 ? required_entries : DEFAULT_REQUIRED_ENTRIES;
ASSERT_MSG(reserve < frame_payload_size, "Descriptor reservation {} >= frame capacity {}",
@@ -15,15 +15,23 @@ namespace Vulkan {
class Device;
class Scheduler;
struct DescriptorAddress {
VkDeviceAddress address;
VkDeviceSize range;
VkFormat format;
};
union DescriptorUpdateEntry {
DescriptorUpdateEntry() = default;
DescriptorUpdateEntry(VkDescriptorImageInfo image_) : image{image_} {}
DescriptorUpdateEntry(VkDescriptorBufferInfo buffer_) : buffer{buffer_} {}
DescriptorUpdateEntry(VkBufferView texel_buffer_) : texel_buffer{texel_buffer_} {}
DescriptorUpdateEntry(DescriptorAddress address_) : address{address_} {}
std::monostate empty{};
VkDescriptorImageInfo image;
VkDescriptorBufferInfo buffer;
VkBufferView texel_buffer;
DescriptorAddress address;
};
class UpdateDescriptorQueue final {
@@ -35,11 +43,17 @@ public:
static constexpr size_t GUEST_FRAME_PAYLOAD_SIZE = 0x80000;
static constexpr size_t COMPUTE_FRAME_PAYLOAD_SIZE = 0x20000;
explicit UpdateDescriptorQueue(const Device& device_, size_t frame_payload_size_);
explicit UpdateDescriptorQueue(const Device& device_, size_t frame_payload_size_,
bool supports_descriptor_buffer_ = false);
~UpdateDescriptorQueue();
[[nodiscard]] bool UsesDescriptorBuffer() const noexcept {
return use_descriptor_buffer;
}
void TickFrame();
void Acquire(Scheduler& scheduler, size_t required_entries = 0);
void Acquire(Scheduler& scheduler, size_t required_entries = 0,
bool use_descriptor_buffer_ = false);
const DescriptorUpdateEntry* UpdateData() const noexcept {
return upload_start;
@@ -69,13 +83,41 @@ public:
};
}
void AddBuffer(VkBuffer buffer, VkDeviceAddress base_address, VkDeviceSize offset,
VkDeviceSize size) {
if (!use_descriptor_buffer) {
AddBuffer(buffer, offset, size);
return;
}
*(payload_cursor++) = DescriptorAddress{
.address = base_address == 0 ? 0 : base_address + offset,
.range = base_address == 0 ? VK_WHOLE_SIZE : size,
.format = VK_FORMAT_UNDEFINED,
};
}
void AddTexelBuffer(VkBufferView texel_buffer) {
*(payload_cursor++) = texel_buffer;
}
void AddTexelBuffer(VkBufferView texel_buffer, VkDeviceAddress base_address,
VkDeviceSize offset, VkDeviceSize size, VkFormat format) {
if (!use_descriptor_buffer) {
AddTexelBuffer(texel_buffer);
return;
}
*(payload_cursor++) = DescriptorAddress{
.address = base_address == 0 ? 0 : base_address + offset,
.range = base_address == 0 ? VK_WHOLE_SIZE : size,
.format = format,
};
}
private:
const Device& device;
const size_t frame_payload_size;
const bool supports_descriptor_buffer;
bool use_descriptor_buffer{false};
size_t frame_index{0};
DescriptorUpdateEntry* payload_cursor = nullptr;
DescriptorUpdateEntry* payload_start = nullptr;
+1 -18
View File
@@ -284,24 +284,7 @@ std::optional<u64> GenericEnvironment::TryFindSize() {
Tegra::Texture::TICEntry GenericEnvironment::ReadTextureInfo(GPUVAddr tic_addr, u32 tic_limit,
bool via_header_index, u32 raw) {
const auto handle{Tegra::Texture::TexturePair(raw, via_header_index)};
if (handle.first > tic_limit) {
LOG_CRITICAL(Shader,
"TIC index out of range: raw=0x{:08x} tic_index={} tsc_index={} tic_limit={} "
"tic_addr=0x{:x} via_header_index={} stage={} program_base=0x{:x} "
"start_address=0x{:x}",
raw, handle.first, handle.second, tic_limit, tic_addr, via_header_index,
static_cast<u32>(stage), program_base, start_address);
ASSERT(handle.first <= tic_limit);
Tegra::Texture::TICEntry fallback{};
fallback.format.Assign(Tegra::Texture::TextureFormat::A8B8G8R8);
fallback.r_type.Assign(Tegra::Texture::ComponentType::UNORM);
fallback.g_type.Assign(Tegra::Texture::ComponentType::UNORM);
fallback.b_type.Assign(Tegra::Texture::ComponentType::UNORM);
fallback.a_type.Assign(Tegra::Texture::ComponentType::UNORM);
fallback.texture_type.Assign(Tegra::Texture::TextureType::Texture2D);
fallback.normalized_coords.Assign(1);
return fallback;
}
ASSERT(handle.first <= tic_limit);
const GPUVAddr descriptor_addr{tic_addr + handle.first * sizeof(Tegra::Texture::TICEntry)};
Tegra::Texture::TICEntry entry;
gpu_memory->ReadBlock(descriptor_addr, &entry, sizeof(entry));
+85 -168
View File
@@ -58,9 +58,23 @@ TextureCache<P>::TextureCache(Runtime& runtime_, Tegra::MaxwellDeviceMemoryManag
void(slot_samplers.insert(runtime, sampler_descriptor));
if constexpr (HAS_DEVICE_MEMORY_INFO) {
memory_budget = runtime.GetDeviceLocalMemory();
const s64 device_local_memory = static_cast<s64>(runtime.GetDeviceLocalMemory());
const s64 min_spacing_expected = device_local_memory - 1_GiB;
const s64 min_spacing_critical = device_local_memory - 512_MiB;
const s64 mem_threshold = (std::min)(device_local_memory, TARGET_THRESHOLD);
const s64 min_vacancy_expected = (6 * mem_threshold) / 10;
const s64 min_vacancy_critical = (2 * mem_threshold) / 10;
expected_memory = static_cast<u64>(
(std::max)((std::min)(device_local_memory - min_vacancy_expected, min_spacing_expected),
DEFAULT_EXPECTED_MEMORY));
critical_memory = static_cast<u64>(
(std::max)((std::min)(device_local_memory - min_vacancy_critical, min_spacing_critical),
DEFAULT_CRITICAL_MEMORY));
minimum_memory = static_cast<u64>((device_local_memory - mem_threshold) / 2);
} else {
memory_budget = FALLBACK_MEMORY_BUDGET;
expected_memory = DEFAULT_EXPECTED_MEMORY + 512_MiB;
critical_memory = DEFAULT_CRITICAL_MEMORY + 1_GiB;
minimum_memory = 0;
}
const bool gpu_unswizzle_enabled = Settings::values.gpu_unswizzle_enabled.GetValue();
@@ -100,154 +114,71 @@ TextureCache<P>::TextureCache(Runtime& runtime_, Tegra::MaxwellDeviceMemoryManag
}
template <class P>
void TextureCache<P>::QueueEvictionDownload(Image& image) {
auto copies = FullDownloadCopies(image.info);
auto staging = runtime.DownloadStagingBuffer(image.unswizzled_size_bytes, true);
image.DownloadMemory(staging, FixSmallVectorADL(copies));
pending_eviction_downloads.push_back(PendingEvictionDownload{
.staging = staging,
.gpu_memory = gpu_memory,
.copies = std::move(copies),
.info = image.info,
.gpu_addr = image.gpu_addr,
.sync_point = runtime.CurrentSyncPoint(),
});
}
template <class P>
void TextureCache<P>::TickEvictionDownloads(u64 completed_sync_point) {
while (!pending_eviction_downloads.empty() &&
pending_eviction_downloads.front().sync_point <= completed_sync_point) {
auto& entry = pending_eviction_downloads.front();
SwizzleImage(*entry.gpu_memory, entry.gpu_addr, entry.info, FixSmallVectorADL(entry.copies),
entry.staging.mapped_span.subspan(entry.staging.offset), swizzle_data_buffer);
runtime.FreeDeferredStagingBuffer(entry.staging);
pending_eviction_downloads.pop_front();
}
}
template <class P>
void TextureCache<P>::FlushEvictionDownloads() {
if (pending_eviction_downloads.empty()) {
return;
}
const u64 last_sync_point = pending_eviction_downloads.back().sync_point;
runtime.WaitSyncPoint(last_sync_point);
TickEvictionDownloads(last_sync_point);
}
template <class P>
u64 TextureCache<P>::ImageSizeBytes(const ImageBase& image) {
u64 tentative_size = (std::max)(image.guest_size_bytes, image.unswizzled_size_bytes);
if ((IsPixelFormatASTC(image.info.format) &&
True(image.flags & ImageFlagBits::AcceleratedUpload)) ||
True(image.flags & ImageFlagBits::Converted)) {
tentative_size = TranscodedAstcSize(tentative_size, image.info.format);
}
u64 size = Common::AlignUp(tentative_size, 1024);
if (image.HasScaled()) {
size += GetScaledImageSizeBytes(image);
}
return size;
}
template <class P>
u64 TextureCache<P>::DeviceUsage(bool force_refresh) {
if (!runtime.CanReportAllocationUsage()) {
return total_used_memory;
}
if (force_refresh || usage_refresh_countdown == 0) {
cached_device_usage = runtime.GetDeviceAllocationUsage();
usage_refresh_countdown = USAGE_REFRESH_INTERVAL;
} else {
--usage_refresh_countdown;
}
return cached_device_usage;
}
template <class P>
u64 TextureCache<P>::ReclaimMemory(u64 target_bytes, bool allow_download) {
if (target_bytes == 0 || in_reclaim) {
return 0;
}
in_reclaim = true;
const u64 drain_point = runtime.CompletedSyncPoint();
TickEvictionDownloads(drain_point);
sentenced_images.Reclaim(drain_point);
sentenced_image_view.Reclaim(drain_point);
sentenced_framebuffers.Reclaim(drain_point);
u64 freed = 0;
const auto evict = [&](ImageId image_id) {
if (freed >= target_bytes) {
void TextureCache<P>::RunGarbageCollector() {
bool high_priority_mode = false;
bool aggressive_mode = false;
u64 ticks_to_destroy = 0;
size_t num_iterations = 0;
const auto Configure = [&](bool allow_aggressive) {
high_priority_mode = total_used_memory >= expected_memory;
aggressive_mode = allow_aggressive && total_used_memory >= critical_memory;
ticks_to_destroy = aggressive_mode ? 10ULL : high_priority_mode ? 25ULL : 50ULL;
num_iterations = aggressive_mode ? 40 : (high_priority_mode ? 20 : 10);
};
const auto Cleanup = [this, &num_iterations, &high_priority_mode, &aggressive_mode](ImageId image_id) {
if (num_iterations == 0) {
return true;
}
--num_iterations;
auto& image = slot_images[image_id];
if (True(image.flags & ImageFlagBits::IsDecoding)) {
return false;
}
const bool must_download =
image.IsSafeDownload() && False(image.flags & ImageFlagBits::BadOverlap);
bool queued_download = false;
if (must_download) {
if constexpr (HAS_TIMELINE_SYNC_POINTS) {
if (!allow_download) {
return false;
}
QueueEvictionDownload(image);
queued_download = true;
} else {
return false;
}
const bool must_download = image.IsSafeDownload() && False(image.flags & ImageFlagBits::BadOverlap);
if ((!aggressive_mode && True(image.flags & ImageFlagBits::CostlyLoad)) || (!high_priority_mode && must_download)) {
return false;
}
if (must_download) {
auto map = runtime.DownloadStagingBuffer(image.unswizzled_size_bytes);
const auto copies = FixSmallVectorADL(FullDownloadCopies(image.info));
image.DownloadMemory(map, copies);
runtime.Finish();
SwizzleImage(*gpu_memory, image.gpu_addr, image.info, copies, map.mapped_span, swizzle_data_buffer);
}
const u64 image_bytes = ImageSizeBytes(image);
if (True(image.flags & ImageFlagBits::Tracked)) {
UntrackImage(image, image_id);
}
UnregisterImage(image_id);
DeleteImage(image_id, !queued_download && image.scale_tick > frame_tick + 5);
freed += image_bytes;
DeleteImage(image_id, image.scale_tick > frame_tick + 5);
if (aggressive_mode && total_used_memory < critical_memory) {
num_iterations >>= 2;
aggressive_mode = false;
} else if (high_priority_mode && total_used_memory < expected_memory) {
num_iterations >>= 1;
high_priority_mode = false;
}
return false;
};
const u64 cold_tick =
frame_tick > RECLAIM_GUARD_FRAMES ? frame_tick - RECLAIM_GUARD_FRAMES : 0;
lru_cache.ForEachItemBelow(cold_tick, evict);
if (freed < target_bytes) {
lru_cache.ForEachItemBelow(frame_tick > 0 ? frame_tick - 1 : 0, evict);
Configure(false);
lru_cache.ForEachItemBelow(frame_tick - ticks_to_destroy, Cleanup);
if (total_used_memory >= critical_memory) {
Configure(true);
lru_cache.ForEachItemBelow(frame_tick - ticks_to_destroy, Cleanup);
}
const u64 exit_point = runtime.CompletedSyncPoint();
sentenced_images.Reclaim(exit_point);
sentenced_image_view.Reclaim(exit_point);
sentenced_framebuffers.Reclaim(exit_point);
in_reclaim = false;
usage_refresh_countdown = 0;
reclaim_stalled = freed == 0;
return freed;
}
template <class P>
void TextureCache<P>::EnsureHeadroom(bool allow_download) {
if (reclaim_stalled) {
return;
}
const u64 limit = memory_budget > RECLAIM_HEADROOM ? memory_budget - RECLAIM_HEADROOM : 0;
const u64 usage = DeviceUsage(false);
if (usage <= limit) {
return;
}
const u64 target = (limit / 100) * RECLAIM_TARGET_PERCENT;
ReclaimMemory((std::min)(usage - target, total_used_memory), allow_download);
}
template <class P>
void TextureCache<P>::TickFrame() {
usage_refresh_countdown = 0;
reclaim_stalled = false;
EnsureHeadroom(true);
const u64 completed_sync_point = runtime.CompletedSyncPoint();
TickEvictionDownloads(completed_sync_point);
sentenced_images.Reclaim(completed_sync_point);
sentenced_framebuffers.Reclaim(completed_sync_point);
sentenced_image_view.Reclaim(completed_sync_point);
// If we can obtain the memory info, use it instead of the estimate.
if (runtime.CanReportMemoryUsage()) {
total_used_memory = runtime.GetDeviceMemoryUsage();
}
if (total_used_memory > minimum_memory) {
RunGarbageCollector();
}
sentenced_images.Tick();
sentenced_framebuffers.Tick();
sentenced_image_view.Tick();
TickAsyncDecode();
TickAsyncUnswizzle();
@@ -665,7 +596,6 @@ void TextureCache<P>::WriteMemory(DAddr cpu_addr, size_t size) {
template <class P>
void TextureCache<P>::DownloadMemory(DAddr cpu_addr, size_t size) {
FlushEvictionDownloads();
boost::container::small_vector<ImageId, 16> images;
ForEachImageInRegion(cpu_addr, size, [&images](ImageId image_id, ImageBase& image) {
if (!image.IsSafeDownload()) {
@@ -964,7 +894,6 @@ void TextureCache<P>::CommitAsyncFlushes() {
template <class P>
void TextureCache<P>::PopAsyncFlushes() {
TickEvictionDownloads(runtime.CompletedSyncPoint());
if (committed_downloads.empty()) {
return;
}
@@ -1365,9 +1294,8 @@ void TextureCache<P>::InvalidateScale(Image& image) {
}
RemoveImageViewReferences(image_view_ids);
RemoveFramebuffers(image_view_ids);
const u64 sync_point = runtime.CurrentSyncPoint();
for (const ImageViewId image_view_id : image_view_ids) {
sentenced_image_view.Push(std::move(slot_image_views[image_view_id]), sync_point);
sentenced_image_view.Push(std::move(slot_image_views[image_view_id]));
slot_image_views.erase(image_view_id);
}
image.image_view_ids.clear();
@@ -1403,7 +1331,6 @@ void TextureCache<P>::QueueAsyncDecode(Image& image, ImageId image_id) {
LOG_INFO(HW_GPU, "Queuing async texture decode");
image.flags |= ImageFlagBits::IsDecoding;
runtime.TransitionImageLayout(image);
auto decode = std::make_unique<AsyncDecodeContext>();
auto* decode_ptr = decode.get();
decode->image_id = image_id;
@@ -1436,7 +1363,6 @@ void TextureCache<P>::QueueAsyncUnswizzle(Image& image, ImageId image_id) {
}
image.flags |= ImageFlagBits::IsDecoding;
runtime.TransitionImageLayout(image);
unswizzle_queue.push_back({
.image_id = image_id,
@@ -1597,7 +1523,6 @@ ImageId TextureCache<P>::InsertImage(const ImageInfo& info, GPUVAddr gpu_addr,
template <class P>
ImageId TextureCache<P>::JoinImages(const ImageInfo& info, GPUVAddr gpu_addr, DAddr cpu_addr) {
EnsureHeadroom(false);
ImageInfo new_info = info;
const size_t size_bytes = CalculateGuestSizeInBytes(new_info);
const bool broken_views = runtime.HasBrokenTextureViewFormats();
@@ -1706,28 +1631,7 @@ ImageId TextureCache<P>::JoinImages(const ImageInfo& info, GPUVAddr gpu_addr, DA
for (const ImageId overlap_id : join_ignore_textures) {
Image& overlap = slot_images[overlap_id];
if (True(overlap.flags & ImageFlagBits::GpuModified)) {
if (new_image.TryFindBase(overlap.gpu_addr) &&
(!can_rescale || ImageCanRescale(overlap))) {
if (can_rescale) {
ScaleUp(overlap);
} else {
ScaleDown(overlap);
}
join_copies_to_do.emplace_back(JoinCopy{false, overlap_id});
continue;
}
if (overlap.IsSafeDownload() && False(overlap.flags & ImageFlagBits::BadOverlap) &&
gpu_memory->GpuToCpuAddress(overlap.gpu_addr).has_value()) {
QueueEvictionDownload(overlap);
} else {
LOG_WARNING(HW_GPU,
"Dropping GPU modified overlap, contents are not recoverable: "
"gpu_addr=0x{:x} format={} size={}x{}x{} levels={} layers={}",
overlap.gpu_addr, static_cast<int>(overlap.info.format),
overlap.info.size.width, overlap.info.size.height,
overlap.info.size.depth, overlap.info.resources.levels,
overlap.info.resources.layers);
}
UNIMPLEMENTED();
}
if (True(overlap.flags & ImageFlagBits::Tracked)) {
UntrackImage(overlap, overlap_id);
@@ -2281,7 +2185,13 @@ void TextureCache<P>::RegisterImage(ImageId image_id) {
ASSERT_MSG(False(image.flags & ImageFlagBits::Registered),
"Trying to register an already registered image");
image.flags |= ImageFlagBits::Registered;
total_used_memory += ImageSizeBytes(image);
u64 tentative_size = (std::max)(image.guest_size_bytes, image.unswizzled_size_bytes);
if ((IsPixelFormatASTC(image.info.format) &&
True(image.flags & ImageFlagBits::AcceleratedUpload)) ||
True(image.flags & ImageFlagBits::Converted)) {
tentative_size = TranscodedAstcSize(tentative_size, image.info.format);
}
total_used_memory += Common::AlignUp(tentative_size, 1024);
image.lru_index = lru_cache.Insert(image_id, frame_tick);
ForEachGPUPage(image.gpu_addr, image.guest_size_bytes, [this, image_id](u64 page) {
@@ -2444,7 +2354,16 @@ void TextureCache<P>::UntrackImage(ImageBase& image, ImageId image_id) {
template <class P>
void TextureCache<P>::DeleteImage(ImageId image_id, bool immediate_delete) {
ImageBase& image = slot_images[image_id];
total_used_memory -= std::min<u64>(total_used_memory, ImageSizeBytes(image));
if (image.HasScaled()) {
total_used_memory -= GetScaledImageSizeBytes(image);
}
u64 tentative_size = (std::max)(image.guest_size_bytes, image.unswizzled_size_bytes);
if ((IsPixelFormatASTC(image.info.format) &&
True(image.flags & ImageFlagBits::AcceleratedUpload)) ||
True(image.flags & ImageFlagBits::Converted)) {
tentative_size = TranscodedAstcSize(tentative_size, image.info.format);
}
total_used_memory -= Common::AlignUp(tentative_size, 1024);
const GPUVAddr gpu_addr = image.gpu_addr;
const auto alloc_it = image_allocs_table.find(gpu_addr);
if (alloc_it == image_allocs_table.end()) {
@@ -2498,15 +2417,14 @@ void TextureCache<P>::DeleteImage(ImageId image_id, bool immediate_delete) {
ASSERT_MSG(num_removed_overlaps == 1, "Invalid number of removed overlapps: {}",
num_removed_overlaps);
}
const u64 sync_point = runtime.CurrentSyncPoint();
for (const ImageViewId image_view_id : image_view_ids) {
if (!immediate_delete) {
sentenced_image_view.Push(std::move(slot_image_views[image_view_id]), sync_point);
sentenced_image_view.Push(std::move(slot_image_views[image_view_id]));
}
slot_image_views.erase(image_view_id);
}
if (!immediate_delete) {
sentenced_images.Push(std::move(slot_images[image_id]), sync_point);
sentenced_images.Push(std::move(slot_images[image_id]));
}
slot_images.erase(image_id);
@@ -2552,8 +2470,7 @@ void TextureCache<P>::RemoveFramebuffers(std::span<const ImageViewId> removed_vi
last_framebuffer_id = {};
last_framebuffer_serial = 0;
}
sentenced_framebuffers.Push(std::move(slot_framebuffers[framebuffer_id]),
runtime.CurrentSyncPoint());
sentenced_framebuffers.Push(std::move(slot_framebuffers[framebuffer_id]));
it = framebuffers.erase(it);
} else {
++it;
@@ -30,7 +30,7 @@
#include "common/thread_worker.h"
#include "video_core/compatible_formats.h"
#include "video_core/control/channel_state_cache.h"
#include "video_core/deferred_destruction_queue.h"
#include "video_core/delayed_destruction_ring.h"
#include "video_core/engines/fermi_2d.h"
#include "video_core/surface.h"
#include "video_core/texture_cache/descriptor_table.h"
@@ -108,20 +108,18 @@ class TextureCache : public VideoCommon::ChannelSetupCaches<TextureCacheChannelI
static constexpr bool HAS_DEVICE_MEMORY_INFO = P::HAS_DEVICE_MEMORY_INFO;
/// True when the API can do asynchronous texture downloads.
static constexpr bool IMPLEMENTS_ASYNC_DOWNLOADS = P::IMPLEMENTS_ASYNC_DOWNLOADS;
static constexpr bool HAS_TIMELINE_SYNC_POINTS = P::HAS_TIMELINE_SYNC_POINTS;
static constexpr size_t UNSET_CHANNEL{(std::numeric_limits<size_t>::max)()};
#ifdef YUZU_LEGACY
static constexpr u64 RECLAIM_HEADROOM = 384_MiB;
static constexpr s64 TARGET_THRESHOLD = 3_GiB;
#else
static constexpr u64 RECLAIM_HEADROOM = 512_MiB;
static constexpr s64 TARGET_THRESHOLD = 4_GiB;
#endif
static constexpr u64 FALLBACK_MEMORY_BUDGET = 2_GiB;
static constexpr u32 USAGE_REFRESH_INTERVAL = 16;
static constexpr u64 RECLAIM_GUARD_FRAMES = 8;
static constexpr u64 RECLAIM_TARGET_PERCENT = 88;
static constexpr s64 DEFAULT_EXPECTED_MEMORY = 1_GiB + 125_MiB;
static constexpr s64 DEFAULT_CRITICAL_MEMORY = 1_GiB + 625_MiB;
static constexpr size_t GC_EMERGENCY_COUNTS = 2;
using Runtime = typename P::Runtime;
using Image = typename P::Image;
@@ -156,8 +154,6 @@ public:
/// Notify the cache that a new frame has been queued
void TickFrame();
u64 ReclaimMemory(u64 target_bytes, bool allow_download);
/// Return a constant reference to the given image view id
[[nodiscard]] const ImageView& GetImageView(ImageViewId id) const noexcept;
@@ -297,17 +293,8 @@ private:
void OnGPUASRegister(size_t map_id) final override;
u64 ImageSizeBytes(const ImageBase& image);
u64 DeviceUsage(bool force_refresh);
void EnsureHeadroom(bool allow_download);
void QueueEvictionDownload(Image& image);
void TickEvictionDownloads(u64 completed_sync_point);
void FlushEvictionDownloads();
/// Runs the Garbage Collector.
void RunGarbageCollector();
/// Find or create an image view in the guest descriptor table
ImageViewId VisitImageView(u32 index, bool compute);
@@ -464,11 +451,9 @@ private:
bool has_deleted_images = false;
bool is_rescaling = false;
u64 total_used_memory = 0;
u64 memory_budget = 0;
u64 cached_device_usage = 0;
u32 usage_refresh_countdown = 0;
bool in_reclaim = false;
bool reclaim_stalled = false;
u64 minimum_memory;
u64 expected_memory;
u64 critical_memory;
size_t gpu_unswizzle_maxsize = 0;
size_t swizzle_chunk_size = 0;
u32 swizzle_slices_per_batch = 0;
@@ -506,19 +491,14 @@ private:
};
Common::LeastRecentlyUsedCache<LRUItemParams> lru_cache;
DeferredDestructionQueue<Image> sentenced_images;
DeferredDestructionQueue<ImageView> sentenced_image_view;
DeferredDestructionQueue<Framebuffer> sentenced_framebuffers;
struct PendingEvictionDownload {
AsyncBuffer staging;
Tegra::MemoryManager* gpu_memory;
boost::container::small_vector<VideoCommon::BufferImageCopy, 16> copies;
VideoCommon::ImageInfo info;
GPUVAddr gpu_addr;
u64 sync_point;
};
std::deque<PendingEvictionDownload> pending_eviction_downloads;
#ifdef YUZU_LEGACY
static constexpr size_t TICKS_TO_DESTROY = 6;
#else
static constexpr size_t TICKS_TO_DESTROY = 8;
#endif
DelayedDestructionRing<Image, TICKS_TO_DESTROY> sentenced_images;
DelayedDestructionRing<ImageView, TICKS_TO_DESTROY> sentenced_image_view;
DelayedDestructionRing<Framebuffer, TICKS_TO_DESTROY> sentenced_framebuffers;
ankerl::unordered_dense::map<GPUVAddr, ImageAllocId> image_allocs_table;
@@ -529,8 +509,7 @@ private:
u64 frame_tick = 0;
u64 last_sampler_gc_frame = (std::numeric_limits<u64>::max)();
Common::ThreadWorker texture_decode_worker{1, "TextureDecoder", {},
Common::ThreadPlacement::Background};
Common::ThreadWorker texture_decode_worker{1, "TextureDecoder"};
std::vector<std::unique_ptr<AsyncDecodeContext>> async_decodes;
std::deque<PendingUnswizzle> unswizzle_queue;
+2 -2
View File
@@ -1,4 +1,4 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
@@ -747,7 +747,7 @@ boost::container::small_vector<ImageCopy, 16> MakeShrinkImageCopies(const ImageI
const bool is_dst_3d = dst.type == ImageType::e3D;
if (is_dst_3d) {
ASSERT(src.type == ImageType::e3D || src.resources.layers == 1);
ASSERT(src.type == ImageType::e3D);
ASSERT(src.resources.levels == 1);
}
const bool both_2d{src.type == ImageType::e2D && dst.type == ImageType::e2D};
+11
View File
@@ -1769,6 +1769,12 @@ static void DecompressBlock(std::span<const u8, 16> inBuf, const u32 blockWidth,
return;
}
if (weightParams.GetNumWeightValues() > 64) {
assert(false && "Too many weights in the weight grid");
FillError(outBuf, blockWidth, blockHeight);
return;
}
// Read num partitions
u32 nPartitions = strm.ReadBits<2>() + 1;
assert(nPartitions <= 4);
@@ -1805,6 +1811,11 @@ static void DecompressBlock(std::span<const u8, 16> inBuf, const u32 blockWidth,
// Remaining bits are color endpoint data...
u32 nWeightBits = weightParams.GetPackedBitSize();
if (nWeightBits < 24 || nWeightBits > 96) {
assert(false && "Invalid weight bit count");
FillError(outBuf, blockWidth, blockHeight);
return;
}
s32 remainingBits = 128 - nWeightBits - static_cast<int>(strm.GetBitsRead());
// Consider extra bits prior to texel data...
+2 -3
View File
@@ -1,4 +1,4 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
@@ -10,8 +10,7 @@ namespace Tegra::Texture {
Common::ThreadWorker& GetThreadWorkers() {
static Common::ThreadWorker workers{(std::max)(std::thread::hardware_concurrency(), 2U) / 2,
"ImageTranscode", {},
Common::ThreadPlacement::Background};
"ImageTranscode"};
return workers;
}
+60 -63
View File
@@ -5,7 +5,6 @@
// SPDX-License-Identifier: GPL-2.0-or-later
#include <algorithm>
#include <array>
#include <bitset>
#include <chrono>
#include <optional>
@@ -507,6 +506,8 @@ Device::Device(VkInstance instance_, vk::PhysicalDevice physical_, VkSurfaceKHR
if (is_qualcomm) {
must_emulate_scaled_formats = true;
LOG_WARNING(Render_Vulkan, "Qualcomm drivers require scaled vertex format emulation.");
has_broken_descriptor_aliasing = true;
LOG_WARNING(Render_Vulkan, "Qualcomm drivers have broken descriptor aliasing.");
LOG_WARNING(Render_Vulkan, "Qualcomm drivers have broken custom border color.");
RemoveExtensionFeature(extensions.custom_border_color, features.custom_border_color,
VK_EXT_CUSTOM_BORDER_COLOR_EXTENSION_NAME);
@@ -714,6 +715,37 @@ Device::Device(VkInstance instance_, vk::PhysicalDevice physical_, VkSurfaceKHR
RemoveExtensionFeature(extensions.vertex_input_dynamic_state, features.vertex_input_dynamic_state, VK_EXT_VERTEX_INPUT_DYNAMIC_STATE_EXTENSION_NAME);
}
// Descriptors feature list
{
auto& descriptor_indexing = features.descriptor_indexing;
descriptor_indexing.shaderInputAttachmentArrayDynamicIndexing = false;
descriptor_indexing.shaderUniformTexelBufferArrayDynamicIndexing = false;
descriptor_indexing.shaderStorageTexelBufferArrayDynamicIndexing = false;
descriptor_indexing.shaderUniformBufferArrayNonUniformIndexing = false;
descriptor_indexing.shaderStorageBufferArrayNonUniformIndexing = false;
descriptor_indexing.shaderInputAttachmentArrayNonUniformIndexing = false;
descriptor_indexing.descriptorBindingUniformBufferUpdateAfterBind = false;
descriptor_indexing.descriptorBindingSampledImageUpdateAfterBind = false;
descriptor_indexing.descriptorBindingStorageImageUpdateAfterBind = false;
descriptor_indexing.descriptorBindingStorageBufferUpdateAfterBind = false;
descriptor_indexing.descriptorBindingUniformTexelBufferUpdateAfterBind = false;
descriptor_indexing.descriptorBindingStorageTexelBufferUpdateAfterBind = false;
descriptor_indexing.descriptorBindingUpdateUnusedWhilePending = false;
descriptor_indexing.descriptorBindingVariableDescriptorCount = false;
descriptor_indexing.runtimeDescriptorArray = false;
}
// VK_EXT_descriptor_buffer requires VK_KHR_buffer_device_address
if (extensions.descriptor_buffer && !features.buffer_device_address.bufferDeviceAddress) {
LOG_WARNING(Render_Vulkan, "Descriptor buffer needs buffer device address, disabling.");
RemoveExtensionFeature(extensions.descriptor_buffer, features.descriptor_buffer,
VK_EXT_DESCRIPTOR_BUFFER_EXTENSION_NAME);
}
if (!extensions.descriptor_buffer) {
RemoveExtensionFeature(extensions.buffer_device_address, features.buffer_device_address,
VK_KHR_BUFFER_DEVICE_ADDRESS_EXTENSION_NAME);
}
logical = vk::Device::Create(physical, queue_cis, ExtensionListForVulkan(loaded_extensions), first_next, dld);
graphics_queue = logical.GetQueue(graphics_family);
@@ -727,13 +759,16 @@ Device::Device(VkInstance instance_, vk::PhysicalDevice physical_, VkSurfaceKHR
if (extensions.memory_budget) {
flags |= VMA_ALLOCATOR_CREATE_EXT_MEMORY_BUDGET_BIT;
}
if (extensions.buffer_device_address) {
flags |= VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT;
}
const VmaAllocatorCreateInfo allocator_info{
.flags = flags,
.physicalDevice = physical,
.device = *logical,
.preferredLargeHeapBlockSize = is_integrated
? (64u * 1024u * 1024u)
: (128u * 1024u * 1024u),
: (256u * 1024u * 1024u),
.pAllocationCallbacks = nullptr,
.pDeviceMemoryCallbacks = nullptr,
.pHeapSizeLimit = nullptr,
@@ -745,32 +780,12 @@ Device::Device(VkInstance instance_, vk::PhysicalDevice physical_, VkSurfaceKHR
vk::Check(vmaCreateAllocator(&allocator_info, &allocator));
{
const auto& limits = properties.properties.limits;
LOG_INFO(Render_Vulkan, "MSAA sample count support:");
LOG_INFO(Render_Vulkan, " framebufferColorSampleCounts: {:#x}",
limits.framebufferColorSampleCounts);
LOG_INFO(Render_Vulkan, " framebufferDepthSampleCounts: {:#x}",
limits.framebufferDepthSampleCounts);
LOG_INFO(Render_Vulkan, " framebufferStencilSampleCounts: {:#x}",
limits.framebufferStencilSampleCounts);
LOG_INFO(Render_Vulkan, " sampledImageColorSampleCounts: {:#x}",
limits.sampledImageColorSampleCounts);
LOG_INFO(Render_Vulkan, " sampledImageDepthSampleCounts: {:#x}",
limits.sampledImageDepthSampleCounts);
LOG_INFO(Render_Vulkan, " sampledImageIntegerSampleCounts:{:#x}",
limits.sampledImageIntegerSampleCounts);
LOG_INFO(Render_Vulkan, " storageImageSampleCounts: {:#x}",
limits.storageImageSampleCounts);
}
// Initialize GPU logging if enabled
InitializeGPULogging();
}
Device::~Device() {
ShutdownGPULogging();
vk::FlushDeletionQueue();
vmaDestroyAllocator(allocator);
}
@@ -838,18 +853,16 @@ bool Device::ComputeIsOptimalAstcSupported() const {
VK_FORMAT_ASTC_12x10_UNORM_BLOCK, VK_FORMAT_ASTC_12x10_SRGB_BLOCK,
VK_FORMAT_ASTC_12x12_UNORM_BLOCK, VK_FORMAT_ASTC_12x12_SRGB_BLOCK,
};
if (!features.features.textureCompressionASTC_LDR ||
!features.texture_compression_astc_hdr.textureCompressionASTC_HDR) {
if (!features.features.textureCompressionASTC_LDR) {
return false;
}
const auto format_feature_usage{VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT |
VK_FORMAT_FEATURE_BLIT_SRC_BIT |
VK_FORMAT_FEATURE_BLIT_DST_BIT |
VK_FORMAT_FEATURE_TRANSFER_SRC_BIT |
VK_FORMAT_FEATURE_TRANSFER_DST_BIT};
const VkFormatFeatureFlags format_feature_usage{
VK_FORMAT_FEATURE_SAMPLED_IMAGE_BIT | VK_FORMAT_FEATURE_SAMPLED_IMAGE_FILTER_LINEAR_BIT |
VK_FORMAT_FEATURE_TRANSFER_SRC_BIT | VK_FORMAT_FEATURE_TRANSFER_DST_BIT};
for (const auto format : astc_formats) {
const auto physical_format_properties{physical.GetFormatProperties(format)};
if ((physical_format_properties.optimalTilingFeatures & format_feature_usage) == 0) {
if ((physical_format_properties.optimalTilingFeatures & format_feature_usage) !=
format_feature_usage) {
return false;
}
}
@@ -995,6 +1008,10 @@ bool Device::GetSuitability(bool requires_swapchain) {
CHECK_EXTENSION(VK_KHR_SWAPCHAIN_EXTENSION_NAME);
}
if (instance_version < VK_API_VERSION_1_2) {
CHECK_EXTENSION(VK_EXT_DESCRIPTOR_INDEXING_EXTENSION_NAME);
}
#undef LOG_EXTENSION
#undef CHECK_EXTENSION
@@ -1105,6 +1122,11 @@ bool Device::GetSuitability(bool requires_swapchain) {
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PUSH_DESCRIPTOR_PROPERTIES_KHR;
SetNext(next, properties.push_descriptor);
}
if (extensions.descriptor_buffer) {
properties.descriptor_buffer.sType =
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_DESCRIPTOR_BUFFER_PROPERTIES_EXT;
SetNext(next, properties.descriptor_buffer);
}
if (extensions.subgroup_size_control || features.subgroup_size_control.subgroupSizeControl) {
properties.subgroup_size_control.sType =
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SUBGROUP_SIZE_CONTROL_PROPERTIES;
@@ -1231,20 +1253,15 @@ void Device::RemoveUnsuitableExtensions() {
RemoveExtensionFeatureIfUnsuitable(extensions.depth_bias_control, features.depth_bias_control,
VK_EXT_DEPTH_BIAS_CONTROL_EXTENSION_NAME);
// VK_EXT_depth_clamp_zero_one
extensions.depth_clamp_zero_one = features.depth_clamp_zero_one.depthClampZeroOne;
RemoveExtensionFeatureIfUnsuitable(extensions.depth_clamp_zero_one,
features.depth_clamp_zero_one,
VK_EXT_DEPTH_CLAMP_ZERO_ONE_EXTENSION_NAME);
// VK_EXT_depth_clip_control
extensions.depth_clip_control = features.depth_clip_control.depthClipControl;
RemoveExtensionFeatureIfUnsuitable(extensions.depth_clip_control, features.depth_clip_control,
VK_EXT_DEPTH_CLIP_CONTROL_EXTENSION_NAME);
// VK_EXT_depth_clip_enable
extensions.depth_clip_enable = features.depth_clip_enable.depthClipEnable;
RemoveExtensionFeatureIfUnsuitable(extensions.depth_clip_enable, features.depth_clip_enable,
VK_EXT_DEPTH_CLIP_ENABLE_EXTENSION_NAME);
// VK_EXT_descriptor_buffer
extensions.descriptor_buffer = features.descriptor_buffer.descriptorBuffer;
RemoveExtensionFeatureIfUnsuitable(extensions.descriptor_buffer, features.descriptor_buffer,
VK_EXT_DESCRIPTOR_BUFFER_EXTENSION_NAME);
// VK_EXT_extended_dynamic_state
extensions.extended_dynamic_state = features.extended_dynamic_state.extendedDynamicState;
@@ -1476,23 +1493,6 @@ std::optional<size_t> Device::GetSamplerHeapBudget() const {
return sampler_heap_budget;
}
Device::MemoryBudgetInfo Device::GetMemoryBudgetInfo() const {
std::array<VmaBudget, VK_MAX_MEMORY_HEAPS> budgets{};
vmaGetHeapBudgets(allocator, budgets.data());
MemoryBudgetInfo info{};
for (const size_t heap : valid_heap_memory) {
info.usage += budgets[heap].usage;
info.budget += budgets[heap].budget;
info.block_bytes += budgets[heap].statistics.blockBytes;
info.allocation_bytes += budgets[heap].statistics.allocationBytes;
}
return info;
}
void Device::TickAllocatorFrame() const {
vmaSetCurrentFrameIndex(allocator, ++allocator_frame_index);
}
u64 Device::GetDeviceMemoryUsage() const {
VkPhysicalDeviceMemoryBudgetPropertiesEXT budget;
budget.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MEMORY_BUDGET_PROPERTIES_EXT;
@@ -1542,12 +1542,9 @@ void Device::CollectPhysicalMemoryInfo() {
device_access_memory -= reserve_memory;
if (Settings::values.vram_usage_mode.GetValue() != Settings::VramUsageMode::Aggressive) {
// Account for resolution scaling in memory limits
const u64 normal_memory = 6_GiB;
const u64 scaler_memory = 1_GiB * Settings::values.resolution_info.ScaleUp(1);
const u64 baseline = normal_memory + scaler_memory;
const u64 proportional = (device_access_memory / 4) * 3;
device_access_memory =
std::min<u64>(device_access_memory, std::max<u64>(baseline, proportional));
const size_t normal_memory = 6_GiB;
const size_t scaler_memory = 1_GiB * Settings::values.resolution_info.ScaleUp(1);
device_access_memory = std::min<u64>(device_access_memory, normal_memory + scaler_memory);
}
}
}
+38 -64
View File
@@ -36,6 +36,7 @@ VK_DEFINE_HANDLE(VmaAllocator)
FEATURE(EXT, DescriptorIndexing, DESCRIPTOR_INDEXING, descriptor_indexing) \
FEATURE(EXT, HostQueryReset, HOST_QUERY_RESET, host_query_reset) \
FEATURE(KHR, 8BitStorage, 8BIT_STORAGE, bit8_storage) \
FEATURE(KHR, BufferDeviceAddress, BUFFER_DEVICE_ADDRESS, buffer_device_address) \
FEATURE(KHR, TimelineSemaphore, TIMELINE_SEMAPHORE, timeline_semaphore)
#define FOR_EACH_VK_FEATURE_1_3(FEATURE) \
@@ -54,9 +55,8 @@ VK_DEFINE_HANDLE(VmaAllocator)
FEATURE(EXT, ColorWriteEnable, COLOR_WRITE_ENABLE, color_write_enable) \
FEATURE(EXT, CustomBorderColor, CUSTOM_BORDER_COLOR, custom_border_color) \
FEATURE(EXT, DepthBiasControl, DEPTH_BIAS_CONTROL, depth_bias_control) \
FEATURE(EXT, DepthClampZeroOne, DEPTH_CLAMP_ZERO_ONE, depth_clamp_zero_one) \
FEATURE(EXT, DepthClipControl, DEPTH_CLIP_CONTROL, depth_clip_control) \
FEATURE(EXT, DepthClipEnable, DEPTH_CLIP_ENABLE, depth_clip_enable) \
FEATURE(EXT, DescriptorBuffer, DESCRIPTOR_BUFFER, descriptor_buffer) \
FEATURE(EXT, ExtendedDynamicState, EXTENDED_DYNAMIC_STATE, extended_dynamic_state) \
FEATURE(EXT, ExtendedDynamicState2, EXTENDED_DYNAMIC_STATE_2, extended_dynamic_state2) \
FEATURE(EXT, ExtendedDynamicState3, EXTENDED_DYNAMIC_STATE_3, extended_dynamic_state3) \
@@ -74,13 +74,12 @@ VK_DEFINE_HANDLE(VmaAllocator)
FEATURE(KHR, PipelineExecutableProperties, PIPELINE_EXECUTABLE_PROPERTIES, \
pipeline_executable_properties) \
FEATURE(KHR, WorkgroupMemoryExplicitLayout, WORKGROUP_MEMORY_EXPLICIT_LAYOUT, \
workgroup_memory_explicit_layout) \
FEATURE(EXT, TextureCompressionASTCHDR, TEXTURE_COMPRESSION_ASTC_HDR, \
texture_compression_astc_hdr)
workgroup_memory_explicit_layout)
// Define miscellaneous extensions which may be used by the implementation here.
#define FOR_EACH_VK_EXTENSION(EXTENSION) \
EXTENSION(EXT, ASTC_DECODE_MODE, astc_decode_mode) \
EXTENSION(EXT, CONDITIONAL_RENDERING, conditional_rendering) \
EXTENSION(EXT, CONSERVATIVE_RASTERIZATION, conservative_rasterization) \
EXTENSION(EXT, DEPTH_RANGE_UNRESTRICTED, depth_range_unrestricted) \
@@ -178,6 +177,8 @@ VK_DEFINE_HANDLE(VmaAllocator)
FEATURE_NAME(depth_bias_control, depthBiasControl) \
FEATURE_NAME(depth_bias_control, leastRepresentableValueForceUnormRepresentation) \
FEATURE_NAME(depth_bias_control, depthBiasExact) \
FEATURE_NAME(descriptor_indexing, descriptorBindingPartiallyBound) \
FEATURE_NAME(descriptor_indexing, shaderSampledImageArrayNonUniformIndexing) \
FEATURE_NAME(extended_dynamic_state, extendedDynamicState) \
FEATURE_NAME(format_a4b4g4r4, formatA4B4G4R4) \
FEATURE_NAME(robust_image_access, robustImageAccess) \
@@ -257,17 +258,6 @@ public:
return allocator;
}
struct MemoryBudgetInfo {
u64 usage;
u64 budget;
u64 block_bytes;
u64 allocation_bytes;
};
MemoryBudgetInfo GetMemoryBudgetInfo() const;
void TickAllocatorFrame() const;
/// Returns the logical device.
const vk::Device& GetLogical() const {
return logical;
@@ -397,7 +387,7 @@ FN_MAX_LIMIT_LIST
/// Returns true if descriptor aliasing is natively supported.
bool IsDescriptorAliasingSupported() const {
return GetDriverID() != VK_DRIVER_ID_QUALCOMM_PROPRIETARY;
return !has_broken_descriptor_aliasing;
}
bool IsSampledImageArrayNonUniformIndexingSupported() const {
@@ -480,6 +470,26 @@ FN_MAX_LIMIT_LIST
return properties.push_descriptor.maxPushDescriptors;
}
/// Returns true if robust buffer access is enabled on the device.
bool IsRobustBufferAccessEnabled() const {
return features.features.robustBufferAccess == VK_TRUE;
}
/// Returns true if the device supports descriptor buffers.
bool IsExtDescriptorBufferSupported() const {
return extensions.descriptor_buffer;
}
/// Returns the descriptor buffer properties of the device.
const VkPhysicalDeviceDescriptorBufferPropertiesEXT& DescriptorBufferProperties() const {
return properties.descriptor_buffer;
}
/// Returns true if the device supports buffer device address.
bool IsBufferDeviceAddressSupported() const {
return extensions.buffer_device_address;
}
/// Returns true if formatless image load is supported.
bool IsFormatlessImageLoadSupported() const {
return features.features.shaderStorageImageReadWithoutFormat;
@@ -614,16 +624,6 @@ FN_MAX_LIMIT_LIST
return extensions.depth_clip_control;
}
/// Returns true if the device supports VK_EXT_depth_clamp_zero_one.
bool IsExtDepthClampZeroOneSupported() const {
return extensions.depth_clamp_zero_one;
}
/// Returns true if the device supports VK_EXT_depth_clip_enable.
bool IsExtDepthClipEnableSupported() const {
return extensions.depth_clip_enable;
}
/// Returns true if the device supports VK_EXT_depth_bias_control.
bool IsExtDepthBiasControlSupported() const {
return extensions.depth_bias_control;
@@ -756,38 +756,6 @@ FN_MAX_LIMIT_LIST
return features.line_rasterization.stippledRectangularLines != VK_FALSE;
}
VkLineRasterizationModeEXT GetLineRasterizationMode(bool wants_smooth) const {
if (wants_smooth && SupportsSmoothLines()) {
return VK_LINE_RASTERIZATION_MODE_RECTANGULAR_SMOOTH_EXT;
}
if (SupportsRectangularLines()) {
return VK_LINE_RASTERIZATION_MODE_RECTANGULAR_EXT;
}
return VK_LINE_RASTERIZATION_MODE_DEFAULT_EXT;
}
bool SupportsStippleForMode(VkLineRasterizationModeEXT mode) const {
switch (mode) {
case VK_LINE_RASTERIZATION_MODE_RECTANGULAR_SMOOTH_EXT:
return features.line_rasterization.stippledSmoothLines != VK_FALSE;
case VK_LINE_RASTERIZATION_MODE_BRESENHAM_EXT:
return features.line_rasterization.stippledBresenhamLines != VK_FALSE;
default:
return features.line_rasterization.stippledRectangularLines != VK_FALSE;
}
}
float ClampLineWidth(float width) const {
if (!features.features.wideLines) {
return 1.0f;
}
const auto& range = properties.properties.limits.lineWidthRange;
if (!(width >= range[0])) {
return range[0];
}
return width > range[1] ? range[1] : width;
}
bool SupportsAlphaToOne() const {
return features.features.alphaToOne != VK_FALSE;
}
@@ -870,6 +838,15 @@ FN_MAX_LIMIT_LIST
return extensions.conditional_rendering;
}
bool IsExtAstcDecodeModeSupported() const {
return extensions.astc_decode_mode;
}
/// Returns true if descriptor bindings is partially bound.
bool IsDescriptorBindingPartiallyBoundSupported() const {
return features.descriptor_indexing.descriptorBindingPartiallyBound;
}
bool HasTimelineSemaphore() const;
/// Returns true if the device supports VK_KHR_synchronization2.
@@ -930,10 +907,6 @@ FN_MAX_LIMIT_LIST
u64 GetDeviceMemoryUsage() const;
VkSampleCountFlags GetStorageImageSampleCounts() const {
return properties.properties.limits.storageImageSampleCounts;
}
u32 GetSetsPerPool() const {
return sets_per_pool;
}
@@ -1118,7 +1091,6 @@ private:
private:
VkInstance instance; ///< Vulkan instance.
VmaAllocator allocator; ///< VMA allocator.
mutable u32 allocator_frame_index{};
vk::DeviceDispatch dld; ///< Device function pointers.
vk::PhysicalDevice physical; ///< Physical device.
vk::Device logical; ///< Logical device.
@@ -1166,6 +1138,7 @@ private:
VkPhysicalDeviceSubgroupProperties subgroup_properties{};
VkPhysicalDeviceFloatControlsProperties float_controls{};
VkPhysicalDevicePushDescriptorPropertiesKHR push_descriptor{};
VkPhysicalDeviceDescriptorBufferPropertiesEXT descriptor_buffer{};
VkPhysicalDeviceSubgroupSizeControlProperties subgroup_size_control{};
VkPhysicalDeviceTransformFeedbackPropertiesEXT transform_feedback{};
VkPhysicalDeviceMaintenance5PropertiesKHR maintenance5{};
@@ -1190,6 +1163,7 @@ private:
bool is_non_gpu{}; ///< Is SoftwareRasterizer, FPGA, non-GPU device.
bool has_broken_compute{}; ///< Compute shaders can cause crashes
bool has_broken_cube_compatibility{}; ///< Has broken cube compatibility bit
bool has_broken_descriptor_aliasing{}; ///< Miscompiles descriptors aliased on one binding
bool has_broken_parallel_compiling{}; ///< Has broken parallel shader compiling.
bool has_renderdoc{}; ///< Has RenderDoc attached
bool has_nsight_graphics{}; ///< Has Nsight Graphics attached
@@ -30,6 +30,26 @@ namespace Vulkan {
// Helpers translating MemoryUsage to flags/usage
[[maybe_unused]] VkMemoryPropertyFlags MemoryUsagePropertyFlags(MemoryUsage usage) {
switch (usage) {
case MemoryUsage::DeviceLocal:
return VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
case MemoryUsage::Upload:
return VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT |
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
case MemoryUsage::Download:
return VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT |
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT |
VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
case MemoryUsage::Stream:
return VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT |
VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT |
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
}
ASSERT_MSG(false, "Invalid memory usage={}", usage);
return VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
}
[[nodiscard]] VkMemoryPropertyFlags MemoryUsagePreferredVmaFlags(MemoryUsage usage) {
if (usage == MemoryUsage::Download) {
return VK_MEMORY_PROPERTY_HOST_CACHED_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
@@ -66,11 +86,125 @@ namespace Vulkan {
}
// This avoids calling vkGetBufferMemoryRequirements* directly.
template<typename T>
static VkBuffer GetVkHandleFromBuffer(const T &buf) {
if constexpr (requires { static_cast<VkBuffer>(buf); }) {
return static_cast<VkBuffer>(buf);
} else if constexpr (requires {{ buf.GetHandle() } -> std::convertible_to<VkBuffer>; }) {
return buf.GetHandle();
} else if constexpr (requires {{ buf.Handle() } -> std::convertible_to<VkBuffer>; }) {
return buf.Handle();
} else if constexpr (requires {{ buf.vk_handle() } -> std::convertible_to<VkBuffer>; }) {
return buf.vk_handle();
} else {
static_assert(sizeof(T) == 0, "Cannot extract VkBuffer handle from vk::Buffer");
return VK_NULL_HANDLE;
}
}
} // namespace
//MemoryCommit is now VMA-backed
MemoryCommit::MemoryCommit(VmaAllocator alloc, VmaAllocation a,
const VmaAllocationInfo &info) noexcept
: allocator{alloc}, allocation{a}, memory{info.deviceMemory},
offset{info.offset}, size{info.size}, mapped_ptr{info.pMappedData} {
// Log GPU memory allocation
if (GPU::Logging::IsActive() &&
Settings::values.gpu_log_memory_tracking.GetValue()) {
GPU::Logging::GPULogger::GetInstance().LogMemoryAllocation(
reinterpret_cast<uintptr_t>(memory),
static_cast<u64>(size),
0 // Memory property flags (not easily available from VMA)
);
}
}
MemoryCommit::~MemoryCommit() { Release(); }
MemoryCommit::MemoryCommit(MemoryCommit &&rhs) noexcept
: allocator{std::exchange(rhs.allocator, nullptr)},
allocation{std::exchange(rhs.allocation, nullptr)},
memory{std::exchange(rhs.memory, VK_NULL_HANDLE)},
offset{std::exchange(rhs.offset, 0)},
size{std::exchange(rhs.size, 0)},
mapped_ptr{std::exchange(rhs.mapped_ptr, nullptr)} {}
MemoryCommit &MemoryCommit::operator=(MemoryCommit &&rhs) noexcept {
if (this != &rhs) {
Release();
allocator = std::exchange(rhs.allocator, nullptr);
allocation = std::exchange(rhs.allocation, nullptr);
memory = std::exchange(rhs.memory, VK_NULL_HANDLE);
offset = std::exchange(rhs.offset, 0);
size = std::exchange(rhs.size, 0);
mapped_ptr = std::exchange(rhs.mapped_ptr, nullptr);
}
return *this;
}
std::span<u8> MemoryCommit::Map()
{
if (!allocation) return {};
if (!mapped_ptr) {
if (vmaMapMemory(allocator, allocation, &mapped_ptr) != VK_SUCCESS) return {};
}
const size_t n = static_cast<size_t>(std::min<VkDeviceSize>(size,
(std::numeric_limits<size_t>::max)()));
return std::span<u8>{static_cast<u8 *>(mapped_ptr), n};
}
std::span<const u8> MemoryCommit::Map() const
{
if (!allocation) return {};
if (!mapped_ptr) {
void *p = nullptr;
if (vmaMapMemory(allocator, allocation, &p) != VK_SUCCESS) return {};
const_cast<MemoryCommit *>(this)->mapped_ptr = p;
}
const size_t n = static_cast<size_t>(std::min<VkDeviceSize>(size,
(std::numeric_limits<size_t>::max)()));
return std::span<const u8>{static_cast<const u8 *>(mapped_ptr), n};
}
void MemoryCommit::Unmap()
{
if (allocation && mapped_ptr) {
vmaUnmapMemory(allocator, allocation);
mapped_ptr = nullptr;
}
}
void MemoryCommit::Release() {
if (allocation && allocator) {
// Log GPU memory deallocation
if (GPU::Logging::IsActive() &&
Settings::values.gpu_log_memory_tracking.GetValue() &&
memory != VK_NULL_HANDLE) {
GPU::Logging::GPULogger::GetInstance().LogMemoryDeallocation(
reinterpret_cast<uintptr_t>(memory)
);
}
if (mapped_ptr) {
vmaUnmapMemory(allocator, allocation);
mapped_ptr = nullptr;
}
vmaFreeMemory(allocator, allocation);
}
allocation = nullptr;
allocator = nullptr;
memory = VK_NULL_HANDLE;
offset = 0;
size = 0;
}
MemoryAllocator::MemoryAllocator(const Device &device_)
: device{device_}, allocator{device.GetAllocator()},
properties{device_.GetPhysical().GetMemoryProperties().memoryProperties} {
properties{device_.GetPhysical().GetMemoryProperties().memoryProperties},
buffer_image_granularity{
device_.GetPhysical().GetProperties().limits.bufferImageGranularity} {
// Preserve the previous "RenderDoc small heap" trimming behavior that we had in original vma minus the heap bug
if (device.HasDebuggingToolAttached())
@@ -90,21 +224,6 @@ namespace Vulkan {
MemoryAllocator::~MemoryAllocator() = default;
void MemoryAllocator::SetReclaimCallback(ReclaimCallback callback) {
reclaim_callback = std::move(callback);
vk::SetAllocatorOwnerThread();
}
bool MemoryAllocator::ReclaimAtLeast(u64 hint_bytes) const {
if (!reclaim_callback || in_reclaim) {
return false;
}
in_reclaim = true;
const u64 freed = reclaim_callback(hint_bytes);
in_reclaim = false;
return freed > 0;
}
vk::Image MemoryAllocator::CreateImage(const VkImageCreateInfo &ci) const
{
const VmaAllocationCreateInfo alloc_ci = {
@@ -121,26 +240,7 @@ namespace Vulkan {
VkImage handle{};
VmaAllocation allocation{};
VmaAllocationInfo alloc_info{};
DEBUG_ASSERT(vk::OnAllocatorOwnerThread());
VkResult res = vmaCreateImage(allocator, &ci, &alloc_ci, &handle, &allocation, &alloc_info);
if (res != VK_SUCCESS && ReclaimAtLeast(IMAGE_RECLAIM_HINT)) {
res = vmaCreateImage(allocator, &ci, &alloc_ci, &handle, &allocation, &alloc_info);
}
if (res != VK_SUCCESS) {
auto relaxed_ci = alloc_ci;
relaxed_ci.flags &= ~VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT;
res = vmaCreateImage(allocator, &ci, &relaxed_ci, &handle, &allocation, &alloc_info);
if (res != VK_SUCCESS) {
relaxed_ci.preferredFlags &= ~VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
res = vmaCreateImage(allocator, &ci, &relaxed_ci, &handle, &allocation, &alloc_info);
}
}
vk::Check(res);
vk::Check(vmaCreateImage(allocator, &ci, &alloc_ci, &handle, &allocation, &alloc_info));
// Log GPU memory allocation for images
if (GPU::Logging::IsActive() &&
@@ -177,28 +277,7 @@ namespace Vulkan {
VmaAllocation allocation{};
VkMemoryPropertyFlags property_flags{};
DEBUG_ASSERT(vk::OnAllocatorOwnerThread());
VkResult res = vmaCreateBuffer(allocator, &ci, &alloc_ci, &handle, &allocation, &alloc_info);
if (res != VK_SUCCESS && ReclaimAtLeast(ci.size)) {
res = vmaCreateBuffer(allocator, &ci, &alloc_ci, &handle, &allocation, &alloc_info);
}
if (res != VK_SUCCESS) {
auto relaxed_ci = alloc_ci;
relaxed_ci.flags &= ~VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT;
res = vmaCreateBuffer(allocator, &ci, &relaxed_ci, &handle, &allocation, &alloc_info);
if (res != VK_SUCCESS &&
(relaxed_ci.preferredFlags & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT)) {
relaxed_ci.preferredFlags &= ~VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
res = vmaCreateBuffer(allocator, &ci, &relaxed_ci, &handle, &allocation,
&alloc_info);
}
}
vk::Check(res);
vk::Check(vmaCreateBuffer(allocator, &ci, &alloc_ci, &handle, &allocation, &alloc_info));
vmaGetAllocationMemoryProperties(allocator, allocation, &property_flags);
// Log GPU memory allocation for buffers
@@ -220,4 +299,77 @@ namespace Vulkan {
device.GetDispatchLoader());
}
MemoryCommit MemoryAllocator::Commit(const VkMemoryRequirements &reqs, MemoryUsage usage)
{
const auto vma_usage = MemoryUsageVma(usage);
VmaAllocationCreateInfo ci{};
ci.flags = VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT | MemoryUsageVmaFlags(usage);
ci.usage = vma_usage;
ci.memoryTypeBits = reqs.memoryTypeBits & valid_memory_types;
ci.requiredFlags = 0;
ci.preferredFlags = MemoryUsagePreferredVmaFlags(usage);
VmaAllocation a{};
VmaAllocationInfo info{};
VkResult res = vmaAllocateMemory(allocator, &reqs, &ci, &a, &info);
if (res != VK_SUCCESS) {
// Relax 1: drop budget constraint
auto ci2 = ci;
ci2.flags &= ~VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT;
res = vmaAllocateMemory(allocator, &reqs, &ci2, &a, &info);
// Relax 2: if we preferred DEVICE_LOCAL, drop that preference
if (res != VK_SUCCESS && (ci.preferredFlags & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT)) {
auto ci3 = ci2;
ci3.preferredFlags &= ~VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
res = vmaAllocateMemory(allocator, &reqs, &ci3, &a, &info);
}
}
vk::Check(res);
return MemoryCommit(allocator, a, info);
}
MemoryCommit MemoryAllocator::Commit(const vk::Buffer &buffer, MemoryUsage usage) {
// Allocate memory appropriate for this buffer automatically
const auto vma_usage = MemoryUsageVma(usage);
VmaAllocationCreateInfo ci{};
ci.flags = VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT | MemoryUsageVmaFlags(usage);
ci.usage = vma_usage;
ci.requiredFlags = 0;
ci.preferredFlags = MemoryUsagePreferredVmaFlags(usage);
ci.pool = VK_NULL_HANDLE;
ci.pUserData = nullptr;
ci.priority = 0.0f;
const VkBuffer raw = *buffer;
VmaAllocation a{};
VmaAllocationInfo info{};
// Let VMA infer memory requirements from the buffer
VkResult res = vmaAllocateMemoryForBuffer(allocator, raw, &ci, &a, &info);
if (res != VK_SUCCESS) {
auto ci2 = ci;
ci2.flags &= ~VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT;
res = vmaAllocateMemoryForBuffer(allocator, raw, &ci2, &a, &info);
if (res != VK_SUCCESS && (ci.preferredFlags & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT)) {
auto ci3 = ci2;
ci3.preferredFlags &= ~VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
res = vmaAllocateMemoryForBuffer(allocator, raw, &ci3, &a, &info);
}
}
vk::Check(res);
vk::Check(vmaBindBufferMemory2(allocator, a, 0, raw, nullptr));
return MemoryCommit(allocator, a, info);
}
} // namespace Vulkan
@@ -1,4 +1,4 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2019 yuzu Emulator Project
@@ -6,7 +6,6 @@
#pragma once
#include <functional>
#include <memory>
#include <span>
#include <vector>
@@ -40,6 +39,51 @@ namespace Vulkan {
}
}
/// Ownership handle of a memory commitment (real VMA allocation).
class MemoryCommit {
public:
MemoryCommit() noexcept = default;
MemoryCommit(VmaAllocator allocator, VmaAllocation allocation,
const VmaAllocationInfo &info) noexcept;
~MemoryCommit();
MemoryCommit(const MemoryCommit &) = delete;
MemoryCommit &operator=(const MemoryCommit &) = delete;
MemoryCommit(MemoryCommit &&) noexcept;
MemoryCommit &operator=(MemoryCommit &&) noexcept;
[[nodiscard]] std::span<u8> Map();
[[nodiscard]] std::span<const u8> Map() const;
void Unmap();
explicit operator bool() const noexcept { return allocation != nullptr; }
VkDeviceMemory Memory() const noexcept { return memory; }
VkDeviceSize Offset() const noexcept { return offset; }
VkDeviceSize Size() const noexcept { return size; }
VmaAllocation Allocation() const noexcept { return allocation; }
private:
void Release();
VmaAllocator allocator{}; ///< VMA allocator
VmaAllocation allocation{}; ///< VMA allocation handle
VkDeviceMemory memory{}; ///< Underlying VkDeviceMemory chosen by VMA
VkDeviceSize offset{}; ///< Offset of this allocation inside VkDeviceMemory
VkDeviceSize size{}; ///< Size of the allocation
void *mapped_ptr{}; ///< Optional persistent mapped pointer
};
/// Memory allocator container.
/// Allocates and releases memory allocations on demand.
class MemoryAllocator {
@@ -63,21 +107,36 @@ namespace Vulkan {
vk::Buffer CreateBuffer(const VkBufferCreateInfo &ci, MemoryUsage usage) const;
using ReclaimCallback = std::function<u64(u64)>;
/**
* Commits a memory with the specified requirements.
*
* @param requirements Requirements returned from a Vulkan call.
* @param usage Indicates how the memory will be used.
*
* @returns A memory commit.
*/
MemoryCommit Commit(const VkMemoryRequirements &requirements, MemoryUsage usage);
void SetReclaimCallback(ReclaimCallback callback);
/// Commits memory required by the buffer and binds it (for buffers created outside VMA).
MemoryCommit Commit(const vk::Buffer &buffer, MemoryUsage usage);
private:
bool ReclaimAtLeast(u64 hint_bytes) const;
static constexpr u64 IMAGE_RECLAIM_HINT = 64ULL * 1024 * 1024;
static bool IsAutoUsage(VmaMemoryUsage u) noexcept {
switch (u) {
case VMA_MEMORY_USAGE_AUTO:
case VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE:
case VMA_MEMORY_USAGE_AUTO_PREFER_HOST:
return true;
default:
return false;
}
}
const Device &device; ///< Device handle.
VmaAllocator allocator; ///< VMA allocator.
const VkPhysicalDeviceMemoryProperties properties; ///< Physical device memory properties.
VkDeviceSize buffer_image_granularity; ///< Adjacent buffer/image granularity
u32 valid_memory_types{~0u};
ReclaimCallback reclaim_callback;
mutable bool in_reclaim{false};
};
} // namespace Vulkan
@@ -5,16 +5,11 @@
// SPDX-License-Identifier: GPL-2.0-or-later
#include <algorithm>
#include <atomic>
#include <limits>
#include <memory>
#include <mutex>
#include <optional>
#include <thread>
#include <utility>
#include <vector>
#include "common/assert.h"
#include "common/common_types.h"
#include "common/logging.h"
#include "video_core/vulkan_common/vk_enum_string_helper.h"
@@ -25,60 +20,6 @@ namespace Vulkan::vk {
namespace {
std::thread::id allocator_owner_thread;
template <typename HandleType>
struct PendingRelease {
VmaAllocator allocator;
HandleType handle;
VmaAllocation allocation;
u64 timeline;
};
std::mutex deletion_mutex;
std::atomic<u64> deletion_timeline{1};
std::vector<PendingRelease<VkImage>> pending_images;
std::vector<PendingRelease<VkBuffer>> pending_buffers;
template <typename HandleType>
void PushPendingRelease(std::vector<PendingRelease<HandleType>>& pending, VmaAllocator allocator,
HandleType handle, VmaAllocation allocation) noexcept {
std::scoped_lock lock{deletion_mutex};
pending.push_back(PendingRelease<HandleType>{
.allocator = allocator,
.handle = handle,
.allocation = allocation,
.timeline = deletion_timeline.load(std::memory_order_acquire),
});
}
template <typename HandleType>
void ExtractReleased(std::vector<PendingRelease<HandleType>>& pending,
std::vector<PendingRelease<HandleType>>& released, u64 completed_value) {
const auto split = std::partition(pending.begin(), pending.end(),
[completed_value](const PendingRelease<HandleType>& entry) {
return entry.timeline > completed_value;
});
released.assign(split, pending.end());
pending.erase(split, pending.end());
}
void DrainDeletionQueue(u64 completed_value) noexcept {
std::vector<PendingRelease<VkImage>> images;
std::vector<PendingRelease<VkBuffer>> buffers;
{
std::scoped_lock lock{deletion_mutex};
ExtractReleased(pending_images, images, completed_value);
ExtractReleased(pending_buffers, buffers, completed_value);
}
for (const auto& entry : images) {
vmaDestroyImage(entry.allocator, entry.handle, entry.allocation);
}
for (const auto& entry : buffers) {
vmaDestroyBuffer(entry.allocator, entry.handle, entry.allocation);
}
}
template <typename Func>
void SortPhysicalDevices(std::vector<VkPhysicalDevice>& devices, const InstanceDispatch& dld,
Func&& func) {
@@ -296,6 +237,12 @@ void Load(VkDevice device, DeviceDispatch& dld) noexcept {
X(vkUnmapMemory);
X(vkUpdateDescriptorSetWithTemplate);
X(vkUpdateDescriptorSets);
X(vkGetBufferDeviceAddress);
X(vkGetDescriptorSetLayoutSizeEXT);
X(vkGetDescriptorSetLayoutBindingOffsetEXT);
X(vkGetDescriptorEXT);
X(vkCmdBindDescriptorBuffersEXT);
X(vkCmdSetDescriptorBufferOffsetsEXT);
X(vkWaitForFences);
X(vkWaitSemaphores);
@@ -561,35 +508,13 @@ DebugReportCallback Instance::CreateDebugReportCallback(
return DebugReportCallback(object, handle, *dld);
}
void SetAllocatorOwnerThread() {
allocator_owner_thread = std::this_thread::get_id();
}
bool OnAllocatorOwnerThread() noexcept {
return allocator_owner_thread == std::thread::id{} ||
allocator_owner_thread == std::this_thread::get_id();
}
void SetDeletionTimeline(u64 value) noexcept {
deletion_timeline.store(value, std::memory_order_release);
}
void TickDeletionQueue(u64 completed_value) noexcept {
DEBUG_ASSERT(OnAllocatorOwnerThread());
DrainDeletionQueue(completed_value);
}
void FlushDeletionQueue() noexcept {
DrainDeletionQueue((std::numeric_limits<u64>::max)());
}
void Image::SetObjectNameEXT(const char* name) const {
SetObjectName(dld, owner, handle, VK_OBJECT_TYPE_IMAGE, name);
}
void Image::Release() const noexcept {
if (handle) {
PushPendingRelease(pending_images, allocator, handle, allocation);
vmaDestroyImage(allocator, handle, allocation);
}
}
@@ -611,7 +536,7 @@ void Buffer::SetObjectNameEXT(const char* name) const {
void Buffer::Release() const noexcept {
if (handle) {
PushPendingRelease(pending_buffers, allocator, handle, allocation);
vmaDestroyBuffer(allocator, handle, allocation);
}
}
+51 -10
View File
@@ -131,16 +131,6 @@ private:
VkResult result;
};
void SetAllocatorOwnerThread();
[[nodiscard]] bool OnAllocatorOwnerThread() noexcept;
void SetDeletionTimeline(u64 value) noexcept;
void TickDeletionQueue(u64 completed_value) noexcept;
void FlushDeletionQueue() noexcept;
/// Throws a Vulkan exception if result is not success.
inline void Check(VkResult result) {
if (result != VK_SUCCESS) {
@@ -362,6 +352,12 @@ struct DeviceDispatch : InstanceDispatch {
PFN_vkSetDebugUtilsObjectTagEXT vkSetDebugUtilsObjectTagEXT{};
PFN_vkUnmapMemory vkUnmapMemory{};
PFN_vkUpdateDescriptorSetWithTemplate vkUpdateDescriptorSetWithTemplate{};
PFN_vkGetBufferDeviceAddress vkGetBufferDeviceAddress{};
PFN_vkGetDescriptorSetLayoutSizeEXT vkGetDescriptorSetLayoutSizeEXT{};
PFN_vkGetDescriptorSetLayoutBindingOffsetEXT vkGetDescriptorSetLayoutBindingOffsetEXT{};
PFN_vkGetDescriptorEXT vkGetDescriptorEXT{};
PFN_vkCmdBindDescriptorBuffersEXT vkCmdBindDescriptorBuffersEXT{};
PFN_vkCmdSetDescriptorBufferOffsetsEXT vkCmdSetDescriptorBufferOffsetsEXT{};
PFN_vkUpdateDescriptorSets vkUpdateDescriptorSets{};
PFN_vkWaitForFences vkWaitForFences{};
PFN_vkWaitSemaphores vkWaitSemaphores{};
@@ -803,6 +799,11 @@ public:
return !mapped.empty();
}
/// Returns true if host writes are visible to the device.
bool IsHostCoherent() const noexcept {
return is_coherent;
}
void Flush() const;
void Invalidate() const;
@@ -1097,6 +1098,34 @@ public:
dld->vkUpdateDescriptorSetWithTemplate(handle, set, update_template, data);
}
[[nodiscard]] VkDeviceAddress GetBufferDeviceAddress(VkBuffer buffer) const noexcept {
const VkBufferDeviceAddressInfo info{
.sType = VK_STRUCTURE_TYPE_BUFFER_DEVICE_ADDRESS_INFO,
.pNext = nullptr,
.buffer = buffer,
};
return dld->vkGetBufferDeviceAddress(handle, &info);
}
[[nodiscard]] VkDeviceSize GetDescriptorSetLayoutSizeEXT(
VkDescriptorSetLayout layout) const noexcept {
VkDeviceSize size{};
dld->vkGetDescriptorSetLayoutSizeEXT(handle, layout, &size);
return size;
}
[[nodiscard]] VkDeviceSize GetDescriptorSetLayoutBindingOffsetEXT(
VkDescriptorSetLayout layout, u32 binding) const noexcept {
VkDeviceSize offset{};
dld->vkGetDescriptorSetLayoutBindingOffsetEXT(handle, layout, binding, &offset);
return offset;
}
void GetDescriptorEXT(const VkDescriptorGetInfoEXT& info, size_t size,
void* descriptor) const noexcept {
dld->vkGetDescriptorEXT(handle, &info, size, descriptor);
}
VkResult AcquireNextImageKHR(VkSwapchainKHR swapchain, u64 timeout, VkSemaphore semaphore,
VkFence fence, u32* image_index) const noexcept {
return dld->vkAcquireNextImageKHR(handle, swapchain, timeout, semaphore, fence,
@@ -1410,6 +1439,18 @@ public:
PipelineBarrier(src_stage_mask, dst_stage_mask, dependency_flags, {}, {}, image_barrier);
}
void BindDescriptorBuffersEXT(Span<VkDescriptorBufferBindingInfoEXT> bindings) const noexcept {
dld->vkCmdBindDescriptorBuffersEXT(handle, bindings.size(), bindings.data());
}
void SetDescriptorBufferOffsetsEXT(VkPipelineBindPoint bind_point, VkPipelineLayout layout,
u32 first_set, Span<u32> buffer_indices,
Span<VkDeviceSize> offsets) const noexcept {
dld->vkCmdSetDescriptorBufferOffsetsEXT(handle, bind_point, layout, first_set,
buffer_indices.size(), buffer_indices.data(),
offsets.data());
}
void CopyBufferToImage(VkBuffer src_buffer, VkImage dst_image, VkImageLayout dst_image_layout,
Span<VkBufferImageCopy> regions) const noexcept {
dld->vkCmdCopyBufferToImage(handle, src_buffer, dst_image, dst_image_layout, regions.size(),