Compare commits

..

42 Commits

Author SHA1 Message Date
CamilleLaVey f2f05293a6 Pure and simply meow 2026-08-27 02:30:35 -04:00
CamilleLaVey ad075400a4 Let's try this on memory reclamations 2026-08-27 01:21:10 -04:00
CamilleLaVey 06dba55923 Extended the impact on AHB fallback 2026-08-26 22:10:00 -04:00
CamilleLaVey 8d19ee6dc5 What if we just rely entirely on SGSR common than using EdgeDir? 2026-08-26 19:06:23 -04:00
CamilleLaVey 3e91569d36 Another intent 2026-08-26 18:40:10 -04:00
CamilleLaVey 6010f60803 Just cries internally 2026-08-26 18:05:03 -04:00
CamilleLaVey c14b1de71e Meow. 2026-08-26 17:47:12 -04:00
CamilleLaVey 14cc49e727 Adjustments on upload for textures 2026-08-26 17:00:18 -04:00
CamilleLaVey e88923331c Fix build 2026-08-26 15:04:18 -04:00
CamilleLaVey 78918e995d Fix msvc build 2026-08-26 14:50:42 -04:00
CamilleLaVey 522440bcb2 [TEST] Bring the unswizzle work from tiled-gpu-v2 x2 + revert of vertex, indexes from AHB path 2026-08-26 14:49:01 -04:00
CamilleLaVey c1d8b7dd42 Just a quick test 2026-08-26 05:09:12 -04:00
CamilleLaVey 28d7c97d71 Added vertex, uniform and indexes on the upload path 2026-08-26 04:52:52 -04:00
CamilleLaVey 8a8763d975 Meowly The Build Destroyer 2026-08-26 04:11:59 -04:00
CamilleLaVey 54d8d5b0a8 Another intent on write/read on the fly 2026-08-26 04:06:25 -04:00
CamilleLaVey b9b9a0d704 Changes on the maps counts again 2026-08-26 03:12:06 -04:00
CamilleLaVey b8ce711575 Step back + keeping fixes from unified download + shared mem refactor 2026-08-25 21:36:17 -04:00
CamilleLaVey 46cd1e5d65 another take 2026-08-25 21:36:17 -04:00
CamilleLaVey ae561d49aa Keep AHB limitations 2026-08-25 21:36:17 -04:00
CamilleLaVey 9f45d357e9 [TEST] Another increased on windows size 2026-08-25 21:36:17 -04:00
CamilleLaVey 2c8c3062c4 Increased windows size 2026-08-25 21:36:17 -04:00
CamilleLaVey 51bdb62d74 [TEST] Increased window limiter on AHB 2026-08-25 21:36:17 -04:00
CamilleLaVey ed3838ccbb [TEST] Adjustment AHB to tiled-gpu-v2 state 2026-08-25 21:36:17 -04:00
CamilleLaVey e8c253f511 [memory, vulkan] Initial implementation for Unified Memory 2026-08-25 21:36:17 -04:00
CamilleLaVey 0295dc5fff [vulkan] Removal of QCOM sampler limiters + CustomBorderColor and ColorBorderSwizzle adjustments (#4301)
This PR removes the artificial limit added for QCOM drivers based on a sampler budget limit (based on #3280 work), removes the ban on CustomBorderColor/BorderColorSwizzle for also QCOM driver, reduce the amount of combination required to created a sampler with custom border color/ color border swizzle; adds cases on R16 formats not able to swizzle on BGR565 formats; degrades samplers and color combinations when there's no color border swizzle available; fixes the srgb (LUT) values from legacy and now all the color variations are cached in the same key, instead of having 7 different handlers for them. In resume, rather than performance (due to the reduced amount of process on duplicated/ synced code on pipeline/texture cache), it's a graphica accuracy work that will improve image quality at the cost of almost 0 performance hit, rather than certain hitch if the color combination wasn't cached before. This applies to all platform.

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4301
Reviewed-by: Samuel <lizzie@eden-emu.dev>
Reviewed-by: MaranBr <maranbr@eden-emu.dev>
2026-08-25 20:03:18 +02:00
CamilleLaVey 60a474b8df [vulkan, qcom] Fix shader float controls on QCOM driver (#4297)
Finishes my torture to find the culprit behind the broken behavior with float controls, even if device does support flush denorm on fp32 it doesn't really support it and provokes bad rounding modes by not flushing correctly denorms; this also returns the other working rounding modes on QCOM drivers.

_Special Thanks_

1.- Smoly The Big (@Gidoly)

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4297
Reviewed-by: Samuel <lizzie@eden-emu.dev>
Reviewed-by: MaranBr <maranbr@eden-emu.dev>
2026-08-24 17:57:12 +02:00
lizzie d9336ce6ce [common, settings] Fixes misaligned Aniso levels (#4295)
PR/Commit Owner: CamilleLaVey

Just quick fixes to ANISO bugged level selector, removes x32 and x64 from choices, also fixes the none value to actully use default.

Co-authored-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4295
Reviewed-by: Maufeat <sahyno1996@gmail.com>
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-24 00:06:11 +02:00
PavelBARABANOV 4eb8dd1458 partial revert a41a98028a (#4292)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------
This commit fix launch Assassin's creed 3

changes nvdrv:a - break launch with fw 20+
changes ldr:ro - break skip loading menu with all fw

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4292
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-by: Maufeat <sahyno1996@gmail.com>
2026-08-24 00:01:21 +02:00
PavelBARABANOV 4a3cc9a3c2 [android] Fix Nvdec None (#4288)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------
Nvdec None is working now; I removed the unnecessary comments — the variable names make everything clear enough.

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4288
Reviewed-by: Samuel <lizzie@eden-emu.dev>
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-23 23:58:50 +02:00
Samuel 86ba79b1ba [vulkan, sgsr] Adjustments to SGSR implementation (#4293)
PR/ Commit Owner: CamiileLaVey

This change has specified fixes on the SGSR initial implementation, it brought a wider sharpening level limits and better image quality.

Co-authored-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4293
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-by: crueter <crueter@eden-emu.dev>
2026-08-23 23:36:24 +02:00
lizzie b610b03d29 [common/logging] eliminate uneeded std::string{} allocations per each logging (#4273)
Signed-off-by: lizzie <lizzie@eden-emu.dev>

- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

the idea is simple, `std::string{}` adds pressure to the memory allocator
so whats the best next thing we can do? well of course use our stack
its plenty, but we shouldn't be greedy either, BUFSIZ should be a fair amount of
space for any would-be messages anyways

the main idea behind this PR is to remove std::string{} allocations
done with libfmt, that way we have 0-allocs per logging entry
this should 100% remove pressure uneeded from the memory allocator
we should only allocate things that are important, we can use our
trusty fast stack for any string manip we need to do
not await/global lock or do evil things with the memory allocator

obviously stack is thread local already sooo... this is even better
than having to deal with malloc()/free() in any capacity whatsoever

and no clang can't heap ellide this (how would you even?)

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4273
Reviewed-by: Maufeat <sahyno1996@gmail.com>
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-23 22:37:17 +02:00
lizzie 421e0b834c Revert "[dynarmic] Remove Ignore Global Monitor from CPU Accuracy Auto (#3846)" (#4276)
This reverts commit 3a823de605.

- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

kills performance and doesn't fix graphical glitches
(according to gido)
"It improved nothing but kills performance on default setting"

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4276
Reviewed-by: Maufeat <sahyno1996@gmail.com>
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-23 22:10:22 +02:00
John 55acf5d260 [Vulkan] Apply MK8D Buffer Fix to Linux (#4289)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

Credit: MaranBR
- Adds the Rainbow Graphics Bug Fix to Linux as the game needs MAX_STREAM_BUFFER_SIZE = 256_MiB

![image](/attachments/722d4014-9449-47a0-be47-730b99e0b464)

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4289
Reviewed-by: Maufeat <sahyno1996@gmail.com>
Reviewed-by: Samuel <lizzie@eden-emu.dev>
2026-08-23 17:43:35 +02:00
crueter 3e07b466eb [vulkan] Adjustments on MSAA and BlitHelpers (#4287)
PR/Commit Owner: CamilleLaVey

This PR contains changes complementary to the MSAA refactor from some weeks ago, adds proper shader convert to depth, stencil for msaa and non msaa convertions; removes redundant helpers and unify paths on the resolve functions, just to make the readability and maintainability better; includes fixes for the blit operations on Nvidia, Intel/Windows (previously banned) and ensures Linux retain the fix without extra burden; fixes the QCOM driver bug on resolution upscaling on any games above x1 (includes turnip on A8xx series) and fixes crashes/ resolution upscaled bugs on certain games that used to have wrong viewporting effect on screen or directly device loss on Vulkan. Adds fixes for regressions caused by previous MSAA refactor on games as Fire Emblem: Engage.

Special Thanks

1.- Big Smolio (@Gidoly)

Co-authored-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4287
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-by: Samuel <lizzie@eden-emu.dev>
2026-08-23 17:31:09 +02:00
xbzk 5b86242313 [android, ui] remove the wash (white tint) of focused game cards (#4285)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

This removes the ugly wash from focused game cards, without removing the interaction's cool ripple effects.
I've stumbled upon a reddit complaining about it, and in fact i always hated it but hadn't realized yet.

It happens to any layout.
Joypad users notice more since navigation consists on focusing cards.
Carousel users notice even more coz cards are bigger and always focused.

Border is more than enough to highlight a card, and we want to cherish colors just like creator artists wanted it to be, right?

No chance of bugs. No chance of disagreement (¬¬).
Approve it already so Youtube videos with carousel thumbs get prettier earlier ^^.

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4285
Reviewed-by: Samuel <lizzie@eden-emu.dev>
Reviewed-by: MaranBr <maranbr@eden-emu.dev>
2026-08-23 00:29:59 +02:00
xbzk d24e79c991 [android, ui] refresh game icons when updates are discovered (#4284)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

This small impl aims to fix an android old visual bug where it could cache/read a base game icon before its update was mounted, so games with update-specific icons kept showing the base icon like almost always (in a way that some players may never have seen the updated icons).

Fix: when scan detects an update matching an already found base game, reload that game’s metadata and refresh only its visible icon if the icon actually changed.

I believe it's polished to the max. No find/scans. When base is found it is tracked via hashmap, and when update is found and conditions met the refresh is triggered.

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4284
Reviewed-by: Samuel <lizzie@eden-emu.dev>
Reviewed-by: MaranBr <maranbr@eden-emu.dev>
2026-08-23 00:25:55 +02:00
xbzk df05d3de23 [android, ui] lsfg: make "target frame rate" show/hide "frame multiplier" (#4274)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4274
Reviewed-by: Samuel <lizzie@eden-emu.dev>
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-22 07:52:16 +02:00
PavelBARABANOV c249ff462b [android] Restore the nvdec toggle (#4280)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4280
Reviewed-by: Samuel <lizzie@eden-emu.dev>
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
2026-08-22 02:40:32 +02:00
crueter 056d11027c [internal_network] Fix WiFi scanner compilation on GCC 15.3.0 (#4279)
No idea why this didn't show up earlier.

Signed-off-by: crueter <crueter@eden-emu.dev>
Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4279
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-by: Samuel <lizzie@eden-emu.dev>
2026-08-21 23:29:24 +02:00
xbzk 61ffb309a3 [android,ui] carousel view improvements (#4277)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

One major change to make on carousel, one yoozoo era bug fixed, and some other minor adjustments.

Commits squashed:

[android, ui] Fixed carousel card size scaling over bottom insets
[android, ui] Removed old shapingFunctions
[android, ui] Implemented carousel true elliptic movement model
[android, ui] Fixed overheighted flicker on cards after refresh
[android, ui] Fixed full opaque secondary cards after refresh
[android, ui] Fixed flickers due to doubled list redraw after refreshes

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4277
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-by: Samuel <lizzie@eden-emu.dev>
2026-08-21 23:29:04 +02:00
crueter 65a3cfd2be [common, yuzu] Fix compilation on newer httplib versions (#4278)
Required for newer httplib versions to compile.

has_header is at least present on 0.38.0 on my local machine. Let's hope
it works on Trixie

Signed-off-by: crueter <crueter@eden-emu.dev>

- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4278
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-by: Samuel <lizzie@eden-emu.dev>
2026-08-21 23:17:46 +02:00
Maufeat 5c54abf353 [nim] unstub IShopServiceAsync (#4272)
- [x] I have read and followed the [Contribution Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CONTRIBUTING.md#code-contributions).
- [x] I have read and followed the [AI Policy](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/AI.md)
- [x] I have read and followed the [Coding Guidelines](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/docs/policies/Coding.md) to the best of my ability.

-------------------
 unstub IShopServiceAsync (but still stub Request, with empty JSON response)

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4272
Reviewed-by: Samuel <lizzie@eden-emu.dev>
Reviewed-by: MaranBr <maranbr@eden-emu.dev>
2026-08-19 19:56:06 +02:00
113 changed files with 6501 additions and 1615 deletions
@@ -27,6 +27,7 @@ enum class BooleanSetting(override val key: String) : AbstractBooleanSetting {
RENDERER_ASYNCHRONOUS_GPU_EMULATION("use_asynchronous_gpu_emulation"), RENDERER_ASYNCHRONOUS_GPU_EMULATION("use_asynchronous_gpu_emulation"),
RENDERER_ASYNC_PRESENTATION("async_presentation"), RENDERER_ASYNC_PRESENTATION("async_presentation"),
RENDERER_ASYNCHRONOUS_SHADERS("use_asynchronous_shaders"), RENDERER_ASYNCHRONOUS_SHADERS("use_asynchronous_shaders"),
RENDERER_UNIFIED_MEMORY("use_unified_memory"),
RENDERER_REACTIVE_FLUSHING("use_reactive_flushing"), RENDERER_REACTIVE_FLUSHING("use_reactive_flushing"),
ENABLE_BUFFER_HISTORY("enable_buffer_history"), ENABLE_BUFFER_HISTORY("enable_buffer_history"),
USE_OPTIMIZED_VERTEX_BUFFERS("use_optimized_vertex_buffers"), USE_OPTIMIZED_VERTEX_BUFFERS("use_optimized_vertex_buffers"),
@@ -624,6 +624,7 @@ abstract class SettingsItem(
IntSetting.FSR_SHARPENING_SLIDER, IntSetting.FSR_SHARPENING_SLIDER,
titleId = R.string.fsr_sharpness, titleId = R.string.fsr_sharpness,
descriptionId = R.string.fsr_sharpness_description, descriptionId = R.string.fsr_sharpness_description,
max = 200,
units = "%" units = "%"
) )
) )
@@ -788,6 +789,13 @@ abstract class SettingsItem(
descriptionId = R.string.renderer_asynchronous_shaders_description descriptionId = R.string.renderer_asynchronous_shaders_description
) )
) )
put(
SwitchSetting(
BooleanSetting.RENDERER_UNIFIED_MEMORY,
titleId = R.string.renderer_unified_memory,
descriptionId = R.string.renderer_unified_memory_description
)
)
put( put(
SingleChoiceSetting( SingleChoiceSetting(
IntSetting.FAST_GPU_TIME, IntSetting.FAST_GPU_TIME,
@@ -99,7 +99,12 @@ class SettingsFragmentPresenter(
add(BooleanSetting.RENDERER_FRAME_GEN.key) add(BooleanSetting.RENDERER_FRAME_GEN.key)
add(IntSetting.RENDERER_FRAME_GEN_TARGET_RATE.key) add(IntSetting.RENDERER_FRAME_GEN_TARGET_RATE.key)
add(IntSetting.RENDERER_FRAME_GEN_MULTIPLIER.key) if (IntSetting.RENDERER_FRAME_GEN_TARGET_RATE.getInt(
getNeedsGlobalForKey(IntSetting.RENDERER_FRAME_GEN_TARGET_RATE.key)
) == 0
) {
add(IntSetting.RENDERER_FRAME_GEN_MULTIPLIER.key)
}
add(IntSetting.RENDERER_FRAME_GEN_QUEUE_TARGET.key) add(IntSetting.RENDERER_FRAME_GEN_QUEUE_TARGET.key)
add(BooleanSetting.RENDERER_FRAME_GEN_FLOW_SCALE_AUTO.key) add(BooleanSetting.RENDERER_FRAME_GEN_FLOW_SCALE_AUTO.key)
if (!BooleanSetting.RENDERER_FRAME_GEN_FLOW_SCALE_AUTO.getBoolean( if (!BooleanSetting.RENDERER_FRAME_GEN_FLOW_SCALE_AUTO.getBoolean(
@@ -307,8 +312,6 @@ class SettingsFragmentPresenter(
// TODO(crueter): sub-submenus? // TODO(crueter): sub-submenus?
private fun addGraphicsSettings(sl: ArrayList<SettingsItem>) { private fun addGraphicsSettings(sl: ArrayList<SettingsItem>) {
sl.apply { sl.apply {
// add(IntSetting.RENDERER_NVDEC_EMULATION.key)
add(IntSetting.RENDERER_RESOLUTION.key) add(IntSetting.RENDERER_RESOLUTION.key)
add(IntSetting.RENDERER_VSYNC.key) add(IntSetting.RENDERER_VSYNC.key)
add(IntSetting.RENDERER_SCALING_FILTER.key) add(IntSetting.RENDERER_SCALING_FILTER.key)
@@ -325,6 +328,7 @@ class SettingsFragmentPresenter(
add(IntSetting.MAX_ANISOTROPY.key) add(IntSetting.MAX_ANISOTROPY.key)
add(IntSetting.RENDERER_VRAM_USAGE_MODE.key) add(IntSetting.RENDERER_VRAM_USAGE_MODE.key)
add(IntSetting.RENDERER_ASTC_DECODE_METHOD.key) add(IntSetting.RENDERER_ASTC_DECODE_METHOD.key)
add(IntSetting.RENDERER_NVDEC_EMULATION.key)
add(BooleanSetting.SYNC_MEMORY_OPERATIONS.key) add(BooleanSetting.SYNC_MEMORY_OPERATIONS.key)
add(BooleanSetting.RENDERER_USE_DISK_SHADER_CACHE.key) add(BooleanSetting.RENDERER_USE_DISK_SHADER_CACHE.key)
@@ -340,6 +344,7 @@ class SettingsFragmentPresenter(
add(BooleanSetting.FIX_BLOOM_EFFECTS.key) add(BooleanSetting.FIX_BLOOM_EFFECTS.key)
add(BooleanSetting.EMULATE_BGR565.key) add(BooleanSetting.EMULATE_BGR565.key)
add(BooleanSetting.RENDERER_ASYNCHRONOUS_SHADERS.key) add(BooleanSetting.RENDERER_ASYNCHRONOUS_SHADERS.key)
add(BooleanSetting.RENDERER_UNIFIED_MEMORY.key)
add(IntSetting.ANDROID_PIPELINE_WORKERS.key) add(IntSetting.ANDROID_PIPELINE_WORKERS.key)
add(BooleanSetting.RENDERER_ASYNCHRONOUS_GPU_EMULATION.key) add(BooleanSetting.RENDERER_ASYNCHRONOUS_GPU_EMULATION.key)
add(BooleanSetting.RENDERER_ASYNC_PRESENTATION.key) add(BooleanSetting.RENDERER_ASYNC_PRESENTATION.key)
@@ -1182,7 +1182,7 @@ class EmulationFragment : Fragment(), SurfaceHolder.Callback {
container, container,
IntSetting.FSR_SHARPENING_SLIDER, IntSetting.FSR_SHARPENING_SLIDER,
minValue = 0, minValue = 0,
maxValue = 100, maxValue = 200,
units = "%" units = "%"
) )
} }
@@ -18,6 +18,7 @@ import androidx.activity.result.contract.ActivityResultContracts
import androidx.appcompat.app.AppCompatActivity import androidx.appcompat.app.AppCompatActivity
import androidx.core.view.ViewCompat import androidx.core.view.ViewCompat
import androidx.core.view.WindowInsetsCompat import androidx.core.view.WindowInsetsCompat
import androidx.core.view.doOnPreDraw
import androidx.core.view.updatePadding import androidx.core.view.updatePadding
import androidx.core.widget.doOnTextChanged import androidx.core.widget.doOnTextChanged
import androidx.fragment.app.Fragment import androidx.fragment.app.Fragment
@@ -59,6 +60,10 @@ class GamesFragment : Fragment() {
private var lastViewType: Int = GameAdapter.VIEW_TYPE_GRID private var lastViewType: Int = GameAdapter.VIEW_TYPE_GRID
private var fallbackBottomInset: Int = 0 private var fallbackBottomInset: Int = 0
private var pendingPostReloadListSettle = false
private var pendingPostReloadListSettleGeneration = 0
private var gameListSubmitGeneration = 0
private var committedGameListSubmitGeneration = 0
companion object { companion object {
private const val SEARCH_TEXT = "SearchText" private const val SEARCH_TEXT = "SearchText"
@@ -168,10 +173,9 @@ class GamesFragment : Fragment() {
gamesViewModel.shouldScrollAfterReload.collect(viewLifecycleOwner) { shouldScroll -> gamesViewModel.shouldScrollAfterReload.collect(viewLifecycleOwner) { shouldScroll ->
if (shouldScroll) { if (shouldScroll) {
binding.gridGames.post { pendingPostReloadListSettle = true
(binding.gridGames as? CarouselRecyclerView)?.pendingScrollAfterReload = true pendingPostReloadListSettleGeneration = gameListSubmitGeneration
gameAdapter.notifyDataSetChanged() schedulePostReloadListSettle()
}
gamesViewModel.setShouldScrollAfterReload(false) gamesViewModel.setShouldScrollAfterReload(false)
} }
} }
@@ -273,11 +277,42 @@ class GamesFragment : Fragment() {
lastSearchText = currentSearchText lastSearchText = currentSearchText
lastFilter = currentFilter lastFilter = currentFilter
} else { } else {
((binding.gridGames as? RecyclerView)?.adapter as? GameAdapter)?.submitList(games) submitGameList(games)
gamesViewModel.setFilteredGames(games) gamesViewModel.setFilteredGames(games)
} }
} }
private fun submitGameList(games: List<Game>) {
val adapter = (binding.gridGames as? RecyclerView)?.adapter as? GameAdapter
if (adapter == null) {
schedulePostReloadListSettle()
return
}
val submitGeneration = ++gameListSubmitGeneration
adapter.submitList(games) {
if (committedGameListSubmitGeneration < submitGeneration) {
committedGameListSubmitGeneration = submitGeneration
}
schedulePostReloadListSettle()
}
}
private fun schedulePostReloadListSettle() {
if (!pendingPostReloadListSettle || _binding == null) return
binding.gridGames.doOnPreDraw {
if (!pendingPostReloadListSettle || _binding == null) return@doOnPreDraw
if (committedGameListSubmitGeneration < pendingPostReloadListSettleGeneration) {
schedulePostReloadListSettle()
return@doOnPreDraw
}
pendingPostReloadListSettle = false
(binding.gridGames as? CarouselRecyclerView)?.refreshView()
}
}
private fun setupTopView() { private fun setupTopView() {
binding.searchText.doOnTextChanged() { text: CharSequence?, _: Int, _: Int, _: Int -> binding.searchText.doOnTextChanged() { text: CharSequence?, _: Int, _: Int, _: Int ->
if (text.toString().isNotEmpty()) { if (text.toString().isNotEmpty()) {
@@ -414,9 +449,7 @@ class GamesFragment : Fragment() {
val searchTerm = binding.searchText.text.toString().lowercase(Locale.getDefault()) val searchTerm = binding.searchText.text.toString().lowercase(Locale.getDefault())
if (searchTerm.isEmpty()) { if (searchTerm.isEmpty()) {
((binding.gridGames as? RecyclerView)?.adapter as? GameAdapter)?.submitList( submitGameList(filteredList)
filteredList
)
gamesViewModel.setFilteredGames(filteredList) gamesViewModel.setFilteredGames(filteredList)
return return
} }
@@ -432,7 +465,7 @@ class GamesFragment : Fragment() {
} }
}.sortedByDescending { it.score }.map { it.item } }.sortedByDescending { it.score }.map { it.item }
((binding.gridGames as? RecyclerView)?.adapter as? GameAdapter)?.submitList(sortedList) submitGameList(sortedList)
gamesViewModel.setFilteredGames(sortedList) gamesViewModel.setFilteredGames(sortedList)
} }
@@ -31,6 +31,7 @@ object GameHelper {
fun getGames(): List<Game> { fun getGames(): List<Game> {
val games = mutableListOf<Game>() val games = mutableListOf<Game>()
val gamesByProgramId = mutableMapOf<String, Game>()
val context = YuzuApplication.appContext val context = YuzuApplication.appContext
preferences = PreferenceManager.getDefaultSharedPreferences(context) preferences = PreferenceManager.getDefaultSharedPreferences(context)
@@ -63,6 +64,7 @@ object GameHelper {
addGamesRecursive( addGamesRecursive(
games, games,
gamesByProgramId,
FileUtil.listFiles(gameDirUri), FileUtil.listFiles(gameDirUri),
scanDepth, scanDepth,
mountedContainerUris mountedContainerUris
@@ -136,6 +138,7 @@ object GameHelper {
private fun addGamesRecursive( private fun addGamesRecursive(
games: MutableList<Game>, games: MutableList<Game>,
gamesByProgramId: MutableMap<String, Game>,
files: Array<MinimalDocumentFile>, files: Array<MinimalDocumentFile>,
depth: Int, depth: Int,
mountedContainerUris: MutableSet<String> mountedContainerUris: MutableSet<String>
@@ -148,6 +151,7 @@ object GameHelper {
if (it.isDirectory) { if (it.isDirectory) {
addGamesRecursive( addGamesRecursive(
games, games,
gamesByProgramId,
FileUtil.listFiles(it.uri), FileUtil.listFiles(it.uri),
depth - 1, depth - 1,
mountedContainerUris mountedContainerUris
@@ -156,8 +160,9 @@ object GameHelper {
val extension = FileUtil.getExtension(it.uri).lowercase() val extension = FileUtil.getExtension(it.uri).lowercase()
val filePath = it.uri.toString() val filePath = it.uri.toString()
if (externalContentExtensions.contains(extension) && val mountedContainer = externalContentExtensions.contains(extension) &&
mountedContainerUris.add(filePath)) { mountedContainerUris.add(filePath)
if (mountedContainer) {
NativeLibrary.addGameFolderFileToFilesystemProvider(filePath) NativeLibrary.addGameFolderFileToFilesystemProvider(filePath)
} }
@@ -165,6 +170,20 @@ object GameHelper {
val game = getGame(it.uri, true, false) val game = getGame(it.uri, true, false)
if (game != null) { if (game != null) {
games.add(game) games.add(game)
if (game.programId != "0") {
gamesByProgramId[game.programId] = game
}
} else if (mountedContainer) {
GameMetadata.getProgramId(filePath).toLongOrNull()?.let { programId ->
gamesByProgramId[(programId and 0x800L.inv()).toString()]
}?.let { existingGame ->
NativeLibrary.getPatchesForFile(existingGame.path, existingGame.programId)
existingGame.version = GameMetadata.getVersion(
existingGame.path,
true
)
GameIconUtils.refreshGameIcon(existingGame)
}
} }
} }
} }
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: 2023 yuzu Emulator Project // SPDX-FileCopyrightText: 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later // SPDX-License-Identifier: GPL-2.0-or-later
@@ -24,6 +27,15 @@ import coil.request.Options
import org.yuzu.yuzu_emu.R import org.yuzu.yuzu_emu.R
import org.yuzu.yuzu_emu.YuzuApplication import org.yuzu.yuzu_emu.YuzuApplication
import org.yuzu.yuzu_emu.model.Game import org.yuzu.yuzu_emu.model.Game
import java.util.Collections
import java.util.WeakHashMap
private val gameIconHashes = Collections.synchronizedMap(mutableMapOf<String, Int>())
private val gameIconTargets = Collections.synchronizedMap(WeakHashMap<ImageView, GameIconTarget>())
private fun Game.iconCacheKey(): String = "$path|$version"
private data class GameIconTarget(val game: Game, var iconHash: Int? = null)
class GameIconFetcher( class GameIconFetcher(
private val game: Game, private val game: Game,
@@ -31,14 +43,15 @@ class GameIconFetcher(
) : Fetcher { ) : Fetcher {
override suspend fun fetch(): FetchResult { override suspend fun fetch(): FetchResult {
return DrawableResult( return DrawableResult(
drawable = decodeGameIcon(game.path)!!.toDrawable(options.context.resources), drawable = decodeGameIcon(game)!!.toDrawable(options.context.resources),
isSampled = false, isSampled = false,
dataSource = DataSource.DISK dataSource = DataSource.DISK
) )
} }
private fun decodeGameIcon(uri: String): Bitmap? { private fun decodeGameIcon(game: Game): Bitmap? {
val data = GameMetadata.getIcon(uri) val data = GameMetadata.getIcon(game.path)
gameIconHashes[game.iconCacheKey()] = data.contentHashCode()
return BitmapFactory.decodeByteArray( return BitmapFactory.decodeByteArray(
data, data,
0, 0,
@@ -54,7 +67,7 @@ class GameIconFetcher(
} }
class GameIconKeyer : Keyer<Game> { class GameIconKeyer : Keyer<Game> {
override fun key(data: Game, options: Options): String = data.path override fun key(data: Game, options: Options): String = data.iconCacheKey()
} }
object GameIconUtils { object GameIconUtils {
@@ -71,14 +84,58 @@ object GameIconUtils {
.build() .build()
fun loadGameIcon(game: Game, imageView: ImageView) { fun loadGameIcon(game: Game, imageView: ImageView) {
gameIconTargets[imageView] = GameIconTarget(game)
val request = ImageRequest.Builder(YuzuApplication.appContext) val request = ImageRequest.Builder(YuzuApplication.appContext)
.data(game) .data(game)
.target(imageView) .target(imageView)
.error(R.drawable.default_icon) .error(R.drawable.default_icon)
.listener(
onSuccess = { _, _ ->
val target = gameIconTargets[imageView]
if (target?.game?.iconCacheKey() == game.iconCacheKey()) {
gameIconHashes[game.iconCacheKey()]?.let {
target.iconHash = it
}
}
},
onError = { _, _ ->
gameIconTargets[imageView]?.iconHash = null
}
)
.build() .build()
imageLoader.enqueue(request) imageLoader.enqueue(request)
} }
fun refreshGameIcon(game: Game) {
val targets = synchronized(gameIconTargets) {
gameIconTargets
.filterValues { it.game.path == game.path && it.game.programId == game.programId }
.keys
.toList()
}
if (targets.isEmpty()) {
return
}
val iconHash = GameMetadata.getIcon(game.path).contentHashCode()
val targetsToRefresh = targets.filter { gameIconTargets[it]?.iconHash != iconHash }
if (targetsToRefresh.isEmpty()) {
return
}
imageLoader.memoryCache?.remove(MemoryCache.Key(game.iconCacheKey()))
targetsToRefresh.forEach { imageView ->
imageView.post {
val target = gameIconTargets[imageView] ?: return@post
if (target.game.path == game.path && target.game.programId == game.programId) {
if (target.iconHash != iconHash) {
loadGameIcon(game, imageView)
}
}
}
}
}
suspend fun getGameIcon(lifecycleOwner: LifecycleOwner, game: Game): Bitmap { suspend fun getGameIcon(lifecycleOwner: LifecycleOwner, game: Game): Bitmap {
val request = ImageRequest.Builder(YuzuApplication.appContext) val request = ImageRequest.Builder(YuzuApplication.appContext)
.data(game) .data(game)
@@ -1,4 +1,4 @@
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project // SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later // SPDX-License-Identifier: GPL-3.0-or-later
package org.yuzu.yuzu_emu.ui package org.yuzu.yuzu_emu.ui
@@ -11,6 +11,8 @@ import androidx.recyclerview.widget.LinearLayoutManager
import androidx.recyclerview.widget.PagerSnapHelper import androidx.recyclerview.widget.PagerSnapHelper
import androidx.recyclerview.widget.RecyclerView import androidx.recyclerview.widget.RecyclerView
import kotlin.math.abs import kotlin.math.abs
import kotlin.math.cos
import kotlin.math.sin
import org.yuzu.yuzu_emu.R import org.yuzu.yuzu_emu.R
import org.yuzu.yuzu_emu.adapters.GameAdapter import org.yuzu.yuzu_emu.adapters.GameAdapter
import androidx.core.view.doOnNextLayout import androidx.core.view.doOnNextLayout
@@ -34,6 +36,7 @@ class CarouselRecyclerView @JvmOverloads constructor(
private var overlapDecoration: OverlappingDecoration? = null private var overlapDecoration: OverlappingDecoration? = null
private var pagerSnapHelper: PagerSnapHelper? = null private var pagerSnapHelper: PagerSnapHelper? = null
private var scalingScrollListener: OnScrollListener? = null private var scalingScrollListener: OnScrollListener? = null
private var savedItemAnimator: RecyclerView.ItemAnimator? = null
companion object { companion object {
private const val CAROUSEL_CARD_SIZE_FACTOR = "CarouselCardSizeMultiplier" private const val CAROUSEL_CARD_SIZE_FACTOR = "CarouselCardSizeMultiplier"
@@ -42,8 +45,13 @@ class CarouselRecyclerView @JvmOverloads constructor(
private const val CAROUSEL_OVERLAP_FACTOR = "CarouselOverlapFactor" private const val CAROUSEL_OVERLAP_FACTOR = "CarouselOverlapFactor"
private const val CAROUSEL_MAX_FLING_COUNT = "CarouselMaxFlingCount" private const val CAROUSEL_MAX_FLING_COUNT = "CarouselMaxFlingCount"
private const val CAROUSEL_FLING_MULTIPLIER = "CarouselFlingMultiplier" private const val CAROUSEL_FLING_MULTIPLIER = "CarouselFlingMultiplier"
private const val CAROUSEL_CARDS_SCALING_SHAPE = "CarouselCardsScalingShape" private const val CAROUSEL_ARC_ANGLE_STEP_DEGREES = 15.0
private const val CAROUSEL_CARDS_ALPHA_SHAPE = "CarouselCardsAlphaShape" private const val CAROUSEL_ARC_MAX_ANGLE_DEGREES = 165.0
private const val CAROUSEL_ARC_DEPTH_MAX_ANGLE_DEGREES = 85.0
private const val CAROUSEL_ARC_DEPTH_STRETCH = 5.0f
private const val CAROUSEL_ARC_X_DEPTH_FACTOR = 0.55f
private const val CAROUSEL_ARC_FADE_OUT_START_DEGREES = 60.0
private const val CAROUSEL_ARC_FADE_OUT_END_DEGREES = 95.0
const val CAROUSEL_LAST_SCROLL_POSITION = "CarouselLastScrollPosition" const val CAROUSEL_LAST_SCROLL_POSITION = "CarouselLastScrollPosition"
const val CAROUSEL_VIEW_TYPE_PORTRAIT = "GamesViewTypePortrait" const val CAROUSEL_VIEW_TYPE_PORTRAIT = "GamesViewTypePortrait"
const val CAROUSEL_VIEW_TYPE_LANDSCAPE = "GamesViewTypeLandscape" const val CAROUSEL_VIEW_TYPE_LANDSCAPE = "GamesViewTypeLandscape"
@@ -160,46 +168,52 @@ class CarouselRecyclerView @JvmOverloads constructor(
} }
} }
fun shapingFunction(x: Float, option: Int = 0): Float {
return when (option) {
0 -> 1f // Off
1 -> 1f - x // linear descending
2 -> (1f - x) * (1f - x) // Ease out
3 -> if (x < 0.05f) 1f else (1f - x) * 0.8f
4 -> kotlin.math.cos(x * Math.PI).toFloat() // Cosine
5 -> kotlin.math.cos((1.5f * x).coerceIn(0f, 1f) * Math.PI).toFloat() // Cosine 1.5x trimmed
else -> 1f // Default to Off
}
}
fun updateChildScaleAndAlphaForPosition(child: View) { fun updateChildScaleAndAlphaForPosition(child: View) {
val cardSize = (adapter as? GameAdapter ?: return).cardSize val cardSize = (adapter as? GameAdapter ?: return).cardSize
val position = getChildViewHolder(child).bindingAdapterPosition val position = getChildViewHolder(child).bindingAdapterPosition
if (position == RecyclerView.NO_POSITION || cardSize <= 0) { if (position == RecyclerView.NO_POSITION || cardSize <= 0) {
return // No valid position or card size return // No valid position or card size
} }
child.layoutParams.width = cardSize val layoutParams = child.layoutParams
child.layoutParams.height = cardSize if (layoutParams.width != cardSize || layoutParams.height != cardSize) {
child.layoutParams = layoutParams.apply {
width = cardSize
height = cardSize
}
}
val signedDistance = getChildDistanceToCenter(child)
val itemStep = (cardSize - overlapPx).toFloat().coerceAtLeast(1f)
val angleStep = Math.toRadians(CAROUSEL_ARC_ANGLE_STEP_DEGREES).toFloat()
val maxAngle = Math.toRadians(CAROUSEL_ARC_MAX_ANGLE_DEGREES).toFloat()
val depthMaxAngle = Math.toRadians(CAROUSEL_ARC_DEPTH_MAX_ANGLE_DEGREES).toFloat()
val fadeOutStartAngle = Math.toRadians(CAROUSEL_ARC_FADE_OUT_START_DEGREES).toFloat()
val fadeOutEndAngle = Math.toRadians(CAROUSEL_ARC_FADE_OUT_END_DEGREES).toFloat()
val angle = (signedDistance / itemStep * angleStep).coerceIn(-maxAngle, maxAngle)
val arcRadius = itemStep / angleStep
val arcX = sin(angle) * arcRadius
val absoluteAngle = abs(angle)
val rawDepthInput = ((1f - cos(absoluteAngle)) / (1f - cos(depthMaxAngle)))
.coerceIn(0f, 1f)
val easedDepthTail = Math.pow(
(1f - rawDepthInput).toDouble(),
CAROUSEL_ARC_DEPTH_STRETCH.toDouble()
).toFloat()
val depthInput = (1f - easedDepthTail).coerceIn(0f, 1f)
val projectedArcX = arcX * (1f - rawDepthInput * CAROUSEL_ARC_X_DEPTH_FACTOR)
child.animate().cancel()
child.translationX = projectedArcX - signedDistance
val center = getRecyclerViewCenter()
val distance = abs(getChildDistanceToCenter(child))
val internalBorderScale = resources.getFraction(R.fraction.carousel_bordercards_scale, 1, 1) val internalBorderScale = resources.getFraction(R.fraction.carousel_bordercards_scale, 1, 1)
val borderScale = preferences.getFloat(CAROUSEL_BORDERCARDS_SCALE, internalBorderScale).coerceIn( val borderScale = preferences.getFloat(CAROUSEL_BORDERCARDS_SCALE, internalBorderScale).coerceIn(
0f, 0f,
1f 1f
) )
val shapeInput = (distance / center).coerceIn(0f, 1f) val shapedScaling = 1f - depthInput
val internalShapeSetting = resources.getInteger(R.integer.carousel_cards_scaling_shape)
val scalingShapeSetting = preferences.getInt(
CAROUSEL_CARDS_SCALING_SHAPE,
internalShapeSetting
)
val shapedScaling = shapingFunction(shapeInput, scalingShapeSetting)
val scale = (borderScale + (1f - borderScale) * shapedScaling).coerceIn(0f, 1f) val scale = (borderScale + (1f - borderScale) * shapedScaling).coerceIn(0f, 1f)
val maxDistance = width / 2f
val alphaInput = (distance / maxDistance).coerceIn(0f, 1f)
val internalBordersAlpha = resources.getFraction( val internalBordersAlpha = resources.getFraction(
R.fraction.carousel_bordercards_alpha, R.fraction.carousel_bordercards_alpha,
1, 1,
@@ -209,15 +223,12 @@ class CarouselRecyclerView @JvmOverloads constructor(
0f, 0f,
1f 1f
) )
val internalAlphaShapeSetting = resources.getInteger(R.integer.carousel_cards_alpha_shape) val shapedAlpha = cos(depthInput * Math.PI).toFloat()
val alphaShapeSetting = preferences.getInt( val baseAlpha = (borderAlpha + (1f - borderAlpha) * shapedAlpha).coerceIn(0f, 1f)
CAROUSEL_CARDS_ALPHA_SHAPE, val rearPresence = (1f - (absoluteAngle - fadeOutStartAngle) /
internalAlphaShapeSetting (fadeOutEndAngle - fadeOutStartAngle)).coerceIn(0f, 1f)
) val alpha = (baseAlpha * rearPresence).coerceIn(0f, 1f)
val shapedAlpha = shapingFunction(alphaInput, alphaShapeSetting)
val alpha = (borderAlpha + (1f - borderAlpha) * shapedAlpha).coerceIn(0f, 1f)
child.animate().cancel()
child.alpha = alpha child.alpha = alpha
child.scaleX = scale child.scaleX = scale
child.scaleY = scale child.scaleY = scale
@@ -273,7 +284,9 @@ class CarouselRecyclerView @JvmOverloads constructor(
0f, 0f,
1f 1f
) )
return (userFactor * (height - bottomInset)).toInt() val scaledHeight = height * userFactor
val availableHeight = height - bottomInset
return minOf(scaledHeight.toInt(), availableHeight.toInt())
} }
fun setupCarousel(enabled: Boolean) { fun setupCarousel(enabled: Boolean) {
@@ -282,6 +295,13 @@ class CarouselRecyclerView @JvmOverloads constructor(
if (gameAdapter.cardSize == 0) return if (gameAdapter.cardSize == 0) return
if (bottomInset < 0) return if (bottomInset < 0) return
itemAnimator?.let {
if (savedItemAnimator == null) {
savedItemAnimator = it
}
itemAnimator = null
}
useCustomDrawingOrder = true useCustomDrawingOrder = true
val cardSize = gameAdapter.cardSize val cardSize = gameAdapter.cardSize
@@ -336,6 +356,12 @@ class CarouselRecyclerView @JvmOverloads constructor(
// Detach PagerSnapHelper // Detach PagerSnapHelper
pagerSnapHelper?.attachToRecyclerView(null) pagerSnapHelper?.attachToRecyclerView(null)
pagerSnapHelper = null pagerSnapHelper = null
savedItemAnimator?.let {
if (itemAnimator == null) {
itemAnimator = it
}
savedItemAnimator = null
}
useCustomDrawingOrder = false useCustomDrawingOrder = false
// Reset padding and fling // Reset padding and fling
setPadding(0, 0, 0, 0) setPadding(0, 0, 0, 0)
@@ -344,6 +370,7 @@ class CarouselRecyclerView @JvmOverloads constructor(
// Reset scaling // Reset scaling
for (i in 0 until childCount) { for (i in 0 until childCount) {
val child = getChildAt(i) val child = getChildAt(i)
child?.translationX = 0f
child?.scaleX = 1f child?.scaleX = 1f
child?.scaleY = 1f child?.scaleY = 1f
child?.alpha = 1f child?.alpha = 1f
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="utf-8"?>
<selector xmlns:android="http://schemas.android.com/apk/res/android">
<item android:state_pressed="true" android:color="?attr/colorControlHighlight" />
<item android:state_focused="true" android:color="@android:color/transparent" />
<item android:state_selected="true" android:color="@android:color/transparent" />
<item android:state_hovered="true" android:color="@android:color/transparent" />
<item android:color="@android:color/transparent" />
</selector>
@@ -10,6 +10,7 @@
android:clipChildren="true" android:clipChildren="true"
android:layout_margin="0dp" android:layout_margin="0dp"
app:cardBackgroundColor="@color/eden_card_background" app:cardBackgroundColor="@color/eden_card_background"
app:rippleColor="@color/game_card_ripple"
app:strokeWidth="1dp" app:strokeWidth="1dp"
app:strokeColor="@color/eden_border"> app:strokeColor="@color/eden_border">
@@ -11,6 +11,7 @@
app:cardCornerRadius="16dp" app:cardCornerRadius="16dp"
app:cardPreventCornerOverlap="true" app:cardPreventCornerOverlap="true"
android:clipChildren="true" android:clipChildren="true"
app:rippleColor="@color/game_card_ripple"
android:layout_margin="4dp"> android:layout_margin="4dp">
<androidx.constraintlayout.widget.ConstraintLayout <androidx.constraintlayout.widget.ConstraintLayout
@@ -22,6 +22,7 @@
android:focusable="true" android:focusable="true"
android:transitionName="card_game" android:transitionName="card_game"
app:cardCornerRadius="16dp" app:cardCornerRadius="16dp"
app:rippleColor="@color/game_card_ripple"
android:foreground="@color/eden_border_gradient_start"> android:foreground="@color/eden_border_gradient_start">
<androidx.constraintlayout.widget.ConstraintLayout <androidx.constraintlayout.widget.ConstraintLayout
@@ -22,6 +22,7 @@
android:focusable="true" android:focusable="true"
android:transitionName="card_game_compact" android:transitionName="card_game_compact"
app:cardCornerRadius="16dp" app:cardCornerRadius="16dp"
app:rippleColor="@color/game_card_ripple"
android:foreground="@color/eden_border_gradient_start"> android:foreground="@color/eden_border_gradient_start">
<androidx.constraintlayout.widget.ConstraintLayout <androidx.constraintlayout.widget.ConstraintLayout
@@ -12,6 +12,7 @@
app:cardCornerRadius="16dp" app:cardCornerRadius="16dp"
app:cardElevation="0dp" app:cardElevation="0dp"
app:cardBackgroundColor="@android:color/transparent" app:cardBackgroundColor="@android:color/transparent"
app:rippleColor="@color/game_card_ripple"
app:strokeWidth="0dp"> app:strokeWidth="0dp">
<androidx.constraintlayout.widget.ConstraintLayout <androidx.constraintlayout.widget.ConstraintLayout
@@ -1107,7 +1107,6 @@
<string name="theme_mode_light">فاتح</string> <string name="theme_mode_light">فاتح</string>
<string name="theme_mode_dark">داكن</string> <string name="theme_mode_dark">داكن</string>
<string name="multiplier_none">لا شيء</string>
<!-- Black backgrounds theme --> <!-- Black backgrounds theme -->
<string name="use_black_backgrounds">خلفيات سوداء</string> <string name="use_black_backgrounds">خلفيات سوداء</string>
@@ -991,7 +991,6 @@ Wirklich fortfahren?</string>
<string name="theme_mode_light">Hell</string> <string name="theme_mode_light">Hell</string>
<string name="theme_mode_dark">Dunkel</string> <string name="theme_mode_dark">Dunkel</string>
<string name="multiplier_none">Keine</string>
<!-- Black backgrounds theme --> <!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Schwarze Hintergründe</string> <string name="use_black_backgrounds">Schwarze Hintergründe</string>
@@ -1092,7 +1092,6 @@
<string name="theme_mode_light">Claro</string> <string name="theme_mode_light">Claro</string>
<string name="theme_mode_dark">Oscuro</string> <string name="theme_mode_dark">Oscuro</string>
<string name="multiplier_none">Nada</string>
<!-- Black backgrounds theme --> <!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Fondos oscuros</string> <string name="use_black_backgrounds">Fondos oscuros</string>
@@ -808,9 +808,6 @@
<string name="multiplier_x4">x4</string> <string name="multiplier_x4">x4</string>
<string name="multiplier_x8">x8</string> <string name="multiplier_x8">x8</string>
<string name="multiplier_x16">x16</string> <string name="multiplier_x16">x16</string>
<string name="multiplier_x32">x32</string>
<string name="multiplier_x64">x64</string>
<string name="multiplier_none">None</string>
<!-- Black backgrounds theme --> <!-- Black backgrounds theme -->
<string name="use_black_backgrounds">پس‌زمینه مشکی</string> <string name="use_black_backgrounds">پس‌زمینه مشکی</string>
@@ -1004,7 +1004,6 @@
<string name="theme_mode_light">Lumineux</string> <string name="theme_mode_light">Lumineux</string>
<string name="theme_mode_dark">Sombre</string> <string name="theme_mode_dark">Sombre</string>
<string name="multiplier_none">Aucun</string>
<!-- Black backgrounds theme --> <!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Arrière-plan noir</string> <string name="use_black_backgrounds">Arrière-plan noir</string>
@@ -935,7 +935,6 @@
<string name="theme_mode_light">Jasny</string> <string name="theme_mode_light">Jasny</string>
<string name="theme_mode_dark">Ciemny</string> <string name="theme_mode_dark">Ciemny</string>
<string name="multiplier_none">Brak</string>
<!-- Black backgrounds theme --> <!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Czarne tła</string> <string name="use_black_backgrounds">Czarne tła</string>
@@ -891,7 +891,6 @@
<string name="theme_mode_light">Claro</string> <string name="theme_mode_light">Claro</string>
<string name="theme_mode_dark">Escuro</string> <string name="theme_mode_dark">Escuro</string>
<string name="multiplier_none">Nenhum</string>
<!-- Black backgrounds theme --> <!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Planos de fundo pretos</string> <string name="use_black_backgrounds">Planos de fundo pretos</string>
@@ -1071,7 +1071,6 @@
<string name="theme_mode_light">Светлая</string> <string name="theme_mode_light">Светлая</string>
<string name="theme_mode_dark">Темная</string> <string name="theme_mode_dark">Темная</string>
<string name="multiplier_none">Отключено</string>
<!-- Black backgrounds theme --> <!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Чёрный фон</string> <string name="use_black_backgrounds">Чёрный фон</string>
@@ -1053,7 +1053,6 @@
<string name="theme_mode_light">Світла</string> <string name="theme_mode_light">Світла</string>
<string name="theme_mode_dark">Темна</string> <string name="theme_mode_dark">Темна</string>
<string name="multiplier_none">Жодного</string>
<!-- Black backgrounds theme --> <!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Чорний фон</string> <string name="use_black_backgrounds">Чорний фон</string>
@@ -1081,7 +1081,6 @@
<string name="theme_mode_light">浅色</string> <string name="theme_mode_light">浅色</string>
<string name="theme_mode_dark">深色</string> <string name="theme_mode_dark">深色</string>
<string name="multiplier_none"></string>
<!-- Black backgrounds theme --> <!-- Black backgrounds theme -->
<string name="use_black_backgrounds">使用黑色背景</string> <string name="use_black_backgrounds">使用黑色背景</string>
@@ -1006,7 +1006,6 @@
<string name="theme_mode_light">淺色</string> <string name="theme_mode_light">淺色</string>
<string name="theme_mode_dark">深色</string> <string name="theme_mode_dark">深色</string>
<string name="multiplier_none"></string>
<!-- Black backgrounds theme --> <!-- Black backgrounds theme -->
<string name="use_black_backgrounds">黑色背景</string> <string name="use_black_backgrounds">黑色背景</string>
+12 -23
View File
@@ -111,43 +111,38 @@
<item>1</item> <item>1</item>
</integer-array> </integer-array>
<!-- VRAM USAGE MODE CHOICES -->
<string-array name="vramUsageMethodNames"> <string-array name="vramUsageMethodNames">
<item>@string/vram_usage_conservative</item> <item>@string/vram_usage_conservative</item>
<item>@string/vram_usage_aggressive</item> <item>@string/vram_usage_aggressive</item>
</string-array> </string-array>
<!-- VRAM USAGE MODE VALUES -->
<integer-array name="vramUsageMethodValues"> <integer-array name="vramUsageMethodValues">
<item>0</item> <!-- Conservative --> <item>0</item>
<item>1</item> <!-- Aggressive --> <item>1</item>
</integer-array> </integer-array>
<!-- ASTC Decoding Method Choices -->
<string-array name="astcDecodingMethodNames"> <string-array name="astcDecodingMethodNames">
<item>@string/accelerate_astc_cpu</item> <item>@string/accelerate_astc_cpu</item>
<item>@string/accelerate_astc_gpu</item> <item>@string/accelerate_astc_gpu</item>
<item>@string/accelerate_astc_async</item> <item>@string/accelerate_astc_async</item>
</string-array> </string-array>
<!-- ASTC Decoding Method Values -->
<integer-array name="astcDecodingMethodValues"> <integer-array name="astcDecodingMethodValues">
<item>0</item> <!-- CPU --> <item>0</item>
<item>1</item> <!-- GPU --> <item>1</item>
<item>2</item> <!-- CPU Asynchronously --> <item>2</item>
</integer-array> </integer-array>
<!-- NVDEC Emulation Choices -->
<string-array name="rendererNvdecNames"> <string-array name="rendererNvdecNames">
<item>@string/nvdec_emulation_none</item> <!-- Off --> <item>@string/nvdec_emulation_none</item>
<item>@string/nvdec_emulation_cpu</item> <!-- Cpu --> <item>@string/nvdec_emulation_cpu</item>
<item>@string/nvdec_emulation_gpu</item> <!-- Gpu --> <item>@string/nvdec_emulation_gpu</item>
</string-array> </string-array>
<!-- NVDEC Emulation Values -->
<integer-array name="rendererNvdecValues"> <integer-array name="rendererNvdecValues">
<item>3</item> <!-- Off value --> <item>0</item>
<item>1</item> <!-- CPU value --> <item>1</item>
<item>2</item> <!-- GPU value --> <item>2</item>
</integer-array> </integer-array>
<string-array name="rendererResolutionNames"> <string-array name="rendererResolutionNames">
@@ -513,9 +508,6 @@
<item>@string/multiplier_x4</item> <item>@string/multiplier_x4</item>
<item>@string/multiplier_x8</item> <item>@string/multiplier_x8</item>
<item>@string/multiplier_x16</item> <item>@string/multiplier_x16</item>
<item>@string/multiplier_x32</item>
<item>@string/multiplier_x64</item>
<item>@string/multiplier_none</item>
</string-array> </string-array>
<integer-array name="anisoValues"> <integer-array name="anisoValues">
<item>0</item> <item>0</item>
@@ -524,9 +516,6 @@
<item>3</item> <item>3</item>
<item>4</item> <item>4</item>
<item>5</item> <item>5</item>
<item>6</item>
<item>7</item>
<item>8</item>
</integer-array> </integer-array>
<string-array name="verticalAlignmentEntries"> <string-array name="verticalAlignmentEntries">
@@ -5,8 +5,6 @@
<integer name="game_columns_grid">2</integer> <integer name="game_columns_grid">2</integer>
<integer name="carousel_max_fling_count">4</integer> <integer name="carousel_max_fling_count">4</integer>
<integer name="carousel_focus_search_repeat_threshold_ms">100</integer> <integer name="carousel_focus_search_repeat_threshold_ms">100</integer>
<integer name="carousel_cards_scaling_shape">1</integer>
<integer name="carousel_cards_alpha_shape">4</integer>
<!-- Default SWITCH landscape layout --> <!-- Default SWITCH landscape layout -->
<integer name="BUTTON_A_X">760</integer> <integer name="BUTTON_A_X">760</integer>
@@ -111,7 +111,7 @@
<!-- NVDEC Emulation --> <!-- NVDEC Emulation -->
<string name="nvdec_emulation">NVDEC Emulation</string> <string name="nvdec_emulation">NVDEC Emulation</string>
<string name="nvdec_emulation_description">Select how video decoding (NVDEC) is handled during cutscenes and intros.</string> <string name="nvdec_emulation_description">Change to CPU if a crash occurs on cinematics.</string>
<string name="nvdec_emulation_cpu" translatable="false">CPU</string> <string name="nvdec_emulation_cpu" translatable="false">CPU</string>
<string name="nvdec_emulation_gpu" translatable="false">GPU</string> <string name="nvdec_emulation_gpu" translatable="false">GPU</string>
<string name="nvdec_emulation_none">None</string> <string name="nvdec_emulation_none">None</string>
@@ -585,6 +585,8 @@
<string name="rescale_hack_description">Enables a legacy handling for the rescale configuration pass for games by using a quick rescale path</string> <string name="rescale_hack_description">Enables a legacy handling for the rescale configuration pass for games by using a quick rescale path</string>
<string name="renderer_asynchronous_shaders">Use asynchronous shaders</string> <string name="renderer_asynchronous_shaders">Use asynchronous shaders</string>
<string name="renderer_asynchronous_shaders_description">Compiles shaders asynchronously. This may reduce stutters but may also introduce glitches.</string> <string name="renderer_asynchronous_shaders_description">Compiles shaders asynchronously. This may reduce stutters but may also introduce glitches.</string>
<string name="renderer_unified_memory">Unified memory access</string>
<string name="renderer_unified_memory_description">Allows GPU write buffer readbacks directly into guest memory, skipping the CPU staging copy.</string>
<string name="gpu_unswizzle_settings">GPU Unswizzle Settings</string> <string name="gpu_unswizzle_settings">GPU Unswizzle Settings</string>
<string name="gpu_unswizzle_settings_description">Configure GPU-based texture unswizzling parameters or disable it entirely. Adjust these settings to balance performance and texture loading quality.</string> <string name="gpu_unswizzle_settings_description">Configure GPU-based texture unswizzling parameters or disable it entirely. Adjust these settings to balance performance and texture loading quality.</string>
<string name="gpu_unswizzle_enable">Enable GPU Unswizzle</string> <string name="gpu_unswizzle_enable">Enable GPU Unswizzle</string>
@@ -1246,9 +1248,6 @@
<string name="multiplier_x4" translatable="false">x4</string> <string name="multiplier_x4" translatable="false">x4</string>
<string name="multiplier_x8" translatable="false">x8</string> <string name="multiplier_x8" translatable="false">x8</string>
<string name="multiplier_x16" translatable="false">x16</string> <string name="multiplier_x16" translatable="false">x16</string>
<string name="multiplier_x32" translatable="false">x32</string>
<string name="multiplier_x64" translatable="false">x64</string>
<string name="multiplier_none">None</string>
<!-- Black backgrounds theme --> <!-- Black backgrounds theme -->
<string name="use_black_backgrounds">Black backgrounds</string> <string name="use_black_backgrounds">Black backgrounds</string>
+22 -28
View File
@@ -4,34 +4,17 @@
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project // SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later // SPDX-License-Identifier: GPL-2.0-or-later
#include <fstream>
#include "common/heap_tracker.h" #include "common/heap_tracker.h"
#include "common/logging.h" #include "common/logging.h"
#include "common/memory_detect.h"
#include "common/assert.h" #include "common/assert.h"
namespace Common { namespace Common {
namespace {
s64 GetMaxPermissibleResidentMapCount() {
// Default value.
s64 value = 65530;
// Try to read how many mappings we can make.
std::ifstream s("/proc/sys/vm/max_map_count");
s >> value;
// Print, for debug.
LOG_INFO(HW_Memory, "Current maximum map count: {}", value);
// Allow 20000 maps for other code and to account for split inaccuracy.
return std::max<s64>(value - 20000, 0);
}
} // namespace
HeapTracker::HeapTracker(Common::HostMemory& buffer) HeapTracker::HeapTracker(Common::HostMemory& buffer)
: m_buffer(buffer), m_max_resident_map_count(GetMaxPermissibleResidentMapCount()) {} : m_buffer(buffer),
m_has_hardware_buffer_backing(!buffer.BackingHardwareBuffers().empty()),
m_max_resident_map_count(static_cast<s64>(GetPermissibleMapCount())) {}
HeapTracker::~HeapTracker() = default; HeapTracker::~HeapTracker() = default;
void HeapTracker::Map(size_t virtual_offset, size_t host_offset, size_t length, void HeapTracker::Map(size_t virtual_offset, size_t host_offset, size_t length,
@@ -85,7 +68,8 @@ void HeapTracker::Unmap(size_t virtual_offset, size_t size, bool is_separate_hea
// If resident, erase from resident map. // If resident, erase from resident map.
if (item->is_resident) { if (item->is_resident) {
ASSERT(--m_resident_map_count >= 0); m_resident_map_count -= this->HostMapCount(item->paddr, item->size);
ASSERT(m_resident_map_count >= 0);
m_resident_mappings.erase(m_resident_mappings.iterator_to(*item)); m_resident_mappings.erase(m_resident_mappings.iterator_to(*item));
} }
@@ -191,7 +175,7 @@ bool HeapTracker::DeferredMapSeparateHeap(size_t virtual_offset) {
// This map is now resident. // This map is now resident.
it->is_resident = true; it->is_resident = true;
m_resident_map_count++; m_resident_map_count += this->HostMapCount(it->paddr, it->size);
m_resident_mappings.insert(*it); m_resident_mappings.insert(*it);
} }
@@ -213,17 +197,17 @@ void HeapTracker::RebuildSeparateHeapAddressSpace() {
// Despite being worse in theory, this has proven to be better in practice than more // Despite being worse in theory, this has proven to be better in practice than more
// regularly dumping a smaller amount, because it significantly reduces average case // regularly dumping a smaller amount, because it significantly reduces average case
// lock contention. // lock contention.
std::size_t const desired_count = (std::min)(m_resident_map_count, m_max_resident_map_count) / 2; s64 const desired_count = (std::min)(m_resident_map_count, m_max_resident_map_count) / 2;
std::size_t const evict_count = m_resident_map_count - desired_count;
auto it = m_resident_mappings.begin(); auto it = m_resident_mappings.begin();
for (size_t i = 0; i < evict_count && it != m_resident_mappings.end(); i++) { while (m_resident_map_count > desired_count && it != m_resident_mappings.end()) {
// Unmark and unmap. // Unmark and unmap.
it->is_resident = false; it->is_resident = false;
m_buffer.Unmap(it->vaddr, it->size, false); m_buffer.Unmap(it->vaddr, it->size, false);
// Advance. // Advance.
ASSERT(--m_resident_map_count >= 0); m_resident_map_count -= this->HostMapCount(it->paddr, it->size);
ASSERT(m_resident_map_count >= 0);
it = m_resident_mappings.erase(it); it = m_resident_mappings.erase(it);
} }
} }
@@ -245,6 +229,7 @@ void HeapTracker::SplitHeapMapLocked(VAddr offset) {
// Cache the original values. // Cache the original values.
auto* const left = std::addressof(*it); auto* const left = std::addressof(*it);
const size_t orig_size = left->size; const size_t orig_size = left->size;
const s64 orig_host_map_count = this->HostMapCount(left->paddr, orig_size);
// Adjust the left map. // Adjust the left map.
const size_t left_size = offset - left->vaddr; const size_t left_size = offset - left->vaddr;
@@ -266,11 +251,20 @@ void HeapTracker::SplitHeapMapLocked(VAddr offset) {
// If resident, also insert into resident map. // If resident, also insert into resident map.
if (right->is_resident) { if (right->is_resident) {
m_resident_map_count++; m_resident_map_count += this->HostMapCount(left->paddr, left->size) +
this->HostMapCount(right->paddr, right->size) -
orig_host_map_count;
m_resident_mappings.insert(*right); m_resident_mappings.insert(*right);
} }
} }
s64 HeapTracker::HostMapCount(PAddr paddr, size_t size) const {
if (!m_has_hardware_buffer_backing) {
return size != 0 ? 1 : 0;
}
return static_cast<s64>(m_buffer.BackingMapCount(paddr, size));
}
HeapTracker::AddrTree::iterator HeapTracker::GetNearestHeapMapLocked(VAddr offset) { HeapTracker::AddrTree::iterator HeapTracker::GetNearestHeapMapLocked(VAddr offset) {
const SeparateHeapMap key{ const SeparateHeapMap key{
.vaddr = offset, .vaddr = offset,
+6
View File
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project // SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later // SPDX-License-Identifier: GPL-2.0-or-later
@@ -82,10 +85,13 @@ private:
AddrTree::iterator GetNearestHeapMapLocked(VAddr offset); AddrTree::iterator GetNearestHeapMapLocked(VAddr offset);
s64 HostMapCount(PAddr paddr, size_t size) const;
void RebuildSeparateHeapAddressSpace(); void RebuildSeparateHeapAddressSpace();
private: private:
Common::HostMemory& m_buffer; Common::HostMemory& m_buffer;
const bool m_has_hardware_buffer_backing;
const s64 m_max_resident_map_count; const s64 m_max_resident_map_count;
std::shared_mutex m_rebuild_lock{}; std::shared_mutex m_rebuild_lock{};
+393 -6
View File
@@ -51,14 +51,65 @@
#endif // ^^^ POSIX ^^^ #endif // ^^^ POSIX ^^^
#include <atomic>
#include <mutex> #include <mutex>
#include <random> #include <random>
#include <vector>
#include "common/alignment.h" #include "common/alignment.h"
#include "common/assert.h" #include "common/assert.h"
#include "common/free_region_manager.h" #include "common/free_region_manager.h"
#include "common/host_memory.h" #include "common/host_memory.h"
#include "common/logging.h" #include "common/logging.h"
#include "common/memory_detect.h"
#include "common/settings.h"
#ifdef __ANDROID__
#include <cerrno>
#include <dlfcn.h>
#include <sys/ioctl.h>
#include <android/hardware_buffer.h>
namespace {
struct NativeHandle {
int version;
int numFds;
int numInts;
int data[1];
};
using PFN_AHardwareBuffer_getNativeHandle = const NativeHandle* (*)(const AHardwareBuffer*);
PFN_AHardwareBuffer_getNativeHandle ResolveGetNativeHandle() {
void* const lib = dlopen("libnativewindow.so", RTLD_NOW);
if (lib == nullptr) {
return nullptr;
}
return reinterpret_cast<PFN_AHardwareBuffer_getNativeHandle>(
dlsym(lib, "AHardwareBuffer_getNativeHandle"));
}
struct DmaBufSync {
u64 flags;
};
constexpr u64 DmaBufSyncRead = 1ULL << 0;
constexpr u64 DmaBufSyncWrite = 1ULL << 1;
constexpr u64 DmaBufSyncStart = 0ULL << 2;
constexpr u64 DmaBufSyncEnd = 1ULL << 2;
void SyncDmaBufCpuAccess(int fd, u64 phase) {
DmaBufSync sync{.flags = phase | DmaBufSyncRead | DmaBufSyncWrite};
while (ioctl(fd, _IOW('b', 0, DmaBufSync), &sync) != 0) {
if (errno != EINTR) {
return;
}
}
}
} // namespace
#endif
#if defined(__ANDROID__) && __ANDROID_API__ < 30 #if defined(__ANDROID__) && __ANDROID_API__ < 30
#include <sys/syscall.h> #include <sys/syscall.h>
@@ -75,6 +126,12 @@ namespace Common {
[[maybe_unused]] constexpr size_t PageAlignment = 0x1000; [[maybe_unused]] constexpr size_t PageAlignment = 0x1000;
[[maybe_unused]] constexpr size_t HugePageSize = 0x200000; [[maybe_unused]] constexpr size_t HugePageSize = 0x200000;
static std::atomic<u64> committed_backing_size{};
u64 GetCommittedBackingSize() noexcept {
return committed_backing_size.load(std::memory_order_relaxed);
}
#ifdef _WIN32 #ifdef _WIN32
// Manually imported for MinGW compatibility // Manually imported for MinGW compatibility
@@ -123,7 +180,7 @@ static void GetFuncAddress(Common::DynamicLibrary& dll, const char* name, T& pfn
class HostMemory::Impl { class HostMemory::Impl {
public: public:
explicit Impl(size_t backing_size_, size_t virtual_size_) explicit Impl(size_t backing_size_, size_t virtual_size_, size_t)
: backing_size{backing_size_} : backing_size{backing_size_}
, virtual_size{virtual_size_} , virtual_size{virtual_size_}
, process{GetCurrentProcess()} , process{GetCurrentProcess()}
@@ -229,6 +286,10 @@ public:
UNREACHABLE(); UNREACHABLE();
} }
bool IsBackingShared() const noexcept {
return true;
}
const size_t backing_size; ///< Size of the backing memory in bytes const size_t backing_size; ///< Size of the backing memory in bytes
const size_t virtual_size; ///< Size of the virtual address placeholder in bytes const size_t virtual_size; ///< Size of the virtual address placeholder in bytes
@@ -501,9 +562,10 @@ static int shm_open_anon(int flags, mode_t mode) {
class HostMemory::Impl { class HostMemory::Impl {
public: public:
explicit Impl(size_t backing_size_, size_t virtual_size_) explicit Impl(size_t backing_size_, size_t virtual_size_, size_t preferred_offset_)
: backing_size{backing_size_} : backing_size{backing_size_}
, virtual_size{virtual_size_} , virtual_size{virtual_size_}
, preferred_offset{preferred_offset_}
{} {}
bool Init() { bool Init() {
@@ -543,10 +605,15 @@ public:
LOG_WARNING(Common_Memory, "Using private mappings instead of shared ones"); LOG_WARNING(Common_Memory, "Using private mappings instead of shared ones");
backing_base = static_cast<u8*>(mmap(nullptr, backing_size, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0)); backing_base = static_cast<u8*>(mmap(nullptr, backing_size, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0));
if (fd > 0) { if (fd > 0) {
fd = -1;
close(fd); close(fd);
} }
fd = -1;
} else { } else {
#ifdef __ANDROID__
if (InitAhbBacking()) {
return InitVirtual();
}
#endif
backing_base = static_cast<u8*>(mmap(nullptr, backing_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0)); backing_base = static_cast<u8*>(mmap(nullptr, backing_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0));
} }
if (backing_base == MAP_FAILED) { if (backing_base == MAP_FAILED) {
@@ -554,7 +621,10 @@ public:
return false; return false;
} }
// Virtual memory initialization return InitVirtual();
}
bool InitVirtual() {
virtual_base = virtual_map_base = static_cast<u8*>(ChooseVirtualBase(virtual_size)); virtual_base = virtual_map_base = static_cast<u8*>(ChooseVirtualBase(virtual_size));
if (virtual_base == MAP_FAILED) { if (virtual_base == MAP_FAILED) {
LOG_CRITICAL(HW_Memory, "mmap failed: {}", strerror(errno)); LOG_CRITICAL(HW_Memory, "mmap failed: {}", strerror(errno));
@@ -567,6 +637,244 @@ public:
return true; return true;
} }
#ifdef __ANDROID__
static AHardwareBuffer_Desc MakeBlobDesc(size_t len) {
return AHardwareBuffer_Desc{
.width = static_cast<u32>(len),
.height = 1,
.layers = 1,
.format = AHARDWAREBUFFER_FORMAT_BLOB,
.usage = AHARDWAREBUFFER_USAGE_CPU_READ_OFTEN |
AHARDWAREBUFFER_USAGE_CPU_WRITE_OFTEN |
AHARDWAREBUFFER_USAGE_GPU_DATA_BUFFER,
.stride = 0,
.rfu0 = 0,
.rfu1 = 0,
};
}
static bool ProbeAhbBacking(PFN_AHardwareBuffer_getNativeHandle get_native_handle) {
const AHardwareBuffer_Desc desc = MakeBlobDesc(PageAlignment * 2);
AHardwareBuffer* buffer{};
if (AHardwareBuffer_allocate(&desc, &buffer) != 0 || buffer == nullptr) {
return false;
}
const NativeHandle* const handle = get_native_handle(buffer);
if (handle == nullptr || handle->numFds < 1) {
AHardwareBuffer_release(buffer);
return false;
}
const int probe_fd = handle->data[0];
bool ok = true;
const auto try_map = [&](int prot, off_t offset) {
if (!ok) {
return;
}
void* const ptr = mmap(nullptr, PageAlignment, prot, MAP_SHARED, probe_fd, offset);
if (ptr == MAP_FAILED) {
ok = false;
return;
}
munmap(ptr, PageAlignment);
};
try_map(PROT_READ | PROT_WRITE, 0);
try_map(PROT_READ | PROT_WRITE, static_cast<off_t>(PageAlignment));
#ifdef ARCHITECTURE_arm64
try_map(PROT_READ | PROT_EXEC, 0);
#endif
AHardwareBuffer_release(buffer);
return ok;
}
size_t ComputeAhbBudget(size_t window_size) const {
const u64 total_physical = Common::GetMemInfo().TotalPhysicalMemory;
constexpr u64 BaselineFootprint = 6ULL << 30;
if (total_physical <= BaselineFootprint) {
return 0;
}
const u64 permissible_maps = Common::GetPermissibleMapCount();
if (permissible_maps == 0) {
return 0;
}
u64 budget = (total_physical - BaselineFootprint) / 2;
constexpr u64 MapSlotsPerWindow = 64;
const u64 affordable_windows = permissible_maps / MapSlotsPerWindow;
budget = (std::min)(budget, affordable_windows * window_size);
const u64 available = Common::GetAvailablePhysicalMemory();
if (available != 0) {
budget = (std::min)(budget, available / 2);
}
budget = (std::min)(budget, static_cast<u64>(backing_size));
budget = Common::AlignDown(budget, window_size);
constexpr u64 MinimumBudget = 256ULL << 20;
if (budget < MinimumBudget) {
return 0;
}
return static_cast<size_t>(budget);
}
bool InitAhbBacking() {
if (!Settings::values.use_unified_memory.GetValue()) {
return false;
}
static const PFN_AHardwareBuffer_getNativeHandle get_native_handle =
ResolveGetNativeHandle();
if (get_native_handle == nullptr) {
return false;
}
constexpr size_t window_size = 256ULL << 20;
const size_t budget = ComputeAhbBudget(window_size);
if (budget == 0) {
return false;
}
if (!ProbeAhbBacking(get_native_handle)) {
return false;
}
const size_t aligned_backing = Common::AlignDown(backing_size, window_size);
const size_t max_windows = (std::min)(budget, aligned_backing) / window_size;
std::vector<AHardwareBuffer*> buffers;
std::vector<int> buffer_fds;
const auto cleanup = [&] {
for (AHardwareBuffer* buffer : buffers) {
AHardwareBuffer_release(buffer);
}
buffers.clear();
buffer_fds.clear();
};
for (size_t i = 0; i < max_windows; ++i) {
const AHardwareBuffer_Desc desc = MakeBlobDesc(window_size);
AHardwareBuffer* buffer{};
if (AHardwareBuffer_allocate(&desc, &buffer) != 0 || buffer == nullptr) {
break;
}
const NativeHandle* const handle = get_native_handle(buffer);
if (handle == nullptr || handle->numFds < 1) {
AHardwareBuffer_release(buffer);
break;
}
const int buffer_fd = handle->data[0];
const off_t buffer_len = lseek(buffer_fd, 0, SEEK_END);
if (buffer_len < static_cast<off_t>(window_size)) {
AHardwareBuffer_release(buffer);
break;
}
buffers.push_back(buffer);
buffer_fds.push_back(buffer_fd);
}
const size_t num_windows = buffers.size();
if (num_windows == 0) {
return false;
}
const size_t region_size = num_windows * window_size;
const size_t region_base = Common::AlignDown(
(std::min)(preferred_offset, aligned_backing - region_size), window_size);
u8* const base = static_cast<u8*>(mmap(nullptr, backing_size, PROT_NONE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0));
if (base == MAP_FAILED) {
cleanup();
return false;
}
const auto map_over_reservation = [&](size_t offset, size_t len, int map_fd,
off_t map_offset) {
if (len == 0) {
return true;
}
if (mmap(base + offset, len, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_FIXED, map_fd,
map_offset) == MAP_FAILED) {
munmap(base, backing_size);
cleanup();
return false;
}
return true;
};
if (!map_over_reservation(0, region_base, fd, 0)) {
return false;
}
for (size_t i = 0; i < num_windows; ++i) {
if (!map_over_reservation(region_base + i * window_size, window_size, buffer_fds[i],
0)) {
return false;
}
}
const size_t tail_offset = region_base + region_size;
if (!map_over_reservation(tail_offset, backing_size - tail_offset, fd,
static_cast<off_t>(tail_offset))) {
return false;
}
backing_base = base;
ahb_windows = std::move(buffers);
ahb_fds = std::move(buffer_fds);
ahb_window_size = window_size;
ahb_base = region_base;
ahb_bytes = region_size;
committed_backing_size.store(region_size, std::memory_order_relaxed);
for (const int window_fd : ahb_fds) {
SyncDmaBufCpuAccess(window_fd, DmaBufSyncStart);
}
return true;
}
void MapBackingRange(size_t virtual_offset, size_t host_offset, size_t length, int prot_flags) {
while (length > 0) {
int map_fd = fd;
off_t map_offset = static_cast<off_t>(host_offset);
size_t chunk = length;
if (host_offset < ahb_base) {
chunk = (std::min)(chunk, ahb_base - host_offset);
} else if (host_offset < ahb_base + ahb_bytes) {
const size_t relative = host_offset - ahb_base;
const size_t window = relative / ahb_window_size;
const size_t local = relative % ahb_window_size;
map_fd = ahb_fds[window];
map_offset = static_cast<off_t>(local);
chunk = (std::min)(chunk, ahb_window_size - local);
}
void* const ret = mmap(virtual_base + virtual_offset, chunk, prot_flags,
MAP_SHARED | MAP_FIXED, map_fd, map_offset);
ASSERT_MSG(ret != MAP_FAILED, "mmap: {}", strerror(errno));
virtual_offset += chunk;
host_offset += chunk;
length -= chunk;
}
}
size_t BackingMapCount(size_t host_offset, size_t length) const noexcept {
if (length == 0) {
return 0;
}
if (ahb_bytes == 0) {
return 1;
}
size_t count = 0;
while (length > 0) {
size_t chunk = length;
if (host_offset < ahb_base) {
chunk = (std::min)(chunk, ahb_base - host_offset);
} else if (host_offset < ahb_base + ahb_bytes) {
const size_t local = (host_offset - ahb_base) % ahb_window_size;
chunk = (std::min)(chunk, ahb_window_size - local);
}
host_offset += chunk;
length -= chunk;
++count;
}
return count;
}
std::span<AHardwareBuffer* const> AhbWindows() const noexcept {
return ahb_windows;
}
size_t AhbWindowSize() const noexcept {
return ahb_bytes != 0 ? ahb_window_size : 0;
}
size_t AhbBase() const noexcept {
return ahb_base;
}
#endif
~Impl() { ~Impl() {
Release(); Release();
} }
@@ -587,6 +895,12 @@ public:
#ifdef ARCHITECTURE_arm64 #ifdef ARCHITECTURE_arm64
if (True(perms & MemoryPermission::Execute)) if (True(perms & MemoryPermission::Execute))
prot_flags |= PROT_EXEC; prot_flags |= PROT_EXEC;
#endif
#ifdef __ANDROID__
if (ahb_bytes != 0) {
MapBackingRange(virtual_offset, host_offset, length, prot_flags);
return;
}
#endif #endif
int flags = (fd >= 0 ? MAP_SHARED : MAP_PRIVATE) | MAP_FIXED; int flags = (fd >= 0 ? MAP_SHARED : MAP_PRIVATE) | MAP_FIXED;
void* ret = mmap(virtual_base + virtual_offset, length, prot_flags, flags, fd, host_offset); void* ret = mmap(virtual_base + virtual_offset, length, prot_flags, flags, fd, host_offset);
@@ -632,8 +946,18 @@ public:
virtual_base = nullptr; virtual_base = nullptr;
} }
bool IsBackingShared() const noexcept {
#ifdef __ANDROID__
if (ahb_bytes != 0) {
return true;
}
#endif
return fd >= 0;
}
const size_t backing_size; ///< Size of the backing memory in bytes const size_t backing_size; ///< Size of the backing memory in bytes
const size_t virtual_size; ///< Size of the virtual address placeholder in bytes const size_t virtual_size; ///< Size of the virtual address placeholder in bytes
const size_t preferred_offset;
u8* backing_base{reinterpret_cast<u8*>(MAP_FAILED)}; u8* backing_base{reinterpret_cast<u8*>(MAP_FAILED)};
u8* virtual_base{reinterpret_cast<u8*>(MAP_FAILED)}; u8* virtual_base{reinterpret_cast<u8*>(MAP_FAILED)};
@@ -656,6 +980,21 @@ private:
int ret = close(fd); int ret = close(fd);
ASSERT_MSG(ret == 0, "close failed: {}", strerror(errno)); ASSERT_MSG(ret == 0, "close failed: {}", strerror(errno));
} }
#ifdef __ANDROID__
for (const int window_fd : ahb_fds) {
SyncDmaBufCpuAccess(window_fd, DmaBufSyncEnd);
}
for (AHardwareBuffer* buffer : ahb_windows) {
AHardwareBuffer_release(buffer);
}
ahb_windows.clear();
ahb_fds.clear();
if (ahb_bytes != 0) {
committed_backing_size.store(0, std::memory_order_relaxed);
ahb_bytes = 0;
}
#endif
} }
void AdjustMap(size_t* virtual_offset, size_t* length) { void AdjustMap(size_t* virtual_offset, size_t* length) {
@@ -681,11 +1020,19 @@ private:
int fd{-1}; // memfd file descriptor, -1 is the error value of memfd_create int fd{-1}; // memfd file descriptor, -1 is the error value of memfd_create
FreeRegionManager free_manager{}; FreeRegionManager free_manager{};
#ifdef __ANDROID__
std::vector<AHardwareBuffer*> ahb_windows;
std::vector<int> ahb_fds;
size_t ahb_window_size{};
size_t ahb_base{};
size_t ahb_bytes{};
#endif
}; };
#endif // ^^^ POSIX ^^^ #endif // ^^^ POSIX ^^^
HostMemory::HostMemory(size_t backing_size_, size_t virtual_size_) HostMemory::HostMemory(size_t backing_size_, size_t virtual_size_, size_t preferred_offset_)
: backing_size(backing_size_) : backing_size(backing_size_)
, virtual_size(virtual_size_) , virtual_size(virtual_size_)
{ {
@@ -697,7 +1044,7 @@ HostMemory::HostMemory(size_t backing_size_, size_t virtual_size_)
#else #else
// Try to allocate a fastmem arena. // Try to allocate a fastmem arena.
// The implementation will fail with std::bad_alloc on errors. // The implementation will fail with std::bad_alloc on errors.
impl = std::make_unique<HostMemory::Impl>(AlignUp(backing_size, PageAlignment), AlignUp(virtual_size, PageAlignment) + HugePageSize); impl = std::make_unique<HostMemory::Impl>(AlignUp(backing_size, PageAlignment), AlignUp(virtual_size, PageAlignment) + HugePageSize, preferred_offset_);
if (impl->Init()) { if (impl->Init()) {
backing_base = impl->backing_base; backing_base = impl->backing_base;
virtual_base = impl->virtual_base; virtual_base = impl->virtual_base;
@@ -767,6 +1114,46 @@ void HostMemory::ClearBackingRegion(size_t physical_offset, size_t length, u32 f
std::memset(backing_base + physical_offset, fill_value, length); std::memset(backing_base + physical_offset, fill_value, length);
} }
std::span<AHardwareBuffer* const> HostMemory::BackingHardwareBuffers() const noexcept {
#ifdef __ANDROID__
return impl ? impl->AhbWindows() : std::span<AHardwareBuffer* const>{};
#else
return {};
#endif
}
size_t HostMemory::BackingMapCount(size_t host_offset, size_t length) const noexcept {
#ifdef __ANDROID__
return impl ? impl->BackingMapCount(host_offset, length) : (length != 0 ? 1 : 0);
#else
return length != 0 ? 1 : 0;
#endif
}
size_t HostMemory::BackingHardwareBufferWindowSize() const noexcept {
#ifdef __ANDROID__
return impl ? impl->AhbWindowSize() : 0;
#else
return 0;
#endif
}
bool HostMemory::IsBackingShared() const noexcept {
#if defined(__OPENORBIS__) || defined(__managarm__)
return false;
#else
return impl && impl->IsBackingShared();
#endif
}
size_t HostMemory::BackingHardwareBufferBase() const noexcept {
#ifdef __ANDROID__
return impl ? impl->AhbBase() : 0;
#else
return 0;
#endif
}
void HostMemory::EnableDirectMappedAddress() { void HostMemory::EnableDirectMappedAddress() {
#if !(defined(__OPENORBIS__) || defined(__managarm__)) #if !(defined(__OPENORBIS__) || defined(__managarm__))
if (impl) { if (impl) {
+20 -1
View File
@@ -8,12 +8,17 @@
#include <memory> #include <memory>
#include <optional> #include <optional>
#include <span>
#include "common/common_funcs.h" #include "common/common_funcs.h"
#include "common/common_types.h" #include "common/common_types.h"
#include "common/virtual_buffer.h" #include "common/virtual_buffer.h"
struct AHardwareBuffer;
namespace Common { namespace Common {
[[nodiscard]] u64 GetCommittedBackingSize() noexcept;
enum class MemoryPermission : u32 { enum class MemoryPermission : u32 {
Read = 1 << 0, Read = 1 << 0,
Write = 1 << 1, Write = 1 << 1,
@@ -28,7 +33,7 @@ DECLARE_ENUM_FLAG_OPERATORS(MemoryPermission)
*/ */
class HostMemory { class HostMemory {
public: public:
explicit HostMemory(size_t backing_size_, size_t virtual_size_); explicit HostMemory(size_t backing_size_, size_t virtual_size_, size_t preferred_offset_ = 0);
~HostMemory(); ~HostMemory();
/** /**
@@ -62,6 +67,20 @@ public:
return backing_base; return backing_base;
} }
[[nodiscard]] size_t BackingSize() const noexcept {
return backing_size;
}
[[nodiscard]] size_t BackingMapCount(size_t host_offset, size_t length) const noexcept;
[[nodiscard]] std::span<AHardwareBuffer* const> BackingHardwareBuffers() const noexcept;
[[nodiscard]] size_t BackingHardwareBufferWindowSize() const noexcept;
[[nodiscard]] size_t BackingHardwareBufferBase() const noexcept;
[[nodiscard]] bool IsBackingShared() const noexcept;
[[nodiscard]] u8* VirtualBasePointer() noexcept { [[nodiscard]] u8* VirtualBasePointer() noexcept {
return virtual_base; return virtual_base;
} }
+56 -45
View File
@@ -39,6 +39,19 @@
namespace Common::Log { namespace Common::Log {
/// @brief A log entry. Log entries are store in a structured format to permit more varied output
/// formatting on different frontends, as well as facilitating filtering and aggregation.
struct Entry {
char const* message = nullptr;
size_t message_len = 0;
std::chrono::microseconds timestamp;
Class log_class{};
Level log_level{};
const char* filename = nullptr;
const char* function = nullptr;
uint32_t line_num = 0;
};
namespace { namespace {
/// @brief Returns the name of the passed log class as a C-string. Subclasses are separated by periods /// @brief Returns the name of the passed log class as a C-string. Subclasses are separated by periods
@@ -70,8 +83,6 @@ const char* GetLevelName(Level log_level) {
} }
} }
}
// Some IDEs prefer <file>:<line> instead, so let's just do that :) // Some IDEs prefer <file>:<line> instead, so let's just do that :)
std::string FormatLogMessage(const Entry& entry) noexcept { std::string FormatLogMessage(const Entry& entry) noexcept {
if (!entry.filename) return ""; if (!entry.filename) return "";
@@ -79,10 +90,9 @@ std::string FormatLogMessage(const Entry& entry) noexcept {
auto const time_fractional = uint32_t(entry.timestamp.count() % 1000000); auto const time_fractional = uint32_t(entry.timestamp.count() % 1000000);
auto const class_name = GetLogClassName(entry.log_class); auto const class_name = GetLogClassName(entry.log_class);
auto const level_name = GetLevelName(entry.log_level); auto const level_name = GetLevelName(entry.log_level);
return fmt::format("[{:4d}.{:06d}] {} <{}> {}:{}:{}: {}", time_seconds, time_fractional, class_name, level_name, entry.filename, entry.line_num, entry.function, entry.message); return fmt::format("[{:4d}.{:06d}] {} <{}> {}:{}:{}: {}\n", time_seconds, time_fractional, class_name, level_name, entry.filename, entry.line_num, entry.function, entry.message);
} }
namespace {
template <typename It> template <typename It>
Level GetLevelByName(const It begin, const It end) { Level GetLevelByName(const It begin, const It end) {
for (u32 i = 0; i < u32(Level::Count); ++i) { for (u32 i = 0; i < u32(Level::Count); ++i) {
@@ -127,25 +137,6 @@ bool ParseFilterRule(Filter& instance, Iterator begin, Iterator end) {
instance.SetClassLevel(log_class, level); instance.SetClassLevel(log_class, level);
return true; return true;
} }
} // Anonymous namespace
void Filter::ParseFilterString(std::string_view filter_view) {
auto clause_begin = filter_view.cbegin();
while (clause_begin != filter_view.cend()) {
auto clause_end = std::find(clause_begin, filter_view.cend(), ' ');
// If clause isn't empty
if (clause_end != clause_begin) {
ParseFilterRule(*this, clause_begin, clause_end);
}
if (clause_end != filter_view.cend()) {
// Skip over the whitespace
++clause_end;
}
clause_begin = clause_end;
}
}
namespace {
/// @brief Trims up to and including the last of ../, ..\, src/, src\ in a string /// @brief Trims up to and including the last of ../, ..\, src/, src\ in a string
/// do not be fooled this isn't generating new strings on .rodata :) /// do not be fooled this isn't generating new strings on .rodata :)
@@ -208,7 +199,7 @@ struct ColorConsoleBackend final : public Backend {
}()); }());
SetConsoleTextAttribute(console_handle, color); SetConsoleTextAttribute(console_handle, color);
auto const df = GetDirectFormatArgs(entry); auto const df = GetDirectFormatArgs(entry);
std::fprintf(stdout, CCB_PRINTF_FMT "\n", df.time_seconds, df.time_fractional, df.class_name, df.level_name, entry.filename, entry.line_num, entry.function, entry.message.c_str()); std::fprintf(stdout, CCB_PRINTF_FMT "\n", df.time_seconds, df.time_fractional, df.class_name, df.level_name, entry.filename, entry.line_num, entry.function, entry.message);
} }
} }
void Flush() noexcept override {} void Flush() noexcept override {}
@@ -220,22 +211,24 @@ struct ColorConsoleBackend final : public Backend {
~ColorConsoleBackend() noexcept override {} ~ColorConsoleBackend() noexcept override {}
void Write(const Entry& entry) noexcept override { void Write(const Entry& entry) noexcept override {
if (enabled) { if (enabled) {
#define ESC "\x1b"
auto const color_str = [&entry]() -> const char* { auto const color_str = [&entry]() -> const char* {
switch (entry.log_level) { switch (entry.log_level) {
#define CCB_MAKE_COLOR_FMT(X) ESC X CCB_PRINTF_FMT ESC "[0m\n" case Level::Debug: return "[0;36m"; // Cyan
case Level::Debug: return CCB_MAKE_COLOR_FMT("[0;36m"); // Cyan case Level::Info: return "[0;37m"; // Bright gray
case Level::Info: return CCB_MAKE_COLOR_FMT("[0;37m"); // Bright gray case Level::Warning: return "[1;33m"; // Bright yellow
case Level::Warning: return CCB_MAKE_COLOR_FMT("[1;33m"); // Bright yellow case Level::Error: return "[1;31m"; // Bright red
case Level::Error: return CCB_MAKE_COLOR_FMT("[1;31m"); // Bright red case Level::Critical: return "[1;35m"; // Bright magenta
case Level::Critical: return CCB_MAKE_COLOR_FMT("[1;35m"); // Bright magenta default: return "[1;30m"; // Grey
default: return CCB_MAKE_COLOR_FMT("[1;30m"); // Grey
#undef CCB_MAKE_COLOR_FMT
} }
}(); }();
auto const df = GetDirectFormatArgs(entry); auto const df = GetDirectFormatArgs(entry);
std::fprintf(stdout, color_str, df.time_seconds, df.time_fractional, df.class_name, df.level_name, entry.filename, entry.line_num, entry.function, entry.message.c_str()); // more restrictive, because take for example this simple prelude:
#undef ESC // [ 50.872256] Config <Info> common/settings.cpp:142:LogSettings:
char buffer[128];
auto result = fmt::format_to_n(buffer, sizeof(buffer) - 1, "\x1b{}[{:4d}.{:06d}] {} <{}> {}:{}:{}: ", color_str, df.time_seconds, df.time_fractional, df.class_name, df.level_name, entry.filename, entry.line_num, entry.function, entry.message);
std::fwrite(buffer, 1, (std::min)(sizeof(buffer) - 1, result.size), stdout);
std::fwrite(entry.message, 1, entry.message_len, stdout);
std::fwrite("\x1b[0m\n", 1, sizeof("\x1b[0m\n"), stdout);
} }
} }
void Flush() noexcept override {} void Flush() noexcept override {}
@@ -246,7 +239,7 @@ struct ColorConsoleBackend final : public Backend {
#ifndef __OPENORBIS__ #ifndef __OPENORBIS__
/// @brief Backend that writes to a file passed into the constructor /// @brief Backend that writes to a file passed into the constructor
struct FileBackend final : public Backend { struct FileBackend final : public Backend {
explicit FileBackend(const std::filesystem::path& filename) noexcept { explicit FileBackend(const std::filesystem::path filename) noexcept {
auto old_filename = filename; auto old_filename = filename;
old_filename += ".old.txt"; old_filename += ".old.txt";
// Existence checks are done within the functions themselves. // Existence checks are done within the functions themselves.
@@ -261,7 +254,7 @@ struct FileBackend final : public Backend {
if (!enabled) if (!enabled)
return; return;
auto message = FormatLogMessage(entry).append(1, '\n'); auto message = FormatLogMessage(entry);
#ifndef __ANDROID__ #ifndef __ANDROID__
if (Settings::values.censor_username.GetValue()) { if (Settings::values.censor_username.GetValue()) {
// This must be a static otherwise it would get checked on EVERY // This must be a static otherwise it would get checked on EVERY
@@ -269,8 +262,7 @@ struct FileBackend final : public Backend {
static std::string username = []() -> std::string { static std::string username = []() -> std::string {
// in order of precedence // in order of precedence
// LOGNAME usually works on UNIX, USERNAME on Windows // LOGNAME usually works on UNIX, USERNAME on Windows
// Some UNIX systems suck and don't use LOGNAME so we also // Some UNIX systems suck and don't use LOGNAME so we also need USER :(
// need USER :(
for (auto const var : { "LOGNAME", "USERNAME", "USER", }) for (auto const var : { "LOGNAME", "USERNAME", "USER", })
if (auto const s = ::getenv(var); s != nullptr) if (auto const s = ::getenv(var); s != nullptr)
return std::string{s}; return std::string{s};
@@ -280,7 +272,7 @@ struct FileBackend final : public Backend {
boost::replace_all(message, username, "user"); boost::replace_all(message, username, "user");
} }
#endif #endif
bytes_written += file->WriteString(message); bytes_written += file->WriteSpan(std::span<const char>{message.begin(), message.end()});
// Option to log each line rather than 4k buffers // Option to log each line rather than 4k buffers
if (Settings::values.log_flush_line.GetValue()) if (Settings::values.log_flush_line.GetValue())
@@ -308,14 +300,13 @@ private:
bool enabled = true; bool enabled = true;
}; };
#endif #endif
#ifdef _WIN32 #ifdef _WIN32
/// @brief Backend that writes to Visual Studio's output window /// @brief Backend that writes to Visual Studio's output window
struct DebuggerBackend final : public Backend { struct DebuggerBackend final : public Backend {
explicit DebuggerBackend() noexcept = default; explicit DebuggerBackend() noexcept = default;
~DebuggerBackend() noexcept override = default; ~DebuggerBackend() noexcept override = default;
void Write(const Entry& entry) noexcept override { void Write(const Entry& entry) noexcept override {
::OutputDebugStringW(UTF8ToUTF16W(FormatLogMessage(entry).append(1, '\n')).c_str()); ::OutputDebugStringW(UTF8ToUTF16W(FormatLogMessage(entry)).c_str());
} }
void Flush() noexcept override {} void Flush() noexcept override {}
}; };
@@ -338,7 +329,7 @@ struct LogcatBackend : public Backend {
} }
}(); }();
auto const df = GetDirectFormatArgs(entry); auto const df = GetDirectFormatArgs(entry);
__android_log_print(android_log_priority, "YuzuNative", CCB_PRINTF_FMT, df.time_seconds, df.time_fractional, df.class_name, df.level_name, entry.filename, entry.line_num, entry.function, entry.message.c_str()); __android_log_print(android_log_priority, "YuzuNative", CCB_PRINTF_FMT, df.time_seconds, df.time_fractional, df.class_name, df.level_name, entry.filename, entry.line_num, entry.function, entry.message);
} }
void Flush() noexcept override {} void Flush() noexcept override {}
}; };
@@ -377,7 +368,23 @@ struct Impl {
#endif #endif
std::chrono::steady_clock::time_point time_origin{std::chrono::steady_clock::now()}; std::chrono::steady_clock::time_point time_origin{std::chrono::steady_clock::now()};
}; };
} // namespace } // Anonymous namespace
void Filter::ParseFilterString(std::string_view filter_view) {
auto clause_begin = filter_view.cbegin();
while (clause_begin < filter_view.cend()) {
auto clause_end = std::find(clause_begin, filter_view.cend(), ' ');
// If clause isn't empty
if (clause_end != clause_begin) {
ParseFilterRule(*this, clause_begin, clause_end);
}
if (clause_end != filter_view.cend()) {
// Skip over the whitespace
++clause_end;
}
clause_begin = clause_end;
}
}
// Constructor shall NOT depend upon Settings() or whatever // Constructor shall NOT depend upon Settings() or whatever
// it's ran at global static ctor() time... so BE CAREFUL MFER! // it's ran at global static ctor() time... so BE CAREFUL MFER!
@@ -418,10 +425,14 @@ void SetColorConsoleBackendEnabled(bool enabled) {
void FmtLogMessageImpl(Class log_class, Level log_level, const char* filename, unsigned int line_num, const char* function, fmt::string_view format, const fmt::format_args& args) { void FmtLogMessageImpl(Class log_class, Level log_level, const char* filename, unsigned int line_num, const char* function, fmt::string_view format, const fmt::format_args& args) {
if (logging_instance && logging_instance->filter.CheckMessage(log_class, log_level)) { if (logging_instance && logging_instance->filter.CheckMessage(log_class, log_level)) {
char buffer[BUFSIZ];
auto result = fmt::vformat_to_n(buffer, sizeof(buffer) - 1, format, args);
buffer[result.size] = '\0';
auto const flush = ::Settings::values.log_flush_line.GetValue(); auto const flush = ::Settings::values.log_flush_line.GetValue();
logging_instance->ForEachBackend([=](Backend& backend) { logging_instance->ForEachBackend([=](Backend& backend) {
backend.Write(Entry{ backend.Write(Entry{
.message = fmt::vformat(format, args), .message = buffer,
.message_len = (std::min)(sizeof(buffer) - 1, result.size),
.timestamp = std::chrono::duration_cast<std::chrono::microseconds>(std::chrono::steady_clock::now() - logging_instance->time_origin), .timestamp = std::chrono::duration_cast<std::chrono::microseconds>(std::chrono::steady_clock::now() - logging_instance->time_origin),
.log_class = log_class, .log_class = log_class,
.log_level = log_level, .log_level = log_level,
-21
View File
@@ -140,25 +140,4 @@ void Stop();
void SetGlobalFilter(const Filter& filter); void SetGlobalFilter(const Filter& filter);
void SetColorConsoleBackendEnabled(bool enabled); void SetColorConsoleBackendEnabled(bool enabled);
/// @brief A log entry. Log entries are store in a structured format to permit more varied output
/// formatting on different frontends, as well as facilitating filtering and aggregation.
struct Entry {
std::string message;
std::chrono::microseconds timestamp;
Class log_class{};
Level log_level{};
const char* filename = nullptr;
const char* function = nullptr;
unsigned int line_num = 0;
};
/// Formats a log entry into the provided text buffer.
std::string FormatLogMessage(const Entry& entry) noexcept;
/// Prints the same message as `PrintMessage`, but colored according to the severity level.
void PrintColoredMessage(const Entry& entry) noexcept;
/// Formats and prints a log entry to the android logcat.
void PrintMessageToLogcat(const Entry& entry) noexcept;
} // namespace Common::Log } // namespace Common::Log
+64
View File
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project // SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later // SPDX-License-Identifier: GPL-2.0-or-later
@@ -17,6 +20,10 @@
#endif #endif
#endif #endif
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include "common/memory_detect.h" #include "common/memory_detect.h"
namespace Common { namespace Common {
@@ -69,4 +76,61 @@ const MemoryInfo& GetMemInfo() {
return mem_info; return mem_info;
} }
u64 GetPermissibleMapCount() {
constexpr u64 DefaultMapCount = 65530;
constexpr u64 ReservedMaps = 20000;
u64 count = DefaultMapCount;
#ifdef __linux__
if (std::FILE* const file = std::fopen("/proc/sys/vm/max_map_count", "re")) {
char line[32];
if (std::fgets(line, sizeof(line), file) != nullptr) {
const u64 parsed = std::strtoull(line, nullptr, 10);
if (parsed != 0) {
count = parsed;
}
}
std::fclose(file);
}
#endif
if (count <= ReservedMaps) {
return 0;
}
return count - ReservedMaps;
}
u64 GetAvailablePhysicalMemory() {
#ifdef _WIN32
MEMORYSTATUSEX memorystatus;
memorystatus.dwLength = sizeof(memorystatus);
if (GlobalMemoryStatusEx(&memorystatus) == 0) {
return 0;
}
return memorystatus.ullAvailPhys;
#elif defined(__linux__)
static constexpr char AvailableKey[] = "MemAvailable:";
if (std::FILE* const file = std::fopen("/proc/meminfo", "re")) {
char line[256];
u64 available = 0;
while (std::fgets(line, sizeof(line), file) != nullptr) {
if (std::strncmp(line, AvailableKey, sizeof(AvailableKey) - 1) != 0) {
continue;
}
available = std::strtoull(line + sizeof(AvailableKey) - 1, nullptr, 10) * 1024;
break;
}
std::fclose(file);
if (available != 0) {
return available;
}
}
struct sysinfo meminfo;
if (sysinfo(&meminfo) != 0) {
return 0;
}
return static_cast<u64>(meminfo.freeram) * static_cast<u64>(meminfo.mem_unit);
#else
return 0;
#endif
}
} // namespace Common } // namespace Common
+7
View File
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project // SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later // SPDX-License-Identifier: GPL-2.0-or-later
@@ -18,4 +21,8 @@ struct MemoryInfo {
*/ */
[[nodiscard]] const MemoryInfo& GetMemInfo(); [[nodiscard]] const MemoryInfo& GetMemInfo();
[[nodiscard]] u64 GetPermissibleMapCount();
[[nodiscard]] u64 GetAvailablePhysicalMemory();
} // namespace Common } // namespace Common
+1 -1
View File
@@ -241,7 +241,7 @@ std::optional<std::string> MakeRequest(const std::string& url, const std::string
response.status); response.status);
return {}; return {};
} }
if (!response.headers.contains("content-type")) { if (!response.has_header("content-type")) {
LOG_ERROR(Common, "GET to {}{} returned no content", url, path); LOG_ERROR(Common, "GET to {}{} returned no content", url, path);
return {}; return {};
} }
+3
View File
@@ -645,6 +645,9 @@ struct Values {
SwitchableSetting<bool> use_asynchronous_shaders{linkage, false, "use_asynchronous_shaders", SwitchableSetting<bool> use_asynchronous_shaders{linkage, false, "use_asynchronous_shaders",
Category::RendererHacks}; Category::RendererHacks};
SwitchableSetting<bool> use_unified_memory{linkage, false, "use_unified_memory",
Category::RendererHacks};
SwitchableSetting<GpuUnswizzleSize> gpu_unswizzle_texture_size{linkage, SwitchableSetting<GpuUnswizzleSize> gpu_unswizzle_texture_size{linkage,
GpuUnswizzleSize::Large, GpuUnswizzleSize::Large,
"gpu_unswizzle_texture_size", "gpu_unswizzle_texture_size",
+1 -1
View File
@@ -128,7 +128,7 @@ ENUM(TimeZone, Auto, Default, Cet, Cst6Cdt, Cuba, Eet, Egypt, Eire, Est, Est5Edt
GmtPlusZero, GmtMinusZero, GmtZero, Greenwich, Hongkong, Hst, Iceland, Iran, Israel, Jamaica, GmtPlusZero, GmtMinusZero, GmtZero, Greenwich, Hongkong, Hst, Iceland, Iran, Israel, Jamaica,
Japan, Kwajalein, Libya, Met, Mst, Mst7Mdt, Navajo, Nz, NzChat, Poland, Portugal, Prc, Pst8Pdt, Japan, Kwajalein, Libya, Met, Mst, Mst7Mdt, Navajo, Nz, NzChat, Poland, Portugal, Prc, Pst8Pdt,
Roc, Rok, Singapore, Turkey, Uct, Universal, Utc, WSu, Wet, Zulu); Roc, Rok, Singapore, Turkey, Uct, Universal, Utc, WSu, Wet, Zulu);
ENUM(AnisotropyMode, Automatic, Default, X2, X4, X8, X16, X32, X64, None); ENUM(AnisotropyMode, Automatic, Default, X2, X4, X8, X16);
ENUM(AstcDecodeMode, Cpu, Gpu, CpuAsynchronous); ENUM(AstcDecodeMode, Cpu, Gpu, CpuAsynchronous);
ENUM(AstcRecompression, Uncompressed, Bc1, Bc3); ENUM(AstcRecompression, Uncompressed, Bc1, Bc3);
ENUM(FramePacingMode, Target_Auto, Target_30, Target_60, Target_90, Target_120); ENUM(FramePacingMode, Target_Auto, Target_30, Target_60, Target_90, Target_120);
+17
View File
@@ -1267,6 +1267,23 @@ if (ARCHITECTURE_x86_64 OR ARCHITECTURE_arm64 OR ARCHITECTURE_riscv64 OR ARCHITE
target_link_libraries(core PRIVATE dynarmic::dynarmic) target_link_libraries(core PRIVATE dynarmic::dynarmic)
endif() endif()
target_sources(core PRIVATE hle/service/ssl/ssl_backend_openssl.cpp)
target_link_libraries(core PRIVATE OpenSSL::SSL OpenSSL::Crypto) target_link_libraries(core PRIVATE OpenSSL::SSL OpenSSL::Crypto)
# TODO
# elseif (APPLE)
# target_sources(core PRIVATE
# hle/service/ssl/ssl_backend_securetransport.cpp)
# target_link_libraries(core PRIVATE "-framework Security")
# elseif (WIN32)
# target_sources(core PRIVATE
# hle/service/ssl/ssl_backend_schannel.cpp)
# target_link_libraries(core PRIVATE crypt32 secur32)
# else()
# target_sources(core PRIVATE
# hle/service/ssl/ssl_backend_none.cpp)
# endif()
create_target_directory_groups(core) create_target_directory_groups(core)
@@ -286,6 +286,7 @@ void ArmDynarmic32::MakeJit(Common::PageTable* page_table) {
config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_UnfuseFMA; config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_UnfuseFMA;
config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_IgnoreStandardFPCRValue; config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_IgnoreStandardFPCRValue;
config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_InaccurateNaN; config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_InaccurateNaN;
config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_IgnoreGlobalMonitor;
break; break;
// Paranoia mode for debugging optimizations // Paranoia mode for debugging optimizations
case Settings::CpuAccuracy::Paranoid: case Settings::CpuAccuracy::Paranoid:
@@ -338,6 +338,7 @@ void ArmDynarmic64::MakeJit(Common::PageTable* page_table, std::size_t address_s
config.unsafe_optimizations = true; config.unsafe_optimizations = true;
config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_UnfuseFMA; config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_UnfuseFMA;
config.fastmem_address_space_bits = 64; config.fastmem_address_space_bits = 64;
config.optimizations |= Dynarmic::OptimizationFlag::Unsafe_IgnoreGlobalMonitor;
break; break;
// Paranoia mode for debugging optimizations // Paranoia mode for debugging optimizations
case Settings::CpuAccuracy::Paranoid: case Settings::CpuAccuracy::Paranoid:
+5 -1
View File
@@ -119,6 +119,7 @@ struct System::Impl {
is_multicore = Settings::values.use_multi_core.GetValue(); is_multicore = Settings::values.use_multi_core.GetValue();
extended_memory_layout = Settings::values.memory_layout_mode.GetValue() != Settings::MemoryLayout::Memory_4Gb; extended_memory_layout = Settings::values.memory_layout_mode.GetValue() != Settings::MemoryLayout::Memory_4Gb;
unified_memory = Settings::values.use_unified_memory.GetValue();
core_timing.SetMulticore(is_multicore); core_timing.SetMulticore(is_multicore);
core_timing.Initialize([&system]() { system.RegisterHostThread(); }); core_timing.Initialize([&system]() { system.RegisterHostThread(); });
@@ -146,7 +147,8 @@ struct System::Impl {
!device_memory.has_value() || !device_memory.has_value() ||
is_multicore != Settings::values.use_multi_core.GetValue() || is_multicore != Settings::values.use_multi_core.GetValue() ||
extended_memory_layout != (Settings::values.memory_layout_mode.GetValue() != extended_memory_layout != (Settings::values.memory_layout_mode.GetValue() !=
Settings::MemoryLayout::Memory_4Gb); Settings::MemoryLayout::Memory_4Gb) ||
unified_memory != Settings::values.use_unified_memory.GetValue();
if (!must_reinitialize) { if (!must_reinitialize) {
return; return;
@@ -157,6 +159,7 @@ struct System::Impl {
is_multicore = Settings::values.use_multi_core.GetValue(); is_multicore = Settings::values.use_multi_core.GetValue();
extended_memory_layout = extended_memory_layout =
Settings::values.memory_layout_mode.GetValue() != Settings::MemoryLayout::Memory_4Gb; Settings::values.memory_layout_mode.GetValue() != Settings::MemoryLayout::Memory_4Gb;
unified_memory = Settings::values.use_unified_memory.GetValue();
Initialize(system); Initialize(system);
} }
@@ -503,6 +506,7 @@ struct System::Impl {
std::atomic_bool is_powered_on{}; std::atomic_bool is_powered_on{};
bool is_multicore : 1 = false; bool is_multicore : 1 = false;
bool extended_memory_layout : 1 = false; bool extended_memory_layout : 1 = false;
bool unified_memory : 1 = false;
bool exit_locked : 1 = false; bool exit_locked : 1 = false;
bool exit_requested : 1 = false; bool exit_requested : 1 = false;
bool nvdec_active : 1 = false; bool nvdec_active : 1 = false;
+13 -1
View File
@@ -1,3 +1,6 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project // SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later // SPDX-License-Identifier: GPL-2.0-or-later
@@ -12,9 +15,18 @@ constexpr size_t VirtualReserveSize = 1ULL << 38;
constexpr size_t VirtualReserveSize = 1ULL << 39; constexpr size_t VirtualReserveSize = 1ULL << 39;
#endif #endif
namespace {
size_t ApplicationPoolOffset() {
using Init = Kernel::Board::Nintendo::Nx::KSystemControl::Init;
const size_t dram_size = Init::GetIntendedMemorySize();
const size_t application_pool_size = Init::GetApplicationPoolSize();
return dram_size > application_pool_size ? dram_size - application_pool_size : 0;
}
}
DeviceMemory::DeviceMemory() DeviceMemory::DeviceMemory()
: buffer{Kernel::Board::Nintendo::Nx::KSystemControl::Init::GetIntendedMemorySize(), : buffer{Kernel::Board::Nintendo::Nx::KSystemControl::Init::GetIntendedMemorySize(),
VirtualReserveSize} {} VirtualReserveSize, ApplicationPoolOffset()} {}
DeviceMemory::~DeviceMemory() = default; DeviceMemory::~DeviceMemory() = default;
+35
View File
@@ -20,6 +20,8 @@
#include "common/scratch_buffer.h" #include "common/scratch_buffer.h"
#include "common/virtual_buffer.h" #include "common/virtual_buffer.h"
struct AHardwareBuffer;
namespace Core { namespace Core {
constexpr size_t DEVICE_PAGEBITS = 12ULL; constexpr size_t DEVICE_PAGEBITS = 12ULL;
@@ -95,6 +97,34 @@ public:
ApplyOpOnPAddr(address, buffer, operation); ApplyOpOnPAddr(address, buffer, operation);
} }
u8* GetPhysicalBase() noexcept {
return reinterpret_cast<u8*>(physical_base);
}
const u8* GetPhysicalBase() const noexcept {
return reinterpret_cast<const u8*>(physical_base);
}
size_t GetPhysicalSize() const noexcept {
return physical_size;
}
std::span<AHardwareBuffer* const> GetBackingHardwareBuffers() const noexcept {
return ahb_windows;
}
size_t GetBackingHardwareBufferWindowSize() const noexcept {
return ahb_window_size;
}
size_t GetBackingHardwareBufferBase() const noexcept {
return ahb_base;
}
bool IsBackingShared() const noexcept {
return backing_is_shared;
}
PAddr GetPhysicalRawAddressFromDAddr(DAddr address) const { PAddr GetPhysicalRawAddressFromDAddr(DAddr address) const {
PAddr subbits = PAddr(address & page_mask); PAddr subbits = PAddr(address & page_mask);
auto paddr = tracked_entries[(address >> page_bits)].compressed_physical_ptr; auto paddr = tracked_entries[(address >> page_bits)].compressed_physical_ptr;
@@ -171,6 +201,11 @@ private:
std::unique_ptr<DeviceMemoryManagerAllocator<Traits>> impl; std::unique_ptr<DeviceMemoryManagerAllocator<Traits>> impl;
const uintptr_t physical_base; const uintptr_t physical_base;
const size_t physical_size;
const std::span<AHardwareBuffer* const> ahb_windows;
const size_t ahb_window_size;
const size_t ahb_base;
const bool backing_is_shared;
DeviceInterface* device_inter; DeviceInterface* device_inter;
struct TrackedEntry { struct TrackedEntry {
+5
View File
@@ -171,6 +171,11 @@ struct DeviceMemoryManagerAllocator {
template <typename Traits> template <typename Traits>
DeviceMemoryManager<Traits>::DeviceMemoryManager(const DeviceMemory& device_memory_) DeviceMemoryManager<Traits>::DeviceMemoryManager(const DeviceMemory& device_memory_)
: physical_base{uintptr_t(device_memory_.buffer.BackingBasePointer())} : physical_base{uintptr_t(device_memory_.buffer.BackingBasePointer())}
, physical_size{device_memory_.buffer.BackingSize()}
, ahb_windows{device_memory_.buffer.BackingHardwareBuffers()}
, ahb_window_size{device_memory_.buffer.BackingHardwareBufferWindowSize()}
, ahb_base{device_memory_.buffer.BackingHardwareBufferBase()}
, backing_is_shared{device_memory_.buffer.IsBackingShared()}
, device_inter{nullptr} , device_inter{nullptr}
, compressed_device_addr(1ULL << ((Settings::values.memory_layout_mode.GetValue() == Settings::MemoryLayout::Memory_4Gb ? physical_min_bits : physical_max_bits) - Memory::YUZU_PAGEBITS)) , compressed_device_addr(1ULL << ((Settings::values.memory_layout_mode.GetValue() == Settings::MemoryLayout::Memory_4Gb ? physical_min_bits : physical_max_bits) - Memory::YUZU_PAGEBITS))
, tracked_entries(device_as_size >> Memory::YUZU_PAGEBITS) , tracked_entries(device_as_size >> Memory::YUZU_PAGEBITS)
+118 -13
View File
@@ -5,33 +5,136 @@
// SPDX-License-Identifier: GPL-2.0-or-later // SPDX-License-Identifier: GPL-2.0-or-later
#include <chrono> #include <chrono>
#include <ctime> #include <mutex>
#include <string>
#include <vector>
#include "core/core.h" #include "core/core.h"
#include "core/hle/kernel/k_event.h" #include "core/hle/kernel/k_event.h"
#include "core/hle/service/cmif_serialization.h"
#include "core/hle/service/cmif_types.h"
#include "core/hle/service/ipc_helpers.h" #include "core/hle/service/ipc_helpers.h"
#include "core/hle/service/kernel_helpers.h" #include "core/hle/service/kernel_helpers.h"
#include "core/hle/service/nim/nim.h" #include "core/hle/service/nim/nim.h"
#include "core/hle/service/os/event.h"
#include "core/hle/service/server_manager.h" #include "core/hle/service/server_manager.h"
#include "core/hle/service/service.h" #include "core/hle/service/service.h"
namespace Service::NIM { namespace Service::NIM {
class IShopServiceAsync final : public ServiceFramework<IShopServiceAsync> { class IShopServiceAsync final : public ServiceFramework<IShopServiceAsync> {
public: public:
explicit IShopServiceAsync(Core::System& system_) explicit IShopServiceAsync(Core::System& system_)
: ServiceFramework{system_, "IShopServiceAsync"} { : ServiceFramework{system_, "IShopServiceAsync"},
service_context{system_, "IShopServiceAsync"} {
// clang-format off // clang-format off
static const FunctionInfo functions[] = { static const FunctionInfo functions[] = {
{0, nullptr, "Cancel"}, {0, D<&IShopServiceAsync::Cancel>, "Cancel"},
{1, nullptr, "GetSize"}, {1, D<&IShopServiceAsync::GetSize>, "GetSize"},
{2, nullptr, "Read"}, {2, D<&IShopServiceAsync::Read>, "Read"},
{3, nullptr, "GetErrorCode"}, {3, D<&IShopServiceAsync::GetErrorCode>, "GetErrorCode"},
{4, nullptr, "Request"}, {4, D<&IShopServiceAsync::Request>, "Request"},
{5, nullptr, "Prepare"}, {5, D<&IShopServiceAsync::Prepare>, "Prepare"},
}; };
// clang-format on // clang-format on
RegisterHandlers(functions); RegisterHandlers(functions);
completion_event = service_context.CreateEvent("IShopServiceAsync:Completion");
}
~IShopServiceAsync() override {
CancelImpl();
service_context.CloseEvent(completion_event);
}
Kernel::KReadableEvent* GetEvent() const {
return &completion_event->GetReadableEvent();
}
private:
KernelHelpers::ServiceContext service_context;
Kernel::KEvent* completion_event;
std::jthread worker;
std::atomic<u32> error_code{0};
std::mutex data_mutex;
std::vector<u8> download_data;
void CancelImpl() {
worker.request_stop();
if (worker.joinable()) {
worker.join();
}
}
Result Cancel() {
LOG_DEBUG(Service_NIM, "called");
CancelImpl();
R_SUCCEED();
}
Result GetSize(Out<u64> out_size) {
LOG_DEBUG(Service_NIM, "called");
std::scoped_lock lock{data_mutex};
*out_size = download_data.size();
R_SUCCEED();
}
Result Read(Out<u64> out_size, u64 offset, OutBuffer<BufferAttr_HipcAutoSelect> out_buffer) {
std::scoped_lock lock{data_mutex};
u64 actual_read = 0;
if (offset < download_data.size()) {
actual_read = std::min<u64>(out_buffer.size(), download_data.size() - offset);
std::memcpy(out_buffer.data(), download_data.data() + offset, actual_read);
}
*out_size = actual_read;
R_SUCCEED();
}
Result GetErrorCode(Out<u32> out_error_code) {
LOG_DEBUG(Service_NIM, "called");
*out_error_code = error_code.load();
R_SUCCEED();
}
Result Request() {
LOG_DEBUG(Service_NIM, "(STUBBED) called");
CancelImpl();
error_code.store(0);
completion_event->Clear(system.Kernel());
{
std::scoped_lock lock{data_mutex};
download_data.clear();
}
worker = std::jthread([this](const std::stop_token& stop_token) {
if (stop_token.stop_requested()) {
error_code.store(1);
} else {
std::scoped_lock lock{data_mutex};
// Dummy JSON response, else it fails...
const std::string dummy_response = "{}";
download_data.assign(dummy_response.begin(), dummy_response.end());
error_code.store(0);
}
completion_event->Signal(system.Kernel());
});
R_SUCCEED();
}
Result Prepare(InArray<char, BufferAttr_HipcMapAlias> in_path, InArray<char, BufferAttr_HipcMapAlias> in_post) {
LOG_DEBUG(Service_NIM, "called");
if (!in_path.empty()) {
std::string url(in_path.data(), in_path.size());
LOG_INFO(Service_NIM, "Preparing request for URL: {}", url);
}
R_SUCCEED();
} }
}; };
@@ -49,11 +152,13 @@ public:
} }
private: private:
void CreateAsyncInterface(HLERequestContext& ctx) { void CreateAsyncInterface(HLERequestContext& ctx) {LOG_DEBUG(Service_NIM, "called");
LOG_WARNING(Service_NIM, "(STUBBED) called"); auto async_interface = std::make_shared<IShopServiceAsync>(system);
IPC::ResponseBuilder rb{ctx, 2, 0, 1};
IPC::ResponseBuilder rb{ctx, 2, 1, 1};
rb.Push(ResultSuccess); rb.Push(ResultSuccess);
rb.PushIpcInterface<IShopServiceAsync>(ctx, system); rb.PushCopyObjects(ctx, async_interface->GetEvent());
rb.PushIpcInterface<IShopServiceAsync>(ctx, std::move(async_interface));
} }
}; };
+6 -1
View File
@@ -105,7 +105,12 @@ void LoopProcess(Core::System& system) {
auto server_manager = std::make_unique<ServerManager>(system); auto server_manager = std::make_unique<ServerManager>(system);
auto module = std::make_shared<Module>(system); auto module = std::make_shared<Module>(system);
server_manager->RegisterNamedService("nvdrv", std::make_shared<NVDRV>(system, module, "nvdrv")); server_manager->RegisterNamedService("nvdrv", std::make_shared<NVDRV>(system, module, "nvdrv"));
server_manager->RegisterNamedService("nvdrv:a", std::make_shared<NVDRV>(system, module, "nvdrv:a"));
const auto NvdrvInterfaceFactoryForApplets = [&, module] {
return std::make_shared<NVDRV>(system, module, "nvdrv:a");
};
server_manager->RegisterNamedService("nvdrv:a", NvdrvInterfaceFactoryForApplets);
server_manager->RegisterNamedService("nvdrv:s", std::make_shared<NVDRV>(system, module, "nvdrv:s")); server_manager->RegisterNamedService("nvdrv:s", std::make_shared<NVDRV>(system, module, "nvdrv:s"));
server_manager->RegisterNamedService("nvdrv:t", std::make_shared<NVDRV>(system, module, "nvdrv:t")); server_manager->RegisterNamedService("nvdrv:t", std::make_shared<NVDRV>(system, module, "nvdrv:t"));
server_manager->RegisterNamedService("nvmemp", std::make_shared<NVMEMP>(system)); server_manager->RegisterNamedService("nvmemp", std::make_shared<NVMEMP>(system));
+6 -1
View File
@@ -588,7 +588,12 @@ void LoopProcess(Core::System& system) {
auto server_manager = std::make_unique<ServerManager>(system); auto server_manager = std::make_unique<ServerManager>(system);
auto ro = std::make_shared<RoContext>(); auto ro = std::make_shared<RoContext>();
server_manager->RegisterNamedService("ldr:ro", std::make_shared<RoInterface>(system, "ldr:ro", ro, NrrKind::User));
const auto RoInterfaceFactoryForUser = [&, ro] {
return std::make_shared<RoInterface>(system, "ldr:ro", ro, NrrKind::User);
};
server_manager->RegisterNamedService("ldr:ro", std::move(RoInterfaceFactoryForUser));
server_manager->RegisterNamedService("ro:1", std::make_shared<RoInterface>(system, "ro:1", ro, NrrKind::JitPlugin)); server_manager->RegisterNamedService("ro:1", std::make_shared<RoInterface>(system, "ro:1", ro, NrrKind::JitPlugin));
server_manager->RegisterNamedService("ro:dmnt", std::make_shared<IDebugMonitorInterface>(system)); server_manager->RegisterNamedService("ro:dmnt", std::make_shared<IDebugMonitorInterface>(system));
ServerManager::RunServer(std::move(server_manager)); ServerManager::RunServer(std::move(server_manager));
+29 -424
View File
@@ -4,18 +4,6 @@
// SPDX-FileCopyrightText: Copyright 2018 yuzu Emulator Project // SPDX-FileCopyrightText: Copyright 2018 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later // SPDX-License-Identifier: GPL-2.0-or-later
#include <mutex>
#include <openssl/bio.h>
#include <openssl/err.h>
#include <openssl/ssl.h>
#include <openssl/x509.h>
#ifdef YUZU_BUNDLED_OPENSSL
#include <openssl/cert.h>
#endif
#include "common/fs/file.h"
#include "common/hex_util.h"
#include "common/string_util.h" #include "common/string_util.h"
#include "core/core.h" #include "core/core.h"
@@ -34,377 +22,6 @@
namespace Service::SSL { namespace Service::SSL {
namespace {
std::once_flag one_time_init_flag;
bool one_time_init_success = false;
SSL_CTX* ssl_ctx = nullptr;
BIO_METHOD* bio_meth = nullptr;
Common::FS::IOFile key_log_file; // only open if SSLKEYLOGFILE set in environment
Result CheckOpenSSLErrors();
void OneTimeInit();
void OneTimeInitLogFile();
bool OneTimeInitBIO();
#ifdef YUZU_BUNDLED_OPENSSL
// This is ported from httplib
struct scope_exit {
explicit scope_exit(std::function<void(void)> &&f)
: exit_function(std::move(f)), execute_on_destruction{true} {}
scope_exit(scope_exit &&rhs) noexcept
: exit_function(std::move(rhs.exit_function)),
execute_on_destruction{rhs.execute_on_destruction} {
rhs.release();
}
~scope_exit() {
if (execute_on_destruction) { this->exit_function(); }
}
void release() { this->execute_on_destruction = false; }
private:
scope_exit(const scope_exit &) = delete;
void operator=(const scope_exit &) = delete;
scope_exit &operator=(scope_exit &&) = delete;
std::function<void(void)> exit_function;
bool execute_on_destruction;
};
inline X509_STORE *CreateCaCertStore(const char *ca_cert,
std::size_t size) {
auto mem = BIO_new_mem_buf(ca_cert, static_cast<int>(size));
auto se = scope_exit([&] { BIO_free_all(mem); });
if (!mem) { return nullptr; }
auto inf = PEM_X509_INFO_read_bio(mem, nullptr, nullptr, nullptr);
if (!inf) { return nullptr; }
auto cts = X509_STORE_new();
if (cts) {
for (auto i = 0; i < static_cast<int>(sk_X509_INFO_num(inf)); i++) {
auto itmp = sk_X509_INFO_value(inf, i);
if (!itmp) { continue; }
if (itmp->x509) { X509_STORE_add_cert(cts, itmp->x509); }
if (itmp->crl) { X509_STORE_add_crl(cts, itmp->crl); }
}
}
sk_X509_INFO_pop_free(inf, X509_INFO_free);
return cts;
}
inline void SetCaCertStore(SSL_CTX *ctx, X509_STORE *ca_cert_store) {
if (ca_cert_store) {
if (ctx) {
if (SSL_CTX_get_cert_store(ctx) != ca_cert_store) {
// Free memory allocated for old cert and use new store `ca_cert_store`
SSL_CTX_set_cert_store(ctx, ca_cert_store);
}
} else {
X509_STORE_free(ca_cert_store);
}
}
}
inline void LoadCaCertStore(SSL_CTX* ctx, const char* ca_cert, std::size_t size)
{
SetCaCertStore(ctx, CreateCaCertStore(ca_cert, size));
}
#endif
} // namespace
class SSLConnectionBackend final {
public:
Result Init() {
// on bundled OpenSSL, load ca cert store
#ifdef YUZU_BUNDLED_OPENSSL
LoadCaCertStore(ssl_ctx, kCert, sizeof(kCert));
#endif
std::call_once(one_time_init_flag, OneTimeInit);
if (!one_time_init_success) {
LOG_ERROR(Service_SSL, "Can't create SSL connection because OpenSSL one-time initialization failed");
return ResultInternalError;
}
ssl = SSL_new(ssl_ctx);
if (!ssl) {
LOG_ERROR(Service_SSL, "SSL_new failed");
return CheckOpenSSLErrors();
}
SSL_set_connect_state(ssl);
bio = BIO_new(bio_meth);
if (!bio) {
LOG_ERROR(Service_SSL, "BIO_new failed");
return CheckOpenSSLErrors();
}
BIO_set_data(bio, this);
BIO_set_init(bio, 1);
SSL_set_bio(ssl, bio, bio);
return ResultSuccess;
}
Result SetHostName(const std::string& hostname) {
if (!skip_cert_verification) {
if (!SSL_set1_host(ssl, hostname.c_str())) {
LOG_ERROR(Service_SSL, "SSL_set1_host({}) failed", hostname);
return CheckOpenSSLErrors();
}
}
if (!SSL_set_tlsext_host_name(ssl, hostname.c_str())) { // hostname for SNI
LOG_ERROR(Service_SSL, "SSL_set_tlsext_host_name({}) failed", hostname);
return CheckOpenSSLErrors();
}
return ResultSuccess;
}
void SetVerifyOption(u32 option) {
skip_cert_verification = (option == 0);
LOG_WARNING(Service_SSL, "option={} skip_verification={}", option,
skip_cert_verification);
if (skip_cert_verification) {
SSL_set_verify(ssl, SSL_VERIFY_NONE, nullptr);
SSL_set1_host(ssl, nullptr);
SSL_set_hostflags(ssl, 0);
} else {
SSL_set_verify(ssl, SSL_VERIFY_PEER, nullptr);
}
}
Result DoHandshake() {
SSL_set_verify_result(ssl, X509_V_OK);
const int ret = SSL_do_handshake(ssl);
if (!skip_cert_verification) {
const long verify_result = SSL_get_verify_result(ssl);
if (verify_result != X509_V_OK) {
LOG_ERROR(Service_SSL, "SSL cert verification failed because: {}",
X509_verify_cert_error_string(verify_result));
return CheckOpenSSLErrors();
}
}
if (ret <= 0) {
const int ssl_err = SSL_get_error(ssl, ret);
if (ssl_err == SSL_ERROR_ZERO_RETURN ||
(ssl_err == SSL_ERROR_SYSCALL && got_read_eof)) {
LOG_ERROR(Service_SSL, "SSL handshake failed because server hung up");
return ResultInternalError;
}
}
return HandleReturn("SSL_do_handshake", 0, ret);
}
Result HandleReturn(const char* what, size_t* actual, int ret) {
const int ssl_err = SSL_get_error(ssl, ret);
CheckOpenSSLErrors();
switch (ssl_err) {
case SSL_ERROR_NONE:
return ResultSuccess;
case SSL_ERROR_ZERO_RETURN:
LOG_DEBUG(Service_SSL, "{} => SSL_ERROR_ZERO_RETURN", what);
// DoHandshake special-cases this, but for Read and Write:
*actual = 0;
return ResultSuccess;
case SSL_ERROR_WANT_READ:
LOG_DEBUG(Service_SSL, "{} => SSL_ERROR_WANT_READ", what);
return ResultWouldBlock;
case SSL_ERROR_WANT_WRITE:
LOG_DEBUG(Service_SSL, "{} => SSL_ERROR_WANT_WRITE", what);
return ResultWouldBlock;
default:
if (ssl_err == SSL_ERROR_SYSCALL && got_read_eof) {
LOG_DEBUG(Service_SSL, "{} => SSL_ERROR_SYSCALL because server hung up", what);
*actual = 0;
return ResultSuccess;
}
LOG_ERROR(Service_SSL, "{} => other SSL_get_error return value {}", what, ssl_err);
return ResultInternalError;
}
}
~SSLConnectionBackend() {
// this is null-tolerant:
SSL_free(ssl);
}
static void KeyLogCallback(const ::SSL* ssl, const char* line) {
std::string str(line);
str.push_back('\n');
// Do this in a single WriteString for atomicity if multiple instances
// are running on different threads (though that can't currently
// happen).
if (key_log_file.WriteString(str) != str.size() || !key_log_file.Flush()) {
LOG_CRITICAL(Service_SSL, "Failed to write to SSLKEYLOGFILE");
}
LOG_DEBUG(Service_SSL, "Wrote to SSLKEYLOGFILE: {}", line);
}
static int WriteCallback(BIO* bio, const char* buf, size_t len, size_t* actual_p) {
auto self = static_cast<SSLConnectionBackend*>(BIO_get_data(bio));
ASSERT_OR_EXECUTE_MSG(
self->socket, { return 0; }, "OpenSSL asked to send but we have no socket");
BIO_clear_retry_flags(bio);
auto [actual, err] = self->socket->Send({reinterpret_cast<const u8*>(buf), len}, 0);
switch (err) {
case Network::Errno::SUCCESS:
*actual_p = actual;
return 1;
case Network::Errno::AGAIN:
BIO_set_flags(bio, BIO_FLAGS_WRITE | BIO_FLAGS_SHOULD_RETRY);
return 0;
default:
LOG_ERROR(Service_SSL, "Socket send returned Network::Errno {}", err);
return -1;
}
}
static int ReadCallback(BIO* bio, char* buf, size_t len, size_t* actual_p) {
auto self = static_cast<SSLConnectionBackend*>(BIO_get_data(bio));
ASSERT_OR_EXECUTE_MSG(
self->socket, { return 0; }, "OpenSSL asked to recv but we have no socket");
BIO_clear_retry_flags(bio);
auto [actual, err] = self->socket->Recv(0, {reinterpret_cast<u8*>(buf), len});
switch (err) {
case Network::Errno::SUCCESS:
*actual_p = actual;
if (actual == 0) {
self->got_read_eof = true;
}
return actual ? 1 : 0;
case Network::Errno::AGAIN:
BIO_set_flags(bio, BIO_FLAGS_READ | BIO_FLAGS_SHOULD_RETRY);
return 0;
default:
LOG_ERROR(Service_SSL, "Socket recv returned Network::Errno {}", err);
return -1;
}
}
static long CtrlCallback(BIO* bio, int cmd, long l_arg, void* p_arg) {
switch (cmd) {
case BIO_CTRL_FLUSH:
// Nothing to flush.
return 1;
case BIO_CTRL_PUSH:
case BIO_CTRL_POP:
#ifdef BIO_CTRL_GET_KTLS_SEND
case BIO_CTRL_GET_KTLS_SEND:
case BIO_CTRL_GET_KTLS_RECV:
#endif
// We don't support these operations, but don't bother logging them
// as they're nothing unusual.
return 0;
default:
LOG_DEBUG(Service_SSL, "OpenSSL BIO got ctrl({}, {}, {})", cmd, l_arg, p_arg);
return 0;
}
}
::SSL* ssl = nullptr;
BIO* bio = nullptr;
bool got_read_eof = false;
bool skip_cert_verification = false;
std::shared_ptr<Network::SocketBase> socket;
};
namespace {
Result CreateSSLConnectionBackend(std::unique_ptr<SSLConnectionBackend>* out_backend) {
auto conn = std::make_unique<SSLConnectionBackend>();
R_TRY(conn->Init());
*out_backend = std::move(conn);
return ResultSuccess;
}
Result CheckOpenSSLErrors() {
unsigned long rc;
const char* file;
int line;
const char* func;
const char* data;
int flags;
#if OPENSSL_VERSION_NUMBER >= 0x30000000L
while ((rc = ERR_get_error_all(&file, &line, &func, &data, &flags)))
#else
// Can't get function names from OpenSSL on this version, so use mine:
func = __func__;
while ((rc = ERR_get_error_line_data(&file, &line, &data, &flags)))
#endif
{
std::string msg;
msg.resize(1024, '\0');
ERR_error_string_n(rc, msg.data(), msg.size());
msg.resize(strlen(msg.data()), '\0');
if (flags & ERR_TXT_STRING) {
msg.append(" | ");
msg.append(data);
}
Common::Log::FmtLogMessage(Common::Log::Class::Service_SSL, Common::Log::Level::Error,
file, line, func, "OpenSSL: {}",
msg);
}
return ResultInternalError;
}
void OneTimeInit() {
ssl_ctx = SSL_CTX_new(TLS_client_method());
if (!ssl_ctx) {
LOG_ERROR(Service_SSL, "SSL_CTX_new failed");
CheckOpenSSLErrors();
return;
}
SSL_CTX_set_verify(ssl_ctx, SSL_VERIFY_PEER, nullptr);
if (!SSL_CTX_set_default_verify_paths(ssl_ctx)) {
LOG_ERROR(Service_SSL, "SSL_CTX_set_default_verify_paths failed");
CheckOpenSSLErrors();
return;
}
OneTimeInitLogFile();
if (!OneTimeInitBIO()) {
return;
}
one_time_init_success = true;
}
void OneTimeInitLogFile() {
const char* logfile = getenv("SSLKEYLOGFILE");
if (logfile) {
key_log_file.Open(logfile, Common::FS::FileAccessMode::Append, Common::FS::FileType::TextFile, Common::FS::FileShareFlag::ShareWriteOnly);
if (key_log_file.IsOpen()) {
SSL_CTX_set_keylog_callback(ssl_ctx, &SSLConnectionBackend::KeyLogCallback);
} else {
LOG_CRITICAL(Service_SSL, "SSLKEYLOGFILE was set but file could not be opened; not logging keys!");
}
}
}
bool OneTimeInitBIO() {
bio_meth =
BIO_meth_new(BIO_get_new_index() | BIO_TYPE_SOURCE_SINK, "SSLConnectionBackend");
if (!bio_meth ||
!BIO_meth_set_write_ex(bio_meth, &SSLConnectionBackend::WriteCallback) ||
!BIO_meth_set_read_ex(bio_meth, &SSLConnectionBackend::ReadCallback) ||
!BIO_meth_set_ctrl(bio_meth, &SSLConnectionBackend::CtrlCallback)) {
LOG_ERROR(Service_SSL, "Failed to create BIO_METHOD");
return false;
}
return true;
}
} // namespace
// This is nn::ssl::sf::CertificateFormat // This is nn::ssl::sf::CertificateFormat
enum class CertificateFormat : u32 { enum class CertificateFormat : u32 {
Pem = 1, Pem = 1,
@@ -545,17 +162,20 @@ private:
auto const res_v = bsd->DuplicateSocketImpl(fd); auto const res_v = bsd->DuplicateSocketImpl(fd);
if (auto *res = std::get_if<s32>(&res_v)) { if (auto *res = std::get_if<s32>(&res_v)) {
const s32 dup_fd = *res; const s32 duplicated_fd = *res;
*out_fd = do_not_close_socket ? dup_fd : -1; if (do_not_close_socket) {
if (!do_not_close_socket) *out_fd = duplicated_fd;
fd_to_close = dup_fd; } else {
auto const sock = bsd->GetSocket(dup_fd); *out_fd = -1;
fd_to_close = duplicated_fd;
}
std::optional<std::shared_ptr<Network::SocketBase>> sock = bsd->GetSocket(duplicated_fd);
if (!sock.has_value()) { if (!sock.has_value()) {
LOG_ERROR(Service_SSL, "invalid socket fd {} after duplication", dup_fd); LOG_ERROR(Service_SSL, "invalid socket fd {} after duplication", duplicated_fd);
return ResultInvalidSocket; return ResultInvalidSocket;
} }
socket = std::move(*sock); socket = std::move(*sock);
backend->socket = std::move(socket); backend->SetSocket(socket);
return ResultSuccess; return ResultSuccess;
} }
LOG_ERROR(Service_SSL, "Failed to duplicate socket with fd {}", fd); LOG_ERROR(Service_SSL, "Failed to duplicate socket with fd {}", fd);
@@ -569,11 +189,11 @@ private:
} }
Result SetVerifyOptionImpl(u32 option) { Result SetVerifyOptionImpl(u32 option) {
LOG_DEBUG(Service_SSL, "called. option={} (forcing 0)", option);
ASSERT(!did_handshake); ASSERT(!did_handshake);
LOG_DEBUG(Service_SSL, "called. option={} (forcing 0)", option);
verify_option = 0; verify_option = 0;
backend->SetVerifyOption(0); backend->SetVerifyOption(0);
R_SUCCEED(); return ResultSuccess;
} }
Result SetIoModeImpl(u32 input_mode) { Result SetIoModeImpl(u32 input_mode) {
@@ -586,13 +206,13 @@ private:
if (error != Network::Errno::SUCCESS) { if (error != Network::Errno::SUCCESS) {
LOG_ERROR(Service_SSL, "Failed to set native socket non-block flag to {}", non_block); LOG_ERROR(Service_SSL, "Failed to set native socket non-block flag to {}", non_block);
} }
R_SUCCEED(); return ResultSuccess;
} }
Result SetSessionCacheModeImpl(u32 mode) { Result SetSessionCacheModeImpl(u32 mode) {
ASSERT(!did_handshake); ASSERT(!did_handshake);
LOG_WARNING(Service_SSL, "(STUBBED) called. value={}", mode); LOG_WARNING(Service_SSL, "(STUBBED) called. value={}", mode);
R_SUCCEED(); return ResultSuccess;
} }
Result DoHandshakeImpl() { Result DoHandshakeImpl() {
@@ -614,17 +234,19 @@ private:
}; };
if (!get_server_cert_chain) { if (!get_server_cert_chain) {
// Just return the first one, unencoded. // Just return the first one, unencoded.
ASSERT_OR_EXECUTE_MSG(!certs.empty(), { return {}; }, "Should be at least one server cert"); ASSERT_OR_EXECUTE_MSG(
!certs.empty(), { return {}; }, "Should be at least one server cert");
return certs[0]; return certs[0];
} }
std::vector<u8> ret; std::vector<u8> ret;
Header header{0x4E4D684374726543, u32(certs.size()), 0}; Header header{0x4E4D684374726543, static_cast<u32>(certs.size()), 0};
ret.insert(ret.end(), reinterpret_cast<u8*>(&header), reinterpret_cast<u8*>(&header + 1)); ret.insert(ret.end(), reinterpret_cast<u8*>(&header), reinterpret_cast<u8*>(&header + 1));
size_t data_offset = sizeof(Header) + certs.size() * sizeof(EntryHeader); size_t data_offset = sizeof(Header) + certs.size() * sizeof(EntryHeader);
for (auto& cert : certs) { for (auto& cert : certs) {
EntryHeader entry_header{u32(cert.size()), u32(data_offset)}; EntryHeader entry_header{static_cast<u32>(cert.size()), static_cast<u32>(data_offset)};
data_offset += cert.size(); data_offset += cert.size();
ret.insert(ret.end(), reinterpret_cast<u8*>(&entry_header), reinterpret_cast<u8*>(&entry_header + 1)); ret.insert(ret.end(), reinterpret_cast<u8*>(&entry_header),
reinterpret_cast<u8*>(&entry_header + 1));
} }
for (auto& cert : certs) { for (auto& cert : certs) {
ret.insert(ret.end(), cert.begin(), cert.end()); ret.insert(ret.end(), cert.begin(), cert.end());
@@ -635,8 +257,7 @@ private:
Result ReadImpl(std::vector<u8>* out_data) { Result ReadImpl(std::vector<u8>* out_data) {
ASSERT_OR_EXECUTE(did_handshake, { return ResultInternalError; }); ASSERT_OR_EXECUTE(did_handshake, { return ResultInternalError; });
size_t actual_size{}; size_t actual_size{};
const int ret = SSL_read_ex(backend->ssl, out_data->data(), out_data->size(), &actual_size); Result res = backend->Read(&actual_size, *out_data);
Result res = backend->HandleReturn("SSL_read_ex", &actual_size, ret);
if (res != ResultSuccess) { if (res != ResultSuccess) {
return res; return res;
} }
@@ -646,13 +267,12 @@ private:
Result WriteImpl(size_t* out_size, std::span<const u8> data) { Result WriteImpl(size_t* out_size, std::span<const u8> data) {
ASSERT_OR_EXECUTE(did_handshake, { return ResultInternalError; }); ASSERT_OR_EXECUTE(did_handshake, { return ResultInternalError; });
const int ret = SSL_write_ex(backend->ssl, data.data(), data.size(), out_size); return backend->Write(out_size, data);
return backend->HandleReturn("SSL_write_ex", out_size, ret);
} }
Result PendingImpl(s32* out_pending) { Result PendingImpl(s32* out_pending) {
LOG_WARNING(Service_SSL, "(STUBBED) called."); LOG_WARNING(Service_SSL, "(STUBBED) called.");
*out_pending = SSL_pending(backend->ssl); *out_pending = 0;
return ResultSuccess; return ResultSuccess;
} }
@@ -706,39 +326,24 @@ private:
OutputParameters out{}; OutputParameters out{};
if (res == ResultSuccess) { if (res == ResultSuccess) {
std::vector<std::vector<u8>> certs; std::vector<std::vector<u8>> certs;
STACK_OF(X509)* chain = SSL_get_peer_cert_chain(backend->ssl); res = backend->GetServerCerts(&certs);
if (chain) { if (res == ResultSuccess) {
int count = sk_X509_num(chain);
ASSERT(count >= 0);
for (int i = 0; i < count; i++) {
X509* x509 = sk_X509_value(chain, i);
ASSERT_OR_EXECUTE(x509 != nullptr, { continue; });
unsigned char* buf = nullptr;
int len = i2d_X509(x509, &buf);
ASSERT_OR_EXECUTE(len >= 0 && buf, { continue; });
certs.emplace_back(buf, buf + len);
OPENSSL_free(buf);
}
// succeed!
const std::vector<u8> certs_buf = SerializeServerCerts(certs); const std::vector<u8> certs_buf = SerializeServerCerts(certs);
if (ctx.CanWriteBuffer()) { if (ctx.CanWriteBuffer()) {
const size_t buffer_size = ctx.GetWriteBufferSize(); const size_t buffer_size = ctx.GetWriteBufferSize();
if (certs_buf.size() <= buffer_size) { if (certs_buf.size() <= buffer_size) {
ctx.WriteBuffer(certs_buf); ctx.WriteBuffer(certs_buf);
} else { } else {
LOG_WARNING(Service_SSL, "Certificate buffer too small: {} bytes needed, {} bytes available", certs_buf.size(), buffer_size); LOG_WARNING(Service_SSL, "Certificate buffer too small: {} bytes needed, {} bytes available",
certs_buf.size(), buffer_size);
ctx.WriteBuffer(std::span<const u8>(certs_buf.data(), buffer_size)); ctx.WriteBuffer(std::span<const u8>(certs_buf.data(), buffer_size));
} }
} else { } else {
LOG_DEBUG(Service_SSL, "No output buffer provided for certificates ({} bytes)", certs_buf.size()); LOG_DEBUG(Service_SSL, "No output buffer provided for certificates ({} bytes)", certs_buf.size());
} }
out.certs_count = u32(certs.size()); out.certs_count = static_cast<u32>(certs.size());
out.certs_size = u32(certs_buf.size()); out.certs_size = static_cast<u32>(certs_buf.size());
} else {
LOG_ERROR(Service_SSL, "SSL_get_peer_cert_chain returned nullptr");
res = ResultInternalError;
} }
} }
IPC::ResponseBuilder rb{ctx, 4}; IPC::ResponseBuilder rb{ctx, 4};
+14
View File
@@ -32,4 +32,18 @@ constexpr Result ResultInternalError{ErrorModule::SSLSrv, 999}; // made up
// polling for read (with a timeout). // polling for read (with a timeout).
constexpr Result ResultWouldBlock{ErrorModule::SSLSrv, 204}; constexpr Result ResultWouldBlock{ErrorModule::SSLSrv, 204};
class SSLConnectionBackend {
public:
virtual ~SSLConnectionBackend() {}
virtual void SetSocket(std::shared_ptr<Network::SocketBase> socket) = 0;
virtual Result SetHostName(const std::string& hostname) = 0;
virtual void SetVerifyOption(u32 option) = 0;
virtual Result DoHandshake() = 0;
virtual Result Read(size_t* out_size, std::span<u8> data) = 0;
virtual Result Write(size_t* out_size, std::span<const u8> data) = 0;
virtual Result GetServerCerts(std::vector<std::vector<u8>>* out_certs) = 0;
};
Result CreateSSLConnectionBackend(std::unique_ptr<SSLConnectionBackend>* out_backend);
} // namespace Service::SSL } // namespace Service::SSL
@@ -0,0 +1,19 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "common/logging.h"
#include "core/hle/service/ssl/ssl_backend.h"
namespace Service::SSL {
Result CreateSSLConnectionBackend(std::unique_ptr<SSLConnectionBackend>* out_backend) {
LOG_ERROR(Service_SSL,
"Can't create SSL connection because no SSL backend is available on this platform");
return ResultInternalError;
}
} // namespace Service::SSL
@@ -0,0 +1,450 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <mutex>
#include <openssl/bio.h>
#include <openssl/err.h>
#include <openssl/ssl.h>
#include <openssl/x509.h>
#include "common/fs/file.h"
#include "common/hex_util.h"
#include "common/string_util.h"
#include "core/hle/service/ssl/ssl_backend.h"
#include "core/internal_network/network.h"
#include "core/internal_network/sockets.h"
#ifdef YUZU_BUNDLED_OPENSSL
#include <openssl/cert.h>
#endif
using namespace Common::FS;
namespace Service::SSL {
// Import OpenSSL's `SSL` type into the namespace. This is needed because the
// namespace is also named `SSL`.
using ::SSL;
namespace {
std::once_flag one_time_init_flag;
bool one_time_init_success = false;
SSL_CTX* ssl_ctx;
IOFile key_log_file; // only open if SSLKEYLOGFILE set in environment
BIO_METHOD* bio_meth;
Result CheckOpenSSLErrors();
void OneTimeInit();
void OneTimeInitLogFile();
bool OneTimeInitBIO();
#ifdef YUZU_BUNDLED_OPENSSL
// This is ported from httplib
struct scope_exit {
explicit scope_exit(std::function<void(void)> &&f)
: exit_function(std::move(f)), execute_on_destruction{true} {}
scope_exit(scope_exit &&rhs) noexcept
: exit_function(std::move(rhs.exit_function)),
execute_on_destruction{rhs.execute_on_destruction} {
rhs.release();
}
~scope_exit() {
if (execute_on_destruction) { this->exit_function(); }
}
void release() { this->execute_on_destruction = false; }
private:
scope_exit(const scope_exit &) = delete;
void operator=(const scope_exit &) = delete;
scope_exit &operator=(scope_exit &&) = delete;
std::function<void(void)> exit_function;
bool execute_on_destruction;
};
inline X509_STORE *CreateCaCertStore(const char *ca_cert,
std::size_t size) {
auto mem = BIO_new_mem_buf(ca_cert, static_cast<int>(size));
auto se = scope_exit([&] { BIO_free_all(mem); });
if (!mem) { return nullptr; }
auto inf = PEM_X509_INFO_read_bio(mem, nullptr, nullptr, nullptr);
if (!inf) { return nullptr; }
auto cts = X509_STORE_new();
if (cts) {
for (auto i = 0; i < static_cast<int>(sk_X509_INFO_num(inf)); i++) {
auto itmp = sk_X509_INFO_value(inf, i);
if (!itmp) { continue; }
if (itmp->x509) { X509_STORE_add_cert(cts, itmp->x509); }
if (itmp->crl) { X509_STORE_add_crl(cts, itmp->crl); }
}
}
sk_X509_INFO_pop_free(inf, X509_INFO_free);
return cts;
}
inline void SetCaCertStore(SSL_CTX *ctx, X509_STORE *ca_cert_store) {
if (ca_cert_store) {
if (ctx) {
if (SSL_CTX_get_cert_store(ctx) != ca_cert_store) {
// Free memory allocated for old cert and use new store `ca_cert_store`
SSL_CTX_set_cert_store(ctx, ca_cert_store);
}
} else {
X509_STORE_free(ca_cert_store);
}
}
}
inline void LoadCaCertStore(SSL_CTX* ctx, const char* ca_cert, std::size_t size)
{
SetCaCertStore(ctx, CreateCaCertStore(ca_cert, size));
}
#endif
} // namespace
class SSLConnectionBackendOpenSSL final : public SSLConnectionBackend {
public:
Result Init() {
// on bundled OpenSSL, load ca cert store
#ifdef YUZU_BUNDLED_OPENSSL
LoadCaCertStore(ssl_ctx, kCert, sizeof(kCert));
#endif
std::call_once(one_time_init_flag, OneTimeInit);
if (!one_time_init_success) {
LOG_ERROR(Service_SSL,
"Can't create SSL connection because OpenSSL one-time initialization failed");
return ResultInternalError;
}
ssl = SSL_new(ssl_ctx);
if (!ssl) {
LOG_ERROR(Service_SSL, "SSL_new failed");
return CheckOpenSSLErrors();
}
SSL_set_connect_state(ssl);
bio = BIO_new(bio_meth);
if (!bio) {
LOG_ERROR(Service_SSL, "BIO_new failed");
return CheckOpenSSLErrors();
}
BIO_set_data(bio, this);
BIO_set_init(bio, 1);
SSL_set_bio(ssl, bio, bio);
return ResultSuccess;
}
void SetSocket(std::shared_ptr<Network::SocketBase> socket_in) override {
socket = std::move(socket_in);
}
Result SetHostName(const std::string& hostname) override {
if (!skip_cert_verification) {
if (!SSL_set1_host(ssl, hostname.c_str())) {
LOG_ERROR(Service_SSL, "SSL_set1_host({}) failed", hostname);
return CheckOpenSSLErrors();
}
}
if (!SSL_set_tlsext_host_name(ssl, hostname.c_str())) { // hostname for SNI
LOG_ERROR(Service_SSL, "SSL_set_tlsext_host_name({}) failed", hostname);
return CheckOpenSSLErrors();
}
return ResultSuccess;
}
void SetVerifyOption(u32 option) override {
skip_cert_verification = (option == 0);
LOG_WARNING(Service_SSL, "option={} skip_verification={}", option,
skip_cert_verification);
if (skip_cert_verification) {
SSL_set_verify(ssl, SSL_VERIFY_NONE, nullptr);
SSL_set1_host(ssl, nullptr);
SSL_set_hostflags(ssl, 0);
} else {
SSL_set_verify(ssl, SSL_VERIFY_PEER, nullptr);
}
}
Result DoHandshake() override {
SSL_set_verify_result(ssl, X509_V_OK);
const int ret = SSL_do_handshake(ssl);
if (!skip_cert_verification) {
const long verify_result = SSL_get_verify_result(ssl);
if (verify_result != X509_V_OK) {
LOG_ERROR(Service_SSL, "SSL cert verification failed because: {}",
X509_verify_cert_error_string(verify_result));
return CheckOpenSSLErrors();
}
}
if (ret <= 0) {
const int ssl_err = SSL_get_error(ssl, ret);
if (ssl_err == SSL_ERROR_ZERO_RETURN ||
(ssl_err == SSL_ERROR_SYSCALL && got_read_eof)) {
LOG_ERROR(Service_SSL, "SSL handshake failed because server hung up");
return ResultInternalError;
}
}
return HandleReturn("SSL_do_handshake", 0, ret);
}
Result Read(size_t* out_size, std::span<u8> data) override {
const int ret = SSL_read_ex(ssl, data.data(), data.size(), out_size);
return HandleReturn("SSL_read_ex", out_size, ret);
}
Result Write(size_t* out_size, std::span<const u8> data) override {
const int ret = SSL_write_ex(ssl, data.data(), data.size(), out_size);
return HandleReturn("SSL_write_ex", out_size, ret);
}
Result HandleReturn(const char* what, size_t* actual, int ret) {
const int ssl_err = SSL_get_error(ssl, ret);
CheckOpenSSLErrors();
switch (ssl_err) {
case SSL_ERROR_NONE:
return ResultSuccess;
case SSL_ERROR_ZERO_RETURN:
LOG_DEBUG(Service_SSL, "{} => SSL_ERROR_ZERO_RETURN", what);
// DoHandshake special-cases this, but for Read and Write:
*actual = 0;
return ResultSuccess;
case SSL_ERROR_WANT_READ:
LOG_DEBUG(Service_SSL, "{} => SSL_ERROR_WANT_READ", what);
return ResultWouldBlock;
case SSL_ERROR_WANT_WRITE:
LOG_DEBUG(Service_SSL, "{} => SSL_ERROR_WANT_WRITE", what);
return ResultWouldBlock;
default:
if (ssl_err == SSL_ERROR_SYSCALL && got_read_eof) {
LOG_DEBUG(Service_SSL, "{} => SSL_ERROR_SYSCALL because server hung up", what);
*actual = 0;
return ResultSuccess;
}
LOG_ERROR(Service_SSL, "{} => other SSL_get_error return value {}", what, ssl_err);
return ResultInternalError;
}
}
Result GetServerCerts(std::vector<std::vector<u8>>* out_certs) override {
STACK_OF(X509)* chain = SSL_get_peer_cert_chain(ssl);
if (!chain) {
LOG_ERROR(Service_SSL, "SSL_get_peer_cert_chain returned nullptr");
return ResultInternalError;
}
int count = sk_X509_num(chain);
ASSERT(count >= 0);
for (int i = 0; i < count; i++) {
X509* x509 = sk_X509_value(chain, i);
ASSERT_OR_EXECUTE(x509 != nullptr, { continue; });
unsigned char* buf = nullptr;
int len = i2d_X509(x509, &buf);
ASSERT_OR_EXECUTE(len >= 0 && buf, { continue; });
out_certs->emplace_back(buf, buf + len);
OPENSSL_free(buf);
}
return ResultSuccess;
}
~SSLConnectionBackendOpenSSL() {
// this is null-tolerant:
SSL_free(ssl);
}
static void KeyLogCallback(const SSL* ssl, const char* line) {
std::string str(line);
str.push_back('\n');
// Do this in a single WriteString for atomicity if multiple instances
// are running on different threads (though that can't currently
// happen).
if (key_log_file.WriteString(str) != str.size() || !key_log_file.Flush()) {
LOG_CRITICAL(Service_SSL, "Failed to write to SSLKEYLOGFILE");
}
LOG_DEBUG(Service_SSL, "Wrote to SSLKEYLOGFILE: {}", line);
}
static int WriteCallback(BIO* bio, const char* buf, size_t len, size_t* actual_p) {
auto self = static_cast<SSLConnectionBackendOpenSSL*>(BIO_get_data(bio));
ASSERT_OR_EXECUTE_MSG(
self->socket, { return 0; }, "OpenSSL asked to send but we have no socket");
BIO_clear_retry_flags(bio);
auto [actual, err] = self->socket->Send({reinterpret_cast<const u8*>(buf), len}, 0);
switch (err) {
case Network::Errno::SUCCESS:
*actual_p = actual;
return 1;
case Network::Errno::AGAIN:
BIO_set_flags(bio, BIO_FLAGS_WRITE | BIO_FLAGS_SHOULD_RETRY);
return 0;
default:
LOG_ERROR(Service_SSL, "Socket send returned Network::Errno {}", err);
return -1;
}
}
static int ReadCallback(BIO* bio, char* buf, size_t len, size_t* actual_p) {
auto self = static_cast<SSLConnectionBackendOpenSSL*>(BIO_get_data(bio));
ASSERT_OR_EXECUTE_MSG(
self->socket, { return 0; }, "OpenSSL asked to recv but we have no socket");
BIO_clear_retry_flags(bio);
auto [actual, err] = self->socket->Recv(0, {reinterpret_cast<u8*>(buf), len});
switch (err) {
case Network::Errno::SUCCESS:
*actual_p = actual;
if (actual == 0) {
self->got_read_eof = true;
}
return actual ? 1 : 0;
case Network::Errno::AGAIN:
BIO_set_flags(bio, BIO_FLAGS_READ | BIO_FLAGS_SHOULD_RETRY);
return 0;
default:
LOG_ERROR(Service_SSL, "Socket recv returned Network::Errno {}", err);
return -1;
}
}
static long CtrlCallback(BIO* bio, int cmd, long l_arg, void* p_arg) {
switch (cmd) {
case BIO_CTRL_FLUSH:
// Nothing to flush.
return 1;
case BIO_CTRL_PUSH:
case BIO_CTRL_POP:
#ifdef BIO_CTRL_GET_KTLS_SEND
case BIO_CTRL_GET_KTLS_SEND:
case BIO_CTRL_GET_KTLS_RECV:
#endif
// We don't support these operations, but don't bother logging them
// as they're nothing unusual.
return 0;
default:
LOG_DEBUG(Service_SSL, "OpenSSL BIO got ctrl({}, {}, {})", cmd, l_arg, p_arg);
return 0;
}
}
SSL* ssl = nullptr;
BIO* bio = nullptr;
bool got_read_eof = false;
bool skip_cert_verification = false;
std::shared_ptr<Network::SocketBase> socket;
};
Result CreateSSLConnectionBackend(std::unique_ptr<SSLConnectionBackend>* out_backend) {
auto conn = std::make_unique<SSLConnectionBackendOpenSSL>();
R_TRY(conn->Init());
*out_backend = std::move(conn);
return ResultSuccess;
}
namespace {
Result CheckOpenSSLErrors() {
unsigned long rc;
const char* file;
int line;
const char* func;
const char* data;
int flags;
#if OPENSSL_VERSION_NUMBER >= 0x30000000L
while ((rc = ERR_get_error_all(&file, &line, &func, &data, &flags)))
#else
// Can't get function names from OpenSSL on this version, so use mine:
func = __func__;
while ((rc = ERR_get_error_line_data(&file, &line, &data, &flags)))
#endif
{
std::string msg;
msg.resize(1024, '\0');
ERR_error_string_n(rc, msg.data(), msg.size());
msg.resize(strlen(msg.data()), '\0');
if (flags & ERR_TXT_STRING) {
msg.append(" | ");
msg.append(data);
}
Common::Log::FmtLogMessage(Common::Log::Class::Service_SSL, Common::Log::Level::Error,
file, line, func, "OpenSSL: {}",
msg);
}
return ResultInternalError;
}
void OneTimeInit() {
ssl_ctx = SSL_CTX_new(TLS_client_method());
if (!ssl_ctx) {
LOG_ERROR(Service_SSL, "SSL_CTX_new failed");
CheckOpenSSLErrors();
return;
}
SSL_CTX_set_verify(ssl_ctx, SSL_VERIFY_PEER, nullptr);
if (!SSL_CTX_set_default_verify_paths(ssl_ctx)) {
LOG_ERROR(Service_SSL, "SSL_CTX_set_default_verify_paths failed");
CheckOpenSSLErrors();
return;
}
OneTimeInitLogFile();
if (!OneTimeInitBIO()) {
return;
}
one_time_init_success = true;
}
void OneTimeInitLogFile() {
const char* logfile = getenv("SSLKEYLOGFILE");
if (logfile) {
key_log_file.Open(logfile, FileAccessMode::Append, FileType::TextFile,
FileShareFlag::ShareWriteOnly);
if (key_log_file.IsOpen()) {
SSL_CTX_set_keylog_callback(ssl_ctx, &SSLConnectionBackendOpenSSL::KeyLogCallback);
} else {
LOG_CRITICAL(Service_SSL,
"SSLKEYLOGFILE was set but file could not be opened; not logging keys!");
}
}
}
bool OneTimeInitBIO() {
bio_meth =
BIO_meth_new(BIO_get_new_index() | BIO_TYPE_SOURCE_SINK, "SSLConnectionBackendOpenSSL");
if (!bio_meth ||
!BIO_meth_set_write_ex(bio_meth, &SSLConnectionBackendOpenSSL::WriteCallback) ||
!BIO_meth_set_read_ex(bio_meth, &SSLConnectionBackendOpenSSL::ReadCallback) ||
!BIO_meth_set_ctrl(bio_meth, &SSLConnectionBackendOpenSSL::CtrlCallback)) {
LOG_ERROR(Service_SSL, "Failed to create BIO_METHOD");
return false;
}
return true;
}
} // namespace
} // namespace Service::SSL
@@ -0,0 +1,563 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <mutex>
#include "common/error.h"
#include "common/fs/file.h"
#include "common/hex_util.h"
#include "common/string_util.h"
#include "core/hle/service/ssl/ssl_backend.h"
#include "core/internal_network/network.h"
#include "core/internal_network/sockets.h"
namespace {
// These includes are inside the namespace to avoid a conflict on MinGW where
// the headers define an enum containing Network and Service as enumerators
// (which clash with the correspondingly named namespaces).
#define SECURITY_WIN32
#include <schnlsp.h>
#include <security.h>
#include <wincrypt.h>
std::once_flag one_time_init_flag;
bool one_time_init_success = false;
SCHANNEL_CRED schannel_cred{};
CredHandle cred_handle;
static void OneTimeInit() {
schannel_cred.dwVersion = SCHANNEL_CRED_VERSION;
schannel_cred.dwFlags =
SCH_USE_STRONG_CRYPTO | // don't allow insecure protocols
SCH_CRED_NO_SERVERNAME_CHECK | // don't validate server names
SCH_CRED_NO_DEFAULT_CREDS; // don't automatically present a client certificate
// ^ I'm assuming that nobody would want to connect Yuzu to a
// service that requires some OS-provided corporate client
// certificate, and presenting one to some arbitrary server
// might be a privacy concern? Who knows, though.
const SECURITY_STATUS ret =
AcquireCredentialsHandle(nullptr, const_cast<LPTSTR>(UNISP_NAME), SECPKG_CRED_OUTBOUND,
nullptr, &schannel_cred, nullptr, nullptr, &cred_handle, nullptr);
if (ret != SEC_E_OK) {
// SECURITY_STATUS codes are a type of HRESULT and can be used with NativeErrorToString.
LOG_ERROR(Service_SSL, "AcquireCredentialsHandle failed: {}",
Common::NativeErrorToString(ret));
return;
}
if (getenv("SSLKEYLOGFILE")) {
LOG_CRITICAL(Service_SSL, "SSLKEYLOGFILE was set but Schannel does not support exporting "
"keys; not logging keys!");
// Not fatal.
}
one_time_init_success = true;
}
} // namespace
namespace Service::SSL {
class SSLConnectionBackendSchannel final : public SSLConnectionBackend {
public:
Result Init() {
std::call_once(one_time_init_flag, OneTimeInit);
if (!one_time_init_success) {
LOG_ERROR(
Service_SSL,
"Can't create SSL connection because Schannel one-time initialization failed");
return ResultInternalError;
}
return ResultSuccess;
}
void SetSocket(std::shared_ptr<Network::SocketBase> socket_in) override {
socket = std::move(socket_in);
}
Result SetHostName(const std::string& hostname_in) override {
hostname = hostname_in;
return ResultSuccess;
}
void SetVerifyOption(u32 option) override {
skip_cert_verification = (option == 0);
LOG_WARNING(Service_SSL, "option={} skip_verification={}", option,
skip_cert_verification);
}
Result DoHandshake() override {
while (1) {
Result r;
switch (handshake_state) {
case HandshakeState::Initial:
if ((r = FlushCiphertextWriteBuf()) != ResultSuccess ||
(r = CallInitializeSecurityContext()) != ResultSuccess) {
return r;
}
// CallInitializeSecurityContext updated `handshake_state`.
continue;
case HandshakeState::ContinueNeeded:
case HandshakeState::IncompleteMessage:
if ((r = FlushCiphertextWriteBuf()) != ResultSuccess ||
(r = FillCiphertextReadBuf()) != ResultSuccess) {
return r;
}
if (ciphertext_read_buf.empty()) {
LOG_ERROR(Service_SSL, "SSL handshake failed because server hung up");
return ResultInternalError;
}
if ((r = CallInitializeSecurityContext()) != ResultSuccess) {
return r;
}
// CallInitializeSecurityContext updated `handshake_state`.
continue;
case HandshakeState::DoneAfterFlush:
if ((r = FlushCiphertextWriteBuf()) != ResultSuccess) {
return r;
}
handshake_state = HandshakeState::Connected;
return ResultSuccess;
case HandshakeState::Connected:
LOG_ERROR(Service_SSL, "Called DoHandshake but we already handshook");
return ResultInternalError;
case HandshakeState::Error:
return ResultInternalError;
}
}
}
Result FillCiphertextReadBuf() {
const size_t fill_size = read_buf_fill_size ? read_buf_fill_size : 4096;
read_buf_fill_size = 0;
// This unnecessarily zeroes the buffer; oh well.
const size_t offset = ciphertext_read_buf.size();
ASSERT_OR_EXECUTE(offset + fill_size >= offset, { return ResultInternalError; });
ciphertext_read_buf.resize(offset + fill_size, 0);
const auto read_span = std::span(ciphertext_read_buf).subspan(offset, fill_size);
const auto [actual, err] = socket->Recv(0, read_span);
switch (err) {
case Network::Errno::SUCCESS:
ASSERT(static_cast<size_t>(actual) <= fill_size);
ciphertext_read_buf.resize(offset + actual);
return ResultSuccess;
case Network::Errno::AGAIN:
ciphertext_read_buf.resize(offset);
return ResultWouldBlock;
default:
ciphertext_read_buf.resize(offset);
LOG_ERROR(Service_SSL, "Socket recv returned Network::Errno {}", err);
return ResultInternalError;
}
}
// Returns success if the write buffer has been completely emptied.
Result FlushCiphertextWriteBuf() {
while (!ciphertext_write_buf.empty()) {
const auto [actual, err] = socket->Send(ciphertext_write_buf, 0);
switch (err) {
case Network::Errno::SUCCESS:
ASSERT(static_cast<size_t>(actual) <= ciphertext_write_buf.size());
ciphertext_write_buf.erase(ciphertext_write_buf.begin(),
ciphertext_write_buf.begin() + actual);
break;
case Network::Errno::AGAIN:
return ResultWouldBlock;
default:
LOG_ERROR(Service_SSL, "Socket send returned Network::Errno {}", err);
return ResultInternalError;
}
}
return ResultSuccess;
}
Result CallInitializeSecurityContext() {
unsigned long req = ISC_REQ_ALLOCATE_MEMORY | ISC_REQ_CONFIDENTIALITY |
ISC_REQ_INTEGRITY | ISC_REQ_REPLAY_DETECT |
ISC_REQ_SEQUENCE_DETECT | ISC_REQ_STREAM |
ISC_REQ_USE_SUPPLIED_CREDS;
if (skip_cert_verification) {
req |= ISC_REQ_MANUAL_CRED_VALIDATION;
}
unsigned long attr;
// https://learn.microsoft.com/en-us/windows/win32/secauthn/initializesecuritycontext--schannel
std::array<SecBuffer, 2> input_buffers{{
// only used if `initial_call_done`
{
// [0]
.cbBuffer = static_cast<unsigned long>(ciphertext_read_buf.size()),
.BufferType = SECBUFFER_TOKEN,
.pvBuffer = ciphertext_read_buf.data(),
},
{
// [1] (will be replaced by SECBUFFER_MISSING when SEC_E_INCOMPLETE_MESSAGE is
// returned, or SECBUFFER_EXTRA when SEC_E_CONTINUE_NEEDED is returned if the
// whole buffer wasn't used)
.cbBuffer = 0,
.BufferType = SECBUFFER_EMPTY,
.pvBuffer = nullptr,
},
}};
std::array<SecBuffer, 2> output_buffers{{
{
.cbBuffer = 0,
.BufferType = SECBUFFER_TOKEN,
.pvBuffer = nullptr,
}, // [0]
{
.cbBuffer = 0,
.BufferType = SECBUFFER_ALERT,
.pvBuffer = nullptr,
}, // [1]
}};
SecBufferDesc input_desc{
.ulVersion = SECBUFFER_VERSION,
.cBuffers = static_cast<unsigned long>(input_buffers.size()),
.pBuffers = input_buffers.data(),
};
SecBufferDesc output_desc{
.ulVersion = SECBUFFER_VERSION,
.cBuffers = static_cast<unsigned long>(output_buffers.size()),
.pBuffers = output_buffers.data(),
};
ASSERT_OR_EXECUTE_MSG(
input_buffers[0].cbBuffer == ciphertext_read_buf.size(),
{ return ResultInternalError; }, "read buffer too large");
bool initial_call_done = handshake_state != HandshakeState::Initial;
if (initial_call_done) {
LOG_DEBUG(Service_SSL, "Passing {} bytes into InitializeSecurityContext",
ciphertext_read_buf.size());
}
char* hostname_ptr = hostname ? const_cast<char*>(hostname->c_str()) : nullptr;
const SECURITY_STATUS ret = InitializeSecurityContextA(
&cred_handle, initial_call_done ? &ctxt : nullptr, hostname_ptr, req,
0, // Reserved1
0, // TargetDataRep not used with Schannel
initial_call_done ? &input_desc : nullptr,
0, // Reserved2
initial_call_done ? nullptr : &ctxt, &output_desc, &attr,
nullptr); // ptsExpiry
if (output_buffers[0].pvBuffer) {
const std::span span(static_cast<u8*>(output_buffers[0].pvBuffer),
output_buffers[0].cbBuffer);
ciphertext_write_buf.insert(ciphertext_write_buf.end(), span.begin(), span.end());
FreeContextBuffer(output_buffers[0].pvBuffer);
}
if (output_buffers[1].pvBuffer) {
const std::span span(static_cast<u8*>(output_buffers[1].pvBuffer),
output_buffers[1].cbBuffer);
// The documentation doesn't explain what format this data is in.
LOG_DEBUG(Service_SSL, "Got a {}-byte alert buffer: {}", span.size(),
Common::HexToString(span));
}
switch (ret) {
case SEC_I_CONTINUE_NEEDED:
LOG_DEBUG(Service_SSL, "InitializeSecurityContext => SEC_I_CONTINUE_NEEDED");
if (input_buffers[1].BufferType == SECBUFFER_EXTRA) {
LOG_DEBUG(Service_SSL, "EXTRA of size {}", input_buffers[1].cbBuffer);
ASSERT(input_buffers[1].cbBuffer <= ciphertext_read_buf.size());
ciphertext_read_buf.erase(ciphertext_read_buf.begin(),
ciphertext_read_buf.end() - input_buffers[1].cbBuffer);
} else {
ASSERT(input_buffers[1].BufferType == SECBUFFER_EMPTY);
ciphertext_read_buf.clear();
}
handshake_state = HandshakeState::ContinueNeeded;
return ResultSuccess;
case SEC_E_INCOMPLETE_MESSAGE:
LOG_DEBUG(Service_SSL, "InitializeSecurityContext => SEC_E_INCOMPLETE_MESSAGE");
ASSERT(input_buffers[1].BufferType == SECBUFFER_MISSING);
read_buf_fill_size = input_buffers[1].cbBuffer;
handshake_state = HandshakeState::IncompleteMessage;
return ResultSuccess;
case SEC_E_OK:
LOG_DEBUG(Service_SSL, "InitializeSecurityContext => SEC_E_OK");
ciphertext_read_buf.clear();
handshake_state = HandshakeState::DoneAfterFlush;
return GrabStreamSizes();
default:
LOG_ERROR(Service_SSL,
"InitializeSecurityContext failed (probably certificate/protocol issue): {}",
Common::NativeErrorToString(ret));
handshake_state = HandshakeState::Error;
return ResultInternalError;
}
}
Result GrabStreamSizes() {
const SECURITY_STATUS ret =
QueryContextAttributes(&ctxt, SECPKG_ATTR_STREAM_SIZES, &stream_sizes);
if (ret != SEC_E_OK) {
LOG_ERROR(Service_SSL, "QueryContextAttributes(SECPKG_ATTR_STREAM_SIZES) failed: {}",
Common::NativeErrorToString(ret));
handshake_state = HandshakeState::Error;
return ResultInternalError;
}
return ResultSuccess;
}
Result Read(size_t* out_size, std::span<u8> data) override {
*out_size = 0;
if (handshake_state != HandshakeState::Connected) {
LOG_ERROR(Service_SSL, "Called Read but we did not successfully handshake");
return ResultInternalError;
}
if (data.size() == 0 || got_read_eof) {
return ResultSuccess;
}
while (1) {
if (!cleartext_read_buf.empty()) {
*out_size = (std::min)(cleartext_read_buf.size(), data.size());
std::memcpy(data.data(), cleartext_read_buf.data(), *out_size);
cleartext_read_buf.erase(cleartext_read_buf.begin(),
cleartext_read_buf.begin() + *out_size);
return ResultSuccess;
}
if (!ciphertext_read_buf.empty()) {
SecBuffer empty{
.cbBuffer = 0,
.BufferType = SECBUFFER_EMPTY,
.pvBuffer = nullptr,
};
std::array<SecBuffer, 5> buffers{{
{
.cbBuffer = static_cast<unsigned long>(ciphertext_read_buf.size()),
.BufferType = SECBUFFER_DATA,
.pvBuffer = ciphertext_read_buf.data(),
},
empty,
empty,
empty,
}};
ASSERT_OR_EXECUTE_MSG(
buffers[0].cbBuffer == ciphertext_read_buf.size(),
{ return ResultInternalError; }, "read buffer too large");
SecBufferDesc desc{
.ulVersion = SECBUFFER_VERSION,
.cBuffers = static_cast<unsigned long>(buffers.size()),
.pBuffers = buffers.data(),
};
SECURITY_STATUS ret =
DecryptMessage(&ctxt, &desc, /*MessageSeqNo*/ 0, /*pfQOP*/ nullptr);
switch (ret) {
case SEC_E_OK:
ASSERT_OR_EXECUTE(buffers[0].BufferType == SECBUFFER_STREAM_HEADER,
{ return ResultInternalError; });
ASSERT_OR_EXECUTE(buffers[1].BufferType == SECBUFFER_DATA,
{ return ResultInternalError; });
ASSERT_OR_EXECUTE(buffers[2].BufferType == SECBUFFER_STREAM_TRAILER,
{ return ResultInternalError; });
cleartext_read_buf.assign(static_cast<u8*>(buffers[1].pvBuffer),
static_cast<u8*>(buffers[1].pvBuffer) +
buffers[1].cbBuffer);
if (buffers[3].BufferType == SECBUFFER_EXTRA) {
ASSERT(buffers[3].cbBuffer <= ciphertext_read_buf.size());
ciphertext_read_buf.erase(ciphertext_read_buf.begin(),
ciphertext_read_buf.end() - buffers[3].cbBuffer);
} else {
ASSERT(buffers[3].BufferType == SECBUFFER_EMPTY);
ciphertext_read_buf.clear();
}
continue;
case SEC_E_INCOMPLETE_MESSAGE:
break;
case SEC_I_CONTEXT_EXPIRED:
// Server hung up by sending close_notify.
got_read_eof = true;
*out_size = 0;
return ResultSuccess;
default:
LOG_ERROR(Service_SSL, "DecryptMessage failed: {}",
Common::NativeErrorToString(ret));
return ResultInternalError;
}
}
const Result r = FillCiphertextReadBuf();
if (r != ResultSuccess) {
return r;
}
if (ciphertext_read_buf.empty()) {
got_read_eof = true;
*out_size = 0;
return ResultSuccess;
}
}
}
Result Write(size_t* out_size, std::span<const u8> data) override {
*out_size = 0;
if (handshake_state != HandshakeState::Connected) {
LOG_ERROR(Service_SSL, "Called Write but we did not successfully handshake");
return ResultInternalError;
}
if (data.size() == 0) {
return ResultSuccess;
}
data = data.subspan(0, std::min<size_t>(data.size(), stream_sizes.cbMaximumMessage));
if (!cleartext_write_buf.empty()) {
// Already in the middle of a write. It wouldn't make sense to not
// finish sending the entire buffer since TLS has
// header/MAC/padding/etc.
if (data.size() != cleartext_write_buf.size() ||
std::memcmp(data.data(), cleartext_write_buf.data(), data.size())) {
LOG_ERROR(Service_SSL, "Called Write but buffer does not match previous buffer");
return ResultInternalError;
}
return WriteAlreadyEncryptedData(out_size);
} else {
cleartext_write_buf.assign(data.begin(), data.end());
}
std::vector<u8> header_buf(stream_sizes.cbHeader, 0);
std::vector<u8> tmp_data_buf = cleartext_write_buf;
std::vector<u8> trailer_buf(stream_sizes.cbTrailer, 0);
std::array<SecBuffer, 3> buffers{{
{
.cbBuffer = stream_sizes.cbHeader,
.BufferType = SECBUFFER_STREAM_HEADER,
.pvBuffer = header_buf.data(),
},
{
.cbBuffer = static_cast<unsigned long>(tmp_data_buf.size()),
.BufferType = SECBUFFER_DATA,
.pvBuffer = tmp_data_buf.data(),
},
{
.cbBuffer = stream_sizes.cbTrailer,
.BufferType = SECBUFFER_STREAM_TRAILER,
.pvBuffer = trailer_buf.data(),
},
}};
ASSERT_OR_EXECUTE_MSG(
buffers[1].cbBuffer == tmp_data_buf.size(), { return ResultInternalError; },
"temp buffer too large");
SecBufferDesc desc{
.ulVersion = SECBUFFER_VERSION,
.cBuffers = static_cast<unsigned long>(buffers.size()),
.pBuffers = buffers.data(),
};
const SECURITY_STATUS ret = EncryptMessage(&ctxt, /*fQOP*/ 0, &desc, /*MessageSeqNo*/ 0);
if (ret != SEC_E_OK) {
LOG_ERROR(Service_SSL, "EncryptMessage failed: {}", Common::NativeErrorToString(ret));
return ResultInternalError;
}
ciphertext_write_buf.insert(ciphertext_write_buf.end(), header_buf.begin(),
header_buf.end());
ciphertext_write_buf.insert(ciphertext_write_buf.end(), tmp_data_buf.begin(),
tmp_data_buf.end());
ciphertext_write_buf.insert(ciphertext_write_buf.end(), trailer_buf.begin(),
trailer_buf.end());
return WriteAlreadyEncryptedData(out_size);
}
Result WriteAlreadyEncryptedData(size_t* out_size) {
const Result r = FlushCiphertextWriteBuf();
if (r != ResultSuccess) {
return r;
}
// write buf is empty
*out_size = cleartext_write_buf.size();
cleartext_write_buf.clear();
return ResultSuccess;
}
Result GetServerCerts(std::vector<std::vector<u8>>* out_certs) override {
PCCERT_CONTEXT returned_cert = nullptr;
const SECURITY_STATUS ret =
QueryContextAttributes(&ctxt, SECPKG_ATTR_REMOTE_CERT_CONTEXT, &returned_cert);
if (ret != SEC_E_OK) {
LOG_ERROR(Service_SSL,
"QueryContextAttributes(SECPKG_ATTR_REMOTE_CERT_CONTEXT) failed: {}",
Common::NativeErrorToString(ret));
return ResultInternalError;
}
PCCERT_CONTEXT some_cert = nullptr;
while ((some_cert = CertEnumCertificatesInStore(returned_cert->hCertStore, some_cert)) !=
nullptr) {
out_certs->emplace_back(static_cast<u8*>(some_cert->pbCertEncoded),
static_cast<u8*>(some_cert->pbCertEncoded) +
some_cert->cbCertEncoded);
}
std::reverse(out_certs->begin(),
out_certs->end()); // Windows returns certs in reverse order from what we want
CertFreeCertificateContext(returned_cert);
return ResultSuccess;
}
~SSLConnectionBackendSchannel() {
if (handshake_state != HandshakeState::Initial) {
DeleteSecurityContext(&ctxt);
}
}
enum class HandshakeState {
// Haven't called anything yet.
Initial,
// `SEC_I_CONTINUE_NEEDED` was returned by
// `InitializeSecurityContext`; must finish sending data (if any) in
// the write buffer, then read at least one byte before calling
// `InitializeSecurityContext` again.
ContinueNeeded,
// `SEC_E_INCOMPLETE_MESSAGE` was returned by
// `InitializeSecurityContext`; hopefully the write buffer is empty;
// must read at least one byte before calling
// `InitializeSecurityContext` again.
IncompleteMessage,
// `SEC_E_OK` was returned by `InitializeSecurityContext`; must
// finish sending data in the write buffer before having `DoHandshake`
// report success.
DoneAfterFlush,
// We finished the above and are now connected. At this point, writing
// and reading are separate 'state machines' represented by the
// nonemptiness of the ciphertext and cleartext read and write buffers.
Connected,
// Another error was returned and we shouldn't allow initialization
// to continue.
Error,
} handshake_state = HandshakeState::Initial;
CtxtHandle ctxt;
SecPkgContext_StreamSizes stream_sizes;
std::shared_ptr<Network::SocketBase> socket;
std::optional<std::string> hostname;
std::vector<u8> ciphertext_read_buf;
std::vector<u8> ciphertext_write_buf;
std::vector<u8> cleartext_read_buf;
std::vector<u8> cleartext_write_buf;
bool got_read_eof = false;
bool skip_cert_verification = false;
size_t read_buf_fill_size = 0;
};
Result CreateSSLConnectionBackend(std::unique_ptr<SSLConnectionBackend>* out_backend) {
auto conn = std::make_unique<SSLConnectionBackendSchannel>();
R_TRY(conn->Init());
*out_backend = std::move(conn);
return ResultSuccess;
}
} // namespace Service::SSL
@@ -0,0 +1,236 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2023 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <mutex>
// SecureTransport has been deprecated in its entirety in favor of
// Network.framework, but that does not allow layering TLS on top of an
// arbitrary socket.
#if defined(__GNUC__) || defined(__clang__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
#include <Security/SecureTransport.h>
#pragma GCC diagnostic pop
#endif
#include "core/hle/service/ssl/ssl_backend.h"
#include "core/internal_network/network.h"
#include "core/internal_network/sockets.h"
namespace {
template <typename T>
struct CFReleaser {
T ptr;
YUZU_NON_COPYABLE(CFReleaser);
constexpr CFReleaser() : ptr(nullptr) {}
constexpr CFReleaser(T ptr) : ptr(ptr) {}
constexpr operator T() {
return ptr;
}
~CFReleaser() {
if (ptr) {
CFRelease(ptr);
}
}
};
std::string CFStringToString(CFStringRef cfstr) {
CFReleaser<CFDataRef> cfdata(
CFStringCreateExternalRepresentation(nullptr, cfstr, kCFStringEncodingUTF8, 0));
ASSERT_OR_EXECUTE(cfdata, { return "???"; });
return std::string(reinterpret_cast<const char*>(CFDataGetBytePtr(cfdata)),
CFDataGetLength(cfdata));
}
std::string OSStatusToString(OSStatus status) {
CFReleaser<CFStringRef> cfstr(SecCopyErrorMessageString(status, nullptr));
if (!cfstr) {
return "[unknown error]";
}
return CFStringToString(cfstr);
}
} // namespace
namespace Service::SSL {
class SSLConnectionBackendSecureTransport final : public SSLConnectionBackend {
public:
Result Init() {
static std::once_flag once_flag;
std::call_once(once_flag, []() {
if (getenv("SSLKEYLOGFILE")) {
LOG_CRITICAL(Service_SSL, "SSLKEYLOGFILE was set but SecureTransport does not "
"support exporting keys; not logging keys!");
// Not fatal.
}
});
context.ptr = SSLCreateContext(nullptr, kSSLClientSide, kSSLStreamType);
if (!context) {
LOG_ERROR(Service_SSL, "SSLCreateContext failed");
return ResultInternalError;
}
OSStatus status;
if ((status = SSLSetIOFuncs(context, ReadCallback, WriteCallback)) ||
(status = SSLSetConnection(context, this))) {
LOG_ERROR(Service_SSL, "SSLContext initialization failed: {}",
OSStatusToString(status));
return ResultInternalError;
}
return ResultSuccess;
}
void SetSocket(std::shared_ptr<Network::SocketBase> in_socket) override {
socket = std::move(in_socket);
}
Result SetHostName(const std::string& hostname) override {
OSStatus status = SSLSetPeerDomainName(context, hostname.c_str(), hostname.size());
if (status) {
LOG_ERROR(Service_SSL, "SSLSetPeerDomainName failed: {}", OSStatusToString(status));
return ResultInternalError;
}
return ResultSuccess;
}
void SetVerifyOption(u32 option) override {
skip_cert_verification = (option == 0);
LOG_WARNING(Service_SSL, "option={} skip_verification={}", option,
skip_cert_verification);
if (skip_cert_verification) {
SSLSetSessionOption(context, kSSLSessionOptionBreakOnServerAuth, true);
}
}
Result DoHandshake() override {
OSStatus status = SSLHandshake(context);
if (skip_cert_verification && status == errSSLServerAuthCompleted) {
LOG_DEBUG(Service_SSL, "Skipping certificate verification as requested");
status = SSLHandshake(context);
}
return HandleReturn("SSLHandshake", 0, status);
}
Result Read(size_t* out_size, std::span<u8> data) override {
OSStatus status = SSLRead(context, data.data(), data.size(), out_size);
return HandleReturn("SSLRead", out_size, status);
}
Result Write(size_t* out_size, std::span<const u8> data) override {
OSStatus status = SSLWrite(context, data.data(), data.size(), out_size);
return HandleReturn("SSLWrite", out_size, status);
}
Result HandleReturn(const char* what, size_t* actual, OSStatus status) {
switch (status) {
case 0:
return ResultSuccess;
case errSSLWouldBlock:
return ResultWouldBlock;
default: {
std::string reason;
if (got_read_eof) {
reason = "server hung up";
} else {
reason = OSStatusToString(status);
}
LOG_ERROR(Service_SSL, "{} failed: {}", what, reason);
return ResultInternalError;
}
}
}
Result GetServerCerts(std::vector<std::vector<u8>>* out_certs) override {
CFReleaser<SecTrustRef> trust;
OSStatus status = SSLCopyPeerTrust(context, &trust.ptr);
if (status) {
LOG_ERROR(Service_SSL, "SSLCopyPeerTrust failed: {}", OSStatusToString(status));
return ResultInternalError;
}
for (CFIndex i = 0, count = SecTrustGetCertificateCount(trust); i < count; i++) {
SecCertificateRef cert = SecTrustGetCertificateAtIndex(trust, i);
CFReleaser<CFDataRef> data(SecCertificateCopyData(cert));
ASSERT_OR_EXECUTE(data, { return ResultInternalError; });
const u8* ptr = CFDataGetBytePtr(data);
out_certs->emplace_back(ptr, ptr + CFDataGetLength(data));
}
return ResultSuccess;
}
static OSStatus ReadCallback(SSLConnectionRef connection, void* data, size_t* dataLength) {
return ReadOrWriteCallback(connection, data, dataLength, true);
}
static OSStatus WriteCallback(SSLConnectionRef connection, const void* data,
size_t* dataLength) {
return ReadOrWriteCallback(connection, const_cast<void*>(data), dataLength, false);
}
static OSStatus ReadOrWriteCallback(SSLConnectionRef connection, void* data, size_t* dataLength,
bool is_read) {
auto self =
static_cast<SSLConnectionBackendSecureTransport*>(const_cast<void*>(connection));
ASSERT_OR_EXECUTE_MSG(
self->socket, { return 0; }, "SecureTransport asked to {} but we have no socket",
is_read ? "read" : "write");
// SecureTransport callbacks (unlike OpenSSL BIO callbacks) are
// expected to read/write the full requested dataLength or return an
// error, so we have to add a loop ourselves.
size_t requested_len = *dataLength;
size_t offset = 0;
while (offset < requested_len) {
std::span cur(reinterpret_cast<u8*>(data) + offset, requested_len - offset);
auto [actual, err] = is_read ? self->socket->Recv(0, cur) : self->socket->Send(cur, 0);
LOG_CRITICAL(Service_SSL, "op={}, offset={} actual={}/{} err={}", is_read, offset,
actual, cur.size(), static_cast<s32>(err));
switch (err) {
case Network::Errno::SUCCESS:
offset += actual;
if (actual == 0) {
ASSERT(is_read);
self->got_read_eof = true;
return errSecEndOfData;
}
break;
case Network::Errno::AGAIN:
*dataLength = offset;
return errSSLWouldBlock;
default:
LOG_ERROR(Service_SSL, "Socket {} returned Network::Errno {}",
is_read ? "recv" : "send", err);
return errSecIO;
}
}
ASSERT(offset == requested_len);
return 0;
}
private:
CFReleaser<SSLContextRef> context = nullptr;
bool got_read_eof = false;
bool skip_cert_verification = false;
std::shared_ptr<Network::SocketBase> socket;
};
Result CreateSSLConnectionBackend(std::unique_ptr<SSLConnectionBackend>* out_backend) {
auto conn = std::make_unique<SSLConnectionBackendSecureTransport>();
R_TRY(conn->Init());
*out_backend = std::move(conn);
return ResultSuccess;
}
} // namespace Service::SSL
+5 -5
View File
@@ -114,13 +114,13 @@ std::vector<Network::ScanData> ScanWifiNetworks(std::chrono::milliseconds deadli
char ifname[IFNAMSIZ] = {0}; char ifname[IFNAMSIZ] = {0};
char *args[1] = {ifname}; char *args[1] = {ifname};
iw_enum_devices(sock, [](int skfd, char* ifname, char* args[], int count) -> int { iw_enum_devices(sock, [](int f_skfd, char* f_ifname, char* f_args[], int) -> int {
iwrange range; iwrange range;
int res = iw_get_range_info(skfd, ifname, &range); int res = iw_get_range_info(f_skfd, f_ifname, &range);
LOG_INFO(Network, "ifname {} returned {} on iw_get_range_info", ifname, res); LOG_INFO(Network, "ifname {} returned {} on iw_get_range_info", f_ifname, res);
if (res >= 0) { if (res >= 0) {
strncpy(args[0], ifname, IFNAMSIZ - 1); strncpy(f_args[0], f_ifname, IFNAMSIZ - 1);
args[0][IFNAMSIZ - 1] = 0; f_args[0][IFNAMSIZ - 1] = 0;
return 1; return 1;
} }
return 0; return 0;
+2 -3
View File
@@ -227,6 +227,8 @@ std::unique_ptr<TranslationMap> InitializeTranslations(QObject* parent) {
tr("Preserves GPU-modified data by reading it back before uploading.\nSome games require this to render certain effects properly.")); tr("Preserves GPU-modified data by reading it back before uploading.\nSome games require this to render certain effects properly."));
INSERT(Settings, use_asynchronous_shaders, tr("Enable asynchronous shader compilation"), INSERT(Settings, use_asynchronous_shaders, tr("Enable asynchronous shader compilation"),
tr("May reduce shader stutter.")); tr("May reduce shader stutter."));
INSERT(Settings, use_unified_memory, tr("Enable unified memory access"),
tr("Lets the GPU write buffer readbacks directly into guest memory."));
INSERT(Settings, gpu_clock, tr("GPU Clocks"), INSERT(Settings, gpu_clock, tr("GPU Clocks"),
tr("Makes the game believe GPU work finishes faster than it does, so it stops lowering " tr("Makes the game believe GPU work finishes faster than it does, so it stops lowering "
"resolution and render distance to fit the Switch's clocks.")); "resolution and render distance to fit the Switch's clocks."));
@@ -522,9 +524,6 @@ std::unique_ptr<ComboboxTranslationMap> ComboboxEnumeration(QObject* parent) {
PAIR(AnisotropyMode, X4, tr("4x")), PAIR(AnisotropyMode, X4, tr("4x")),
PAIR(AnisotropyMode, X8, tr("8x")), PAIR(AnisotropyMode, X8, tr("8x")),
PAIR(AnisotropyMode, X16, tr("16x")), PAIR(AnisotropyMode, X16, tr("16x")),
PAIR(AnisotropyMode, X32, tr("32x")),
PAIR(AnisotropyMode, X64, tr("64x")),
PAIR(AnisotropyMode, None, tr("None")),
}}); }});
translations->insert( translations->insert(
{Settings::EnumMetadata<Settings::Language>::Index(), {Settings::EnumMetadata<Settings::Language>::Index(),
@@ -335,7 +335,7 @@ void SetupDenormControl(const Profile& profile, const IR::Program& program, Emit
if (info.uses_fp32_denorms_flush && info.uses_fp32_denorms_preserve) { if (info.uses_fp32_denorms_flush && info.uses_fp32_denorms_preserve) {
LOG_DEBUG(Shader_SPIRV, "Fp32 denorm flush and preserve on the same shader"); LOG_DEBUG(Shader_SPIRV, "Fp32 denorm flush and preserve on the same shader");
} else if (info.uses_fp32_denorms_flush) { } else if (info.uses_fp32_denorms_flush) {
if (profile.support_fp32_denorm_flush) { if (profile.support_fp32_denorm_flush && !profile.has_broken_fp32_denorm_flush) {
ctx.AddCapability(spv::Capability::DenormFlushToZero); ctx.AddCapability(spv::Capability::DenormFlushToZero);
ctx.AddExecutionMode(main_func, spv::ExecutionMode::DenormFlushToZero, 32U); ctx.AddExecutionMode(main_func, spv::ExecutionMode::DenormFlushToZero, 32U);
} else { } else {
@@ -13,6 +13,7 @@
namespace Shader::Backend::SPIRV { namespace Shader::Backend::SPIRV {
namespace { namespace {
Id SharedPointer(EmitContext& ctx, Id offset, u32 index_offset = 0) { Id SharedPointer(EmitContext& ctx, Id offset, u32 index_offset = 0) {
offset = ctx.BoundSharedOffset(offset, 4 + index_offset * 4);
const Id shift_id{ctx.Const(2U)}; const Id shift_id{ctx.Const(2U)};
Id index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift_id)}; Id index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift_id)};
if (index_offset > 0) { if (index_offset > 0) {
@@ -160,7 +161,8 @@ Id EmitSharedAtomicExchange32(EmitContext& ctx, Id offset, Id value) {
Id EmitSharedAtomicExchange64(EmitContext& ctx, Id offset, Id value) { Id EmitSharedAtomicExchange64(EmitContext& ctx, Id offset, Id value) {
if (ctx.profile.support_shared_int64_atomics && ctx.uses_explicit_workgroup_layout) { if (ctx.profile.support_shared_int64_atomics && ctx.uses_explicit_workgroup_layout) {
const Id shift_id{ctx.Const(3U)}; const Id shift_id{ctx.Const(3U)};
const Id index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift_id)}; const Id index{
ctx.OpShiftRightArithmetic(ctx.U32[1], ctx.BoundSharedOffset(offset, 8), shift_id)};
const Id pointer{ const Id pointer{
ctx.OpAccessChain(ctx.shared_u64, ctx.shared_memory_u64, ctx.u32_zero_value, index)}; ctx.OpAccessChain(ctx.shared_u64, ctx.shared_memory_u64, ctx.u32_zero_value, index)};
const auto [scope, semantics]{AtomicArgs(ctx)}; const auto [scope, semantics]{AtomicArgs(ctx)};
@@ -31,6 +31,7 @@ std::pair<Id, Id> ExtractArgs(EmitContext& ctx, Id offset, u32 mask, u32 count)
} // Anonymous namespace } // Anonymous namespace
Id EmitLoadSharedU8(EmitContext& ctx, Id offset) { Id EmitLoadSharedU8(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 1);
if (ctx.uses_explicit_workgroup_layout) { if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{ const Id pointer{
ctx.OpAccessChain(ctx.shared_u8, ctx.shared_memory_u8, ctx.u32_zero_value, offset)}; ctx.OpAccessChain(ctx.shared_u8, ctx.shared_memory_u8, ctx.u32_zero_value, offset)};
@@ -42,6 +43,7 @@ Id EmitLoadSharedU8(EmitContext& ctx, Id offset) {
} }
Id EmitLoadSharedS8(EmitContext& ctx, Id offset) { Id EmitLoadSharedS8(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 1);
if (ctx.uses_explicit_workgroup_layout) { if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{ const Id pointer{
ctx.OpAccessChain(ctx.shared_u8, ctx.shared_memory_u8, ctx.u32_zero_value, offset)}; ctx.OpAccessChain(ctx.shared_u8, ctx.shared_memory_u8, ctx.u32_zero_value, offset)};
@@ -53,6 +55,7 @@ Id EmitLoadSharedS8(EmitContext& ctx, Id offset) {
} }
Id EmitLoadSharedU16(EmitContext& ctx, Id offset) { Id EmitLoadSharedU16(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 2);
if (ctx.uses_explicit_workgroup_layout) { if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u16, ctx.shared_memory_u16, offset, 1)}; const Id pointer{Pointer(ctx, ctx.shared_u16, ctx.shared_memory_u16, offset, 1)};
return ctx.OpUConvert(ctx.U32[1], ctx.OpLoad(ctx.U16, pointer)); return ctx.OpUConvert(ctx.U32[1], ctx.OpLoad(ctx.U16, pointer));
@@ -63,6 +66,7 @@ Id EmitLoadSharedU16(EmitContext& ctx, Id offset) {
} }
Id EmitLoadSharedS16(EmitContext& ctx, Id offset) { Id EmitLoadSharedS16(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 2);
if (ctx.uses_explicit_workgroup_layout) { if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u16, ctx.shared_memory_u16, offset, 1)}; const Id pointer{Pointer(ctx, ctx.shared_u16, ctx.shared_memory_u16, offset, 1)};
return ctx.OpSConvert(ctx.U32[1], ctx.OpLoad(ctx.U16, pointer)); return ctx.OpSConvert(ctx.U32[1], ctx.OpLoad(ctx.U16, pointer));
@@ -73,6 +77,7 @@ Id EmitLoadSharedS16(EmitContext& ctx, Id offset) {
} }
Id EmitLoadSharedU32(EmitContext& ctx, Id offset) { Id EmitLoadSharedU32(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 4);
if (ctx.uses_explicit_workgroup_layout) { if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32, ctx.shared_memory_u32, offset, 2)}; const Id pointer{Pointer(ctx, ctx.shared_u32, ctx.shared_memory_u32, offset, 2)};
return ctx.OpLoad(ctx.U32[1], pointer); return ctx.OpLoad(ctx.U32[1], pointer);
@@ -82,6 +87,7 @@ Id EmitLoadSharedU32(EmitContext& ctx, Id offset) {
} }
Id EmitLoadSharedU64(EmitContext& ctx, Id offset) { Id EmitLoadSharedU64(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 8);
if (ctx.uses_explicit_workgroup_layout) { if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32x2, ctx.shared_memory_u32x2, offset, 3)}; const Id pointer{Pointer(ctx, ctx.shared_u32x2, ctx.shared_memory_u32x2, offset, 3)};
return ctx.OpLoad(ctx.U32[2], pointer); return ctx.OpLoad(ctx.U32[2], pointer);
@@ -97,6 +103,7 @@ Id EmitLoadSharedU64(EmitContext& ctx, Id offset) {
} }
Id EmitLoadSharedU128(EmitContext& ctx, Id offset) { Id EmitLoadSharedU128(EmitContext& ctx, Id offset) {
offset = ctx.BoundSharedOffset(offset, 16);
if (ctx.uses_explicit_workgroup_layout) { if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32x4, ctx.shared_memory_u32x4, offset, 4)}; const Id pointer{Pointer(ctx, ctx.shared_u32x4, ctx.shared_memory_u32x4, offset, 4)};
return ctx.OpLoad(ctx.U32[4], pointer); return ctx.OpLoad(ctx.U32[4], pointer);
@@ -113,6 +120,7 @@ Id EmitLoadSharedU128(EmitContext& ctx, Id offset) {
} }
void EmitWriteSharedU8(EmitContext& ctx, Id offset, Id value) { void EmitWriteSharedU8(EmitContext& ctx, Id offset, Id value) {
offset = ctx.BoundSharedOffset(offset, 1);
if (ctx.uses_explicit_workgroup_layout) { if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{ const Id pointer{
ctx.OpAccessChain(ctx.shared_u8, ctx.shared_memory_u8, ctx.u32_zero_value, offset)}; ctx.OpAccessChain(ctx.shared_u8, ctx.shared_memory_u8, ctx.u32_zero_value, offset)};
@@ -123,6 +131,7 @@ void EmitWriteSharedU8(EmitContext& ctx, Id offset, Id value) {
} }
void EmitWriteSharedU16(EmitContext& ctx, Id offset, Id value) { void EmitWriteSharedU16(EmitContext& ctx, Id offset, Id value) {
offset = ctx.BoundSharedOffset(offset, 2);
if (ctx.uses_explicit_workgroup_layout) { if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u16, ctx.shared_memory_u16, offset, 1)}; const Id pointer{Pointer(ctx, ctx.shared_u16, ctx.shared_memory_u16, offset, 1)};
ctx.OpStore(pointer, ctx.OpUConvert(ctx.U16, value)); ctx.OpStore(pointer, ctx.OpUConvert(ctx.U16, value));
@@ -132,6 +141,7 @@ void EmitWriteSharedU16(EmitContext& ctx, Id offset, Id value) {
} }
void EmitWriteSharedU32(EmitContext& ctx, Id offset, Id value) { void EmitWriteSharedU32(EmitContext& ctx, Id offset, Id value) {
offset = ctx.BoundSharedOffset(offset, 4);
Id pointer{}; Id pointer{};
if (ctx.uses_explicit_workgroup_layout) { if (ctx.uses_explicit_workgroup_layout) {
pointer = Pointer(ctx, ctx.shared_u32, ctx.shared_memory_u32, offset, 2); pointer = Pointer(ctx, ctx.shared_u32, ctx.shared_memory_u32, offset, 2);
@@ -144,6 +154,7 @@ void EmitWriteSharedU32(EmitContext& ctx, Id offset, Id value) {
} }
void EmitWriteSharedU64(EmitContext& ctx, Id offset, Id value) { void EmitWriteSharedU64(EmitContext& ctx, Id offset, Id value) {
offset = ctx.BoundSharedOffset(offset, 8);
if (ctx.uses_explicit_workgroup_layout) { if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32x2, ctx.shared_memory_u32x2, offset, 3)}; const Id pointer{Pointer(ctx, ctx.shared_u32x2, ctx.shared_memory_u32x2, offset, 3)};
ctx.OpStore(pointer, value); ctx.OpStore(pointer, value);
@@ -159,6 +170,7 @@ void EmitWriteSharedU64(EmitContext& ctx, Id offset, Id value) {
} }
void EmitWriteSharedU128(EmitContext& ctx, Id offset, Id value) { void EmitWriteSharedU128(EmitContext& ctx, Id offset, Id value) {
offset = ctx.BoundSharedOffset(offset, 16);
if (ctx.uses_explicit_workgroup_layout) { if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32x4, ctx.shared_memory_u32x4, offset, 4)}; const Id pointer{Pointer(ctx, ctx.shared_u32x4, ctx.shared_memory_u32x4, offset, 4)};
ctx.OpStore(pointer, value); ctx.OpStore(pointer, value);
@@ -600,6 +600,16 @@ void EmitContext::DefineLocalMemory(const IR::Program& program) {
} }
} }
Id EmitContext::BoundSharedOffset(Id offset, u32 access_bytes) {
if (shared_memory_declared_bytes == 0) {
return offset;
}
const u32 last_valid{shared_memory_declared_bytes > access_bytes
? shared_memory_declared_bytes - access_bytes
: 0U};
return OpUMin(U32[1], offset, Const(last_valid));
}
void EmitContext::DefineSharedMemory(const IR::Program& program) { void EmitContext::DefineSharedMemory(const IR::Program& program) {
uses_explicit_workgroup_layout = uses_explicit_workgroup_layout =
profile.support_explicit_workgroup_layout && profile.support_explicit_workgroup_layout &&
@@ -608,8 +618,15 @@ void EmitContext::DefineSharedMemory(const IR::Program& program) {
if (program.shared_memory_size == 0) { if (program.shared_memory_size == 0) {
return; return;
} }
const u32 device_limit{profile.max_shared_memory_size};
const u32 shared_memory_size{device_limit != 0 && program.shared_memory_size > device_limit
? device_limit
: program.shared_memory_size};
if (shared_memory_size != program.shared_memory_size) {
shared_memory_declared_bytes = shared_memory_size;
}
const auto make{[&](Id element_type, u32 element_size) { const auto make{[&](Id element_type, u32 element_size) {
const u32 num_elements{Common::DivCeil(program.shared_memory_size, element_size)}; const u32 num_elements{Common::DivCeil(shared_memory_size, element_size)};
const Id array_type{TypeArray(element_type, Const(num_elements))}; const Id array_type{TypeArray(element_type, Const(num_elements))};
Decorate(array_type, spv::Decoration::ArrayStride, element_size); Decorate(array_type, spv::Decoration::ArrayStride, element_size);
@@ -644,7 +661,7 @@ void EmitContext::DefineSharedMemory(const IR::Program& program) {
std::tie(shared_memory_u32x4, shared_u32x4, std::ignore) = make(U32[4], 16); std::tie(shared_memory_u32x4, shared_u32x4, std::ignore) = make(U32[4], 16);
return; return;
} }
const u32 num_elements{Common::DivCeil(program.shared_memory_size, 4U)}; const u32 num_elements{Common::DivCeil(shared_memory_size, 4U)};
const Id type{TypeArray(U32[1], Const(num_elements))}; const Id type{TypeArray(U32[1], Const(num_elements))};
shared_memory_u32_type = TypePointer(spv::StorageClass::Workgroup, type); shared_memory_u32_type = TypePointer(spv::StorageClass::Workgroup, type);
@@ -312,6 +312,8 @@ public:
Id local_memory{}; Id local_memory{};
bool uses_explicit_workgroup_layout{}; bool uses_explicit_workgroup_layout{};
u32 shared_memory_declared_bytes{};
[[nodiscard]] Id BoundSharedOffset(Id offset, u32 access_bytes);
Id shared_memory_u8{}; Id shared_memory_u8{};
Id shared_memory_u16{}; Id shared_memory_u16{};
Id shared_memory_u32{}; Id shared_memory_u32{};
+5
View File
@@ -86,6 +86,8 @@ struct Profile {
bool has_broken_signed_operations{}; bool has_broken_signed_operations{};
/// Float controls break when fp16 is enabled /// Float controls break when fp16 is enabled
bool has_broken_fp16_float_controls{}; bool has_broken_fp16_float_controls{};
/// Declaring fp32 denorm flush to zero miscompiles on some drivers
bool has_broken_fp32_denorm_flush{};
/// Dynamic vec4 indexing is broken on some OpenGL drivers /// Dynamic vec4 indexing is broken on some OpenGL drivers
bool has_gl_component_indexing_bug{}; bool has_gl_component_indexing_bug{};
/// The precise type qualifier is broken in the fragment stage of some drivers /// The precise type qualifier is broken in the fragment stage of some drivers
@@ -101,6 +103,9 @@ struct Profile {
u32 gl_max_compute_smem_size{}; u32 gl_max_compute_smem_size{};
/// Largest workgroup shared memory allocation the device accepts, 0 when unconstrained
u32 max_shared_memory_size{};
/// Maxwell and earlier nVidia architectures have broken robust support /// Maxwell and earlier nVidia architectures have broken robust support
bool has_broken_robust{}; bool has_broken_robust{};
+272 -37
View File
@@ -571,7 +571,11 @@ void BufferCache<P>::AccumulateFlushes() {
template <class P> template <class P>
bool BufferCache<P>::ShouldWaitAsyncFlushes() const noexcept { bool BufferCache<P>::ShouldWaitAsyncFlushes() const noexcept {
return (!async_buffers.empty() && async_buffers.front().has_value()); if (async_buffers.empty()) {
return false;
}
return async_buffers.front().has_value() ||
!pending_downloads.front().unified_copies.empty();
} }
template <class P> template <class P>
@@ -579,6 +583,7 @@ void BufferCache<P>::CommitAsyncFlushesHigh() {
AccumulateFlushes(); AccumulateFlushes();
if (committed_gpu_modified_ranges.empty()) { if (committed_gpu_modified_ranges.empty()) {
pending_downloads.emplace_back();
async_buffers.emplace_back(std::optional<Async_Buffer>{}); async_buffers.emplace_back(std::optional<Async_Buffer>{});
return; return;
} }
@@ -638,27 +643,84 @@ void BufferCache<P>::CommitAsyncFlushesHigh() {
} }
committed_gpu_modified_ranges.clear(); committed_gpu_modified_ranges.clear();
if (downloads.empty()) { if (downloads.empty()) {
pending_downloads.emplace_back();
async_buffers.emplace_back(std::optional<Async_Buffer>{}); async_buffers.emplace_back(std::optional<Async_Buffer>{});
return; return;
} }
auto download_staging = runtime.DownloadStagingBuffer(total_size_bytes, true);
boost::container::small_vector<BufferCopy, 4> normalized_copies; struct QueuedUnifiedCopy {
runtime.PreCopyBarrier(); u64 window;
BufferId buffer_id;
boost::container::small_vector<BufferCopy, 16> copies;
};
AsyncDownloadBatch batch;
boost::container::small_vector<std::pair<BufferCopy, BufferId>, 16> staging_downloads;
boost::container::small_vector<QueuedUnifiedCopy, 4> unified_copy_queue;
boost::container::small_vector<u64, 4> window_ids;
UnifiedWindowGroups groups;
u64 staging_size_bytes = 0;
for (auto& [copy, buffer_id] : downloads) { for (auto& [copy, buffer_id] : downloads) {
copy.dst_offset += download_staging.offset;
const std::array copies{copy};
BufferCopy second_copy{copy};
Buffer& buffer = slot_buffers[buffer_id]; Buffer& buffer = slot_buffers[buffer_id];
second_copy.src_offset = static_cast<size_t>(buffer.CpuAddr()) + copy.src_offset; const DAddr orig_device_addr = buffer.CpuAddr() + copy.src_offset;
const DAddr orig_device_addr = static_cast<DAddr>(second_copy.src_offset); bool unified = false;
if constexpr (USE_UNIFIED_MEMORY) {
if (runtime.HasUnifiedMemory()) {
window_ids.clear();
groups.clear();
unified = ResolveUnifiedWindows(orig_device_addr, copy.src_offset, copy.size,
window_ids, groups);
}
}
BufferCopy record{copy};
record.src_offset = static_cast<size_t>(orig_device_addr);
if (unified) {
async_downloads.Add(orig_device_addr, copy.size);
buffer.MarkUsage(copy.src_offset, copy.size);
for (size_t i = 0; i < window_ids.size(); ++i) {
unified_copy_queue.push_back(
QueuedUnifiedCopy{window_ids[i], buffer_id, std::move(groups[i])});
}
batch.unified_copies.push_back(record);
continue;
}
copy.dst_offset = staging_size_bytes;
constexpr u64 align = 64ULL;
staging_size_bytes += (copy.size + align - 1) & ~(align - 1ULL);
staging_downloads.push_back({copy, buffer_id});
}
std::optional<Async_Buffer> download_staging;
if (!staging_downloads.empty()) {
download_staging = runtime.DownloadStagingBuffer(staging_size_bytes, true);
}
runtime.PreCopyBarrier();
for (auto& [copy, buffer_id] : staging_downloads) {
copy.dst_offset += download_staging->offset;
const std::array copies{copy};
Buffer& buffer = slot_buffers[buffer_id];
BufferCopy record{copy};
record.src_offset = static_cast<size_t>(buffer.CpuAddr()) + copy.src_offset;
const DAddr orig_device_addr = static_cast<DAddr>(record.src_offset);
async_downloads.Add(orig_device_addr, copy.size); async_downloads.Add(orig_device_addr, copy.size);
buffer.MarkUsage(copy.src_offset, copy.size); buffer.MarkUsage(copy.src_offset, copy.size);
runtime.CopyBuffer(download_staging.buffer, buffer, copies, false); runtime.CopyBuffer(download_staging->buffer, buffer, copies, false);
normalized_copies.push_back(second_copy); batch.staging_copies.push_back(record);
}
if constexpr (USE_UNIFIED_MEMORY) {
for (const auto& queued : unified_copy_queue) {
const std::span<const BufferCopy> group_span(queued.copies.data(),
queued.copies.size());
runtime.CopyToUnifiedMemory(queued.window, slot_buffers[queued.buffer_id], group_span);
}
if (!unified_copy_queue.empty()) {
runtime.FlushUnifiedMemoryCopies();
runtime.UnifiedMemoryHostBarrier();
}
} }
runtime.PostCopyBarrier(); runtime.PostCopyBarrier();
pending_downloads.emplace_back(std::move(normalized_copies)); pending_downloads.emplace_back(std::move(batch));
async_buffers.emplace_back(download_staging); async_buffers.emplace_back(std::move(download_staging));
} }
template <class P> template <class P>
@@ -673,32 +735,49 @@ void BufferCache<P>::PopAsyncFlushes() {
template <class P> template <class P>
void BufferCache<P>::PopAsyncBuffers() { void BufferCache<P>::PopAsyncBuffers() {
if (async_buffers.empty()) { struct Writeback {
return; DAddr addr;
} const u8* src;
if (!async_buffers.front().has_value()) { u64 size;
};
boost::container::small_vector<Writeback, 8> writebacks;
{
std::scoped_lock lock{mutex};
if (async_buffers.empty()) {
return;
}
auto& batch = pending_downloads.front();
auto& async_buffer = async_buffers.front();
if (async_buffer.has_value()) {
const u8* base = async_buffer->mapped_span.data();
const size_t base_offset = async_buffer->offset;
for (const auto& copy : batch.staging_copies) {
const DAddr device_addr = static_cast<DAddr>(copy.src_offset);
const u64 dst_offset = copy.dst_offset - base_offset;
const u8* read_mapped_memory = base + dst_offset;
async_downloads.ForEachInRange(
device_addr, copy.size, [&](DAddr start, DAddr end, s32) {
writebacks.push_back(
{start, &read_mapped_memory[start - device_addr], end - start});
});
async_downloads.Subtract(device_addr, copy.size, [&](DAddr start, DAddr end) {
gpu_modified_ranges.Subtract(start, end - start);
});
}
async_buffers_death_ring.emplace_back(*async_buffer);
}
for (const auto& copy : batch.unified_copies) {
const DAddr device_addr = static_cast<DAddr>(copy.src_offset);
async_downloads.Subtract(device_addr, copy.size, [&](DAddr start, DAddr end) {
gpu_modified_ranges.Subtract(start, end - start);
});
}
async_buffers.pop_front(); async_buffers.pop_front();
return; pending_downloads.pop_front();
} }
auto& downloads = pending_downloads.front(); for (const auto& wb : writebacks) {
auto& async_buffer = async_buffers.front(); device_memory.WriteBlockUnsafe(wb.addr, wb.src, wb.size);
u8* base = async_buffer->mapped_span.data();
const size_t base_offset = async_buffer->offset;
for (const auto& copy : downloads) {
const DAddr device_addr = static_cast<DAddr>(copy.src_offset);
const u64 dst_offset = copy.dst_offset - base_offset;
const u8* read_mapped_memory = base + dst_offset;
async_downloads.ForEachInRange(device_addr, copy.size, [&](DAddr start, DAddr end, s32) {
device_memory.WriteBlockUnsafe(start, &read_mapped_memory[start - device_addr],
end - start);
});
async_downloads.Subtract(device_addr, copy.size, [&](DAddr start, DAddr end) {
gpu_modified_ranges.Subtract(start, end - start);
});
} }
async_buffers_death_ring.emplace_back(*async_buffer);
async_buffers.pop_front();
pending_downloads.pop_front();
} }
template <class P> template <class P>
@@ -1006,11 +1085,25 @@ void BufferCache<P>::BindHostGraphicsStorageBuffers(size_t stage) {
Buffer& buffer = slot_buffers[binding.buffer_id]; Buffer& buffer = slot_buffers[binding.buffer_id];
TouchBuffer(buffer, binding.buffer_id); TouchBuffer(buffer, binding.buffer_id);
const u32 size = binding.size; const u32 size = binding.size;
const bool is_written = ((channel_state->written_storage_buffers[stage] >> index) & 1) != 0;
if constexpr (USE_UNIFIED_MEMORY) {
const auto window = TryResolveUnifiedRange(binding.device_addr, size);
if (window && runtime.IsUnifiedStorageRange(size, window->offset)) {
if (is_written) {
memory_tracker.MarkRegionAsCpuModified(binding.device_addr, size);
}
runtime.BindStorageBuffer(runtime.UnifiedWindowBuffer(window->window),
runtime.UnifiedWindowAddress(window->window),
static_cast<u32>(window->offset), size, is_written);
return;
}
}
SynchronizeBuffer(buffer, binding.device_addr, size); SynchronizeBuffer(buffer, binding.device_addr, size);
const u32 offset = buffer.Offset(binding.device_addr); const u32 offset = buffer.Offset(binding.device_addr);
buffer.MarkUsage(offset, size); buffer.MarkUsage(offset, size);
const bool is_written = ((channel_state->written_storage_buffers[stage] >> index) & 1) != 0;
if (is_written) { if (is_written) {
MarkWrittenBuffer(binding.buffer_id, binding.device_addr, size); MarkWrittenBuffer(binding.buffer_id, binding.device_addr, size);
@@ -1699,6 +1792,142 @@ void BufferCache<P>::ImmediateUploadMemory([[maybe_unused]] Buffer& buffer,
} }
} }
template <class P>
bool BufferCache<P>::ResolveUnifiedWindows(
[[maybe_unused]] DAddr device_addr, [[maybe_unused]] u64 buffer_offset,
[[maybe_unused]] u64 size, [[maybe_unused]] boost::container::small_vector<u64, 4>& window_ids,
[[maybe_unused]] UnifiedWindowGroups& groups) {
if constexpr (USE_UNIFIED_MEMORY) {
const u8* const physical_base = device_memory.GetPhysicalBase();
const u64 unified_base = runtime.UnifiedMemoryBase();
const u64 unified_size = runtime.UnifiedMemorySize();
const u64 window_size = runtime.UnifiedMemoryWindowSize();
if (window_size == 0) {
return false;
}
const auto group_for = [&](u64 window) -> boost::container::small_vector<BufferCopy, 16>& {
for (size_t i = 0; i < window_ids.size(); ++i) {
if (window_ids[i] == window) {
return groups[i];
}
}
window_ids.push_back(window);
groups.emplace_back();
return groups.back();
};
u64 downloaded = 0;
while (downloaded < size) {
const DAddr page_addr = device_addr + downloaded;
const u8* const ptr = device_memory.GetPointer<u8>(page_addr);
if (ptr == nullptr) {
return false;
}
const u64 page_offset = page_addr & Core::DEVICE_PAGEMASK;
u64 chunk = (std::min)(size - downloaded,
static_cast<u64>(Core::DEVICE_PAGESIZE) - page_offset);
const u64 phys_offset = static_cast<u64>(ptr - physical_base);
if (phys_offset < unified_base || phys_offset - unified_base + chunk > unified_size) {
return false;
}
const u64 relative = phys_offset - unified_base;
const u64 window = relative / window_size;
const u64 local_offset = relative % window_size;
chunk = (std::min)(chunk, window_size - local_offset);
auto& group = group_for(window);
if (!group.empty()) {
BufferCopy& last = group.back();
if (last.src_offset + last.size == buffer_offset + downloaded &&
last.dst_offset + last.size == local_offset) {
last.size += chunk;
downloaded += chunk;
continue;
}
}
group.push_back(BufferCopy{
.src_offset = buffer_offset + downloaded,
.dst_offset = local_offset,
.size = chunk,
});
downloaded += chunk;
}
return true;
} else {
return false;
}
}
template <class P>
std::optional<typename BufferCache<P>::UnifiedWindowRange>
BufferCache<P>::TryResolveUnifiedRange([[maybe_unused]] DAddr device_addr,
[[maybe_unused]] u64 size) {
if constexpr (USE_UNIFIED_MEMORY) {
if (size == 0 || !runtime.IsUnifiedMemoryBindable()) {
return std::nullopt;
}
const u64 window_size = runtime.UnifiedMemoryWindowSize();
if (window_size == 0) {
return std::nullopt;
}
const u8* const first = device_memory.GetSpan(device_addr, size);
if (first == nullptr) {
return std::nullopt;
}
const u64 phys_offset = static_cast<u64>(first - device_memory.GetPhysicalBase());
const u64 unified_base = runtime.UnifiedMemoryBase();
if (phys_offset < unified_base) {
return std::nullopt;
}
const u64 relative = phys_offset - unified_base;
const u64 unified_size = runtime.UnifiedMemorySize();
if (relative >= unified_size || unified_size - relative < size) {
return std::nullopt;
}
const u64 local_offset = relative % window_size;
if (window_size - local_offset < size) {
return std::nullopt;
}
if (memory_tracker.IsRegionGpuModified(device_addr, size) ||
IsRegionGpuModified(device_addr, size)) {
return std::nullopt;
}
return UnifiedWindowRange{
.window = static_cast<size_t>(relative / window_size),
.offset = local_offset,
};
} else {
return std::nullopt;
}
}
template <class P>
bool BufferCache<P>::TryUnifiedDownloadMemory([[maybe_unused]] Buffer& buffer,
[[maybe_unused]] std::span<BufferCopy> copies) {
if constexpr (USE_UNIFIED_MEMORY) {
boost::container::small_vector<u64, 4> window_ids;
UnifiedWindowGroups groups;
for (const BufferCopy& copy : copies) {
if (!ResolveUnifiedWindows(buffer.CpuAddr() + copy.src_offset, copy.src_offset,
copy.size, window_ids, groups)) {
return false;
}
}
for (const BufferCopy& copy : copies) {
buffer.MarkUsage(copy.src_offset, copy.size);
}
runtime.PreCopyBarrier();
for (size_t i = 0; i < window_ids.size(); ++i) {
const std::span<const BufferCopy> group_span(groups[i].data(), groups[i].size());
runtime.CopyToUnifiedMemory(window_ids[i], buffer, group_span);
}
runtime.FlushUnifiedMemoryCopies();
runtime.UnifiedMemoryHostBarrier();
runtime.Finish();
return true;
} else {
return false;
}
}
template <class P> template <class P>
void BufferCache<P>::MappedUploadMemory([[maybe_unused]] Buffer& buffer, void BufferCache<P>::MappedUploadMemory([[maybe_unused]] Buffer& buffer,
[[maybe_unused]] u64 total_size_bytes, [[maybe_unused]] u64 total_size_bytes,
@@ -1802,6 +2031,12 @@ void BufferCache<P>::DownloadBufferMemory(Buffer& buffer, DAddr device_addr, u64
} }
if constexpr (USE_MEMORY_MAPS) { if constexpr (USE_MEMORY_MAPS) {
if constexpr (USE_UNIFIED_MEMORY) {
if (runtime.HasUnifiedMemory() &&
TryUnifiedDownloadMemory(buffer, std::span(copies.data(), copies.size()))) {
return;
}
}
auto download_staging = runtime.DownloadStagingBuffer(total_size_bytes); auto download_staging = runtime.DownloadStagingBuffer(total_size_bytes);
const u8* const mapped_memory = download_staging.mapped_span.data(); const u8* const mapped_memory = download_staging.mapped_span.data();
const std::span<BufferCopy> copies_span(copies.data(), copies.data() + copies.size()); const std::span<BufferCopy> copies_span(copies.data(), copies.data() + copies.size());
@@ -12,6 +12,7 @@
#include <functional> #include <functional>
#include <memory> #include <memory>
#include <mutex> #include <mutex>
#include <optional>
#include <numeric> #include <numeric>
#include <span> #include <span>
#include <vector> #include <vector>
@@ -180,6 +181,7 @@ class BufferCache : public VideoCommon::ChannelSetupCaches<BufferCacheChannelInf
static constexpr bool USE_MEMORY_MAPS = P::USE_MEMORY_MAPS; static constexpr bool USE_MEMORY_MAPS = P::USE_MEMORY_MAPS;
static constexpr bool SEPARATE_IMAGE_BUFFERS_BINDINGS = P::SEPARATE_IMAGE_BUFFER_BINDINGS; static constexpr bool SEPARATE_IMAGE_BUFFERS_BINDINGS = P::SEPARATE_IMAGE_BUFFER_BINDINGS;
static constexpr bool USE_MEMORY_MAPS_FOR_UPLOADS = P::USE_MEMORY_MAPS_FOR_UPLOADS; static constexpr bool USE_MEMORY_MAPS_FOR_UPLOADS = P::USE_MEMORY_MAPS_FOR_UPLOADS;
static constexpr bool USE_UNIFIED_MEMORY = P::USE_UNIFIED_MEMORY;
#ifdef YUZU_LEGACY #ifdef YUZU_LEGACY
static constexpr s64 TARGET_THRESHOLD = 3_GiB; static constexpr s64 TARGET_THRESHOLD = 3_GiB;
@@ -443,6 +445,22 @@ private:
void MappedUploadMemory(Buffer& buffer, u64 total_size_bytes, std::span<BufferCopy> copies); void MappedUploadMemory(Buffer& buffer, u64 total_size_bytes, std::span<BufferCopy> copies);
bool TryUnifiedDownloadMemory(Buffer& buffer, std::span<BufferCopy> copies);
struct UnifiedWindowRange {
size_t window;
u64 offset;
};
std::optional<UnifiedWindowRange> TryResolveUnifiedRange(DAddr device_addr, u64 size);
using UnifiedWindowGroups =
boost::container::small_vector<boost::container::small_vector<BufferCopy, 16>, 4>;
bool ResolveUnifiedWindows(DAddr device_addr, u64 buffer_offset, u64 size,
boost::container::small_vector<u64, 4>& window_ids,
UnifiedWindowGroups& groups);
void DownloadBufferMemory(Buffer& buffer_id); void DownloadBufferMemory(Buffer& buffer_id);
void DownloadBufferMemory(Buffer& buffer_id, DAddr device_addr, u64 size); void DownloadBufferMemory(Buffer& buffer_id, DAddr device_addr, u64 size);
@@ -498,9 +516,14 @@ private:
std::deque<Common::RangeSet<DAddr>> committed_gpu_modified_ranges; std::deque<Common::RangeSet<DAddr>> committed_gpu_modified_ranges;
// Async Buffers // Async Buffers
struct AsyncDownloadBatch {
boost::container::small_vector<BufferCopy, 4> staging_copies;
boost::container::small_vector<BufferCopy, 4> unified_copies;
};
Common::OverlapRangeSet<DAddr> async_downloads; Common::OverlapRangeSet<DAddr> async_downloads;
std::deque<std::optional<Async_Buffer>> async_buffers; std::deque<std::optional<Async_Buffer>> async_buffers;
std::deque<boost::container::small_vector<BufferCopy, 4>> pending_downloads; std::deque<AsyncDownloadBatch> pending_downloads;
std::optional<Async_Buffer> current_buffer; std::optional<Async_Buffer> current_buffer;
std::deque<Async_Buffer> async_buffers_death_ring; std::deque<Async_Buffer> async_buffers_death_ring;
@@ -17,11 +17,14 @@ set(SHADER_FILES
${CMAKE_CURRENT_SOURCE_DIR}/astc_decoder.comp ${CMAKE_CURRENT_SOURCE_DIR}/astc_decoder.comp
${CMAKE_CURRENT_SOURCE_DIR}/blit_color_float.frag ${CMAKE_CURRENT_SOURCE_DIR}/blit_color_float.frag
${CMAKE_CURRENT_SOURCE_DIR}/block_linear_unswizzle_2d.comp ${CMAKE_CURRENT_SOURCE_DIR}/block_linear_unswizzle_2d.comp
${CMAKE_CURRENT_SOURCE_DIR}/block_linear_unswizzle_2d_buffer.comp
${CMAKE_CURRENT_SOURCE_DIR}/blit_color_msaa.frag ${CMAKE_CURRENT_SOURCE_DIR}/blit_color_msaa.frag
${CMAKE_CURRENT_SOURCE_DIR}/blit_depth.frag
${CMAKE_CURRENT_SOURCE_DIR}/blit_depth_msaa.frag ${CMAKE_CURRENT_SOURCE_DIR}/blit_depth_msaa.frag
${CMAKE_CURRENT_SOURCE_DIR}/blit_depth_stencil_msaa.frag ${CMAKE_CURRENT_SOURCE_DIR}/blit_depth_stencil_msaa.frag
${CMAKE_CURRENT_SOURCE_DIR}/block_linear_unswizzle_3d.comp ${CMAKE_CURRENT_SOURCE_DIR}/block_linear_unswizzle_3d.comp
${CMAKE_CURRENT_SOURCE_DIR}/block_linear_unswizzle_3d_bcn.comp ${CMAKE_CURRENT_SOURCE_DIR}/block_linear_unswizzle_3d_bcn.comp
${CMAKE_CURRENT_SOURCE_DIR}/block_linear_unswizzle_3d_buffer.comp
${CMAKE_CURRENT_SOURCE_DIR}/convert_abgr8_to_d24s8.frag ${CMAKE_CURRENT_SOURCE_DIR}/convert_abgr8_to_d24s8.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_abgr8_to_d32f.frag ${CMAKE_CURRENT_SOURCE_DIR}/convert_abgr8_to_d32f.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_d32f_to_abgr8.frag ${CMAKE_CURRENT_SOURCE_DIR}/convert_d32f_to_abgr8.frag
@@ -30,8 +33,12 @@ set(SHADER_FILES
${CMAKE_CURRENT_SOURCE_DIR}/convert_float_to_depth.frag ${CMAKE_CURRENT_SOURCE_DIR}/convert_float_to_depth.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_msaa_to_non_msaa.comp ${CMAKE_CURRENT_SOURCE_DIR}/convert_msaa_to_non_msaa.comp
${CMAKE_CURRENT_SOURCE_DIR}/convert_msaa_to_non_msaa.frag ${CMAKE_CURRENT_SOURCE_DIR}/convert_msaa_to_non_msaa.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_msaa_to_non_msaa_depth.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_msaa_to_non_msaa_depth_stencil.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_non_msaa_to_msaa.comp ${CMAKE_CURRENT_SOURCE_DIR}/convert_non_msaa_to_msaa.comp
${CMAKE_CURRENT_SOURCE_DIR}/convert_non_msaa_to_msaa.frag ${CMAKE_CURRENT_SOURCE_DIR}/convert_non_msaa_to_msaa.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_non_msaa_to_msaa_depth.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_non_msaa_to_msaa_depth_stencil.frag
${CMAKE_CURRENT_SOURCE_DIR}/convert_s8d24_to_abgr8.frag ${CMAKE_CURRENT_SOURCE_DIR}/convert_s8d24_to_abgr8.frag
${CMAKE_CURRENT_SOURCE_DIR}/full_screen_triangle.vert ${CMAKE_CURRENT_SOURCE_DIR}/full_screen_triangle.vert
${CMAKE_CURRENT_SOURCE_DIR}/fxaa.frag ${CMAKE_CURRENT_SOURCE_DIR}/fxaa.frag
@@ -163,6 +170,44 @@ foreach(SOURCE_FILE IN ITEMS ${SHADER_FILES})
endif() endif()
endforeach() endforeach()
# Integer variants of the MSAA conversion shaders. They only differ from the float
# source in the sampler and output types, so they are generated from it via defines.
set(SHADER_TYPE_VARIANTS
"convert_msaa_to_non_msaa.frag|sint|isampler2DMS|ivec4"
"convert_msaa_to_non_msaa.frag|uint|usampler2DMS|uvec4"
"convert_non_msaa_to_msaa.frag|sint|isampler2D|ivec4"
"convert_non_msaa_to_msaa.frag|uint|usampler2D|uvec4"
)
foreach(VARIANT IN ITEMS ${SHADER_TYPE_VARIANTS})
string(REPLACE "|" ";" VARIANT_PARTS ${VARIANT})
list(GET VARIANT_PARTS 0 VARIANT_FILENAME)
list(GET VARIANT_PARTS 1 VARIANT_SUFFIX)
list(GET VARIANT_PARTS 2 VARIANT_SAMPLER)
list(GET VARIANT_PARTS 3 VARIANT_TEXEL)
set(VARIANT_SOURCE ${CMAKE_CURRENT_SOURCE_DIR}/${VARIANT_FILENAME})
get_filename_component(VARIANT_STEM ${VARIANT_FILENAME} NAME_WE)
get_filename_component(VARIANT_EXT ${VARIANT_FILENAME} EXT)
string(REPLACE "." "" VARIANT_EXT ${VARIANT_EXT})
set(VARIANT_NAME ${VARIANT_STEM}_${VARIANT_SUFFIX}_${VARIANT_EXT})
string(TOUPPER ${VARIANT_NAME}_SPV VARIANT_VARIABLE_NAME)
set(VARIANT_HEADER_FILE ${SHADER_DIR}/${VARIANT_NAME}_spv.h)
add_custom_command(
OUTPUT
${VARIANT_HEADER_FILE}
COMMAND
${GLSLANGVALIDATOR} -V ${QUIET_FLAG} -I"${FIDELITYFX_INCLUDE_DIR}" ${GLSL_FLAGS}
-DSAMPLER_TYPE=${VARIANT_SAMPLER} -DTEXEL_TYPE=${VARIANT_TEXEL}
--variable-name ${VARIANT_VARIABLE_NAME} -o ${VARIANT_HEADER_FILE} ${VARIANT_SOURCE}
--target-env ${SPIR_V_VERSION}
MAIN_DEPENDENCY
${VARIANT_SOURCE}
)
set(SHADER_HEADERS ${SHADER_HEADERS} ${VARIANT_HEADER_FILE})
endforeach()
foreach(FILEPATH IN ITEMS ${FIDELITYFX_FILES}) foreach(FILEPATH IN ITEMS ${FIDELITYFX_FILES})
get_filename_component(FILENAME ${FILEPATH} NAME) get_filename_component(FILENAME ${FILEPATH} NAME)
string(REPLACE "." "_" HEADER_NAME ${FILENAME}) string(REPLACE "." "_" HEADER_NAME ${FILENAME})
@@ -0,0 +1,12 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 450 core
layout(binding = 0) uniform sampler2D depth_tex;
layout(location = 0) in vec2 texcoord;
void main() {
gl_FragDepth = textureLod(depth_tex, texcoord, 0).r;
}
@@ -8,5 +8,5 @@ layout(binding = 0) uniform sampler2DMS depth_tex;
layout(location = 0) in vec2 texcoord; layout(location = 0) in vec2 texcoord;
void main() { void main() {
gl_FragDepth = texelFetch(depth_tex, ivec2(texcoord), 0).r; gl_FragDepth = texelFetch(depth_tex, ivec2(texcoord), gl_SampleID).r;
} }
@@ -10,6 +10,6 @@ layout(binding = 1) uniform usampler2DMS stencil_tex;
layout(location = 0) in vec2 texcoord; layout(location = 0) in vec2 texcoord;
void main() { void main() {
gl_FragDepth = texelFetch(depth_tex, ivec2(texcoord), 0).r; gl_FragDepth = texelFetch(depth_tex, ivec2(texcoord), gl_SampleID).r;
gl_FragStencilRefARB = int(texelFetch(stencil_tex, ivec2(texcoord), 0).r); gl_FragStencilRefARB = int(texelFetch(stencil_tex, ivec2(texcoord), gl_SampleID).r);
} }
@@ -0,0 +1,104 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 430
#extension GL_EXT_shader_16bit_storage : require
#extension GL_EXT_shader_8bit_storage : require
#define BINDING_INPUT_BUFFER 0
#define BINDING_OUTPUT_BUFFER 1
layout(push_constant) uniform PushConstants {
uvec3 dim;
uint bytes_per_block_log2;
uvec3 origin;
uint layer_stride;
uint block_size;
uint x_shift;
uint block_height;
uint block_height_mask;
} pc;
layout(binding = BINDING_INPUT_BUFFER, std430) buffer InputBufferU32 { uint u32data[]; };
layout(binding = BINDING_INPUT_BUFFER, std430) buffer InputBufferU64 { uvec2 u64data[]; };
layout(binding = BINDING_INPUT_BUFFER, std430) buffer InputBufferU128 { uvec4 u128data[]; };
layout(binding = BINDING_OUTPUT_BUFFER, std430) writeonly buffer OutputBuffer {
uint out_u32[];
};
layout(local_size_x = 16, local_size_y = 8, local_size_z = 1) in;
const uint GOB_SIZE_X = 64;
const uint GOB_SIZE_Y = 8;
const uint GOB_SIZE_X_SHIFT = 6;
const uint GOB_SIZE_Y_SHIFT = 3;
const uint GOB_SIZE_SHIFT = GOB_SIZE_X_SHIFT + GOB_SIZE_Y_SHIFT;
const uvec2 SWIZZLE_MASK = uvec2(GOB_SIZE_X - 1u, GOB_SIZE_Y - 1u);
uint SwizzleTable(uint pos) {
const uint t[8] = uint[](
0x12100200, 0x13110301, 0x16140604, 0x17150705,
0x1a180a08, 0x1b190b09, 0x1e1c0e0c, 0x1f1d0f0d
);
const uint i = pos >> 4;
const uint h = (t[i / 4] >> ((i % 4) * 8)) & 0xff;
return (h << 4) | (pos & 0xf);
}
uint SwizzleOffset(uvec2 pos) {
pos = pos & SWIZZLE_MASK;
return SwizzleTable(pos.y * 64u + pos.x);
}
uvec4 ReadTexel(uint offset) {
switch (pc.bytes_per_block_log2) {
case 2u:
return uvec4(u32data[offset / 4u], 0u, 0u, 0u);
case 3u:
return uvec4(u64data[offset / 8u], 0u, 0u);
case 4u:
return u128data[offset / 16u];
}
return uvec4(0u);
}
void main() {
uvec3 coord = gl_GlobalInvocationID;
if (coord.x >= pc.dim.x || coord.y >= pc.dim.y || coord.z >= pc.dim.z) {
return;
}
uvec3 pos = coord + pc.origin;
pos.x <<= pc.bytes_per_block_log2;
uint swizzle = SwizzleOffset(pos.xy);
uint block_y = pos.y >> GOB_SIZE_Y_SHIFT;
uint offset = 0u;
offset += pos.z * pc.layer_stride;
offset += (block_y >> pc.block_height) * pc.block_size;
offset += (block_y & pc.block_height_mask) << GOB_SIZE_SHIFT;
offset += (pos.x >> GOB_SIZE_X_SHIFT) << pc.x_shift;
offset += swizzle;
uvec4 texel = ReadTexel(offset);
uint words = 1u << (pc.bytes_per_block_log2 - 2u);
uint linear_index = coord.x + coord.y * pc.dim.x + coord.z * pc.dim.x * pc.dim.y;
uint out_idx = linear_index * words;
out_u32[out_idx] = texel.x;
if (words > 1u) {
out_u32[out_idx + 1u] = texel.y;
}
if (words > 2u) {
out_u32[out_idx + 2u] = texel.z;
out_u32[out_idx + 3u] = texel.w;
}
}
@@ -0,0 +1,105 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 430
#define BINDING_INPUT_BUFFER 0
#define BINDING_OUTPUT_BUFFER 1
layout(push_constant) uniform PushConstants {
uvec3 dim;
uint bytes_per_block_log2;
uvec3 origin;
uint slice_size;
uint block_size;
uint x_shift;
uint block_height;
uint block_height_mask;
uint block_depth;
uint block_depth_mask;
} pc;
layout(binding = BINDING_INPUT_BUFFER, std430) buffer InputBufferU32 { uint u32data[]; };
layout(binding = BINDING_INPUT_BUFFER, std430) buffer InputBufferU64 { uvec2 u64data[]; };
layout(binding = BINDING_INPUT_BUFFER, std430) buffer InputBufferU128 { uvec4 u128data[]; };
layout(binding = BINDING_OUTPUT_BUFFER, std430) writeonly buffer OutputBuffer {
uint out_u32[];
};
layout(local_size_x = 8, local_size_y = 8, local_size_z = 4) in;
const uint GOB_SIZE_X = 64;
const uint GOB_SIZE_Y = 8;
const uint GOB_SIZE_X_SHIFT = 6;
const uint GOB_SIZE_Y_SHIFT = 3;
const uint GOB_SIZE_SHIFT = GOB_SIZE_X_SHIFT + GOB_SIZE_Y_SHIFT;
const uvec2 SWIZZLE_MASK = uvec2(GOB_SIZE_X - 1u, GOB_SIZE_Y - 1u);
uint SwizzleTable(uint pos) {
const uint t[8] = uint[](
0x12100200, 0x13110301, 0x16140604, 0x17150705,
0x1a180a08, 0x1b190b09, 0x1e1c0e0c, 0x1f1d0f0d
);
const uint i = pos >> 4;
const uint h = (t[i / 4] >> ((i % 4) * 8)) & 0xff;
return (h << 4) | (pos & 0xf);
}
uint SwizzleOffset(uvec2 pos) {
pos = pos & SWIZZLE_MASK;
return SwizzleTable(pos.y * 64u + pos.x);
}
uvec4 ReadTexel(uint offset) {
switch (pc.bytes_per_block_log2) {
case 2u:
return uvec4(u32data[offset / 4u], 0u, 0u, 0u);
case 3u:
return uvec4(u64data[offset / 8u], 0u, 0u);
case 4u:
return u128data[offset / 16u];
}
return uvec4(0u);
}
void main() {
uvec3 coord = gl_GlobalInvocationID;
if (coord.x >= pc.dim.x || coord.y >= pc.dim.y || coord.z >= pc.dim.z) {
return;
}
uvec3 pos = coord + pc.origin;
pos.x <<= pc.bytes_per_block_log2;
uint swizzle = SwizzleOffset(pos.xy);
uint block_y = pos.y >> GOB_SIZE_Y_SHIFT;
uint offset = 0u;
offset += (pos.z >> pc.block_depth) * pc.slice_size;
offset += (pos.z & pc.block_depth_mask) << (GOB_SIZE_SHIFT + pc.block_height);
offset += (block_y >> pc.block_height) * pc.block_size;
offset += (block_y & pc.block_height_mask) << GOB_SIZE_SHIFT;
offset += (pos.x >> GOB_SIZE_X_SHIFT) << pc.x_shift;
offset += swizzle;
uvec4 texel = ReadTexel(offset);
uint words = 1u << (pc.bytes_per_block_log2 - 2u);
uint linear_index = coord.x + coord.y * pc.dim.x + coord.z * pc.dim.x * pc.dim.y;
uint out_idx = linear_index * words;
out_u32[out_idx] = texel.x;
if (words > 1u) {
out_u32[out_idx + 1u] = texel.y;
}
if (words > 2u) {
out_u32[out_idx + 2u] = texel.z;
out_u32[out_idx + 3u] = texel.w;
}
}
@@ -3,7 +3,14 @@
#version 450 core #version 450 core
layout(binding = 0) uniform sampler2DMS msaa_in; #ifndef SAMPLER_TYPE
#define SAMPLER_TYPE sampler2DMS
#endif
#ifndef TEXEL_TYPE
#define TEXEL_TYPE vec4
#endif
layout(binding = 0) uniform SAMPLER_TYPE msaa_in;
layout(push_constant) uniform PushConstants { layout(push_constant) uniform PushConstants {
ivec2 dst_offset; ivec2 dst_offset;
@@ -11,7 +18,7 @@ layout(push_constant) uniform PushConstants {
ivec2 scale; ivec2 scale;
}; };
layout(location = 0) out vec4 frag_color; layout(location = 0) out TEXEL_TYPE frag_color;
void main() { void main() {
const ivec2 coord = ivec2(gl_FragCoord.xy) - dst_offset + src_offset; const ivec2 coord = ivec2(gl_FragCoord.xy) - dst_offset + src_offset;
@@ -0,0 +1,20 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 450 core
layout(binding = 0) uniform sampler2DMS msaa_in;
layout(push_constant) uniform PushConstants {
ivec2 dst_offset;
ivec2 src_offset;
ivec2 scale;
};
void main() {
const ivec2 coord = ivec2(gl_FragCoord.xy) - dst_offset + src_offset;
const ivec2 msaa_coord = coord / scale;
const ivec2 sample_offset = coord % scale;
const int sample_id = sample_offset.x + scale.x * sample_offset.y;
gl_FragDepth = texelFetch(msaa_in, msaa_coord, sample_id).r;
}
@@ -0,0 +1,23 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 450 core
#extension GL_ARB_shader_stencil_export : require
layout(binding = 0) uniform sampler2DMS depth_tex;
layout(binding = 1) uniform usampler2DMS stencil_tex;
layout(push_constant) uniform PushConstants {
ivec2 dst_offset;
ivec2 src_offset;
ivec2 scale;
};
void main() {
const ivec2 coord = ivec2(gl_FragCoord.xy) - dst_offset + src_offset;
const ivec2 msaa_coord = coord / scale;
const ivec2 sample_offset = coord % scale;
const int sample_id = sample_offset.x + scale.x * sample_offset.y;
gl_FragDepth = texelFetch(depth_tex, msaa_coord, sample_id).r;
gl_FragStencilRefARB = int(texelFetch(stencil_tex, msaa_coord, sample_id).r);
}
@@ -3,7 +3,14 @@
#version 450 core #version 450 core
layout(binding = 0) uniform sampler2D img_in; #ifndef SAMPLER_TYPE
#define SAMPLER_TYPE sampler2D
#endif
#ifndef TEXEL_TYPE
#define TEXEL_TYPE vec4
#endif
layout(binding = 0) uniform SAMPLER_TYPE img_in;
layout(push_constant) uniform PushConstants { layout(push_constant) uniform PushConstants {
ivec2 dst_offset; ivec2 dst_offset;
@@ -11,7 +18,7 @@ layout(push_constant) uniform PushConstants {
ivec2 scale; ivec2 scale;
}; };
layout(location = 0) out vec4 frag_color; layout(location = 0) out TEXEL_TYPE frag_color;
void main() { void main() {
const ivec2 msaa_coord = ivec2(gl_FragCoord.xy) - dst_offset; const ivec2 msaa_coord = ivec2(gl_FragCoord.xy) - dst_offset;
@@ -0,0 +1,19 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 450 core
layout(binding = 0) uniform sampler2D img_in;
layout(push_constant) uniform PushConstants {
ivec2 dst_offset;
ivec2 src_offset;
ivec2 scale;
};
void main() {
const ivec2 msaa_coord = ivec2(gl_FragCoord.xy) - dst_offset;
const ivec2 sample_offset = ivec2(gl_SampleID % scale.x, gl_SampleID / scale.x);
const ivec2 coord = msaa_coord * scale + sample_offset + src_offset;
gl_FragDepth = texelFetch(img_in, coord, 0).r;
}
@@ -0,0 +1,22 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#version 450 core
#extension GL_ARB_shader_stencil_export : require
layout(binding = 0) uniform sampler2D depth_tex;
layout(binding = 1) uniform usampler2D stencil_tex;
layout(push_constant) uniform PushConstants {
ivec2 dst_offset;
ivec2 src_offset;
ivec2 scale;
};
void main() {
const ivec2 msaa_coord = ivec2(gl_FragCoord.xy) - dst_offset;
const ivec2 sample_offset = ivec2(gl_SampleID % scale.x, gl_SampleID / scale.x);
const ivec2 coord = msaa_coord * scale + sample_offset + src_offset;
gl_FragDepth = texelFetch(depth_tex, coord, 0).r;
gl_FragStencilRefARB = int(texelFetch(stencil_tex, coord, 0).r);
}
@@ -7,6 +7,7 @@ layout(push_constant) uniform constants {
vec2 scale; vec2 scale;
vec2 size; vec2 size;
vec2 resize_factor; vec2 resize_factor;
vec2 crop_offset;
float edge_sharpness; float edge_sharpness;
}; };
layout(location = 0) out highp vec2 texcoord; layout(location = 0) out highp vec2 texcoord;
@@ -15,5 +16,5 @@ void main() {
float x = float((gl_VertexIndex & 1) << 2); float x = float((gl_VertexIndex & 1) << 2);
float y = float((gl_VertexIndex & 2) << 1); float y = float((gl_VertexIndex & 2) << 1);
gl_Position = vec4(x - 1.0f, y - 1.0f, 0.0, 1.0f) * vec4(sign(resize_factor), 1.f, 1.f); gl_Position = vec4(x - 1.0f, y - 1.0f, 0.0, 1.0f) * vec4(sign(resize_factor), 1.f, 1.f);
texcoord = vec2(x, y) * abs(resize_factor) * 0.5; texcoord = crop_offset + vec2(x, y) * abs(resize_factor) * 0.5;
} }
@@ -6,34 +6,52 @@
precision highp float; precision highp float;
precision highp int; precision highp int;
// Operation modes: RGBA -> 1, RGBY -> 3, LERP -> 4
#define OPERATION_MODE 1
#define EDGE_THRESHOLD (8.0 / 255.0) #define EDGE_THRESHOLD (8.0 / 255.0)
#define DIRECTION_EPSILON 6.5e-05
#define DEVIATION_FLOOR 6.0e-02
layout(push_constant) uniform constants { layout(push_constant) uniform constants {
vec2 scale; vec2 scale;
vec2 size; vec2 size;
vec2 resize_factor; vec2 resize_factor;
vec2 crop_offset;
float edge_sharpness; float edge_sharpness;
}; };
layout(set = 0, binding = 0) uniform sampler2D sampler0; layout(set = 0, binding = 0) uniform sampler2D sampler0;
layout(location=0) in vec2 texcoord; layout(location=0) in vec2 texcoord;
layout(location=0) out vec4 frag_color; layout(location=0) out vec4 frag_color;
vec4 weightY(vec4 dx, vec4 dy, vec4 std) { mediump vec4 fastLanczos2(mediump vec4 x) {
vec4 x = ((dx * dx) + (dy * dy)) * 0.55f + std; mediump vec4 wA = x - 4.0f;
return (x - 1.f) * (x - 4.f) * 3.8125f; // approx. of (x - 1) * (x - 4)^3 mediump vec4 wB = x * wA - wA;
wA *= wA;
return wB * wA;
}
mediump vec2 edgeDirection(mediump vec4 left, mediump vec4 right) {
mediump float RxLz = right.x - left.z;
mediump float RwLy = right.w - left.y;
mediump vec2 delta = vec2(RxLz + RwLy, RxLz - RwLy);
mediump float length_inv =
inversesqrt((delta.x * delta.x + DIRECTION_EPSILON) + delta.y * delta.y);
return delta * length_inv;
}
mediump vec4 weightY(mediump vec4 dx, mediump vec4 dy, mediump vec4 c, mediump float std,
mediump vec2 dir) {
mediump vec4 edge_dis = dx * dir.y + dy * dir.x;
mediump vec4 x = (dx * dx + dy * dy) +
(edge_dis * edge_dis) * (clamp((c * c) * std, 0.0f, 1.0f) * 0.7f - 1.0f);
return fastLanczos2(x);
} }
void main() { void main() {
vec4 color = textureLod(sampler0, texcoord.xy, 0.0f); mediump vec4 color = textureLod(sampler0, texcoord.xy, 0.0f);
// image coord highp vec2 icoord = (texcoord * size + vec2(-0.5f, 0.5f));
vec2 icoord = (texcoord * size + vec2(-0.5f, 0.5f)); highp vec2 icoord_pixel = floor(icoord);
vec2 icoord_pixel = floor(icoord); highp vec2 coord = icoord_pixel * scale;
vec2 coord = icoord_pixel * scale; mediump vec2 pl = icoord - icoord_pixel;
vec2 pl = icoord - icoord_pixel; mediump mat3x4 dg = mat3x4(
// left: 0, right: 1, upDown: 2
mat3x4 dg = mat3x4(
textureGather(sampler0, coord, 1), textureGather(sampler0, coord, 1),
textureGather(sampler0, coord + vec2(2.f * scale.x, 0.0f), 1), textureGather(sampler0, coord + vec2(2.f * scale.x, 0.0f), 1),
vec4( vec4(
@@ -41,42 +59,40 @@ void main() {
textureGather(sampler0, coord + vec2(scale.x, +scale.y), 1).yx textureGather(sampler0, coord + vec2(scale.x, +scale.y), 1).yx
) )
); );
float edgeVote = abs(dg[0].z - dg[0].y) + abs(color.y - dg[0].y) + abs(color.y - dg[0].z); mediump float edgeVote =
abs(dg[0].z - dg[0].y) + abs(color.y - dg[0].y) + abs(color.y - dg[0].z);
if (edgeVote > EDGE_THRESHOLD) { if (edgeVote > EDGE_THRESHOLD) {
float mean = (dg[0].y + dg[0].z + dg[1].x + dg[1].w) * 0.25f; mediump float mean = (dg[0].y + dg[0].z + dg[1].x + dg[1].w) * 0.25f;
dg = dg - mean; dg = dg - mean;
vec4 sum = abs(dg[0]) + abs(dg[1]) + abs(dg[2]); mediump float sum = dot(abs(dg[0]) + abs(dg[1]) + abs(dg[2]), vec4(1.0f));
float std = 2.181818f / (sum.x + sum.y + sum.z + sum.w); mediump float sum_mean = 1.014185e+01f / max(sum, DEVIATION_FLOOR);
mat2x4 w = mat2x4( mediump float std = sum_mean * sum_mean;
weightY( mediump vec2 dir = edgeDirection(dg[0], dg[1]);
pl.xxxx + vec4(+1.0f, +0.0f, +0.0f, +1.0f), mediump vec4 w0 = weightY(
pl.yyyy + vec4(-1.0f, -1.0f, +0.0f, +0.0f), pl.xxxx + vec4(+1.0f, +0.0f, +0.0f, +1.0f),
clamp(abs(dg[0]) * std, 0.0f, 1.0f) pl.yyyy + vec4(-1.0f, -1.0f, +0.0f, +0.0f),
) + weightY( dg[0], std, dir
pl.xxxx + vec4(-1.0f, -2.0f, -2.0f, -1.0f),
pl.yyyy + vec4(-1.0f, -1.0f, +0.0f, +0.0f),
clamp(abs(dg[1]) * std, 0.0f, 1.0f)
) + weightY(
pl.xxxx + vec4(+0.0f, -1.0f, -1.0f, +0.0f),
pl.yyyy + vec4(+1.0f, +1.0f, -2.0f, -2.0f),
clamp(abs(dg[2]) * std, 0.0f, 1.0f)
),
dg[0] + dg[1] + dg[2]
); );
// compute final y with bounds mediump vec4 w1 = weightY(
vec2 yb = vec2( pl.xxxx + vec4(-1.0f, -2.0f, -2.0f, -1.0f),
min(min(dg[0].y, dg[0].z), min(dg[1].x, dg[1].w)), // min pl.yyyy + vec4(-1.0f, -1.0f, +0.0f, +0.0f),
max(max(dg[0].y, dg[0].z), max(dg[1].x, dg[1].w)) // max dg[1], std, dir
); );
vec2 fvy = vec2( mediump vec4 w2 = weightY(
w[0].x + w[0].y + w[0].z + w[0].w, pl.xxxx + vec4(+0.0f, -1.0f, -1.0f, +0.0f),
w[1].x + w[1].y + w[1].z + w[1].w pl.yyyy + vec4(+1.0f, +1.0f, -2.0f, -2.0f),
dg[2], std, dir
); );
float fy = clamp((fvy.y / fvy.x) * edge_sharpness, yb[0], yb[1]); mediump float sum_w = dot(w0 + w1 + w2, vec4(1.0f));
// Smooth high contrast input mediump float sum_wc = dot(w0 * dg[0] + w1 * dg[1] + w2 * dg[2], vec4(1.0f));
float dy = clamp(fy - color.y + mean, -23.0f / 255.0f, 23.0f / 255.0f); mediump vec2 yb = vec2(
min(min(dg[0].y, dg[0].z), min(dg[1].x, dg[1].w)),
max(max(dg[0].y, dg[0].z), max(dg[1].x, dg[1].w))
);
mediump float fy = clamp((sum_wc / sum_w) * edge_sharpness, yb[0], yb[1]);
mediump float dy = clamp(fy - color.y + mean, -23.0f / 255.0f, 23.0f / 255.0f);
color = clamp(color + dy, 0.0f, 1.0f); color = clamp(color + dy, 0.0f, 1.0f);
} }
color.w = 1.0f; //assume alpha channel is not used color.w = 1.0f;
frag_color.xyzw = color; frag_color.xyzw = color;
} }
@@ -13,6 +13,7 @@
layout( push_constant ) uniform constants { layout( push_constant ) uniform constants {
vec4 ViewportInfo[1]; vec4 ViewportInfo[1];
vec2 ResizeFactor; vec2 ResizeFactor;
vec2 CropOffset;
float EdgeSharpness; float EdgeSharpness;
}; };
layout(set = 0, binding = 0) uniform sampler2D ps0; layout(set = 0, binding = 0) uniform sampler2D ps0;
@@ -261,6 +261,7 @@ struct BufferCacheParams {
// TODO: Investigate why OpenGL seems to perform worse with persistently mapped buffer uploads // TODO: Investigate why OpenGL seems to perform worse with persistently mapped buffer uploads
static constexpr bool USE_MEMORY_MAPS_FOR_UPLOADS = false; static constexpr bool USE_MEMORY_MAPS_FOR_UPLOADS = false;
static constexpr bool USE_UNIFIED_MEMORY = false;
}; };
using BufferCache = VideoCommon::BufferCache<BufferCacheParams>; using BufferCache = VideoCommon::BufferCache<BufferCacheParams>;
@@ -231,12 +231,14 @@ ShaderCache::ShaderCache(Tegra::MaxwellDeviceMemoryManager& device_memory_,
.has_broken_unsigned_image_offsets = true, .has_broken_unsigned_image_offsets = true,
.has_broken_signed_operations = true, .has_broken_signed_operations = true,
.has_broken_fp16_float_controls = false, .has_broken_fp16_float_controls = false,
.has_broken_fp32_denorm_flush = false,
.has_gl_component_indexing_bug = device.HasComponentIndexingBug(), .has_gl_component_indexing_bug = device.HasComponentIndexingBug(),
.has_gl_precise_bug = device.HasPreciseBug(), .has_gl_precise_bug = device.HasPreciseBug(),
.has_gl_cbuf_ftou_bug = device.HasCbufFtouBug(), .has_gl_cbuf_ftou_bug = device.HasCbufFtouBug(),
.has_gl_bool_ref_bug = device.HasBoolRefBug(), .has_gl_bool_ref_bug = device.HasBoolRefBug(),
.ignore_nan_fp_comparisons = true, .ignore_nan_fp_comparisons = true,
.gl_max_compute_smem_size = device.GetMaxComputeSharedMemorySize(), .gl_max_compute_smem_size = device.GetMaxComputeSharedMemorySize(),
.max_shared_memory_size = device.GetMaxComputeSharedMemorySize(),
.min_ssbo_alignment = device.GetShaderStorageBufferAlignment(), .min_ssbo_alignment = device.GetShaderStorageBufferAlignment(),
// Use the host limit, but never more than the guest can produce. Maxwell exposes 8 clip // Use the host limit, but never more than the guest can produce. Maxwell exposes 8 clip
// distances and the SPIR-V output array is sized for at most 8, so clamping here keeps a // distances and the SPIR-V output array is sized for at most 8, so clamping here keeps a
@@ -75,6 +75,8 @@ public:
void Finish(); void Finish();
void FlushDeferredClear() {}
StagingBufferMap UploadStagingBuffer(size_t size, bool deferred = false); StagingBufferMap UploadStagingBuffer(size_t size, bool deferred = false);
StagingBufferMap DownloadStagingBuffer(size_t size, bool deferred = false); StagingBufferMap DownloadStagingBuffer(size_t size, bool deferred = false);
@@ -370,6 +372,8 @@ struct TextureCacheParams {
static constexpr bool HAS_EMULATED_COPIES = true; static constexpr bool HAS_EMULATED_COPIES = true;
static constexpr bool HAS_DEVICE_MEMORY_INFO = true; static constexpr bool HAS_DEVICE_MEMORY_INFO = true;
static constexpr bool IMPLEMENTS_ASYNC_DOWNLOADS = true; static constexpr bool IMPLEMENTS_ASYNC_DOWNLOADS = true;
static constexpr bool HAS_MSAA_DOWNLOADS = false;
static constexpr bool USE_UNIFIED_MEMORY = false;
using Runtime = OpenGL::TextureCacheRuntime; using Runtime = OpenGL::TextureCacheRuntime;
using Image = OpenGL::Image; using Image = OpenGL::Image;
+555 -237
View File
@@ -12,6 +12,7 @@
#include "common/settings.h" #include "common/settings.h"
#include "video_core/host_shaders/blit_color_float_frag_spv.h" #include "video_core/host_shaders/blit_color_float_frag_spv.h"
#include "video_core/host_shaders/blit_color_msaa_frag_spv.h" #include "video_core/host_shaders/blit_color_msaa_frag_spv.h"
#include "video_core/host_shaders/blit_depth_frag_spv.h"
#include "video_core/host_shaders/blit_depth_msaa_frag_spv.h" #include "video_core/host_shaders/blit_depth_msaa_frag_spv.h"
#include "video_core/host_shaders/blit_depth_stencil_msaa_frag_spv.h" #include "video_core/host_shaders/blit_depth_stencil_msaa_frag_spv.h"
#include "video_core/host_shaders/convert_abgr8_to_d24s8_frag_spv.h" #include "video_core/host_shaders/convert_abgr8_to_d24s8_frag_spv.h"
@@ -21,7 +22,15 @@
#include "video_core/host_shaders/convert_depth_to_float_frag_spv.h" #include "video_core/host_shaders/convert_depth_to_float_frag_spv.h"
#include "video_core/host_shaders/convert_float_to_depth_frag_spv.h" #include "video_core/host_shaders/convert_float_to_depth_frag_spv.h"
#include "video_core/host_shaders/convert_msaa_to_non_msaa_frag_spv.h" #include "video_core/host_shaders/convert_msaa_to_non_msaa_frag_spv.h"
#include "video_core/host_shaders/convert_msaa_to_non_msaa_depth_frag_spv.h"
#include "video_core/host_shaders/convert_msaa_to_non_msaa_depth_stencil_frag_spv.h"
#include "video_core/host_shaders/convert_msaa_to_non_msaa_sint_frag_spv.h"
#include "video_core/host_shaders/convert_msaa_to_non_msaa_uint_frag_spv.h"
#include "video_core/host_shaders/convert_non_msaa_to_msaa_frag_spv.h" #include "video_core/host_shaders/convert_non_msaa_to_msaa_frag_spv.h"
#include "video_core/host_shaders/convert_non_msaa_to_msaa_sint_frag_spv.h"
#include "video_core/host_shaders/convert_non_msaa_to_msaa_uint_frag_spv.h"
#include "video_core/host_shaders/convert_non_msaa_to_msaa_depth_frag_spv.h"
#include "video_core/host_shaders/convert_non_msaa_to_msaa_depth_stencil_frag_spv.h"
#include "video_core/host_shaders/convert_s8d24_to_abgr8_frag_spv.h" #include "video_core/host_shaders/convert_s8d24_to_abgr8_frag_spv.h"
#include "video_core/host_shaders/full_screen_triangle_vert_spv.h" #include "video_core/host_shaders/full_screen_triangle_vert_spv.h"
#include "video_core/host_shaders/vulkan_blit_depth_stencil_frag_spv.h" #include "video_core/host_shaders/vulkan_blit_depth_stencil_frag_spv.h"
@@ -518,8 +527,19 @@ void RecordShaderReadBarrier(Scheduler& scheduler, const ImageView& image_view)
} }
} }
[[nodiscard]] MSAACopyFormatClass FormatClass(VideoCore::Surface::PixelFormat format) {
if (!VideoCore::Surface::IsPixelFormatInteger(format)) {
return MSAACopyFormatClass::Float;
}
if (VideoCore::Surface::IsPixelFormatSignedInteger(format)) {
return MSAACopyFormatClass::SignedInteger;
}
return MSAACopyFormatClass::UnsignedInteger;
}
[[nodiscard]] vk::ImageView MakeMSAACopyView(const vk::Device& device, VkImage image, [[nodiscard]] vk::ImageView MakeMSAACopyView(const vk::Device& device, VkImage image,
VkFormat format, u32 base_level) { VkFormat format, u32 base_level, u32 base_layer,
VkImageAspectFlags aspect_mask) {
return device.CreateImageView(VkImageViewCreateInfo{ return device.CreateImageView(VkImageViewCreateInfo{
.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO, .sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO,
.pNext = nullptr, .pNext = nullptr,
@@ -534,10 +554,10 @@ void RecordShaderReadBarrier(Scheduler& scheduler, const ImageView& image_view)
.a = VK_COMPONENT_SWIZZLE_IDENTITY, .a = VK_COMPONENT_SWIZZLE_IDENTITY,
}, },
.subresourceRange{ .subresourceRange{
.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT, .aspectMask = aspect_mask,
.baseMipLevel = base_level, .baseMipLevel = base_level,
.levelCount = 1, .levelCount = 1,
.baseArrayLayer = 0, .baseArrayLayer = base_layer,
.layerCount = 1, .layerCount = 1,
}, },
}); });
@@ -586,12 +606,17 @@ BlitImageHelper::BlitImageHelper(const Device& device_, Scheduler& scheduler_,
msaa_copy_pipeline_layout(device.GetLogical().CreatePipelineLayout(PipelineLayoutCreateInfo( msaa_copy_pipeline_layout(device.GetLogical().CreatePipelineLayout(PipelineLayoutCreateInfo(
one_texture_set_layout.address(), one_texture_set_layout.address(),
PUSH_CONSTANT_RANGE<VK_SHADER_STAGE_FRAGMENT_BIT, sizeof(MSAACopyPushConstants)>))), PUSH_CONSTANT_RANGE<VK_SHADER_STAGE_FRAGMENT_BIT, sizeof(MSAACopyPushConstants)>))),
msaa_copy_depth_stencil_pipeline_layout(
device.GetLogical().CreatePipelineLayout(PipelineLayoutCreateInfo(
two_textures_set_layout.address(),
PUSH_CONSTANT_RANGE<VK_SHADER_STAGE_FRAGMENT_BIT, sizeof(MSAACopyPushConstants)>))),
full_screen_vert(BuildShader(device, FULL_SCREEN_TRIANGLE_VERT_SPV)), full_screen_vert(BuildShader(device, FULL_SCREEN_TRIANGLE_VERT_SPV)),
blit_color_to_color_frag(BuildShader(device, BLIT_COLOR_FLOAT_FRAG_SPV)), blit_color_to_color_frag(BuildShader(device, BLIT_COLOR_FLOAT_FRAG_SPV)),
blit_color_msaa_frag(BuildShader(device, BLIT_COLOR_MSAA_FRAG_SPV)), blit_color_msaa_frag(BuildShader(device, BLIT_COLOR_MSAA_FRAG_SPV)),
blit_depth_stencil_frag(device.IsExtShaderStencilExportSupported() blit_depth_stencil_frag(device.IsExtShaderStencilExportSupported()
? BuildShader(device, VULKAN_BLIT_DEPTH_STENCIL_FRAG_SPV) ? BuildShader(device, VULKAN_BLIT_DEPTH_STENCIL_FRAG_SPV)
: vk::ShaderModule{}), : vk::ShaderModule{}),
blit_depth_frag(BuildShader(device, BLIT_DEPTH_FRAG_SPV)),
blit_depth_msaa_frag(BuildShader(device, BLIT_DEPTH_MSAA_FRAG_SPV)), blit_depth_msaa_frag(BuildShader(device, BLIT_DEPTH_MSAA_FRAG_SPV)),
blit_depth_stencil_msaa_frag(device.IsExtShaderStencilExportSupported() blit_depth_stencil_msaa_frag(device.IsExtShaderStencilExportSupported()
? BuildShader(device, BLIT_DEPTH_STENCIL_MSAA_FRAG_SPV) ? BuildShader(device, BLIT_DEPTH_STENCIL_MSAA_FRAG_SPV)
@@ -609,7 +634,25 @@ BlitImageHelper::BlitImageHelper(const Device& device_, Scheduler& scheduler_,
convert_d24s8_to_abgr8_frag(BuildShader(device, CONVERT_D24S8_TO_ABGR8_FRAG_SPV)), convert_d24s8_to_abgr8_frag(BuildShader(device, CONVERT_D24S8_TO_ABGR8_FRAG_SPV)),
convert_s8d24_to_abgr8_frag(BuildShader(device, CONVERT_S8D24_TO_ABGR8_FRAG_SPV)), convert_s8d24_to_abgr8_frag(BuildShader(device, CONVERT_S8D24_TO_ABGR8_FRAG_SPV)),
convert_msaa_to_non_msaa_frag(BuildShader(device, CONVERT_MSAA_TO_NON_MSAA_FRAG_SPV)), convert_msaa_to_non_msaa_frag(BuildShader(device, CONVERT_MSAA_TO_NON_MSAA_FRAG_SPV)),
convert_msaa_to_non_msaa_sint_frag(
BuildShader(device, CONVERT_MSAA_TO_NON_MSAA_SINT_FRAG_SPV)),
convert_msaa_to_non_msaa_uint_frag(
BuildShader(device, CONVERT_MSAA_TO_NON_MSAA_UINT_FRAG_SPV)),
convert_msaa_to_non_msaa_depth_frag(
BuildShader(device, CONVERT_MSAA_TO_NON_MSAA_DEPTH_FRAG_SPV)),
convert_msaa_to_non_msaa_depth_stencil_frag(
BuildShader(device, CONVERT_MSAA_TO_NON_MSAA_DEPTH_STENCIL_FRAG_SPV)),
convert_non_msaa_to_msaa_frag(BuildShader(device, CONVERT_NON_MSAA_TO_MSAA_FRAG_SPV)), convert_non_msaa_to_msaa_frag(BuildShader(device, CONVERT_NON_MSAA_TO_MSAA_FRAG_SPV)),
convert_non_msaa_to_msaa_sint_frag(
BuildShader(device, CONVERT_NON_MSAA_TO_MSAA_SINT_FRAG_SPV)),
convert_non_msaa_to_msaa_uint_frag(
BuildShader(device, CONVERT_NON_MSAA_TO_MSAA_UINT_FRAG_SPV)),
convert_non_msaa_to_msaa_depth_frag(
BuildShader(device, CONVERT_NON_MSAA_TO_MSAA_DEPTH_FRAG_SPV)),
convert_non_msaa_to_msaa_depth_stencil_frag(
device.IsExtShaderStencilExportSupported()
? BuildShader(device, CONVERT_NON_MSAA_TO_MSAA_DEPTH_STENCIL_FRAG_SPV)
: vk::ShaderModule{}),
linear_sampler(device.GetLogical().CreateSampler(SAMPLER_CREATE_INFO<VK_FILTER_LINEAR>)), linear_sampler(device.GetLogical().CreateSampler(SAMPLER_CREATE_INFO<VK_FILTER_LINEAR>)),
nearest_sampler(device.GetLogical().CreateSampler(SAMPLER_CREATE_INFO<VK_FILTER_NEAREST>)) {} nearest_sampler(device.GetLogical().CreateSampler(SAMPLER_CREATE_INFO<VK_FILTER_NEAREST>)) {}
@@ -624,24 +667,13 @@ void BlitImageHelper::BlitColor(const Framebuffer* dst_framebuffer, const ImageV
.renderpass = dst_framebuffer->RenderPass(), .renderpass = dst_framebuffer->RenderPass(),
.operation = operation, .operation = operation,
}; };
const VkPipelineLayout layout = *one_texture_pipeline_layout; VkSampler sampler = *nearest_sampler;
const VkSampler sampler = is_linear ? *linear_sampler : *nearest_sampler; if (is_linear) {
const VkPipeline pipeline = FindOrEmplaceColorPipeline(key); sampler = *linear_sampler;
const VkImageView src_view = src_image_view.Handle(Shader::TextureType::Color2D); }
BlitImpl(dst_framebuffer, src_image_view, dst_region, src_region,
RecordShaderReadBarrier(scheduler, src_image_view); FindOrEmplaceColorPipeline(key), sampler,
scheduler.RequestRenderpass(dst_framebuffer); src_image_view.Handle(Shader::TextureType::Color2D), VK_NULL_HANDLE, false);
scheduler.Record([this, dst_region, src_region, pipeline, layout, sampler,
src_view](vk::CommandBuffer cmdbuf) {
const VkDescriptorSet descriptor_set = one_texture_descriptor_allocator.Commit();
UpdateOneTextureDescriptorSet(device, descriptor_set, sampler, src_view);
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
BindBlitState(cmdbuf, layout, dst_region, src_region);
cmdbuf.Draw(3, 1, 0, 0);
});
scheduler.InvalidateState();
} }
void BlitImageHelper::BlitColor(const Framebuffer* dst_framebuffer, VkImageView src_image_view, void BlitImageHelper::BlitColor(const Framebuffer* dst_framebuffer, VkImageView src_image_view,
@@ -670,6 +702,38 @@ void BlitImageHelper::BlitColor(const Framebuffer* dst_framebuffer, VkImageView
}); });
} }
void BlitImageHelper::BlitImpl(const Framebuffer* dst_framebuffer,
const ImageView& src_image_view, const Region2D& dst_region,
const Region2D& src_region, VkPipeline pipeline, VkSampler sampler,
VkImageView src_view, VkImageView src_stencil_view,
bool blit_stencil) {
VkPipelineLayout layout = *one_texture_pipeline_layout;
if (blit_stencil) {
layout = *two_textures_pipeline_layout;
}
RecordShaderReadBarrier(scheduler, src_image_view);
scheduler.RequestRenderpass(dst_framebuffer);
scheduler.Record([this, dst_region, src_region, pipeline, layout, sampler, src_view,
src_stencil_view, blit_stencil](vk::CommandBuffer cmdbuf) {
VkDescriptorSet descriptor_set = VK_NULL_HANDLE;
if (blit_stencil) {
descriptor_set = two_textures_descriptor_allocator.Commit();
UpdateTwoTexturesDescriptorSet(device, descriptor_set, sampler, src_view,
src_stencil_view);
} else {
descriptor_set = one_texture_descriptor_allocator.Commit();
UpdateOneTextureDescriptorSet(device, descriptor_set, sampler, src_view);
}
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
BindBlitState(cmdbuf, layout, dst_region, src_region);
cmdbuf.Draw(3, 1, 0, 0);
});
scheduler.InvalidateState();
}
void BlitImageHelper::BlitColorMSAA(const Framebuffer* dst_framebuffer, void BlitImageHelper::BlitColorMSAA(const Framebuffer* dst_framebuffer,
const ImageView& src_image_view, const Region2D& dst_region, const ImageView& src_image_view, const Region2D& dst_region,
const Region2D& src_region) { const Region2D& src_region) {
@@ -677,24 +741,34 @@ void BlitImageHelper::BlitColorMSAA(const Framebuffer* dst_framebuffer,
.renderpass = dst_framebuffer->RenderPass(), .renderpass = dst_framebuffer->RenderPass(),
.samples = dst_framebuffer->Samples(), .samples = dst_framebuffer->Samples(),
}; };
const VkPipelineLayout layout = *one_texture_pipeline_layout; BlitImpl(dst_framebuffer, src_image_view, dst_region, src_region,
const VkSampler sampler = *nearest_sampler; FindOrEmplaceBlitColorMSAAPipeline(key), *nearest_sampler,
const VkPipeline pipeline = FindOrEmplaceBlitColorMSAAPipeline(key); src_image_view.Handle(Shader::TextureType::Color2D), VK_NULL_HANDLE, false);
const VkImageView src_view = src_image_view.Handle(Shader::TextureType::Color2D); }
RecordShaderReadBarrier(scheduler, src_image_view); void BlitImageHelper::BlitDepthStencilMSAA(const Framebuffer* dst_framebuffer,
scheduler.RequestRenderpass(dst_framebuffer); ImageView& src_image_view, const Region2D& dst_region,
scheduler.Record([this, dst_region, src_region, pipeline, layout, sampler, const Region2D& src_region) {
src_view](vk::CommandBuffer cmdbuf) { const bool blit_stencil =
const VkDescriptorSet descriptor_set = one_texture_descriptor_allocator.Commit(); dst_framebuffer->HasAspectStencilBit() && device.IsExtShaderStencilExportSupported();
UpdateOneTextureDescriptorSet(device, descriptor_set, sampler, src_view); const BlitMSAAPipelineKey key{
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline); .renderpass = dst_framebuffer->RenderPass(),
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set, .samples = dst_framebuffer->Samples(),
nullptr); };
BindBlitState(cmdbuf, layout, dst_region, src_region); VkImageView src_stencil_view = VK_NULL_HANDLE;
cmdbuf.Draw(3, 1, 0, 0); if (blit_stencil) {
}); src_stencil_view = src_image_view.StencilView();
scheduler.InvalidateState(); }
BlitImpl(dst_framebuffer, src_image_view, dst_region, src_region,
FindOrEmplaceBlitDepthStencilMSAAPipeline(key, blit_stencil), *nearest_sampler,
src_image_view.DepthView(), src_stencil_view, blit_stencil);
}
void BlitImageHelper::BlitDepth(const Framebuffer* dst_framebuffer, ImageView& src_image_view,
const Region2D& dst_region, const Region2D& src_region) {
BlitImpl(dst_framebuffer, src_image_view, dst_region, src_region,
FindOrEmplaceBlitDepthPipeline(dst_framebuffer->RenderPass()), *nearest_sampler,
src_image_view.DepthView(), VK_NULL_HANDLE, false);
} }
void BlitImageHelper::ResolveDepthStencil(const Framebuffer* dst_framebuffer, void BlitImageHelper::ResolveDepthStencil(const Framebuffer* dst_framebuffer,
@@ -702,36 +776,14 @@ void BlitImageHelper::ResolveDepthStencil(const Framebuffer* dst_framebuffer,
const Region2D& src_region) { const Region2D& src_region) {
const bool resolve_stencil = const bool resolve_stencil =
dst_framebuffer->HasAspectStencilBit() && device.IsExtShaderStencilExportSupported(); dst_framebuffer->HasAspectStencilBit() && device.IsExtShaderStencilExportSupported();
const VkPipeline pipeline = VkImageView src_stencil_view = VK_NULL_HANDLE;
FindOrEmplaceResolveDepthStencilPipeline(dst_framebuffer->RenderPass(), resolve_stencil); if (resolve_stencil) {
const VkPipelineLayout layout = src_stencil_view = src_image_view.StencilView();
resolve_stencil ? *two_textures_pipeline_layout : *one_texture_pipeline_layout; }
const VkSampler sampler = *nearest_sampler; BlitImpl(dst_framebuffer, src_image_view, dst_region, src_region,
const VkImageView src_depth_view = src_image_view.DepthView(); FindOrEmplaceResolveDepthStencilPipeline(dst_framebuffer->RenderPass(),
const VkImageView src_stencil_view = resolve_stencil),
resolve_stencil ? src_image_view.StencilView() : VK_NULL_HANDLE; *nearest_sampler, src_image_view.DepthView(), src_stencil_view, resolve_stencil);
RecordShaderReadBarrier(scheduler, src_image_view);
scheduler.RequestRenderpass(dst_framebuffer);
scheduler.Record([this, dst_region, src_region, pipeline, layout, sampler, src_depth_view,
src_stencil_view, resolve_stencil](vk::CommandBuffer cmdbuf) {
if (resolve_stencil) {
const VkDescriptorSet descriptor_set = two_textures_descriptor_allocator.Commit();
UpdateTwoTexturesDescriptorSet(device, descriptor_set, sampler, src_depth_view,
src_stencil_view);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
} else {
const VkDescriptorSet descriptor_set = one_texture_descriptor_allocator.Commit();
UpdateOneTextureDescriptorSet(device, descriptor_set, sampler, src_depth_view);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
}
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
BindBlitState(cmdbuf, layout, dst_region, src_region);
cmdbuf.Draw(3, 1, 0, 0);
});
scheduler.InvalidateState();
} }
void BlitImageHelper::BlitDepthStencil(const Framebuffer* dst_framebuffer, void BlitImageHelper::BlitDepthStencil(const Framebuffer* dst_framebuffer,
@@ -739,35 +791,23 @@ void BlitImageHelper::BlitDepthStencil(const Framebuffer* dst_framebuffer,
const Region2D& dst_region, const Region2D& src_region, const Region2D& dst_region, const Region2D& src_region,
Tegra::Engines::Fermi2D::Filter filter, Tegra::Engines::Fermi2D::Filter filter,
Tegra::Engines::Fermi2D::Operation operation) { Tegra::Engines::Fermi2D::Operation operation) {
if (!device.IsExtShaderStencilExportSupported()) {
return;
}
ASSERT(filter == Tegra::Engines::Fermi2D::Filter::Point); ASSERT(filter == Tegra::Engines::Fermi2D::Filter::Point);
ASSERT(operation == Tegra::Engines::Fermi2D::Operation::SrcCopy); ASSERT(operation == Tegra::Engines::Fermi2D::Operation::SrcCopy);
const bool blit_stencil = device.IsExtShaderStencilExportSupported();
const BlitImagePipelineKey key{ const BlitImagePipelineKey key{
.renderpass = dst_framebuffer->RenderPass(), .renderpass = dst_framebuffer->RenderPass(),
.operation = operation, .operation = operation,
}; };
const VkPipelineLayout layout = *two_textures_pipeline_layout; VkPipeline pipeline{};
const VkSampler sampler = *nearest_sampler; VkImageView src_stencil_view = VK_NULL_HANDLE;
const VkPipeline pipeline = FindOrEmplaceDepthStencilPipeline(key); if (blit_stencil) {
const VkImageView src_depth_view = src_image_view.DepthView(); pipeline = FindOrEmplaceDepthStencilPipeline(key);
const VkImageView src_stencil_view = src_image_view.StencilView(); src_stencil_view = src_image_view.StencilView();
} else {
RecordShaderReadBarrier(scheduler, src_image_view); pipeline = FindOrEmplaceBlitDepthPipeline(key.renderpass);
scheduler.RequestRenderpass(dst_framebuffer); }
scheduler.Record([dst_region, src_region, pipeline, layout, sampler, src_depth_view, BlitImpl(dst_framebuffer, src_image_view, dst_region, src_region, pipeline, *nearest_sampler,
src_stencil_view, this](vk::CommandBuffer cmdbuf) { src_image_view.DepthView(), src_stencil_view, blit_stencil);
const VkDescriptorSet descriptor_set = two_textures_descriptor_allocator.Commit();
UpdateTwoTexturesDescriptorSet(device, descriptor_set, sampler, src_depth_view,
src_stencil_view);
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
BindBlitState(cmdbuf, layout, dst_region, src_region);
cmdbuf.Draw(3, 1, 0, 0);
});
scheduler.InvalidateState();
} }
void BlitImageHelper::ConvertD32ToR32(const Framebuffer* dst_framebuffer, void BlitImageHelper::ConvertD32ToR32(const Framebuffer* dst_framebuffer,
@@ -882,19 +922,183 @@ void BlitImageHelper::ClearDepthStencil(const Framebuffer* dst_framebuffer, bool
scheduler.InvalidateState(); scheduler.InvalidateState();
} }
void BlitImageHelper::CopyMSAAImpl(VkRenderPass renderpass, VkPipeline pipeline,
VkPipelineLayout layout, VkImage dst_image,
VkFormat dst_vk_format, VkImage src_image,
VkFormat src_vk_format, s32 scale_x, s32 scale_y,
std::span<const VideoCommon::ImageCopy> copies,
const MSAACopyAspectInfo& aspect_info, bool copy_stencil) {
while (!msaa_copy_resources.empty() && scheduler.IsFree(msaa_copy_resources.front().tick)) {
msaa_copy_resources.pop_front();
}
const VkSampler sampler = *nearest_sampler;
for (const VideoCommon::ImageCopy& copy : copies) {
const s32 num_layers = (std::min)(copy.src_subresource.num_layers,
copy.dst_subresource.num_layers);
for (s32 layer = 0; layer < num_layers; ++layer) {
const u32 src_level = static_cast<u32>(copy.src_subresource.base_level);
const u32 src_layer = static_cast<u32>(copy.src_subresource.base_layer + layer);
vk::ImageView src_view =
MakeMSAACopyView(device.GetLogical(), src_image, src_vk_format, src_level,
src_layer, aspect_info.src_view_aspect);
vk::ImageView src_stencil_view;
if (copy_stencil) {
src_stencil_view =
MakeMSAACopyView(device.GetLogical(), src_image, src_vk_format, src_level,
src_layer, VK_IMAGE_ASPECT_STENCIL_BIT);
}
vk::ImageView dst_view =
MakeMSAACopyView(device.GetLogical(), dst_image, dst_vk_format,
static_cast<u32>(copy.dst_subresource.base_level),
static_cast<u32>(copy.dst_subresource.base_layer + layer),
aspect_info.attachment_aspect);
const VkOffset2D dst_offset{copy.dst_offset.x, copy.dst_offset.y};
const VkExtent2D dst_extent{copy.extent.width, copy.extent.height};
const VkRect2D render_area{
.offset = dst_offset,
.extent = dst_extent,
};
vk::Framebuffer framebuffer = device.GetLogical().CreateFramebuffer(VkFramebufferCreateInfo{
.sType = VK_STRUCTURE_TYPE_FRAMEBUFFER_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.renderPass = renderpass,
.attachmentCount = 1,
.pAttachments = dst_view.address(),
.width = static_cast<u32>(dst_offset.x) + dst_extent.width,
.height = static_cast<u32>(dst_offset.y) + dst_extent.height,
.layers = 1,
});
const MSAACopyPushConstants push_constants{
.dst_offset = {dst_offset.x, dst_offset.y},
.src_offset = {copy.src_offset.x, copy.src_offset.y},
.scale = {scale_x, scale_y},
};
VkImageView src_stencil_handle = VK_NULL_HANDLE;
if (copy_stencil) {
src_stencil_handle = *src_stencil_view;
}
scheduler.RequestOutsideRenderPassOperationContext();
scheduler.Record([this, pipeline, layout, sampler, renderpass,
framebuffer_handle = *framebuffer, src_view_handle = *src_view,
src_stencil_handle, src = src_image, dst = dst_image, render_area,
aspect_info, push_constants](vk::CommandBuffer cmdbuf) {
const VkImageSubresourceRange barrier_range{
.aspectMask = aspect_info.barrier_aspect,
.baseMipLevel = 0,
.levelCount = VK_REMAINING_MIP_LEVELS,
.baseArrayLayer = 0,
.layerCount = VK_REMAINING_ARRAY_LAYERS,
};
const std::array pre_barriers{
VkImageMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = aspect_info.pre_src_access,
.dstAccessMask = aspect_info.pre_src_dst_access,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = src,
.subresourceRange = barrier_range,
},
VkImageMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = aspect_info.pre_src_access,
.dstAccessMask = aspect_info.pre_dst_dst_access,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst,
.subresourceRange = barrier_range,
},
};
cmdbuf.PipelineBarrier(aspect_info.pre_src_stages, aspect_info.pre_dst_stages, 0,
nullptr, nullptr, pre_barriers);
const VkRenderPassBeginInfo renderpass_bi{
.sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO,
.pNext = nullptr,
.renderPass = renderpass,
.framebuffer = framebuffer_handle,
.renderArea = render_area,
.clearValueCount = 0,
.pClearValues = nullptr,
};
cmdbuf.BeginRenderPass(renderpass_bi, VK_SUBPASS_CONTENTS_INLINE);
VkDescriptorSet descriptor_set = VK_NULL_HANDLE;
if (src_stencil_handle != VK_NULL_HANDLE) {
descriptor_set = two_textures_descriptor_allocator.Commit();
UpdateTwoTexturesDescriptorSet(device, descriptor_set, sampler, src_view_handle,
src_stencil_handle);
} else {
descriptor_set = one_texture_descriptor_allocator.Commit();
UpdateOneTextureDescriptorSet(device, descriptor_set, sampler, src_view_handle);
}
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
const VkViewport viewport{
.x = static_cast<float>(render_area.offset.x),
.y = static_cast<float>(render_area.offset.y),
.width = static_cast<float>(render_area.extent.width),
.height = static_cast<float>(render_area.extent.height),
.minDepth = 0.0f,
.maxDepth = 1.0f,
};
cmdbuf.SetViewport(0, viewport);
cmdbuf.SetScissor(0, render_area);
cmdbuf.PushConstants(layout, VK_SHADER_STAGE_FRAGMENT_BIT, push_constants);
cmdbuf.Draw(3, 1, 0, 0);
cmdbuf.EndRenderPass();
const VkImageMemoryBarrier post_barrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = aspect_info.post_src_access,
.dstAccessMask = aspect_info.post_dst_access,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst,
.subresourceRange = barrier_range,
};
cmdbuf.PipelineBarrier(aspect_info.post_src_stages, aspect_info.post_dst_stages, 0,
post_barrier);
});
msaa_copy_resources.push_back(MSAACopyResources{
.tick = scheduler.CurrentTick(),
.src_view = std::move(src_view),
.dst_view = std::move(dst_view),
.framebuffer = std::move(framebuffer),
});
if (copy_stencil) {
msaa_copy_resources.push_back(MSAACopyResources{
.tick = scheduler.CurrentTick(),
.src_view = std::move(src_stencil_view),
.dst_view = vk::ImageView{},
.framebuffer = vk::Framebuffer{},
});
}
}
}
scheduler.InvalidateState();
}
void BlitImageHelper::CopyMSAA(RenderPassCache& render_pass_cache, VkImage dst_image, void BlitImageHelper::CopyMSAA(RenderPassCache& render_pass_cache, VkImage dst_image,
VideoCore::Surface::PixelFormat dst_format, VkImage src_image, VideoCore::Surface::PixelFormat dst_format, VkImage src_image,
VideoCore::Surface::PixelFormat src_format, u32 num_samples, VideoCore::Surface::PixelFormat src_format, u32 num_samples,
std::span<const VideoCommon::ImageCopy> copies, std::span<const VideoCommon::ImageCopy> copies,
bool msaa_to_non_msaa) { bool msaa_to_non_msaa) {
while (!msaa_copy_resources.empty() && scheduler.IsFree(msaa_copy_resources.front().tick)) {
msaa_copy_resources.pop_front();
}
const auto [samples_x, samples_y] = VideoCommon::SamplesLog2(static_cast<int>(num_samples)); const auto [samples_x, samples_y] = VideoCommon::SamplesLog2(static_cast<int>(num_samples));
const s32 scale_x = 1 << samples_x; const s32 scale_x = 1 << samples_x;
const s32 scale_y = 1 << samples_y; const s32 scale_y = 1 << samples_y;
const VkSampleCountFlagBits samples = VkSampleCountFlagBits samples = SampleCountFlag(num_samples);
msaa_to_non_msaa ? VK_SAMPLE_COUNT_1_BIT : SampleCountFlag(num_samples); if (msaa_to_non_msaa) {
samples = VK_SAMPLE_COUNT_1_BIT;
}
RenderPassKey renderpass_key{}; RenderPassKey renderpass_key{};
renderpass_key.color_formats.fill(VideoCore::Surface::PixelFormat::Invalid); renderpass_key.color_formats.fill(VideoCore::Surface::PixelFormat::Invalid);
renderpass_key.color_formats[0] = dst_format; renderpass_key.color_formats[0] = dst_format;
@@ -905,149 +1109,35 @@ void BlitImageHelper::CopyMSAA(RenderPassCache& render_pass_cache, VkImage dst_i
.renderpass = renderpass, .renderpass = renderpass,
.samples = samples, .samples = samples,
.msaa_to_non_msaa = msaa_to_non_msaa, .msaa_to_non_msaa = msaa_to_non_msaa,
.format_class = FormatClass(dst_format),
};
const MSAACopyAspectInfo aspect_info{
.src_view_aspect = VK_IMAGE_ASPECT_COLOR_BIT,
.attachment_aspect = VK_IMAGE_ASPECT_COLOR_BIT,
.barrier_aspect = VK_IMAGE_ASPECT_COLOR_BIT,
.pre_src_access = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT | VK_ACCESS_SHADER_WRITE_BIT |
VK_ACCESS_TRANSFER_WRITE_BIT,
.pre_src_dst_access = VK_ACCESS_SHADER_READ_BIT,
.pre_dst_dst_access =
VK_ACCESS_COLOR_ATTACHMENT_READ_BIT | VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
.pre_src_stages = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT |
VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT |
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT | VK_PIPELINE_STAGE_TRANSFER_BIT,
.pre_dst_stages =
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT | VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
.post_src_access = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
.post_dst_access = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_TRANSFER_READ_BIT,
.post_src_stages = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
.post_dst_stages = VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT |
VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT | VK_PIPELINE_STAGE_TRANSFER_BIT,
}; };
const VkPipeline pipeline = FindOrEmplaceMSAACopyPipeline(key);
const VkPipelineLayout layout = *msaa_copy_pipeline_layout;
const VkSampler sampler = *nearest_sampler;
const VkFormat src_vk_format = const VkFormat src_vk_format =
MaxwellToVK::SurfaceFormat(device, FormatType::Optimal, true, src_format).format; MaxwellToVK::SurfaceFormat(device, FormatType::Optimal, true, src_format).format;
const VkFormat dst_vk_format = const VkFormat dst_vk_format =
MaxwellToVK::SurfaceFormat(device, FormatType::Optimal, true, dst_format).format; MaxwellToVK::SurfaceFormat(device, FormatType::Optimal, true, dst_format).format;
for (const VideoCommon::ImageCopy& copy : copies) { CopyMSAAImpl(renderpass, FindOrEmplaceMSAACopyPipeline(key), *msaa_copy_pipeline_layout,
ASSERT(copy.src_subresource.base_layer == 0); dst_image, dst_vk_format, src_image, src_vk_format, scale_x, scale_y, copies,
ASSERT(copy.src_subresource.num_layers == 1); aspect_info, false);
ASSERT(copy.dst_subresource.base_layer == 0);
ASSERT(copy.dst_subresource.num_layers == 1);
vk::ImageView src_view =
MakeMSAACopyView(device.GetLogical(), src_image, src_vk_format,
static_cast<u32>(copy.src_subresource.base_level));
vk::ImageView dst_view =
MakeMSAACopyView(device.GetLogical(), dst_image, dst_vk_format,
static_cast<u32>(copy.dst_subresource.base_level));
const VkOffset2D dst_offset{copy.dst_offset.x, copy.dst_offset.y};
const VkExtent2D dst_extent{copy.extent.width, copy.extent.height};
const VkRect2D render_area{
.offset = dst_offset,
.extent = dst_extent,
};
vk::Framebuffer framebuffer = device.GetLogical().CreateFramebuffer(VkFramebufferCreateInfo{
.sType = VK_STRUCTURE_TYPE_FRAMEBUFFER_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.renderPass = renderpass,
.attachmentCount = 1,
.pAttachments = dst_view.address(),
.width = static_cast<u32>(dst_offset.x) + dst_extent.width,
.height = static_cast<u32>(dst_offset.y) + dst_extent.height,
.layers = 1,
});
const MSAACopyPushConstants push_constants{
.dst_offset = {dst_offset.x, dst_offset.y},
.src_offset = {copy.src_offset.x, copy.src_offset.y},
.scale = {scale_x, scale_y},
};
scheduler.RequestOutsideRenderPassOperationContext();
scheduler.Record([this, pipeline, layout, sampler, renderpass,
framebuffer_handle = *framebuffer, src_view_handle = *src_view,
src = src_image, dst = dst_image, render_area,
push_constants](vk::CommandBuffer cmdbuf) {
constexpr VkImageSubresourceRange color_range{
.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT,
.baseMipLevel = 0,
.levelCount = VK_REMAINING_MIP_LEVELS,
.baseArrayLayer = 0,
.layerCount = VK_REMAINING_ARRAY_LAYERS,
};
const std::array pre_barriers{
VkImageMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT |
VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_SHADER_READ_BIT,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = src,
.subresourceRange = color_range,
},
VkImageMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT |
VK_ACCESS_SHADER_WRITE_BIT | VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_COLOR_ATTACHMENT_READ_BIT |
VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst,
.subresourceRange = color_range,
},
};
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT |
VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT |
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT |
VK_PIPELINE_STAGE_TRANSFER_BIT,
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT |
VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
0, nullptr, nullptr, pre_barriers);
const VkRenderPassBeginInfo renderpass_bi{
.sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO,
.pNext = nullptr,
.renderPass = renderpass,
.framebuffer = framebuffer_handle,
.renderArea = render_area,
.clearValueCount = 0,
.pClearValues = nullptr,
};
cmdbuf.BeginRenderPass(renderpass_bi, VK_SUBPASS_CONTENTS_INLINE);
const VkDescriptorSet descriptor_set = one_texture_descriptor_allocator.Commit();
UpdateOneTextureDescriptorSet(device, descriptor_set, sampler, src_view_handle);
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_GRAPHICS, layout, 0, descriptor_set,
nullptr);
const VkViewport viewport{
.x = static_cast<float>(render_area.offset.x),
.y = static_cast<float>(render_area.offset.y),
.width = static_cast<float>(render_area.extent.width),
.height = static_cast<float>(render_area.extent.height),
.minDepth = 0.0f,
.maxDepth = 1.0f,
};
cmdbuf.SetViewport(0, viewport);
cmdbuf.SetScissor(0, render_area);
cmdbuf.PushConstants(layout, VK_SHADER_STAGE_FRAGMENT_BIT, push_constants);
cmdbuf.Draw(3, 1, 0, 0);
cmdbuf.EndRenderPass();
const VkImageMemoryBarrier post_barrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,
.dstAccessMask = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_TRANSFER_READ_BIT,
.oldLayout = VK_IMAGE_LAYOUT_GENERAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst,
.subresourceRange = color_range,
};
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT |
VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT |
VK_PIPELINE_STAGE_TRANSFER_BIT,
0, post_barrier);
});
msaa_copy_resources.push_back(MSAACopyResources{
.tick = scheduler.CurrentTick(),
.src_view = std::move(src_view),
.dst_view = std::move(dst_view),
.framebuffer = std::move(framebuffer),
});
}
scheduler.InvalidateState();
} }
void BlitImageHelper::Convert(VkPipeline pipeline, const Framebuffer* dst_framebuffer, void BlitImageHelper::Convert(VkPipeline pipeline, const Framebuffer* dst_framebuffer,
@@ -1379,6 +1469,87 @@ VkPipeline BlitImageHelper::FindOrEmplaceBlitColorMSAAPipeline(const BlitMSAAPip
return *blit_msaa_color_pipelines.back(); return *blit_msaa_color_pipelines.back();
} }
VkPipeline BlitImageHelper::FindOrEmplaceBlitDepthStencilMSAAPipeline(
const BlitMSAAPipelineKey& key, bool blit_stencil) {
auto& keys = blit_stencil ? blit_msaa_depth_stencil_keys : blit_msaa_depth_keys;
auto& pipelines = blit_stencil ? blit_msaa_depth_stencil_pipelines : blit_msaa_depth_pipelines;
const auto it = std::ranges::find(keys, key);
if (it != keys.end()) {
return *pipelines[std::distance(keys.begin(), it)];
}
keys.push_back(key);
const std::array stages =
MakeStages(*full_screen_vert,
blit_stencil ? *blit_depth_stencil_msaa_frag : *blit_depth_msaa_frag);
const VkPipelineMultisampleStateCreateInfo multisample_ci{
.sType = VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.rasterizationSamples = key.samples,
.sampleShadingEnable = VK_TRUE,
.minSampleShading = 1.0f,
.pSampleMask = nullptr,
.alphaToCoverageEnable = VK_FALSE,
.alphaToOneEnable = VK_FALSE,
};
const VkPipelineInputAssemblyStateCreateInfo input_assembly_ci = GetPipelineInputAssemblyStateCreateInfo(device);
pipelines.push_back(device.GetLogical().CreateGraphicsPipeline({
.sType = VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.stageCount = static_cast<u32>(stages.size()),
.pStages = stages.data(),
.pVertexInputState = &PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO,
.pInputAssemblyState = &input_assembly_ci,
.pTessellationState = nullptr,
.pViewportState = &PIPELINE_VIEWPORT_STATE_CREATE_INFO,
.pRasterizationState = &PIPELINE_RASTERIZATION_STATE_CREATE_INFO,
.pMultisampleState = &multisample_ci,
.pDepthStencilState = blit_stencil ? &PIPELINE_DEPTH_STENCIL_STATE_CREATE_INFO
: &PIPELINE_DEPTH_ONLY_STATE_CREATE_INFO,
.pColorBlendState = &PIPELINE_COLOR_BLEND_STATE_EMPTY_CREATE_INFO,
.pDynamicState = &PIPELINE_DYNAMIC_STATE_CREATE_INFO,
.layout = blit_stencil ? *two_textures_pipeline_layout : *one_texture_pipeline_layout,
.renderPass = key.renderpass,
.subpass = 0,
.basePipelineHandle = VK_NULL_HANDLE,
.basePipelineIndex = 0,
}));
return *pipelines.back();
}
VkPipeline BlitImageHelper::FindOrEmplaceBlitDepthPipeline(VkRenderPass renderpass) {
const auto it = std::ranges::find(blit_depth_keys, renderpass);
if (it != blit_depth_keys.end()) {
return *blit_depth_pipelines[std::distance(blit_depth_keys.begin(), it)];
}
blit_depth_keys.push_back(renderpass);
const std::array stages = MakeStages(*full_screen_vert, *blit_depth_frag);
const VkPipelineInputAssemblyStateCreateInfo input_assembly_ci = GetPipelineInputAssemblyStateCreateInfo(device);
blit_depth_pipelines.push_back(device.GetLogical().CreateGraphicsPipeline({
.sType = VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.stageCount = static_cast<u32>(stages.size()),
.pStages = stages.data(),
.pVertexInputState = &PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO,
.pInputAssemblyState = &input_assembly_ci,
.pTessellationState = nullptr,
.pViewportState = &PIPELINE_VIEWPORT_STATE_CREATE_INFO,
.pRasterizationState = &PIPELINE_RASTERIZATION_STATE_CREATE_INFO,
.pMultisampleState = &PIPELINE_MULTISAMPLE_STATE_CREATE_INFO,
.pDepthStencilState = &PIPELINE_DEPTH_ONLY_STATE_CREATE_INFO,
.pColorBlendState = &PIPELINE_COLOR_BLEND_STATE_EMPTY_CREATE_INFO,
.pDynamicState = &PIPELINE_DYNAMIC_STATE_CREATE_INFO,
.layout = *one_texture_pipeline_layout,
.renderPass = renderpass,
.subpass = 0,
.basePipelineHandle = VK_NULL_HANDLE,
.basePipelineIndex = 0,
}));
return *blit_depth_pipelines.back();
}
VkPipeline BlitImageHelper::FindOrEmplaceResolveDepthStencilPipeline(VkRenderPass renderpass, VkPipeline BlitImageHelper::FindOrEmplaceResolveDepthStencilPipeline(VkRenderPass renderpass,
bool resolve_stencil) { bool resolve_stencil) {
auto& keys = resolve_stencil ? resolve_depth_stencil_keys : resolve_depth_keys; auto& keys = resolve_stencil ? resolve_depth_stencil_keys : resolve_depth_keys;
@@ -1417,15 +1588,83 @@ VkPipeline BlitImageHelper::FindOrEmplaceResolveDepthStencilPipeline(VkRenderPas
return *pipelines.back(); return *pipelines.back();
} }
void BlitImageHelper::CopyMSAADepth(RenderPassCache& render_pass_cache, VkImage dst_image,
VideoCore::Surface::PixelFormat dst_format, VkImage src_image,
VideoCore::Surface::PixelFormat src_format, u32 num_samples,
std::span<const VideoCommon::ImageCopy> copies,
bool copy_stencil, bool msaa_to_non_msaa) {
const auto [samples_x, samples_y] = VideoCommon::SamplesLog2(static_cast<int>(num_samples));
const s32 scale_x = 1 << samples_x;
const s32 scale_y = 1 << samples_y;
VkSampleCountFlagBits samples = SampleCountFlag(num_samples);
if (msaa_to_non_msaa) {
samples = VK_SAMPLE_COUNT_1_BIT;
}
RenderPassKey renderpass_key{};
renderpass_key.color_formats.fill(VideoCore::Surface::PixelFormat::Invalid);
renderpass_key.depth_format = dst_format;
renderpass_key.samples = samples;
const VkRenderPass renderpass = render_pass_cache.Get(renderpass_key);
const MSAACopyPipelineKey key{
.renderpass = renderpass,
.samples = samples,
.msaa_to_non_msaa = msaa_to_non_msaa,
.format_class = MSAACopyFormatClass::Float,
};
VkImageAspectFlags attachment_aspect = VK_IMAGE_ASPECT_DEPTH_BIT;
if (VideoCore::Surface::GetFormatType(dst_format) ==
VideoCore::Surface::SurfaceType::DepthStencil) {
attachment_aspect |= VK_IMAGE_ASPECT_STENCIL_BIT;
}
VkPipelineLayout layout = *msaa_copy_pipeline_layout;
if (copy_stencil) {
layout = *msaa_copy_depth_stencil_pipeline_layout;
}
const MSAACopyAspectInfo aspect_info{
.src_view_aspect = VK_IMAGE_ASPECT_DEPTH_BIT,
.attachment_aspect = attachment_aspect,
.barrier_aspect = attachment_aspect,
.pre_src_access =
VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT | VK_ACCESS_TRANSFER_WRITE_BIT,
.pre_src_dst_access = VK_ACCESS_SHADER_READ_BIT,
.pre_dst_dst_access = VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT |
VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT,
.pre_src_stages = VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT |
VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT |
VK_PIPELINE_STAGE_TRANSFER_BIT,
.pre_dst_stages =
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT | VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT,
.post_src_access = VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT,
.post_dst_access = VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_TRANSFER_READ_BIT |
VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_READ_BIT,
.post_src_stages = VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT,
.post_dst_stages = vk::PIPELINE_STAGE_GRAPHICS_COMPUTE_TRANSFER,
};
const VkFormat src_vk_format =
MaxwellToVK::SurfaceFormat(device, FormatType::Optimal, true, src_format).format;
const VkFormat dst_vk_format =
MaxwellToVK::SurfaceFormat(device, FormatType::Optimal, true, dst_format).format;
CopyMSAAImpl(renderpass, FindOrEmplaceMSAACopyDepthPipeline(key, copy_stencil), layout,
dst_image, dst_vk_format, src_image, src_vk_format, scale_x, scale_y, copies,
aspect_info, copy_stencil);
}
VkPipeline BlitImageHelper::FindOrEmplaceMSAACopyPipeline(const MSAACopyPipelineKey& key) { VkPipeline BlitImageHelper::FindOrEmplaceMSAACopyPipeline(const MSAACopyPipelineKey& key) {
const auto it = std::ranges::find(msaa_copy_keys, key); const auto it = std::ranges::find(msaa_copy_keys, key);
if (it != msaa_copy_keys.end()) { if (it != msaa_copy_keys.end()) {
return *msaa_copy_pipelines[std::distance(msaa_copy_keys.begin(), it)]; return *msaa_copy_pipelines[std::distance(msaa_copy_keys.begin(), it)];
} }
msaa_copy_keys.push_back(key); msaa_copy_keys.push_back(key);
const std::array stages = MakeStages(*clear_color_vert, key.msaa_to_non_msaa VkShaderModule frag_module = key.msaa_to_non_msaa ? *convert_msaa_to_non_msaa_frag
? *convert_msaa_to_non_msaa_frag : *convert_non_msaa_to_msaa_frag;
: *convert_non_msaa_to_msaa_frag); if (key.format_class == MSAACopyFormatClass::SignedInteger) {
frag_module = key.msaa_to_non_msaa ? *convert_msaa_to_non_msaa_sint_frag
: *convert_non_msaa_to_msaa_sint_frag;
} else if (key.format_class == MSAACopyFormatClass::UnsignedInteger) {
frag_module = key.msaa_to_non_msaa ? *convert_msaa_to_non_msaa_uint_frag
: *convert_non_msaa_to_msaa_uint_frag;
}
const std::array stages = MakeStages(*clear_color_vert, frag_module);
const VkPipelineMultisampleStateCreateInfo multisample_ci{ const VkPipelineMultisampleStateCreateInfo multisample_ci{
.sType = VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO, .sType = VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO,
.pNext = nullptr, .pNext = nullptr,
@@ -1462,6 +1701,85 @@ VkPipeline BlitImageHelper::FindOrEmplaceMSAACopyPipeline(const MSAACopyPipeline
return *msaa_copy_pipelines.back(); return *msaa_copy_pipelines.back();
} }
VkPipeline BlitImageHelper::FindOrEmplaceMSAACopyDepthPipeline(const MSAACopyPipelineKey& key,
bool copy_stencil) {
auto& keys = copy_stencil ? msaa_copy_depth_stencil_keys : msaa_copy_depth_keys;
auto& pipelines = copy_stencil ? msaa_copy_depth_stencil_pipelines : msaa_copy_depth_pipelines;
const auto it = std::ranges::find(keys, key);
if (it != keys.end()) {
return *pipelines[std::distance(keys.begin(), it)];
}
keys.push_back(key);
VkShaderModule frag_module;
if (key.msaa_to_non_msaa) {
frag_module = copy_stencil ? *convert_msaa_to_non_msaa_depth_stencil_frag
: *convert_msaa_to_non_msaa_depth_frag;
} else {
frag_module = copy_stencil ? *convert_non_msaa_to_msaa_depth_stencil_frag
: *convert_non_msaa_to_msaa_depth_frag;
}
const std::array stages = MakeStages(*clear_color_vert, frag_module);
const VkPipelineMultisampleStateCreateInfo multisample_ci{
.sType = VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.rasterizationSamples = key.samples,
.sampleShadingEnable = key.msaa_to_non_msaa ? VK_FALSE : VK_TRUE,
.minSampleShading = key.msaa_to_non_msaa ? 0.0f : 1.0f,
.pSampleMask = nullptr,
.alphaToCoverageEnable = VK_FALSE,
.alphaToOneEnable = VK_FALSE,
};
static constexpr VkStencilOpState REPLACE_STENCIL_OP{
.failOp = VK_STENCIL_OP_REPLACE,
.passOp = VK_STENCIL_OP_REPLACE,
.depthFailOp = VK_STENCIL_OP_REPLACE,
.compareOp = VK_COMPARE_OP_ALWAYS,
.compareMask = 0xFF,
.writeMask = 0xFF,
.reference = 0,
};
const VkPipelineDepthStencilStateCreateInfo depth_stencil_ci{
.sType = VK_STRUCTURE_TYPE_PIPELINE_DEPTH_STENCIL_STATE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.depthTestEnable = VK_TRUE,
.depthWriteEnable = VK_TRUE,
.depthCompareOp = VK_COMPARE_OP_ALWAYS,
.depthBoundsTestEnable = VK_FALSE,
.stencilTestEnable = copy_stencil ? VK_TRUE : VK_FALSE,
.front = copy_stencil ? REPLACE_STENCIL_OP : VkStencilOpState{},
.back = copy_stencil ? REPLACE_STENCIL_OP : VkStencilOpState{},
.minDepthBounds = 0.0f,
.maxDepthBounds = 0.0f,
};
const VkPipelineInputAssemblyStateCreateInfo input_assembly_ci =
GetPipelineInputAssemblyStateCreateInfo(device);
pipelines.push_back(device.GetLogical().CreateGraphicsPipeline({
.sType = VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO,
.pNext = nullptr,
.flags = 0,
.stageCount = static_cast<u32>(stages.size()),
.pStages = stages.data(),
.pVertexInputState = &PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO,
.pInputAssemblyState = &input_assembly_ci,
.pTessellationState = nullptr,
.pViewportState = &PIPELINE_VIEWPORT_STATE_CREATE_INFO,
.pRasterizationState = &PIPELINE_RASTERIZATION_STATE_CREATE_INFO,
.pMultisampleState = &multisample_ci,
.pDepthStencilState = &depth_stencil_ci,
.pColorBlendState = &PIPELINE_COLOR_BLEND_STATE_EMPTY_CREATE_INFO,
.pDynamicState = &PIPELINE_DYNAMIC_STATE_CREATE_INFO,
.layout = copy_stencil ? *msaa_copy_depth_stencil_pipeline_layout
: *msaa_copy_pipeline_layout,
.renderPass = key.renderpass,
.subpass = 0,
.basePipelineHandle = VK_NULL_HANDLE,
.basePipelineIndex = 0,
}));
return *pipelines.back();
}
void BlitImageHelper::ConvertDepthToColorPipeline(vk::Pipeline& pipeline, VkRenderPass renderpass) { void BlitImageHelper::ConvertDepthToColorPipeline(vk::Pipeline& pipeline, VkRenderPass renderpass) {
ConvertPipeline(pipeline, renderpass, false); ConvertPipeline(pipeline, renderpass, false);
} }
@@ -45,12 +45,19 @@ struct BlitDepthStencilPipelineKey {
u32 stencil_ref; u32 stencil_ref;
}; };
enum class MSAACopyFormatClass : u32 {
Float,
SignedInteger,
UnsignedInteger,
};
struct MSAACopyPipelineKey { struct MSAACopyPipelineKey {
constexpr auto operator<=>(const MSAACopyPipelineKey&) const noexcept = default; constexpr auto operator<=>(const MSAACopyPipelineKey&) const noexcept = default;
VkRenderPass renderpass; VkRenderPass renderpass;
VkSampleCountFlagBits samples; VkSampleCountFlagBits samples;
bool msaa_to_non_msaa; bool msaa_to_non_msaa;
MSAACopyFormatClass format_class;
}; };
struct BlitMSAAPipelineKey { struct BlitMSAAPipelineKey {
@@ -78,6 +85,12 @@ public:
void BlitColorMSAA(const Framebuffer* dst_framebuffer, const ImageView& src_image_view, void BlitColorMSAA(const Framebuffer* dst_framebuffer, const ImageView& src_image_view,
const Region2D& dst_region, const Region2D& src_region); const Region2D& dst_region, const Region2D& src_region);
void BlitDepthStencilMSAA(const Framebuffer* dst_framebuffer, ImageView& src_image_view,
const Region2D& dst_region, const Region2D& src_region);
void BlitDepth(const Framebuffer* dst_framebuffer, ImageView& src_image_view,
const Region2D& dst_region, const Region2D& src_region);
void ResolveDepthStencil(const Framebuffer* dst_framebuffer, ImageView& src_image_view, void ResolveDepthStencil(const Framebuffer* dst_framebuffer, ImageView& src_image_view,
const Region2D& dst_region, const Region2D& src_region); const Region2D& dst_region, const Region2D& src_region);
@@ -116,7 +129,39 @@ public:
VideoCore::Surface::PixelFormat src_format, u32 num_samples, VideoCore::Surface::PixelFormat src_format, u32 num_samples,
std::span<const VideoCommon::ImageCopy> copies, bool msaa_to_non_msaa); std::span<const VideoCommon::ImageCopy> copies, bool msaa_to_non_msaa);
void CopyMSAADepth(RenderPassCache& render_pass_cache, VkImage dst_image,
VideoCore::Surface::PixelFormat dst_format, VkImage src_image,
VideoCore::Surface::PixelFormat src_format, u32 num_samples,
std::span<const VideoCommon::ImageCopy> copies, bool copy_stencil,
bool msaa_to_non_msaa);
private: private:
struct MSAACopyAspectInfo {
VkImageAspectFlags src_view_aspect;
VkImageAspectFlags attachment_aspect;
VkImageAspectFlags barrier_aspect;
VkAccessFlags pre_src_access;
VkAccessFlags pre_src_dst_access;
VkAccessFlags pre_dst_dst_access;
VkPipelineStageFlags pre_src_stages;
VkPipelineStageFlags pre_dst_stages;
VkAccessFlags post_src_access;
VkAccessFlags post_dst_access;
VkPipelineStageFlags post_src_stages;
VkPipelineStageFlags post_dst_stages;
};
void BlitImpl(const Framebuffer* dst_framebuffer, const ImageView& src_image_view,
const Region2D& dst_region, const Region2D& src_region, VkPipeline pipeline,
VkSampler sampler, VkImageView src_view, VkImageView src_stencil_view,
bool blit_stencil);
void CopyMSAAImpl(VkRenderPass renderpass, VkPipeline pipeline, VkPipelineLayout layout,
VkImage dst_image, VkFormat dst_vk_format, VkImage src_image,
VkFormat src_vk_format, s32 scale_x, s32 scale_y,
std::span<const VideoCommon::ImageCopy> copies,
const MSAACopyAspectInfo& aspect_info, bool copy_stencil);
void Convert(VkPipeline pipeline, const Framebuffer* dst_framebuffer, void Convert(VkPipeline pipeline, const Framebuffer* dst_framebuffer,
const ImageView& src_image_view); const ImageView& src_image_view);
@@ -131,7 +176,13 @@ private:
[[nodiscard]] VkPipeline FindOrEmplaceClearStencilPipeline( [[nodiscard]] VkPipeline FindOrEmplaceClearStencilPipeline(
const BlitDepthStencilPipelineKey& key); const BlitDepthStencilPipelineKey& key);
[[nodiscard]] VkPipeline FindOrEmplaceMSAACopyPipeline(const MSAACopyPipelineKey& key); [[nodiscard]] VkPipeline FindOrEmplaceMSAACopyPipeline(const MSAACopyPipelineKey& key);
[[nodiscard]] VkPipeline FindOrEmplaceMSAACopyDepthPipeline(const MSAACopyPipelineKey& key,
bool copy_stencil);
[[nodiscard]] VkPipeline FindOrEmplaceBlitColorMSAAPipeline(const BlitMSAAPipelineKey& key); [[nodiscard]] VkPipeline FindOrEmplaceBlitColorMSAAPipeline(const BlitMSAAPipelineKey& key);
[[nodiscard]] VkPipeline FindOrEmplaceBlitDepthStencilMSAAPipeline(
const BlitMSAAPipelineKey& key, bool blit_stencil);
[[nodiscard]] VkPipeline FindOrEmplaceBlitDepthPipeline(VkRenderPass renderpass);
[[nodiscard]] VkPipeline FindOrEmplaceResolveDepthStencilPipeline(VkRenderPass renderpass, [[nodiscard]] VkPipeline FindOrEmplaceResolveDepthStencilPipeline(VkRenderPass renderpass,
bool resolve_stencil); bool resolve_stencil);
@@ -162,10 +213,12 @@ private:
vk::PipelineLayout two_textures_pipeline_layout; vk::PipelineLayout two_textures_pipeline_layout;
vk::PipelineLayout clear_color_pipeline_layout; vk::PipelineLayout clear_color_pipeline_layout;
vk::PipelineLayout msaa_copy_pipeline_layout; vk::PipelineLayout msaa_copy_pipeline_layout;
vk::PipelineLayout msaa_copy_depth_stencil_pipeline_layout;
vk::ShaderModule full_screen_vert; vk::ShaderModule full_screen_vert;
vk::ShaderModule blit_color_to_color_frag; vk::ShaderModule blit_color_to_color_frag;
vk::ShaderModule blit_color_msaa_frag; vk::ShaderModule blit_color_msaa_frag;
vk::ShaderModule blit_depth_stencil_frag; vk::ShaderModule blit_depth_stencil_frag;
vk::ShaderModule blit_depth_frag;
vk::ShaderModule blit_depth_msaa_frag; vk::ShaderModule blit_depth_msaa_frag;
vk::ShaderModule blit_depth_stencil_msaa_frag; vk::ShaderModule blit_depth_stencil_msaa_frag;
vk::ShaderModule clear_color_vert; vk::ShaderModule clear_color_vert;
@@ -179,7 +232,15 @@ private:
vk::ShaderModule convert_d24s8_to_abgr8_frag; vk::ShaderModule convert_d24s8_to_abgr8_frag;
vk::ShaderModule convert_s8d24_to_abgr8_frag; vk::ShaderModule convert_s8d24_to_abgr8_frag;
vk::ShaderModule convert_msaa_to_non_msaa_frag; vk::ShaderModule convert_msaa_to_non_msaa_frag;
vk::ShaderModule convert_msaa_to_non_msaa_sint_frag;
vk::ShaderModule convert_msaa_to_non_msaa_uint_frag;
vk::ShaderModule convert_msaa_to_non_msaa_depth_frag;
vk::ShaderModule convert_msaa_to_non_msaa_depth_stencil_frag;
vk::ShaderModule convert_non_msaa_to_msaa_frag; vk::ShaderModule convert_non_msaa_to_msaa_frag;
vk::ShaderModule convert_non_msaa_to_msaa_sint_frag;
vk::ShaderModule convert_non_msaa_to_msaa_uint_frag;
vk::ShaderModule convert_non_msaa_to_msaa_depth_frag;
vk::ShaderModule convert_non_msaa_to_msaa_depth_stencil_frag;
vk::Sampler linear_sampler; vk::Sampler linear_sampler;
vk::Sampler nearest_sampler; vk::Sampler nearest_sampler;
@@ -193,8 +254,18 @@ private:
std::vector<vk::Pipeline> clear_stencil_pipelines; std::vector<vk::Pipeline> clear_stencil_pipelines;
std::vector<MSAACopyPipelineKey> msaa_copy_keys; std::vector<MSAACopyPipelineKey> msaa_copy_keys;
std::vector<vk::Pipeline> msaa_copy_pipelines; std::vector<vk::Pipeline> msaa_copy_pipelines;
std::vector<MSAACopyPipelineKey> msaa_copy_depth_keys;
std::vector<vk::Pipeline> msaa_copy_depth_pipelines;
std::vector<MSAACopyPipelineKey> msaa_copy_depth_stencil_keys;
std::vector<vk::Pipeline> msaa_copy_depth_stencil_pipelines;
std::vector<BlitMSAAPipelineKey> blit_msaa_color_keys; std::vector<BlitMSAAPipelineKey> blit_msaa_color_keys;
std::vector<vk::Pipeline> blit_msaa_color_pipelines; std::vector<vk::Pipeline> blit_msaa_color_pipelines;
std::vector<VkRenderPass> blit_depth_keys;
std::vector<vk::Pipeline> blit_depth_pipelines;
std::vector<BlitMSAAPipelineKey> blit_msaa_depth_keys;
std::vector<vk::Pipeline> blit_msaa_depth_pipelines;
std::vector<BlitMSAAPipelineKey> blit_msaa_depth_stencil_keys;
std::vector<vk::Pipeline> blit_msaa_depth_stencil_pipelines;
std::vector<VkRenderPass> resolve_depth_keys; std::vector<VkRenderPass> resolve_depth_keys;
std::vector<vk::Pipeline> resolve_depth_pipelines; std::vector<vk::Pipeline> resolve_depth_pipelines;
std::vector<VkRenderPass> resolve_depth_stencil_keys; std::vector<VkRenderPass> resolve_depth_stencil_keys;
@@ -370,20 +370,9 @@ inline void PushImageDescriptors(TextureCache& texture_cache,
const VkImageView null_image_view{texture_cache.GetImageView(VideoCommon::NULL_IMAGE_VIEW_ID).Handle(desc.type)}; const VkImageView null_image_view{texture_cache.GetImageView(VideoCommon::NULL_IMAGE_VIEW_ID).Handle(desc.type)};
if (null_image_view != VK_NULL_HANDLE) vk_image_view = null_image_view; if (null_image_view != VK_NULL_HANDLE) vk_image_view = null_image_view;
} }
const Sampler& sampler{texture_cache.GetSampler(sampler_id)}; Sampler& sampler{texture_cache.GetSampler(sampler_id)};
const bool use_fallback_sampler{sampler.HasAddedAnisotropy() && guest_descriptor_queue.AddSampledImage(vk_image_view,
!image_view.SupportsAnisotropy()}; sampler.HandleFor(image_view, desc.is_depth));
VkSampler vk_sampler{use_fallback_sampler ? sampler.HandleWithDefaultAnisotropy()
: sampler.Handle()};
if (sampler.HasLinearFiltering() &&
VideoCore::Surface::IsPixelFormatInteger(image_view.format)) {
vk_sampler = sampler.HandleWithNearestFilter();
}
if (desc.is_depth && sampler.HasDepthComparison() &&
!image_view.SupportsDepthComparison()) {
vk_sampler = sampler.HandleWithoutDepthComparison();
}
guest_descriptor_queue.AddSampledImage(vk_image_view, vk_sampler);
const bool element_rescaled{texture_cache.IsRescaling(image_view)}; const bool element_rescaled{texture_cache.IsRescaling(image_view)};
is_rescaled |= element_rescaled; is_rescaled |= element_rescaled;
} }
+18 -14
View File
@@ -1,6 +1,8 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project // SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later // SPDX-License-Identifier: GPL-3.0-or-later
#include <algorithm>
#include "common/common_types.h" #include "common/common_types.h"
#include "common/div_ceil.h" #include "common/div_ceil.h"
#include "common/settings.h" #include "common/settings.h"
@@ -17,7 +19,7 @@
namespace Vulkan { namespace Vulkan {
using PushConstants = std::array<u32, 4 + 2 + 1>; using PushConstants = std::array<u32, 4 + 2 + 2 + 1>;
SGSR::SGSR(const Device& device, MemoryAllocator& memory_allocator, size_t image_count, VkExtent2D extent, bool edge_dir) SGSR::SGSR(const Device& device, MemoryAllocator& memory_allocator, size_t image_count, VkExtent2D extent, bool edge_dir)
: m_memory_allocator{memory_allocator} : m_memory_allocator{memory_allocator}
@@ -100,26 +102,28 @@ VkImageView SGSR::Draw(const Device& device, Scheduler& scheduler, size_t image_
const f32 input_image_width = f32(input_image_extent.width); const f32 input_image_width = f32(input_image_extent.width);
const f32 input_image_height = f32(input_image_extent.height); const f32 input_image_height = f32(input_image_extent.height);
const f32 viewport_width = (crop_rect.right - crop_rect.left) * input_image_width; const f32 crop_width = (crop_rect.right - crop_rect.left) * input_image_width;
const f32 viewport_height = (crop_rect.bottom - crop_rect.top) * input_image_height; const f32 crop_height = (crop_rect.bottom - crop_rect.top) * input_image_height;
// expected [0, 2] static constexpr f32 EDGE_SHARPNESS_MAX = 2.0f;
const f32 sharpening = f32(Settings::values.fsr_sharpening_slider.GetValue()) / 100.0f; const f32 edge_sharpness =
EDGE_SHARPNESS_MAX - f32(Settings::values.fsr_sharpening_slider.GetValue()) / 200.0f;
// p = (tex * viewport) / input = [0,n] (normalized texcoords)
// p * input = [0,1024], [0,768]
// layout( push_constant ) uniform constants { // layout( push_constant ) uniform constants {
// highp vec4 ViewportInfo[1]; // highp vec4 ViewportInfo[1];
// highp vec2 ResizeFactor; // highp vec2 ResizeFactor;
// highp vec2 CropOffset;
// highp float EdgeSharpness; // highp float EdgeSharpness;
// }; // };
PushConstants viewport_con{}; PushConstants viewport_con{};
viewport_con[0] = std::bit_cast<u32>(std::abs(1.f / viewport_width)); viewport_con[0] = std::bit_cast<u32>(1.f / input_image_width);
viewport_con[1] = std::bit_cast<u32>(std::abs(1.f / viewport_height)); viewport_con[1] = std::bit_cast<u32>(1.f / input_image_height);
viewport_con[2] = std::bit_cast<u32>(std::abs(viewport_width)); viewport_con[2] = std::bit_cast<u32>(input_image_width);
viewport_con[3] = std::bit_cast<u32>(std::abs(viewport_height)); viewport_con[3] = std::bit_cast<u32>(input_image_height);
viewport_con[4] = std::bit_cast<u32>(viewport_width / input_image_width); viewport_con[4] = std::bit_cast<u32>(crop_width / input_image_width);
viewport_con[5] = std::bit_cast<u32>(viewport_height / input_image_height); viewport_con[5] = std::bit_cast<u32>(crop_height / input_image_height);
viewport_con[6] = std::bit_cast<u32>(sharpening); viewport_con[6] = std::bit_cast<u32>((std::min)(crop_rect.left, crop_rect.right));
viewport_con[7] = std::bit_cast<u32>((std::min)(crop_rect.top, crop_rect.bottom));
viewport_con[8] = std::bit_cast<u32>(edge_sharpness);
UploadImages(device, scheduler); UploadImages(device, scheduler);
UpdateDescriptorSets(device, source_image_view, image_index); UpdateDescriptorSets(device, source_image_view, image_index);
@@ -7,7 +7,9 @@
#include <algorithm> #include <algorithm>
#include <array> #include <array>
#include <cstring> #include <cstring>
#include <limits>
#include <span> #include <span>
#include <utility>
#include <vector> #include <vector>
#include "video_core/buffer_cache/buffer_cache_base.h" #include "video_core/buffer_cache/buffer_cache_base.h"
@@ -32,6 +34,32 @@ VkBufferCopy MakeBufferCopy(const VideoCommon::BufferCopy& copy) {
}; };
} }
constexpr size_t MAX_WINDOW_BARRIER_RANGES = 8;
using WindowRange = std::pair<VkDeviceSize, VkDeviceSize>;
using WindowRanges = boost::container::small_vector<WindowRange, MAX_WINDOW_BARRIER_RANGES>;
void CoalesceWindowRanges(WindowRanges& ranges) {
if (ranges.size() < 2) {
return;
}
std::sort(ranges.begin(), ranges.end());
size_t merged = 0;
for (size_t index = 1; index < ranges.size(); ++index) {
if (ranges[index].first <= ranges[merged].second) {
ranges[merged].second = (std::max)(ranges[merged].second, ranges[index].second);
} else {
ranges[++merged] = ranges[index];
}
}
ranges.resize(merged + 1);
if (ranges.size() > MAX_WINDOW_BARRIER_RANGES) {
const WindowRange bounding{ranges.front().first, ranges.back().second};
ranges.clear();
ranges.push_back(bounding);
}
}
VkIndexType IndexTypeFromNumElements(const Device& device, u32 num_elements) { VkIndexType IndexTypeFromNumElements(const Device& device, u32 num_elements) {
if (num_elements <= 0xff && device.IsExtIndexTypeUint8Supported()) { if (num_elements <= 0xff && device.IsExtIndexTypeUint8Supported()) {
return VK_INDEX_TYPE_UINT8_EXT; return VK_INDEX_TYPE_UINT8_EXT;
@@ -42,6 +70,16 @@ VkIndexType IndexTypeFromNumElements(const Device& device, u32 num_elements) {
return VK_INDEX_TYPE_UINT32; return VK_INDEX_TYPE_UINT32;
} }
u32 GrowIndexCount(u32 current, u32 requested) {
constexpr u32 MinimumIndices = 4096;
constexpr u32 GrowthLimit = (std::numeric_limits<u32>::max)() / 2;
u32 grown = (std::max)(requested, MinimumIndices);
if (current <= GrowthLimit) {
grown = (std::max)(grown, current * 2);
}
return grown;
}
size_t BytesPerIndex(VkIndexType index_type) { size_t BytesPerIndex(VkIndexType index_type) {
switch (index_type) { switch (index_type) {
case VK_INDEX_TYPE_UINT8_EXT: case VK_INDEX_TYPE_UINT8_EXT:
@@ -158,13 +196,12 @@ public:
virtual ~QuadIndexBuffer() = default; virtual ~QuadIndexBuffer() = default;
void UpdateBuffer(u32 num_indices_) { void UpdateBuffer(u32 num_indices_) {
ReleaseRetiredBuffers();
if (num_indices_ <= num_indices) { if (num_indices_ <= num_indices) {
return; return;
} }
scheduler.Finish(); num_indices = GrowIndexCount(num_indices, num_indices_);
num_indices = num_indices_;
index_type = IndexTypeFromNumElements(device, num_indices); index_type = IndexTypeFromNumElements(device, num_indices);
const u32 num_quads = GetQuadsNum(num_indices); const u32 num_quads = GetQuadsNum(num_indices);
@@ -182,6 +219,12 @@ public:
.queueFamilyIndexCount = 0, .queueFamilyIndexCount = 0,
.pQueueFamilyIndices = nullptr, .pQueueFamilyIndices = nullptr,
}; };
if (buffer) {
retired_buffers.push_back(RetiredBuffer{
.buffer = std::move(buffer),
.tick = scheduler.CurrentTick(),
});
}
buffer = memory_allocator.CreateBuffer(buffer_ci, MemoryUsage::DeviceLocal); buffer = memory_allocator.CreateBuffer(buffer_ci, MemoryUsage::DeviceLocal);
if (device.HasDebuggingToolAttached()) { if (device.HasDebuggingToolAttached()) {
buffer.SetObjectNameEXT("Quad LUT"); buffer.SetObjectNameEXT("Quad LUT");
@@ -249,6 +292,17 @@ protected:
virtual void MakeAndUpdateIndices(u8* staging_data, size_t quad_size, u32 quad, u32 first) = 0; virtual void MakeAndUpdateIndices(u8* staging_data, size_t quad_size, u32 quad, u32 first) = 0;
struct RetiredBuffer {
vk::Buffer buffer;
u64 tick;
};
void ReleaseRetiredBuffers() {
std::erase_if(retired_buffers, [this](const RetiredBuffer& entry) {
return scheduler.IsFree(entry.tick);
});
}
const Device& device; const Device& device;
MemoryAllocator& memory_allocator; MemoryAllocator& memory_allocator;
Scheduler& scheduler; Scheduler& scheduler;
@@ -256,6 +310,7 @@ protected:
vk::Buffer buffer{}; vk::Buffer buffer{};
MemoryCommit memory_commit{}; MemoryCommit memory_commit{};
std::vector<RetiredBuffer> retired_buffers;
VkIndexType index_type{}; VkIndexType index_type{};
u32 num_indices = 0; u32 num_indices = 0;
}; };
@@ -365,6 +420,138 @@ BufferCacheRuntime::BufferCacheRuntime(const Device& device_, MemoryAllocator& m
scheduler_, staging_pool_); scheduler_, staging_pool_);
} }
void BufferCacheRuntime::TryEnableUnifiedMemory(void* base, size_t size,
std::span<AHardwareBuffer* const> hardware_buffers,
size_t hardware_buffer_window,
size_t hardware_buffer_base) {
unified_memory = memory_allocator.CreateHostMemoryImport(
base, size, hardware_buffers, hardware_buffer_window, hardware_buffer_base);
}
void BufferCacheRuntime::CopyToUnifiedMemory(
size_t window_index, VkBuffer src_buffer,
std::span<const VideoCommon::BufferCopy> copies) {
if (!unified_memory || src_buffer == VK_NULL_HANDLE || copies.empty() ||
window_index >= unified_memory->GetWindowCount() ||
unified_memory->GetWindowBuffer(window_index) == VK_NULL_HANDLE) {
return;
}
PendingUnifiedCopy& pending = pending_unified_copies.emplace_back();
pending.window = window_index;
pending.buffer = src_buffer;
pending.copies.resize(copies.size());
std::ranges::transform(copies, pending.copies.begin(), MakeBufferCopy);
}
void BufferCacheRuntime::FlushUnifiedMemoryCopies() {
if (pending_unified_copies.empty()) {
return;
}
struct UnifiedCopyCommand {
VkBuffer buffer;
boost::container::small_vector<VkBufferCopy, 8> copies;
};
std::stable_sort(pending_unified_copies.begin(), pending_unified_copies.end(),
[](const PendingUnifiedCopy& lhs, const PendingUnifiedCopy& rhs) {
return lhs.window < rhs.window;
});
const bool foreign = unified_memory->NeedsForeignOwnershipTransfer();
const u32 queue_family = device.GetGraphicsFamily();
size_t group_begin = 0;
while (group_begin < pending_unified_copies.size()) {
const size_t window = pending_unified_copies[group_begin].window;
size_t group_end = group_begin;
while (group_end < pending_unified_copies.size() &&
pending_unified_copies[group_end].window == window) {
++group_end;
}
const VkBuffer window_buffer = unified_memory->GetWindowBuffer(window);
WindowRanges ranges;
for (size_t index = group_begin; index < group_end; ++index) {
for (const VkBufferCopy& copy : pending_unified_copies[index].copies) {
ranges.emplace_back(copy.dstOffset, copy.dstOffset + copy.size);
}
}
CoalesceWindowRanges(ranges);
boost::container::small_vector<VkBufferMemoryBarrier, MAX_WINDOW_BARRIER_RANGES> acquire;
boost::container::small_vector<VkBufferMemoryBarrier, MAX_WINDOW_BARRIER_RANGES> release;
if (foreign) {
for (const WindowRange& range : ranges) {
acquire.push_back(VkBufferMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = 0,
.dstAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_FOREIGN_EXT,
.dstQueueFamilyIndex = queue_family,
.buffer = window_buffer,
.offset = range.first,
.size = range.second - range.first,
});
release.push_back(VkBufferMemoryBarrier{
.sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = 0,
.srcQueueFamilyIndex = queue_family,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_FOREIGN_EXT,
.buffer = window_buffer,
.offset = range.first,
.size = range.second - range.first,
});
}
}
boost::container::small_vector<UnifiedCopyCommand, 4> commands;
commands.reserve(group_end - group_begin);
for (size_t index = group_begin; index < group_end; ++index) {
PendingUnifiedCopy& pending = pending_unified_copies[index];
commands.push_back(UnifiedCopyCommand{pending.buffer, std::move(pending.copies)});
}
scheduler.RequestOutsideRenderPassOperationContext();
scheduler.Record([window_buffer, acquire = std::move(acquire), release = std::move(release),
commands = std::move(commands)](vk::CommandBuffer cmdbuf) {
if (!acquire.empty()) {
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT,
VK_PIPELINE_STAGE_TRANSFER_BIT, 0, {},
VideoCommon::FixSmallVectorADL(acquire), {});
}
for (const UnifiedCopyCommand& command : commands) {
cmdbuf.CopyBuffer(command.buffer, window_buffer,
VideoCommon::FixSmallVectorADL(command.copies));
}
if (!release.empty()) {
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_TRANSFER_BIT,
VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, 0, {},
VideoCommon::FixSmallVectorADL(release), {});
}
});
group_begin = group_end;
}
pending_unified_copies.clear();
}
void BufferCacheRuntime::UnifiedMemoryHostBarrier() {
static constexpr VkMemoryBarrier HOST_BARRIER{
.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_HOST_READ_BIT,
};
scheduler.RequestOutsideRenderPassOperationContext();
scheduler.Record([](vk::CommandBuffer cmdbuf) {
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_HOST_BIT, 0,
HOST_BARRIER);
});
}
StagingBufferRef BufferCacheRuntime::UploadStagingBuffer(size_t size) { StagingBufferRef BufferCacheRuntime::UploadStagingBuffer(size_t size) {
return staging_pool.Request(size, MemoryUsage::Upload); return staging_pool.Request(size, MemoryUsage::Upload);
} }
@@ -402,6 +589,7 @@ u32 BufferCacheRuntime::GetStorageBufferAlignment() const {
} }
void BufferCacheRuntime::TickFrame(Common::SlotVector<Buffer>& slot_buffers) noexcept { void BufferCacheRuntime::TickFrame(Common::SlotVector<Buffer>& slot_buffers) noexcept {
FlushUnifiedMemoryCopies();
for (auto it = slot_buffers.begin(); it != slot_buffers.end(); it++) { for (auto it = slot_buffers.begin(); it != slot_buffers.end(); it++) {
if (scheduler.IsFree(it->LastUsageTick())) { if (scheduler.IsFree(it->LastUsageTick())) {
it->ResetUsageTracking(); it->ResetUsageTracking();
@@ -7,6 +7,10 @@
#pragma once #pragma once
#include <limits> #include <limits>
#include <memory>
#include <span>
#include <boost/container/small_vector.hpp>
#include "video_core/buffer_cache/buffer_cache_base.h" #include "video_core/buffer_cache/buffer_cache_base.h"
#include "video_core/buffer_cache/memory_tracker_base.h" #include "video_core/buffer_cache/memory_tracker_base.h"
@@ -97,6 +101,33 @@ public:
void TickFrame(Common::SlotVector<Buffer>& slot_buffers) noexcept; void TickFrame(Common::SlotVector<Buffer>& slot_buffers) noexcept;
void TryEnableUnifiedMemory(void* base, size_t size,
std::span<AHardwareBuffer* const> hardware_buffers,
size_t hardware_buffer_window, size_t hardware_buffer_base);
[[nodiscard]] bool HasUnifiedMemory() const noexcept {
return unified_memory != nullptr && unified_memory->IsValid();
}
[[nodiscard]] u64 UnifiedMemorySize() const noexcept {
return unified_memory ? unified_memory->GetSize() : 0;
}
[[nodiscard]] u64 UnifiedMemoryBase() const noexcept {
return unified_memory ? unified_memory->GetBaseOffset() : 0;
}
[[nodiscard]] u64 UnifiedMemoryWindowSize() const noexcept {
return unified_memory ? unified_memory->GetWindowSize() : 0;
}
void CopyToUnifiedMemory(size_t window_index, VkBuffer src_buffer,
std::span<const VideoCommon::BufferCopy> copies);
void FlushUnifiedMemoryCopies();
void UnifiedMemoryHostBarrier();
u64 CurrentTick(); u64 CurrentTick();
u64 KnownGpuTick(); u64 KnownGpuTick();
@@ -164,6 +195,29 @@ public:
BindBuffer(buffer, offset, size); BindBuffer(buffer, offset, size);
} }
void BindStorageBuffer(VkBuffer buffer, VkDeviceAddress address, u32 offset, u32 size,
[[maybe_unused]] bool is_written) {
guest_descriptor_queue.AddBuffer(buffer, address, offset, size);
}
[[nodiscard]] bool IsUnifiedMemoryBindable() const noexcept {
return unified_memory != nullptr && unified_memory->IsValid() &&
unified_memory->IsBindable();
}
[[nodiscard]] VkBuffer UnifiedWindowBuffer(size_t index) const noexcept {
return unified_memory->GetWindowBuffer(index);
}
[[nodiscard]] VkDeviceAddress UnifiedWindowAddress(size_t index) const noexcept {
return unified_memory->GetWindowAddress(index);
}
[[nodiscard]] bool IsUnifiedStorageRange(u32 size, u64 offset) const {
return size <= device.GetMaxStorageBufferRange() &&
(offset % device.GetStorageBufferAlignment()) == 0;
}
void BindTextureBuffer(Buffer& buffer, u32 offset, u32 size, void BindTextureBuffer(Buffer& buffer, u32 offset, u32 size,
VideoCore::Surface::PixelFormat format) { VideoCore::Surface::PixelFormat format) {
guest_descriptor_queue.AddTexelBuffer(buffer.View(offset, size, format), guest_descriptor_queue.AddTexelBuffer(buffer.View(offset, size, format),
@@ -180,6 +234,12 @@ public:
} }
private: private:
struct PendingUnifiedCopy {
size_t window;
VkBuffer buffer;
boost::container::small_vector<VkBufferCopy, 8> copies;
};
void BindBuffer(const Buffer& buffer, u32 offset, u32 size) { void BindBuffer(const Buffer& buffer, u32 offset, u32 size) {
const VkBuffer handle = buffer.Handle(); const VkBuffer handle = buffer.Handle();
if (handle == VK_NULL_HANDLE) { if (handle == VK_NULL_HANDLE) {
@@ -204,6 +264,8 @@ private:
std::shared_ptr<QuadStripIndexBuffer> quad_strip_index_buffer; std::shared_ptr<QuadStripIndexBuffer> quad_strip_index_buffer;
vk::Buffer null_buffer; vk::Buffer null_buffer;
HostMemoryImport* unified_memory{};
boost::container::small_vector<PendingUnifiedCopy, 8> pending_unified_copies;
std::unique_ptr<Uint8Pass> uint8_pass; std::unique_ptr<Uint8Pass> uint8_pass;
QuadIndexedPass quad_index_pass; QuadIndexedPass quad_index_pass;
@@ -226,6 +288,7 @@ struct BufferCacheParams {
static constexpr bool USE_MEMORY_MAPS = true; static constexpr bool USE_MEMORY_MAPS = true;
static constexpr bool SEPARATE_IMAGE_BUFFER_BINDINGS = false; static constexpr bool SEPARATE_IMAGE_BUFFER_BINDINGS = false;
static constexpr bool USE_MEMORY_MAPS_FOR_UPLOADS = true; static constexpr bool USE_MEMORY_MAPS_FOR_UPLOADS = true;
static constexpr bool USE_UNIFIED_MEMORY = true;
}; };
using BufferCache = VideoCommon::BufferCache<BufferCacheParams>; using BufferCache = VideoCommon::BufferCache<BufferCacheParams>;
@@ -4,6 +4,7 @@
// SPDX-FileCopyrightText: Copyright 2019 yuzu Emulator Project // SPDX-FileCopyrightText: Copyright 2019 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later // SPDX-License-Identifier: GPL-2.0-or-later
#include <algorithm>
#include <array> #include <array>
#include <memory> #include <memory>
#include <numeric> #include <numeric>
@@ -12,6 +13,7 @@
#include "video_core/renderer_vulkan/vk_texture_cache.h" #include "video_core/renderer_vulkan/vk_texture_cache.h"
#include "common/alignment.h"
#include "common/assert.h" #include "common/assert.h"
#include "common/common_types.h" #include "common/common_types.h"
#include "common/div_ceil.h" #include "common/div_ceil.h"
@@ -22,7 +24,9 @@
#include "video_core/host_shaders/resolve_conditional_render_comp_spv.h" #include "video_core/host_shaders/resolve_conditional_render_comp_spv.h"
#include "video_core/host_shaders/vulkan_quad_indexed_comp_spv.h" #include "video_core/host_shaders/vulkan_quad_indexed_comp_spv.h"
#include "video_core/host_shaders/vulkan_uint8_comp_spv.h" #include "video_core/host_shaders/vulkan_uint8_comp_spv.h"
#include "video_core/host_shaders/block_linear_unswizzle_2d_buffer_comp_spv.h"
#include "video_core/host_shaders/block_linear_unswizzle_3d_bcn_comp_spv.h" #include "video_core/host_shaders/block_linear_unswizzle_3d_bcn_comp_spv.h"
#include "video_core/host_shaders/block_linear_unswizzle_3d_buffer_comp_spv.h"
#include "video_core/renderer_vulkan/vk_compute_pass.h" #include "video_core/renderer_vulkan/vk_compute_pass.h"
#include "video_core/surface.h" #include "video_core/surface.h"
#include "video_core/renderer_vulkan/vk_descriptor_pool.h" #include "video_core/renderer_vulkan/vk_descriptor_pool.h"
@@ -872,4 +876,537 @@ void BlockLinearUnswizzle3DPass::UnswizzleChunk(
}); });
} }
namespace {
constexpr u32 BL2D_BINDING_INPUT_BUFFER = 0;
constexpr u32 BL2D_BINDING_OUTPUT_BUFFER = 1;
struct alignas(16) BlockLinearUnswizzle2DPushConstants {
std::array<u32, 3> dim;
u32 bytes_per_block_log2;
std::array<u32, 3> origin;
u32 layer_stride;
u32 block_size;
u32 x_shift;
u32 block_height;
u32 block_height_mask;
};
static_assert(sizeof(BlockLinearUnswizzle2DPushConstants) <= 128);
constexpr std::array<VkDescriptorSetLayoutBinding, 2> BL2D_BINDINGS{{
{
.binding = BL2D_BINDING_INPUT_BUFFER,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.descriptorCount = 1,
.stageFlags = VK_SHADER_STAGE_COMPUTE_BIT,
.pImmutableSamplers = nullptr,
},
{
.binding = BL2D_BINDING_OUTPUT_BUFFER,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.descriptorCount = 1,
.stageFlags = VK_SHADER_STAGE_COMPUTE_BIT,
.pImmutableSamplers = nullptr,
},
}};
constexpr std::array<VkDescriptorUpdateTemplateEntry, 2> BL2D_TEMPLATE{{
{
.dstBinding = BL2D_BINDING_INPUT_BUFFER,
.dstArrayElement = 0,
.descriptorCount = 1,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.offset = BL2D_BINDING_INPUT_BUFFER * sizeof(DescriptorUpdateEntry),
.stride = sizeof(DescriptorUpdateEntry),
},
{
.dstBinding = BL2D_BINDING_OUTPUT_BUFFER,
.dstArrayElement = 0,
.descriptorCount = 1,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.offset = BL2D_BINDING_OUTPUT_BUFFER * sizeof(DescriptorUpdateEntry),
.stride = sizeof(DescriptorUpdateEntry),
},
}};
constexpr DescriptorBankInfo BL2D_BANK_INFO{
.uniform_buffers = 0,
.storage_buffers = 2,
.texture_buffers = 0,
.image_buffers = 0,
.textures = 0,
.images = 0,
.score = 2,
};
} // Anonymous namespace
BlockLinearUnswizzle2DPass::BlockLinearUnswizzle2DPass(
const Device& device_, Scheduler& scheduler_, DescriptorPool& descriptor_pool_,
StagingBufferPool& staging_buffer_pool_,
ComputePassDescriptorQueue& compute_pass_descriptor_queue_)
: ComputePass(device_, scheduler_, descriptor_pool_, BL2D_BINDINGS, BL2D_TEMPLATE,
BL2D_BANK_INFO,
COMPUTE_PUSH_CONSTANT_RANGE<sizeof(BlockLinearUnswizzle2DPushConstants)>,
BLOCK_LINEAR_UNSWIZZLE_2D_BUFFER_COMP_SPV),
scheduler{scheduler_}, staging_buffer_pool{staging_buffer_pool_},
compute_pass_descriptor_queue{compute_pass_descriptor_queue_} {}
BlockLinearUnswizzle2DPass::~BlockLinearUnswizzle2DPass() = default;
bool BlockLinearUnswizzle2DPass::IsSupported(const Device& device,
const VideoCommon::ImageInfo& info) {
if (info.type != VideoCommon::ImageType::e2D) {
return false;
}
if (info.num_samples > 1) {
return false;
}
if (device.GetStorageBufferAlignment() > Tegra::Texture::GOB_SIZE) {
return false;
}
if (VideoCore::Surface::GetFormatType(info.format) !=
VideoCore::Surface::SurfaceType::ColorTexture) {
return false;
}
if (VideoCore::Surface::IsPixelFormatASTC(info.format) && !device.IsOptimalAstcSupported()) {
return false;
}
if (VideoCore::Surface::IsPixelFormatBCn(info.format) && !device.IsOptimalBcnSupported()) {
return false;
}
const u32 bytes_per_block = VideoCore::Surface::BytesPerBlock(info.format);
return bytes_per_block == 4 || bytes_per_block == 8 || bytes_per_block == 16;
}
void BlockLinearUnswizzle2DPass::Unswizzle(
Image& image, const StagingBufferRef& swizzled,
std::span<const VideoCommon::SwizzleParameters> swizzles) {
UnswizzleFrom(image, swizzled.buffer, swizzled.offset, swizzles);
}
void BlockLinearUnswizzle2DPass::UnswizzleFrom(
Image& image, VkBuffer source_buffer, VkDeviceSize source_offset,
std::span<const VideoCommon::SwizzleParameters> swizzles) {
const VkImage dst_image = image.Handle();
if (swizzles.empty() || source_buffer == VK_NULL_HANDLE || dst_image == VK_NULL_HANDLE) {
return;
}
const u32 layers = image.info.resources.layers;
const VkImageAspectFlags aspect = image.AspectMask();
const VkDeviceSize output_alignment =
(std::max)(device.GetStorageBufferAlignment(), VkDeviceSize{16});
VkDeviceSize total_output = 0;
for (const VideoCommon::SwizzleParameters& sw : swizzles) {
const auto params =
VideoCommon::Accelerated::MakeBlockLinearSwizzle2DParams(sw, image.info);
const VkDeviceSize level_size = static_cast<VkDeviceSize>(sw.num_tiles.width) *
sw.num_tiles.height * layers *
(1ULL << params.bytes_per_block_log2);
if (level_size == 0) {
continue;
}
total_output = Common::AlignUp(total_output, output_alignment) + level_size;
}
if (total_output == 0) {
return;
}
const StagingBufferRef output =
staging_buffer_pool.Request(static_cast<size_t>(total_output), MemoryUsage::DeviceLocal);
const VkBuffer out_buffer = output.buffer;
if (out_buffer == VK_NULL_HANDLE) {
return;
}
VkDeviceSize level_offset = 0;
scheduler.RequestOutsideRenderPassOperationContext();
VkAccessFlags pre_access = VK_ACCESS_NONE;
VkImageLayout pre_layout = VK_IMAGE_LAYOUT_UNDEFINED;
VkPipelineStageFlags pre_stage = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT;
if (image.ExchangeInitialization()) {
pre_access = VK_ACCESS_SHADER_READ_BIT;
pre_layout = VK_IMAGE_LAYOUT_GENERAL;
pre_stage = vk::PIPELINE_STAGE_GRAPHICS_COMPUTE;
}
scheduler.Record([dst_image, aspect, pre_access, pre_layout,
pre_stage](vk::CommandBuffer cmdbuf) {
const VkImageMemoryBarrier barrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = pre_access,
.dstAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.oldLayout = pre_layout,
.newLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst_image,
.subresourceRange{
.aspectMask = aspect,
.baseMipLevel = 0,
.levelCount = VK_REMAINING_MIP_LEVELS,
.baseArrayLayer = 0,
.layerCount = VK_REMAINING_ARRAY_LAYERS,
},
};
cmdbuf.PipelineBarrier(pre_stage, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, {}, {}, barrier);
});
for (const VideoCommon::SwizzleParameters& sw : swizzles) {
const auto params =
VideoCommon::Accelerated::MakeBlockLinearSwizzle2DParams(sw, image.info);
const u32 width = sw.num_tiles.width;
const u32 height = sw.num_tiles.height;
const u32 bytes_per_block = 1u << params.bytes_per_block_log2;
const VkDeviceSize output_size =
static_cast<VkDeviceSize>(width) * height * layers * bytes_per_block;
if (output_size == 0) {
continue;
}
const u32 level = static_cast<u32>(sw.level);
const u32 texel_width = (std::max)(1u, image.info.size.width >> level);
const u32 texel_height = (std::max)(1u, image.info.size.height >> level);
level_offset = Common::AlignUp(level_offset, output_alignment);
const VkDeviceSize out_offset = output.offset + level_offset;
level_offset += output_size;
BlockLinearUnswizzle2DPushConstants pc{};
pc.dim = {width, height, layers};
pc.bytes_per_block_log2 = params.bytes_per_block_log2;
pc.origin = params.origin;
pc.layer_stride = params.layer_stride;
pc.block_size = params.block_size;
pc.x_shift = params.x_shift;
pc.block_height = params.block_height;
pc.block_height_mask = params.block_height_mask;
compute_pass_descriptor_queue.Acquire(scheduler, 2);
compute_pass_descriptor_queue.AddBuffer(source_buffer, sw.buffer_offset + source_offset,
image.guest_size_bytes - sw.buffer_offset);
compute_pass_descriptor_queue.AddBuffer(out_buffer, out_offset, output_size);
const void* descriptor_data = compute_pass_descriptor_queue.UpdateData();
const VkDescriptorSet set = descriptor_allocator.Commit();
const u32 gx = Common::DivCeil(width, 16u);
const u32 gy = Common::DivCeil(height, 8u);
scheduler.Record([this, set, descriptor_data, pc, gx, gy, layers, output_size, out_buffer,
out_offset, dst_image, aspect, texel_width, texel_height,
level](vk::CommandBuffer cmdbuf) {
device.GetLogical().UpdateDescriptorSet(set, *descriptor_template, descriptor_data);
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_COMPUTE, *pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_COMPUTE, *layout, 0, set, {});
cmdbuf.PushConstants(*layout, VK_SHADER_STAGE_COMPUTE_BIT, 0, sizeof(pc), &pc);
cmdbuf.Dispatch(gx, gy, layers);
const VkBufferMemoryBarrier buffer_barrier{
.sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_SHADER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_TRANSFER_READ_BIT,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.buffer = out_buffer,
.offset = out_offset,
.size = output_size,
};
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT,
VK_PIPELINE_STAGE_TRANSFER_BIT, 0, {}, buffer_barrier, {});
const VkBufferImageCopy copy{
.bufferOffset = out_offset,
.bufferRowLength = 0,
.bufferImageHeight = 0,
.imageSubresource{
.aspectMask = aspect,
.mipLevel = level,
.baseArrayLayer = 0,
.layerCount = layers,
},
.imageOffset = {0, 0, 0},
.imageExtent = {texel_width, texel_height, 1},
};
cmdbuf.CopyBufferToImage(out_buffer, dst_image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
copy);
});
}
scheduler.Record([dst_image, aspect](vk::CommandBuffer cmdbuf) {
const VkImageMemoryBarrier barrier{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_SHADER_READ_BIT,
.oldLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst_image,
.subresourceRange{
.aspectMask = aspect,
.baseMipLevel = 0,
.levelCount = VK_REMAINING_MIP_LEVELS,
.baseArrayLayer = 0,
.layerCount = VK_REMAINING_ARRAY_LAYERS,
},
};
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_TRANSFER_BIT,
vk::PIPELINE_STAGE_GRAPHICS_COMPUTE, 0, {}, {}, barrier);
});
}
namespace {
constexpr u32 BL3DB_BINDING_INPUT_BUFFER = 0;
constexpr u32 BL3DB_BINDING_OUTPUT_BUFFER = 1;
struct alignas(16) BlockLinearUnswizzle3DBufferPushConstants {
std::array<u32, 3> dim;
u32 bytes_per_block_log2;
std::array<u32, 3> origin;
u32 slice_size;
u32 block_size;
u32 x_shift;
u32 block_height;
u32 block_height_mask;
u32 block_depth;
u32 block_depth_mask;
};
static_assert(sizeof(BlockLinearUnswizzle3DBufferPushConstants) <= 128);
constexpr std::array<VkDescriptorSetLayoutBinding, 2> BL3DB_BINDINGS{{
{
.binding = BL3DB_BINDING_INPUT_BUFFER,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.descriptorCount = 1,
.stageFlags = VK_SHADER_STAGE_COMPUTE_BIT,
.pImmutableSamplers = nullptr,
},
{
.binding = BL3DB_BINDING_OUTPUT_BUFFER,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.descriptorCount = 1,
.stageFlags = VK_SHADER_STAGE_COMPUTE_BIT,
.pImmutableSamplers = nullptr,
},
}};
constexpr std::array<VkDescriptorUpdateTemplateEntry, 2> BL3DB_TEMPLATE{{
{
.dstBinding = BL3DB_BINDING_INPUT_BUFFER,
.dstArrayElement = 0,
.descriptorCount = 1,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.offset = BL3DB_BINDING_INPUT_BUFFER * sizeof(DescriptorUpdateEntry),
.stride = sizeof(DescriptorUpdateEntry),
},
{
.dstBinding = BL3DB_BINDING_OUTPUT_BUFFER,
.dstArrayElement = 0,
.descriptorCount = 1,
.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER,
.offset = BL3DB_BINDING_OUTPUT_BUFFER * sizeof(DescriptorUpdateEntry),
.stride = sizeof(DescriptorUpdateEntry),
},
}};
constexpr DescriptorBankInfo BL3DB_BANK_INFO{
.uniform_buffers = 0,
.storage_buffers = 2,
.texture_buffers = 0,
.image_buffers = 0,
.textures = 0,
.images = 0,
.score = 2,
};
} // Anonymous namespace
BlockLinearUnswizzle3DBufferPass::BlockLinearUnswizzle3DBufferPass(
const Device& device_, Scheduler& scheduler_, DescriptorPool& descriptor_pool_,
StagingBufferPool& staging_buffer_pool_,
ComputePassDescriptorQueue& compute_pass_descriptor_queue_)
: ComputePass(device_, scheduler_, descriptor_pool_, BL3DB_BINDINGS, BL3DB_TEMPLATE,
BL3DB_BANK_INFO,
COMPUTE_PUSH_CONSTANT_RANGE<sizeof(BlockLinearUnswizzle3DBufferPushConstants)>,
BLOCK_LINEAR_UNSWIZZLE_3D_BUFFER_COMP_SPV),
scheduler{scheduler_}, staging_buffer_pool{staging_buffer_pool_},
compute_pass_descriptor_queue{compute_pass_descriptor_queue_} {}
BlockLinearUnswizzle3DBufferPass::~BlockLinearUnswizzle3DBufferPass() = default;
bool BlockLinearUnswizzle3DBufferPass::IsSupported(const Device& device,
const VideoCommon::ImageInfo& info) {
if (info.type != VideoCommon::ImageType::e3D) {
return false;
}
if (info.resources.levels != 1 || info.resources.layers != 1) {
return false;
}
if (info.num_samples > 1) {
return false;
}
if (info.size.depth <= 1) {
return false;
}
if (VideoCore::Surface::GetFormatType(info.format) !=
VideoCore::Surface::SurfaceType::ColorTexture) {
return false;
}
if (VideoCore::Surface::IsPixelFormatASTC(info.format)) {
return false;
}
if (VideoCore::Surface::IsPixelFormatBCn(info.format) && !device.IsOptimalBcnSupported()) {
return false;
}
const u32 bytes_per_block = VideoCore::Surface::BytesPerBlock(info.format);
return bytes_per_block == 4 || bytes_per_block == 8 || bytes_per_block == 16;
}
void BlockLinearUnswizzle3DBufferPass::Unswizzle(
Image& image, const StagingBufferRef& swizzled,
std::span<const VideoCommon::SwizzleParameters> swizzles) {
if (swizzles.empty()) {
return;
}
const VideoCommon::SwizzleParameters& sw = swizzles.front();
const auto params = VideoCommon::Accelerated::MakeBlockLinearSwizzle3DParams(sw, image.info);
const u32 blocks_x = sw.num_tiles.width;
const u32 blocks_y = sw.num_tiles.height;
const u32 blocks_z = sw.num_tiles.depth;
const u32 bytes_per_block = 1u << params.bytes_per_block_log2;
const VkDeviceSize output_size =
static_cast<VkDeviceSize>(blocks_x) * blocks_y * blocks_z * bytes_per_block;
const StagingBufferRef output =
staging_buffer_pool.Request(static_cast<size_t>(output_size), MemoryUsage::DeviceLocal);
BlockLinearUnswizzle3DBufferPushConstants pc{};
pc.dim = {blocks_x, blocks_y, blocks_z};
pc.bytes_per_block_log2 = params.bytes_per_block_log2;
pc.origin = params.origin;
pc.slice_size = params.slice_size;
pc.block_size = params.block_size;
pc.x_shift = params.x_shift;
pc.block_height = params.block_height;
pc.block_height_mask = params.block_height_mask;
pc.block_depth = params.block_depth;
pc.block_depth_mask = params.block_depth_mask;
scheduler.RequestOutsideRenderPassOperationContext();
compute_pass_descriptor_queue.Acquire(scheduler, 2);
compute_pass_descriptor_queue.AddBuffer(swizzled.buffer, sw.buffer_offset + swizzled.offset,
image.guest_size_bytes - sw.buffer_offset);
compute_pass_descriptor_queue.AddBuffer(output.buffer, output.offset, output_size);
const void* descriptor_data = compute_pass_descriptor_queue.UpdateData();
const VkDescriptorSet set = descriptor_allocator.Commit();
const u32 gx = Common::DivCeil(blocks_x, 8u);
const u32 gy = Common::DivCeil(blocks_y, 8u);
const u32 gz = Common::DivCeil(blocks_z, 4u);
const bool is_initialized = image.ExchangeInitialization();
const VkBuffer out_buffer = output.buffer;
const VkDeviceSize out_offset = output.offset;
const VkImage dst_image = image.Handle();
const VkImageAspectFlags aspect = image.AspectMask();
const VkExtent3D extent{
.width = image.info.size.width,
.height = image.info.size.height,
.depth = image.info.size.depth,
};
scheduler.Record([this, set, descriptor_data, pc, gx, gy, gz, output_size, out_buffer,
out_offset, dst_image, aspect, extent,
is_initialized](vk::CommandBuffer cmdbuf) {
if (dst_image == VK_NULL_HANDLE || out_buffer == VK_NULL_HANDLE) {
return;
}
device.GetLogical().UpdateDescriptorSet(set, *descriptor_template, descriptor_data);
cmdbuf.BindPipeline(VK_PIPELINE_BIND_POINT_COMPUTE, *pipeline);
cmdbuf.BindDescriptorSets(VK_PIPELINE_BIND_POINT_COMPUTE, *layout, 0, set, {});
cmdbuf.PushConstants(*layout, VK_SHADER_STAGE_COMPUTE_BIT, 0, sizeof(pc), &pc);
cmdbuf.Dispatch(gx, gy, gz);
const VkBufferMemoryBarrier buffer_barrier{
.sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_SHADER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_TRANSFER_READ_BIT,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.buffer = out_buffer,
.offset = out_offset,
.size = output_size,
};
VkAccessFlags pre_copy_access = VK_ACCESS_NONE;
VkImageLayout pre_copy_layout = VK_IMAGE_LAYOUT_UNDEFINED;
VkPipelineStageFlags pre_copy_stage = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT;
if (is_initialized) {
pre_copy_access = VK_ACCESS_SHADER_READ_BIT;
pre_copy_layout = VK_IMAGE_LAYOUT_GENERAL;
pre_copy_stage = vk::PIPELINE_STAGE_GRAPHICS_COMPUTE;
}
const VkImageMemoryBarrier pre_copy{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = pre_copy_access,
.dstAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.oldLayout = pre_copy_layout,
.newLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst_image,
.subresourceRange{
.aspectMask = aspect,
.baseMipLevel = 0,
.levelCount = VK_REMAINING_MIP_LEVELS,
.baseArrayLayer = 0,
.layerCount = VK_REMAINING_ARRAY_LAYERS,
},
};
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT | pre_copy_stage,
VK_PIPELINE_STAGE_TRANSFER_BIT, 0, {}, buffer_barrier, pre_copy);
const VkBufferImageCopy copy{
.bufferOffset = out_offset,
.bufferRowLength = 0,
.bufferImageHeight = 0,
.imageSubresource{
.aspectMask = aspect,
.mipLevel = 0,
.baseArrayLayer = 0,
.layerCount = 1,
},
.imageOffset = {0, 0, 0},
.imageExtent = extent,
};
cmdbuf.CopyBufferToImage(out_buffer, dst_image, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, copy);
const VkImageMemoryBarrier post_copy{
.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,
.pNext = nullptr,
.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT,
.dstAccessMask = VK_ACCESS_SHADER_READ_BIT,
.oldLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
.newLayout = VK_IMAGE_LAYOUT_GENERAL,
.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED,
.image = dst_image,
.subresourceRange{
.aspectMask = aspect,
.baseMipLevel = 0,
.levelCount = VK_REMAINING_MIP_LEVELS,
.baseArrayLayer = 0,
.layerCount = VK_REMAINING_ARRAY_LAYERS,
},
};
cmdbuf.PipelineBarrier(VK_PIPELINE_STAGE_TRANSFER_BIT, vk::PIPELINE_STAGE_GRAPHICS_COMPUTE,
0, {}, {}, post_copy);
});
}
} // namespace Vulkan } // namespace Vulkan
@@ -164,4 +164,46 @@ private:
ComputePassDescriptorQueue& compute_pass_descriptor_queue; ComputePassDescriptorQueue& compute_pass_descriptor_queue;
}; };
class BlockLinearUnswizzle2DPass final : public ComputePass {
public:
explicit BlockLinearUnswizzle2DPass(const Device& device_, Scheduler& scheduler_,
DescriptorPool& descriptor_pool_,
StagingBufferPool& staging_buffer_pool_,
ComputePassDescriptorQueue& compute_pass_descriptor_queue_);
~BlockLinearUnswizzle2DPass();
[[nodiscard]] static bool IsSupported(const Device& device,
const VideoCommon::ImageInfo& info);
void Unswizzle(Image& image, const StagingBufferRef& swizzled,
std::span<const VideoCommon::SwizzleParameters> swizzles);
void UnswizzleFrom(Image& image, VkBuffer source_buffer, VkDeviceSize source_offset,
std::span<const VideoCommon::SwizzleParameters> swizzles);
private:
Scheduler& scheduler;
StagingBufferPool& staging_buffer_pool;
ComputePassDescriptorQueue& compute_pass_descriptor_queue;
};
class BlockLinearUnswizzle3DBufferPass final : public ComputePass {
public:
explicit BlockLinearUnswizzle3DBufferPass(
const Device& device_, Scheduler& scheduler_, DescriptorPool& descriptor_pool_,
StagingBufferPool& staging_buffer_pool_,
ComputePassDescriptorQueue& compute_pass_descriptor_queue_);
~BlockLinearUnswizzle3DBufferPass();
[[nodiscard]] static bool IsSupported(const Device& device, const VideoCommon::ImageInfo& info);
void Unswizzle(Image& image, const StagingBufferRef& swizzled,
std::span<const VideoCommon::SwizzleParameters> swizzles);
private:
Scheduler& scheduler;
StagingBufferPool& staging_buffer_pool;
ComputePassDescriptorQueue& compute_pass_descriptor_queue;
};
} // namespace Vulkan } // namespace Vulkan
@@ -444,8 +444,10 @@ PipelineCache::PipelineCache(Tegra::MaxwellDeviceMemoryManager& device_memory_,
.has_broken_unsigned_image_offsets = false, .has_broken_unsigned_image_offsets = false,
.has_broken_signed_operations = false, .has_broken_signed_operations = false,
.has_broken_fp16_float_controls = driver_id == VK_DRIVER_ID_NVIDIA_PROPRIETARY, .has_broken_fp16_float_controls = driver_id == VK_DRIVER_ID_NVIDIA_PROPRIETARY,
.has_broken_fp32_denorm_flush = driver_id == VK_DRIVER_ID_QUALCOMM_PROPRIETARY,
.ignore_nan_fp_comparisons = false, .ignore_nan_fp_comparisons = false,
.has_broken_spirv_subgroup_mask_vector_extract_dynamic = false, .has_broken_spirv_subgroup_mask_vector_extract_dynamic = false,
.max_shared_memory_size = device.GetMaxComputeSharedMemorySize(),
.has_broken_robust = .has_broken_robust =
device.IsNvidia() && device.GetNvidiaArch() <= NvidiaArchitecture::Arch_Pascal, device.IsNvidia() && device.GetNvidiaArch() <= NvidiaArchitecture::Arch_Pascal,
.min_ssbo_alignment = device.GetStorageBufferAlignment(), .min_ssbo_alignment = device.GetStorageBufferAlignment(),
@@ -921,19 +923,6 @@ std::unique_ptr<ComputePipeline> PipelineCache::CreateComputePipeline(
} }
auto program{TranslateProgram(pools.inst, pools.block, env, cfg, host_info)}; auto program{TranslateProgram(pools.inst, pools.block, env, cfg, host_info)};
const VkDriverIdKHR driver_id = device.GetDriverID();
const bool needs_shared_mem_clamp =
driver_id == VK_DRIVER_ID_QUALCOMM_PROPRIETARY ||
driver_id == VK_DRIVER_ID_ARM_PROPRIETARY;
const u32 max_shared_memory = device.GetMaxComputeSharedMemorySize();
if (needs_shared_mem_clamp && program.shared_memory_size > max_shared_memory) {
LOG_WARNING(Render_Vulkan,
"Compute shader {:#016x} requests {}KB shared memory but device max is {}KB - clamping",
key.unique_hash,
program.shared_memory_size / 1024,
max_shared_memory / 1024);
program.shared_memory_size = max_shared_memory;
}
const std::vector<u32> code{EmitSPIRV(profile, program)}; const std::vector<u32> code{EmitSPIRV(profile, program)};
device.SaveShader(code); device.SaveShader(code);
vk::ShaderModule spv_module{BuildShader(device, code)}; vk::ShaderModule spv_module{BuildShader(device, code)};
@@ -225,6 +225,13 @@ RasterizerVulkan::RasterizerVulkan(Core::Frontend::EmuWindow& emu_window_, Tegra
fence_manager(*this, gpu, texture_cache, buffer_cache, query_cache, device, scheduler), fence_manager(*this, gpu, texture_cache, buffer_cache, query_cache, device, scheduler),
wfi_event(device.GetLogical().CreateEvent()) { wfi_event(device.GetLogical().CreateEvent()) {
scheduler.SetQueryCache(query_cache); scheduler.SetQueryCache(query_cache);
if (Settings::values.use_unified_memory.GetValue() && device_memory.IsBackingShared()) {
buffer_cache_runtime.TryEnableUnifiedMemory(
device_memory.GetPhysicalBase(), device_memory.GetPhysicalSize(),
device_memory.GetBackingHardwareBuffers(),
device_memory.GetBackingHardwareBufferWindowSize(),
device_memory.GetBackingHardwareBufferBase());
}
} }
RasterizerVulkan::~RasterizerVulkan() { RasterizerVulkan::~RasterizerVulkan() {
@@ -424,24 +431,12 @@ void RasterizerVulkan::Clear(u32 layer_count) {
const bool ds_deferrable = const bool ds_deferrable =
!ds_used || ((!framebuffer->HasAspectDepthBit() || use_depth) && !ds_used || ((!framebuffer->HasAspectDepthBit() || use_depth) &&
(!framebuffer->HasAspectStencilBit() || use_stencil) && !stencil_partial); (!framebuffer->HasAspectStencilBit() || use_stencil) && !stencil_partial);
const bool can_defer_clear = ENABLE_DEFERRED_CLEAR && !regs.clear_control.use_scissor &&
regs.clear_surface.layer == 0 &&
!scheduler.IsRenderPassActive() &&
(!use_color || color_full_channels) && ds_deferrable;
if (!can_defer_clear) {
scheduler.RequestRenderpass(framebuffer);
}
query_cache.NotifySegment(true);
query_cache.CounterEnable(VideoCommon::QueryType::ZPassPixelCount64, maxwell3d->regs.zpass_pixel_count_enable);
u32 up_scale = 1; u32 up_scale = 1;
u32 down_shift = 0; u32 down_shift = 0;
if (texture_cache.IsRescaling()) { if (texture_cache.IsRescaling()) {
up_scale = Settings::values.resolution_info.up_scale; up_scale = Settings::values.resolution_info.up_scale;
down_shift = Settings::values.resolution_info.down_shift; down_shift = Settings::values.resolution_info.down_shift;
} }
UpdateViewportsState(regs);
VkRect2D default_scissor{}; VkRect2D default_scissor{};
default_scissor.offset.x = 0; default_scissor.offset.x = 0;
default_scissor.offset.y = 0; default_scissor.offset.y = 0;
@@ -497,6 +492,22 @@ void RasterizerVulkan::Clear(u32 layer_count) {
return; return;
} }
const bool clear_covers_render_area =
clear_rect.rect.offset.x == 0 && clear_rect.rect.offset.y == 0 &&
clear_rect.rect.extent.width >= render_area.width &&
clear_rect.rect.extent.height >= render_area.height;
const bool can_defer_clear = ENABLE_DEFERRED_CLEAR && (!regs.clear_control.use_scissor || clear_covers_render_area) &&
regs.clear_surface.layer == 0 &&
!scheduler.IsRenderPassActive() &&
(!use_color || color_full_channels) && ds_deferrable;
if (!can_defer_clear) {
scheduler.RequestRenderpass(framebuffer);
}
query_cache.NotifySegment(true);
query_cache.CounterEnable(VideoCommon::QueryType::ZPassPixelCount64, maxwell3d->regs.zpass_pixel_count_enable);
UpdateViewportsState(regs);
const u32 color_attachment = regs.clear_surface.RT; const u32 color_attachment = regs.clear_surface.RT;
if (use_color && framebuffer->HasAspectColorBit(color_attachment)) { if (use_color && framebuffer->HasAspectColorBit(color_attachment)) {
const auto format = VideoCore::Surface::PixelFormatFromRenderTargetFormat(regs.rt[color_attachment].format); const auto format = VideoCore::Surface::PixelFormatFromRenderTargetFormat(regs.rt[color_attachment].format);
@@ -65,8 +65,72 @@ using VideoCore::Surface::SurfaceType;
.finalLayout = VK_IMAGE_LAYOUT_GENERAL, .finalLayout = VK_IMAGE_LAYOUT_GENERAL,
}; };
} }
struct ResolveAspects {
bool depth;
bool stencil;
};
struct ResolveModes {
VkResolveModeFlagBits depth;
VkResolveModeFlagBits stencil;
};
constexpr ResolveAspects GetResolveAspects(PixelFormat format) {
const SurfaceType surface_type = GetSurfaceType(format);
return ResolveAspects{
.depth = surface_type == SurfaceType::Depth ||
surface_type == SurfaceType::DepthStencil,
.stencil = surface_type == SurfaceType::Stencil ||
surface_type == SurfaceType::DepthStencil,
};
}
ResolveModes PickResolveModes(const Device& device, PixelFormat format) {
constexpr VkResolveModeFlagBits mode = VK_RESOLVE_MODE_SAMPLE_ZERO_BIT;
const ResolveAspects aspects = GetResolveAspects(format);
const bool depth_mode_supported = (device.GetDepthResolveModes() & mode) != 0;
const bool stencil_mode_supported = (device.GetStencilResolveModes() & mode) != 0;
ResolveModes modes{
.depth = VK_RESOLVE_MODE_NONE,
.stencil = VK_RESOLVE_MODE_NONE,
};
if (aspects.depth && depth_mode_supported) {
modes.depth = mode;
}
if (aspects.stencil && stencil_mode_supported) {
modes.stencil = mode;
}
if (modes.depth == modes.stencil || device.SupportsIndependentResolveNone()) {
return modes;
}
if (modes.depth != VK_RESOLVE_MODE_NONE && stencil_mode_supported) {
modes.stencil = mode;
} else if (modes.stencil != VK_RESOLVE_MODE_NONE && depth_mode_supported) {
modes.depth = mode;
}
return modes;
}
} // Anonymous namespace } // Anonymous namespace
bool SupportsDepthStencilResolve(const Device& device, PixelFormat depth_format) {
if (depth_format == PixelFormat::Invalid || !device.IsKhrDepthStencilResolveSupported()) {
return false;
}
const ResolveAspects aspects = GetResolveAspects(depth_format);
if (!aspects.depth && !aspects.stencil) {
return false;
}
const ResolveModes modes = PickResolveModes(device, depth_format);
if ((aspects.depth && modes.depth == VK_RESOLVE_MODE_NONE) ||
(aspects.stencil && modes.stencil == VK_RESOLVE_MODE_NONE)) {
return false;
}
return modes.depth == modes.stencil || device.SupportsIndependentResolveNone();
}
RenderPassCache::RenderPassCache(const Device& device_) : device{&device_} {} RenderPassCache::RenderPassCache(const Device& device_) : device{&device_} {}
VkRenderPass RenderPassCache::Get(const RenderPassKey& key) { VkRenderPass RenderPassCache::Get(const RenderPassKey& key) {
@@ -75,7 +139,9 @@ VkRenderPass RenderPassCache::Get(const RenderPassKey& key) {
if (!is_new) { if (!is_new) {
return *pair->second; return *pair->second;
} }
boost::container::static_vector<VkAttachmentDescription, 9> descriptions; static constexpr size_t MAX_ATTACHMENTS =
2 * std::tuple_size_v<decltype(RenderPassKey::color_formats)> + 2;
boost::container::static_vector<VkAttachmentDescription, MAX_ATTACHMENTS> descriptions;
std::array<VkAttachmentReference, 8> references{}; std::array<VkAttachmentReference, 8> references{};
u32 num_attachments{}; u32 num_attachments{};
u32 num_colors{}; u32 num_colors{};
@@ -109,8 +175,11 @@ VkRenderPass RenderPassCache::Get(const RenderPassKey& key) {
const VkAttachmentLoadOp depth_load_op = key.depth_stencil_clear const VkAttachmentLoadOp depth_load_op = key.depth_stencil_clear
? VK_ATTACHMENT_LOAD_OP_CLEAR ? VK_ATTACHMENT_LOAD_OP_CLEAR
: VK_ATTACHMENT_LOAD_OP_LOAD; : VK_ATTACHMENT_LOAD_OP_LOAD;
const VkAttachmentStoreOp depth_store_op = key.depth_stencil_discard
? VK_ATTACHMENT_STORE_OP_DONT_CARE
: VK_ATTACHMENT_STORE_OP_STORE;
descriptions.push_back(AttachmentDescription(*device, key.depth_format, key.samples, descriptions.push_back(AttachmentDescription(*device, key.depth_format, key.samples,
depth_load_op, VK_ATTACHMENT_STORE_OP_STORE)); depth_load_op, depth_store_op));
} }
std::array<VkAttachmentReference, 8> resolve_references{}; std::array<VkAttachmentReference, 8> resolve_references{};
const bool do_resolve_color = const bool do_resolve_color =
@@ -133,6 +202,21 @@ VkRenderPass RenderPassCache::Get(const RenderPassKey& key) {
} }
} }
} }
const bool do_resolve_depth_stencil = key.resolve_depth_stencil && has_depth &&
key.samples != VK_SAMPLE_COUNT_1_BIT &&
SupportsDepthStencilResolve(*device, key.depth_format);
VkAttachmentReference depth_resolve_reference{};
if (do_resolve_depth_stencil) {
depth_resolve_reference = VkAttachmentReference{
.attachment = static_cast<u32>(descriptions.size()),
.layout = VK_IMAGE_LAYOUT_GENERAL,
};
VkAttachmentDescription resolve_desc =
AttachmentDescription(*device, key.depth_format, VK_SAMPLE_COUNT_1_BIT,
VK_ATTACHMENT_LOAD_OP_DONT_CARE, VK_ATTACHMENT_STORE_OP_STORE);
resolve_desc.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
descriptions.push_back(resolve_desc);
}
const VkSubpassDescription subpass{ const VkSubpassDescription subpass{
.flags = 0, .flags = 0,
.pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS, .pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS,
@@ -157,6 +241,92 @@ VkRenderPass RenderPassCache::Get(const RenderPassKey& key) {
.dstAccessMask = VK_ACCESS_SHADER_READ_BIT, .dstAccessMask = VK_ACCESS_SHADER_READ_BIT,
.dependencyFlags = VK_DEPENDENCY_BY_REGION_BIT .dependencyFlags = VK_DEPENDENCY_BY_REGION_BIT
}; };
if (device->IsKhrCreateRenderPass2Supported()) {
boost::container::static_vector<VkAttachmentDescription2, MAX_ATTACHMENTS> descriptions2;
for (const VkAttachmentDescription& description : descriptions) {
descriptions2.push_back(VkAttachmentDescription2{
.sType = VK_STRUCTURE_TYPE_ATTACHMENT_DESCRIPTION_2,
.pNext = nullptr,
.flags = description.flags,
.format = description.format,
.samples = description.samples,
.loadOp = description.loadOp,
.storeOp = description.storeOp,
.stencilLoadOp = description.stencilLoadOp,
.stencilStoreOp = description.stencilStoreOp,
.initialLayout = description.initialLayout,
.finalLayout = description.finalLayout,
});
}
const auto promote = [](const VkAttachmentReference& reference) {
return VkAttachmentReference2{
.sType = VK_STRUCTURE_TYPE_ATTACHMENT_REFERENCE_2,
.pNext = nullptr,
.attachment = reference.attachment,
.layout = reference.layout,
.aspectMask = 0,
};
};
std::array<VkAttachmentReference2, 8> references2{};
std::array<VkAttachmentReference2, 8> resolve_references2{};
for (size_t index = 0; index < references.size(); ++index) {
references2[index] = promote(references[index]);
resolve_references2[index] = promote(resolve_references[index]);
}
const VkAttachmentReference2 depth_reference2 = promote(depth_reference);
const VkAttachmentReference2 depth_resolve_reference2 = promote(depth_resolve_reference);
const ResolveModes resolve_modes = PickResolveModes(*device, key.depth_format);
const VkSubpassDescriptionDepthStencilResolve depth_stencil_resolve{
.sType = VK_STRUCTURE_TYPE_SUBPASS_DESCRIPTION_DEPTH_STENCIL_RESOLVE,
.pNext = nullptr,
.depthResolveMode = resolve_modes.depth,
.stencilResolveMode = resolve_modes.stencil,
.pDepthStencilResolveAttachment = &depth_resolve_reference2,
};
const VkSubpassDescription2 subpass2{
.sType = VK_STRUCTURE_TYPE_SUBPASS_DESCRIPTION_2,
.pNext = do_resolve_depth_stencil ? &depth_stencil_resolve : nullptr,
.flags = 0,
.pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS,
.viewMask = 0,
.inputAttachmentCount = 0,
.pInputAttachments = nullptr,
.colorAttachmentCount = num_attachments,
.pColorAttachments = references2.data(),
.pResolveAttachments = do_resolve_color ? resolve_references2.data() : nullptr,
.pDepthStencilAttachment = has_depth ? &depth_reference2 : nullptr,
.preserveAttachmentCount = 0,
.pPreserveAttachments = nullptr,
};
const VkSubpassDependency2 dependency2{
.sType = VK_STRUCTURE_TYPE_SUBPASS_DEPENDENCY_2,
.pNext = nullptr,
.srcSubpass = dependency.srcSubpass,
.dstSubpass = dependency.dstSubpass,
.srcStageMask = dependency.srcStageMask,
.dstStageMask = dependency.dstStageMask,
.srcAccessMask = dependency.srcAccessMask,
.dstAccessMask = dependency.dstAccessMask,
.dependencyFlags = dependency.dependencyFlags,
.viewOffset = 0,
};
pair->second = device->GetLogical().CreateRenderPass2({
.sType = VK_STRUCTURE_TYPE_RENDER_PASS_CREATE_INFO_2,
.pNext = nullptr,
.flags = 0,
.attachmentCount = static_cast<u32>(descriptions2.size()),
.pAttachments = descriptions2.empty() ? nullptr : descriptions2.data(),
.subpassCount = 1,
.pSubpasses = &subpass2,
.dependencyCount = 1,
.pDependencies = &dependency2,
.correlatedViewMaskCount = 0,
.pCorrelatedViewMasks = nullptr,
});
return *pair->second;
}
pair->second = device->GetLogical().CreateRenderPass({ pair->second = device->GetLogical().CreateRenderPass({
.sType = VK_STRUCTURE_TYPE_RENDER_PASS_CREATE_INFO, .sType = VK_STRUCTURE_TYPE_RENDER_PASS_CREATE_INFO,
.pNext = nullptr, .pNext = nullptr,
@@ -9,6 +9,7 @@
#include <mutex> #include <mutex>
#include <ankerl/unordered_dense.h> #include <ankerl/unordered_dense.h>
#include "common/container_hash.h"
#include "video_core/surface.h" #include "video_core/surface.h"
#include "video_core/vulkan_common/vulkan_wrapper.h" #include "video_core/vulkan_common/vulkan_wrapper.h"
@@ -21,9 +22,11 @@ struct RenderPassKey {
VideoCore::Surface::PixelFormat depth_format; VideoCore::Surface::PixelFormat depth_format;
VkSampleCountFlagBits samples; VkSampleCountFlagBits samples;
bool resolve_color; bool resolve_color;
bool resolve_depth_stencil;
u32 color_clear_mask; u32 color_clear_mask;
bool depth_stencil_clear; bool depth_stencil_clear;
u32 color_discard_mask; u32 color_discard_mask;
bool depth_stencil_discard;
}; };
} // namespace Vulkan } // namespace Vulkan
@@ -31,17 +34,28 @@ struct RenderPassKey {
namespace std { namespace std {
template <> template <>
struct hash<Vulkan::RenderPassKey> { struct hash<Vulkan::RenderPassKey> {
static_assert(std::tuple_size_v<decltype(Vulkan::RenderPassKey::color_formats)> <= 8);
static_assert(static_cast<u32>(VideoCore::Surface::PixelFormat::Invalid) <= 0xFF);
static_assert(static_cast<u32>(VideoCore::Surface::PixelFormat::Max) <= 0xFF);
static_assert(VK_SAMPLE_COUNT_64_BIT <= 0xFF);
[[nodiscard]] size_t operator()(const Vulkan::RenderPassKey& key) const noexcept { [[nodiscard]] size_t operator()(const Vulkan::RenderPassKey& key) const noexcept {
size_t value = static_cast<size_t>(key.depth_format) << 48; u64 formats = 0;
value ^= static_cast<size_t>(key.samples) << 52; for (size_t index = 0; index < key.color_formats.size(); ++index) {
value ^= static_cast<size_t>(key.resolve_color) << 63; formats |= static_cast<u64>(key.color_formats[index]) << (index * 8);
value ^= static_cast<size_t>(key.color_clear_mask) << 54;
value ^= static_cast<size_t>(key.depth_stencil_clear) << 62;
value ^= static_cast<size_t>(key.color_discard_mask) << 24;
for (size_t i = 0; i < key.color_formats.size(); ++i) {
value ^= static_cast<size_t>(key.color_formats[i]) << (i * 6);
} }
return value; const u64 state = static_cast<u64>(key.depth_format) |
(static_cast<u64>(key.samples) << 8) |
(static_cast<u64>(key.color_clear_mask) << 16) |
(static_cast<u64>(key.color_discard_mask) << 24) |
(static_cast<u64>(key.resolve_color) << 32) |
(static_cast<u64>(key.depth_stencil_clear) << 33) |
(static_cast<u64>(key.resolve_depth_stencil) << 34) |
(static_cast<u64>(key.depth_stencil_discard) << 35);
size_t seed = 0;
Common::HashCombine(seed, formats);
Common::HashCombine(seed, state);
return seed;
} }
}; };
} // namespace std } // namespace std
@@ -50,6 +64,9 @@ namespace Vulkan {
class Device; class Device;
[[nodiscard]] bool SupportsDepthStencilResolve(const Device& device,
VideoCore::Surface::PixelFormat depth_format);
class RenderPassCache { class RenderPassCache {
public: public:
explicit RenderPassCache(const Device& device_); explicit RenderPassCache(const Device& device_);
@@ -132,6 +132,7 @@ void Scheduler::BeginRenderPassImpl(const Framebuffer* framebuffer, VkRenderPass
num_renderpass_images = framebuffer->NumImages(); num_renderpass_images = framebuffer->NumImages();
renderpass_images = framebuffer->Images(); renderpass_images = framebuffer->Images();
renderpass_image_ranges = framebuffer->ImageRanges(); renderpass_image_ranges = framebuffer->ImageRanges();
framebuffer->MarkResolveShadowsUpToDate();
} }
void Scheduler::RealizeDeferredClear() { void Scheduler::RealizeDeferredClear() {
@@ -155,8 +156,10 @@ void Scheduler::RealizeDeferredClear() {
} }
const u32 color_discard_mask = const u32 color_discard_mask =
dc.framebuffer->DiscardsMsaaColor() ? dc.color_clear_mask : 0u; dc.framebuffer->DiscardsMsaaColor() ? dc.color_clear_mask : 0u;
const bool depth_stencil_discard =
dc.depth_stencil && dc.framebuffer->DiscardsMsaaDepthStencil();
const VkRenderPass renderpass = dc.framebuffer->RenderPassVariant( const VkRenderPass renderpass = dc.framebuffer->RenderPassVariant(
dc.color_clear_mask, dc.depth_stencil, color_discard_mask); dc.color_clear_mask, dc.depth_stencil, color_discard_mask, depth_stencil_discard);
EndRenderPass(); EndRenderPass();
BeginRenderPassImpl(dc.framebuffer, renderpass, clear_values.data(), count); BeginRenderPassImpl(dc.framebuffer, renderpass, clear_values.data(), count);
} }
@@ -190,6 +193,14 @@ bool Scheduler::DeferDepthStencilClear(const Framebuffer* framebuffer, const VkC
return true; return true;
} }
void Scheduler::FlushDeferredClear() {
if (deferred_clear.framebuffer == nullptr) {
return;
}
RealizeDeferredClear();
EndRenderPass();
}
void Scheduler::RequestRenderpass(const Framebuffer* framebuffer) { void Scheduler::RequestRenderpass(const Framebuffer* framebuffer) {
if (deferred_clear.framebuffer == framebuffer) { if (deferred_clear.framebuffer == framebuffer) {
RealizeDeferredClear(); RealizeDeferredClear();
@@ -65,6 +65,9 @@ public:
/// Defers a full depth/stencil clear so it becomes the next render pass. /// Defers a full depth/stencil clear so it becomes the next render pass.
bool DeferDepthStencilClear(const Framebuffer* framebuffer, const VkClearValue& value); bool DeferDepthStencilClear(const Framebuffer* framebuffer, const VkClearValue& value);
/// Realizes any pending deferred clear before its framebuffer can be moved or freed.
void FlushDeferredClear();
/// Requests the current execution context to be able to execute operations only allowed outside /// Requests the current execution context to be able to execute operations only allowed outside
/// of a renderpass. /// of a renderpass.
void RequestOutsideRenderPassOperationContext(); void RequestOutsideRenderPassOperationContext();
@@ -126,6 +129,10 @@ public:
return master_semaphore->IsFree(tick); return master_semaphore->IsFree(tick);
} }
void RefreshTick() {
master_semaphore->Refresh();
}
/// Waits for the given GPU tick, optionally pacing frames. /// Waits for the given GPU tick, optionally pacing frames.
void Wait(u64 tick, double target_fps = 0.0) { void Wait(u64 tick, double target_fps = 0.0) {
if (tick > 0) { if (tick > 0) {
@@ -33,10 +33,10 @@ constexpr VkDeviceSize MAX_ALIGNMENT = 256;
// Windows ones however, can intake bigger buffers and generally do not OOM. // Windows ones however, can intake bigger buffers and generally do not OOM.
// - GTX 960 on Windows will not OOM with 256mib // - GTX 960 on Windows will not OOM with 256mib
// - GT 1030 on ^NIX will OOM with 256mib // - GT 1030 on ^NIX will OOM with 256mib
#if defined(_WIN32) || defined(__ANDROID__) #if defined(__FreeBSD__)
constexpr VkDeviceSize MAX_STREAM_BUFFER_SIZE = 256_MiB;
#else
constexpr VkDeviceSize MAX_STREAM_BUFFER_SIZE = 128_MiB; constexpr VkDeviceSize MAX_STREAM_BUFFER_SIZE = 128_MiB;
#else
constexpr VkDeviceSize MAX_STREAM_BUFFER_SIZE = 256_MiB;
#endif #endif
size_t GetStreamBufferSize(const Device& device) { size_t GetStreamBufferSize(const Device& device) {
File diff suppressed because it is too large Load Diff
+162 -42
View File
@@ -60,13 +60,19 @@ public:
void TickFrame(); void TickFrame();
void FlushDeferredClear();
u64 GetDeviceLocalMemory() const; u64 GetDeviceLocalMemory() const;
u64 GetDeviceMemoryUsage() const; u64 GetDeviceMemoryUsage() const;
bool CanReportMemoryUsage() const; bool CanReportMemoryUsage() const;
std::optional<size_t> GetSamplerHeapBudget() const; bool CanDownloadMsaa(const VideoCommon::ImageInfo& info) const;
[[nodiscard]] VkImage AcquireMsaaScratchImage(const VkImageCreateInfo& image_ci);
void ReleaseMsaaScratchImage(VkImage image);
void BlitImage(Framebuffer* dst_framebuffer, ImageView& dst, ImageView& src, void BlitImage(Framebuffer* dst_framebuffer, ImageView& dst, ImageView& src,
const Region2D& dst_region, const Region2D& src_region, const Region2D& dst_region, const Region2D& src_region,
@@ -95,6 +101,23 @@ public:
std::span<const VideoCommon::SwizzleParameters>, std::span<const VideoCommon::SwizzleParameters>,
u32 z_start, u32 z_count); u32 z_start, u32 z_count);
[[nodiscard]] bool IsUnifiedMemoryBindable() const noexcept;
[[nodiscard]] u64 UnifiedMemoryBase() const noexcept;
[[nodiscard]] u64 UnifiedMemorySize() const noexcept;
[[nodiscard]] u64 UnifiedMemoryWindowSize() const noexcept;
[[nodiscard]] bool CanUploadImageDirectly(const VideoCommon::ImageInfo& info) const;
bool UploadImageDirectly(Image& image, size_t window_index, u64 window_offset,
std::span<const VideoCommon::SwizzleParameters> swizzles);
[[nodiscard]] u64 CurrentTick() const noexcept;
[[nodiscard]] bool IsDirectUploadRetired(u64 tick);
void InsertUploadMemoryBarrier() {} void InsertUploadMemoryBarrier() {}
void TransitionImageLayout(Image& image); void TransitionImageLayout(Image& image);
@@ -117,16 +140,20 @@ public:
VkFormat format = VK_FORMAT_UNDEFINED; VkFormat format = VK_FORMAT_UNDEFINED;
VkExtent2D extent{}; VkExtent2D extent{};
u32 layers = 0; u32 layers = 0;
VkImageAspectFlags aspect_mask = VK_IMAGE_ASPECT_COLOR_BIT;
bool up_to_date = false; bool up_to_date = false;
}; };
[[nodiscard]] VkImageView GetOrCreateResolveShadow(VkImage msaa_image, VkFormat format, [[nodiscard]] VkImageView GetOrCreateResolveShadow(VkImage msaa_image, VkFormat format,
VkExtent2D extent, u32 layers); VkExtent2D extent, u32 layers,
VkImageAspectFlags aspect_mask);
[[nodiscard]] const ResolveShadow* GetValidResolveShadow(VkImage msaa_image) const; [[nodiscard]] const ResolveShadow* GetValidResolveShadow(VkImage msaa_image) const;
void InvalidateResolveShadow(VkImage msaa_image); void InvalidateResolveShadow(VkImage msaa_image);
void MarkResolveShadowUpToDate(VkImage msaa_image);
void EraseResolveShadow(VkImage msaa_image); void EraseResolveShadow(VkImage msaa_image);
std::span<const VkFormat> ViewFormats(PixelFormat format) { std::span<const VkFormat> ViewFormats(PixelFormat format) {
@@ -149,13 +176,37 @@ public:
std::optional<ASTCDecoderPass> astc_decoder_pass; std::optional<ASTCDecoderPass> astc_decoder_pass;
std::optional<BlockLinearUnswizzle3DPass> bl3d_unswizzle_pass; std::optional<BlockLinearUnswizzle3DPass> bl3d_unswizzle_pass;
std::optional<BlockLinearUnswizzle2DPass> bl2d_unswizzle_pass;
std::optional<BlockLinearUnswizzle3DBufferPass> bl3db_unswizzle_pass;
const Settings::ResolutionScalingInfo& resolution; const Settings::ResolutionScalingInfo& resolution;
std::array<std::vector<VkFormat>, VideoCore::Surface::MaxPixelFormat> view_formats; std::array<std::vector<VkFormat>, VideoCore::Surface::MaxPixelFormat> view_formats;
static constexpr size_t indexing_slots = 8 * sizeof(size_t); static constexpr size_t indexing_slots = 8 * sizeof(size_t);
std::array<vk::Buffer, indexing_slots> buffers{}; std::array<vk::Buffer, indexing_slots> buffers{};
std::vector<std::pair<u64, vk::Image>> pending_msaa_images; struct MsaaScratchKey {
VkFormat format;
VkImageType type;
u32 width;
u32 height;
u32 depth;
u32 levels;
u32 layers;
VkImageUsageFlags usage;
VkImageCreateFlags flags;
bool operator==(const MsaaScratchKey&) const noexcept = default;
};
struct MsaaScratchImage {
MsaaScratchKey key;
vk::Image image;
u64 tick;
u32 unused_frames;
};
std::vector<MsaaScratchImage> msaa_scratch_images;
ankerl::unordered_dense::map<VkImage, ResolveShadow> resolve_shadows; ankerl::unordered_dense::map<VkImage, ResolveShadow> resolve_shadows;
std::vector<std::pair<u64, ResolveShadow>> pending_resolve_shadows;
}; };
class Framebuffer { class Framebuffer {
@@ -191,7 +242,8 @@ public:
} }
[[nodiscard]] VkRenderPass RenderPassVariant(u32 color_clear_mask, bool depth_stencil_clear, [[nodiscard]] VkRenderPass RenderPassVariant(u32 color_clear_mask, bool depth_stencil_clear,
u32 color_discard_mask) const; u32 color_discard_mask,
bool depth_stencil_discard) const;
[[nodiscard]] VkExtent2D RenderArea() const noexcept { [[nodiscard]] VkExtent2D RenderArea() const noexcept {
return render_area; return render_area;
@@ -233,19 +285,21 @@ public:
return is_rescaled; return is_rescaled;
} }
[[nodiscard]] bool HasResolveColor() const noexcept {
return !resolve_images.empty();
}
[[nodiscard]] VkImage ResolveColorImage(size_t index) const noexcept {
return index < resolve_images.size() ? *resolve_images[index] : VK_NULL_HANDLE;
}
[[nodiscard]] bool DiscardsMsaaColor() const noexcept { [[nodiscard]] bool DiscardsMsaaColor() const noexcept {
return discard_msaa_color; return discard_msaa_color;
} }
[[nodiscard]] bool DiscardsMsaaDepthStencil() const noexcept {
return discard_msaa_depth_stencil;
}
/// Records that a render pass has begun, so its resolve attachments will hold valid contents
/// once it ends.
void MarkResolveShadowsUpToDate() const;
private: private:
static constexpr size_t NUM_MEMOIZED_RENDER_PASS_VARIANTS = 8;
vk::Framebuffer framebuffer; vk::Framebuffer framebuffer;
VkRenderPass renderpass{}; VkRenderPass renderpass{};
VkExtent2D render_area{}; VkExtent2D render_area{};
@@ -258,11 +312,16 @@ private:
bool has_depth{}; bool has_depth{};
bool has_stencil{}; bool has_stencil{};
bool is_rescaled{}; bool is_rescaled{};
std::vector<vk::Image> resolve_images; std::array<VkImage, 9> resolve_shadow_images{};
std::vector<vk::ImageView> resolve_image_views; u32 num_resolve_shadows = 0;
TextureCacheRuntime* runtime_ptr{nullptr};
RenderPassKey render_pass_key{}; RenderPassKey render_pass_key{};
RenderPassCache* render_pass_cache{nullptr}; RenderPassCache* render_pass_cache{nullptr};
bool discard_msaa_color{}; bool discard_msaa_color{};
bool discard_msaa_depth_stencil{};
mutable std::array<u32, NUM_MEMOIZED_RENDER_PASS_VARIANTS> variant_keys{};
mutable std::array<VkRenderPass, NUM_MEMOIZED_RENDER_PASS_VARIANTS> variant_render_passes{};
mutable u32 num_memoized_variants{};
}; };
class Image : public VideoCommon::ImageBase { class Image : public VideoCommon::ImageBase {
@@ -404,6 +463,22 @@ public:
return supports_depth_comparison; return supports_depth_comparison;
} }
[[nodiscard]] bool RequiresBorderColorFormat() const noexcept {
return requires_border_color_format;
}
[[nodiscard]] bool SupportsMinmaxFilter() const noexcept {
return supports_minmax_filter;
}
[[nodiscard]] const VkComponentMapping& Swizzle() const noexcept {
return swizzle_mapping;
}
[[nodiscard]] bool HasIdentitySwizzle() const noexcept {
return has_identity_swizzle;
}
[[nodiscard]] GPUVAddr GpuAddr() const noexcept { [[nodiscard]] GPUVAddr GpuAddr() const noexcept {
return gpu_addr; return gpu_addr;
} }
@@ -436,48 +511,91 @@ private:
VkSampleCountFlagBits samples = VK_SAMPLE_COUNT_1_BIT; VkSampleCountFlagBits samples = VK_SAMPLE_COUNT_1_BIT;
u32 buffer_size = 0; u32 buffer_size = 0;
VkComponentMapping swizzle_mapping{};
bool supports_depth_comparison = false; bool supports_depth_comparison = false;
bool requires_border_color_format = false;
bool supports_minmax_filter = false;
bool has_identity_swizzle = true;
}; };
class ImageAlloc : public VideoCommon::ImageAllocBase {}; class ImageAlloc : public VideoCommon::ImageAllocBase {};
class CustomBorderColorBudget {
public:
CustomBorderColorBudget() = default;
~CustomBorderColorBudget();
CustomBorderColorBudget(const CustomBorderColorBudget&) = delete;
CustomBorderColorBudget& operator=(const CustomBorderColorBudget&) = delete;
CustomBorderColorBudget(CustomBorderColorBudget&& rhs) noexcept;
CustomBorderColorBudget& operator=(CustomBorderColorBudget&& rhs) noexcept;
bool TryAcquire(const Device& device, size_t count);
private:
void Release() noexcept;
const Device* device_ptr = nullptr;
size_t held = 0;
};
class Sampler { class Sampler {
public: public:
explicit Sampler(TextureCacheRuntime&, const Tegra::Texture::TSCEntry&); explicit Sampler(TextureCacheRuntime&, const Tegra::Texture::TSCEntry&);
[[nodiscard]] VkSampler Handle() const noexcept { [[nodiscard]] VkSampler Handle() const noexcept {
return *sampler; return *variants.front().sampler;
} }
[[nodiscard]] VkSampler HandleWithDefaultAnisotropy() const noexcept { [[nodiscard]] VkSampler HandleFor(const ImageView& image_view, bool is_depth);
return *sampler_default_anisotropy;
}
[[nodiscard]] bool HasAddedAnisotropy() const noexcept {
return static_cast<bool>(sampler_default_anisotropy);
}
[[nodiscard]] VkSampler HandleWithNearestFilter() const noexcept {
return *sampler_nearest;
}
[[nodiscard]] bool HasLinearFiltering() const noexcept {
return static_cast<bool>(sampler_nearest);
}
[[nodiscard]] VkSampler HandleWithoutDepthComparison() const noexcept {
return *sampler_noncompare;
}
[[nodiscard]] bool HasDepthComparison() const noexcept {
return static_cast<bool>(sampler_noncompare);
}
private: private:
vk::Sampler sampler; struct VariantKey {
vk::Sampler sampler_default_anisotropy; bool reduce_anisotropy;
vk::Sampler sampler_nearest; bool force_nearest;
vk::Sampler sampler_noncompare; bool drop_depth_comparison;
bool drop_reduction;
bool drop_custom_border;
bool srgb_border;
std::array<VkComponentSwizzle, 4> swizzle;
bool operator==(const VariantKey&) const noexcept = default;
[[nodiscard]] bool HasSwizzle() const noexcept {
return swizzle != std::array<VkComponentSwizzle, 4>{};
}
};
struct Variant {
VariantKey key;
vk::Sampler sampler;
};
static constexpr size_t MAX_VARIANTS = 32;
[[nodiscard]] VariantKey MakeKey(const ImageView& image_view, bool is_depth) const noexcept;
[[nodiscard]] VkSampler Find(const VariantKey& key) const noexcept;
VkSampler Emplace(VariantKey key);
CustomBorderColorBudget custom_border_color_budget;
std::vector<Variant> variants;
const Device* device_ptr{nullptr};
VkSamplerCreateInfo base_ci{};
VkSamplerReductionModeEXT reduction_mode{VK_SAMPLER_REDUCTION_MODE_WEIGHTED_AVERAGE_EXT};
std::array<float, 4> border_color{};
std::array<float, 4> srgb_border_color{};
f32 default_anisotropy{1.0f};
bool has_added_anisotropy{};
bool has_linear_filtering{};
bool has_depth_comparison{};
bool has_minmax_reduction{};
bool has_custom_border_colors{};
bool has_srgb_border_color{};
bool needs_swizzle_mapping{};
}; };
struct TextureCacheParams { struct TextureCacheParams {
@@ -486,6 +604,8 @@ struct TextureCacheParams {
static constexpr bool HAS_EMULATED_COPIES = false; static constexpr bool HAS_EMULATED_COPIES = false;
static constexpr bool HAS_DEVICE_MEMORY_INFO = true; static constexpr bool HAS_DEVICE_MEMORY_INFO = true;
static constexpr bool IMPLEMENTS_ASYNC_DOWNLOADS = true; static constexpr bool IMPLEMENTS_ASYNC_DOWNLOADS = true;
static constexpr bool HAS_MSAA_DOWNLOADS = true;
static constexpr bool USE_UNIFIED_MEMORY = true;
using Runtime = Vulkan::TextureCacheRuntime; using Runtime = Vulkan::TextureCacheRuntime;
using Image = Vulkan::Image; using Image = Vulkan::Image;

Some files were not shown because too many files have changed in this diff Show More