mirror of
https://git.eden-emu.dev/eden-emu/eden.git
synced 2026-08-31 10:36:56 +00:00
[vulkan, android] Mediacodec implementation + Vulkan fixes (#4191)
This is the first step into the conversion on full GPU video decoding for Android; currently due to the VIC structure, I couldn't set it on surface mode to prevent the latency when sending video decoded (which is processed by GPU now), because currently we convert YUV420 into the Nvidia's format each frame constantly on CPU, aside the requirements from other extensions to work + VIC rewrite, which is a work currently not planned to happen on this PR; the actual configuration for video decoding is ByteBuffer, using GPU to decode NVDEC data (all codecs supported, h264, vp8 and vp9) and send it to CPU for display purposes, which is more faster than relying purely on CPU for any drawing task, saving devices resources/ heating, the downside on this will be the slight latency when a new frame is displayed, which is gonna be a black frame for less than a second, nothing major to harm the experience rather than actually trying our best to take advantage on hardware accelerated. Aside this, I also added a bunch of minor Vulkan fixes to grant drivers less thinkering when receiving spir-v instructions, meaning this has new bans for extensions on QCOM (following reported issues on other projects working around Adreno driver behavior), this more than providing performance aims to enhance the stability on the driver, performance it's gonna likely to be hit based on the UBO's (StorageBufferAccess) operations and SSBO (uniformStorageBufferAccess), there was an already existing path for the emulation which forces to wide them into 32bit packed operations. I also included some smaller changes/ bugs + VUID's fixes from earlier changes on my work. Special Thanks: -> Mr. Smoly Gidolard (@gidoly) Sources: 1.- https://github.com/microsoft/DirectXShaderCompiler/issues/2842 2.- https://github.com/mlc-ai/web-llm/issues/836 3.- https://github.com/ggml-org/llama.cpp/issues/5186 4.- https://github.com/encounter/aurora/pull/202 Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4191
This commit is contained in:
@@ -179,7 +179,7 @@ public:
|
||||
~DecoderContext();
|
||||
|
||||
void InitializeHardwareDecoder(const HardwareContext& context, AVPixelFormat hw_pix_fmt);
|
||||
bool OpenContext(const Decoder& decoder);
|
||||
bool OpenContext(const Decoder& decoder, std::span<const u8> extradata = {});
|
||||
bool SendPacket(const Packet& packet);
|
||||
std::shared_ptr<Frame> ReceiveFrame();
|
||||
|
||||
@@ -198,6 +198,18 @@ private:
|
||||
bool m_decode_order{};
|
||||
};
|
||||
|
||||
struct FrameOffsets {
|
||||
bool interlaced{};
|
||||
bool hidden{};
|
||||
u64 luma{};
|
||||
u64 luma_bottom{};
|
||||
};
|
||||
|
||||
struct FrameDimensions {
|
||||
s32 width{};
|
||||
s32 height{};
|
||||
};
|
||||
|
||||
class DecodeApi {
|
||||
public:
|
||||
YUZU_NON_COPYABLE(DecodeApi);
|
||||
@@ -213,13 +225,24 @@ public:
|
||||
return m_decoder_context->UsingDecodeOrder();
|
||||
}
|
||||
|
||||
bool SendPacket(std::span<const u8> packet_data);
|
||||
std::shared_ptr<Frame> ReceiveFrame();
|
||||
bool SendPacket(std::span<const u8> packet_data, const FrameOffsets& offsets,
|
||||
std::optional<FrameDimensions> dimensions = std::nullopt);
|
||||
|
||||
struct DecodedFrame {
|
||||
std::shared_ptr<Frame> frame;
|
||||
FrameOffsets offsets;
|
||||
};
|
||||
std::optional<DecodedFrame> ReceiveFrame();
|
||||
|
||||
private:
|
||||
std::optional<FFmpeg::Decoder> m_decoder;
|
||||
std::optional<FFmpeg::DecoderContext> m_decoder_context;
|
||||
std::optional<FFmpeg::HardwareContext> m_hardware_context;
|
||||
bool m_opened{};
|
||||
bool m_defer_android_mediacodec_open{};
|
||||
bool m_needs_h264_extradata{};
|
||||
s64 m_next_pts{};
|
||||
std::queue<FrameOffsets> m_pending_offsets;
|
||||
};
|
||||
|
||||
} // namespace FFmpeg
|
||||
|
||||
Reference in New Issue
Block a user