Shaders: RDNA 2 → SPIR-V

Shader translation in one picture

A guest shader is a program for the console GPU. The host Vulkan driver accepts SPIR-V, so KytyPS5 decodes the guest instructions, represents their meaning in an intermediate representation (IR), and emits a host shader. A control-flow graph connects basic blocks; an execution mask decides which lanes take part. Resources must also be located and bound correctly. A correct addition instruction with the wrong buffer still draws the wrong result.

For a first pass, explain input → decode → IR → SPIR-V and try the wave model. Then follow D3. Return to opcode widths, resource-analysis passes, mesh stages and LDS fallback when investigating that specific path.

What must remain the same after translation?

The shader's observable operations for active lanes, including values, control flow and memory effects. Binary instructions and register allocation can differ; an image looking plausible is weaker evidence than a checked input/output case.

Use the code-example guide: snippets below are source excerpts or labelled schematic sequences, not standalone programs.

At main commit d2413fc2 (7 October 2026), the recompiler lives under src/graphics/shader/recompiler/. It is organised as an explicit compiler pipeline: frontend/decode, frontend/cfg, frontend/translate, a typed value IR and its passes, then backend/spirv.

Why it is hard, restated precisely

• The input has no source-level structure. It is an instruction stream operating on scalar and vector registers, with no source variables or functions.
• Control flow is hardware-shaped. Guest branches and execution masks must become SPIR-V selections and loops with declared merge and continue blocks.
• Resources are computed values. A descriptor may be inline in user data or reached through shader resource-table loads, so binding requires tracing the shader's own address calculations.
• Wave semantics differ. The guest assumes 32- or 64-lane waves and cross-lane operations; the host exposes Vulkan subgroups and their capabilities.
• Failure can be visual only. A bad translation may draw the wrong pixels without crashing, which is why shader tests dominate the suite.

The current pipeline

Two functions in ShaderRecompiler.cpp are the authoritative reading map. TranslateProgram() runs when a new SPIR-V permutation is needed; the first translation also seeds the cached resource plan; CompileProgram() runs once per resource specialization and emits SPIR-V. Both log the expensive phases with the shader hash:

// Schematic: TranslateProgram(code, options)
// Decode fused halves when present; Local decodes its front half.
// Supported BVH32/BVH64 instructions continue through the ordinary CFG and IR.
auto native_cfg = CFG::BuildGraph(decoded);
auto structured_cfg = CFG::Structurize(native_cfg); // use native CFG for dispatcher fallback
auto ir = Frontend::TranslateProgram(decoded, selected_cfg, translate_options);
IR::RewriteToSsa(ir.blocks);
IR::ConstantPropagationPass(ir.blocks, ir.wave_size);
IR::ResolveControlFlowIdentities(ir);
IR::RemoveIdentities(ir.blocks);
IR::EliminateDeadCode(ir.blocks);
IR::EliminateReadLane(ir, ir.wave_size); // simplify again if reads changed
LowerTessellationMemory(ir, options);
IR::TrackResources(ir, decoded, native_cfg); // includes scalar/SRT planning

IR::ApplyResourceSpecialization(ir, specialization);   // the snapshot chosen at draw time
IR::RemoveIdentities(ir.blocks);
IR::EliminateDeadCode(ir.blocks);
IR::CollectShaderInfo(ir, options.input_info);
IR::AllocateBindings(ir, push_data_start_dword,
    ir.stage == ShaderType::Compute && options.input_info.compute != nullptr &&
        options.input_info.compute->lds_storage);
// EmitProgram constructs the emitter, which analyses requirements.
auto spirv = Spirv::EmitProgram(ir, options.input_info);

Materialisation happens in the renderer's pipeline cache between the two functions above — see Registers, sharps & resources.

Decode, recover control flow, translate

frontend/decode/ShaderDecoder.cpp turns variable-length encodings into typed instructions from the SOP, VOP, SMEM, buffer, image, FLAT, LDS, and export families. frontend/cfg/ShaderCFG.cpp splits those instructions into blocks, computes graph facts, detects irreducible regions, and attempts to recover the merge and continue blocks SPIR-V requires.

Irreducibility or a failed Structurize() is a supported outcome. Kyty restores the unstructured graph, sets dispatcher_fallback and records a fallback_reason; the SPIR-V backend then represents the shader as a loop around a program-counter dispatcher. This is slower, but it gives unsupported graph shapes an explicit fallback instead of silently miscompiling them. LogDispatcherFallback() prints the phase (build or structurize), the failure kind and the block, which is the line to grep for.

The frontend lowers scalar/vector arithmetic, comparisons, buffer/image accesses and atomics, LDS/GDS operations, exports and control flow into typed IR. Examples include FP64 conversions and rounding, packed half operations, DPP/DPP8, CMPX, DS permutation and masked-OR operations. DS_ORDERED_COUNT is lowered to a GDS atomic. Coverage is per opcode and modifier combination: consult the decoder, translator, emitter and focused tests together. MAD and FMA have different rounding behavior, and host floating-point limits still matter.

Operand widths and formatted buffer access

V_MIN3_U16 uses VOP3 opcode 0x353. ApplyNativeVop3B16TernarySelectors() selects the low or high half of each source and the destination; Integer16Ternary() widens those selected unsigned values, emits UMinTri32, then writes one 16-bit half while preserving the other. The same helper uses signed inputs and SMedTri32 for V_MED3_I16. Destination/source overlap and inactive EXEC lanes need tests, not just a correct minimum formula.

void Translator::Integer16Ternary(const Decoder::Instruction& inst, IR::ValueOpcode opcode,
                                  bool sign) {
    const auto result = IR::U32(ir.Emit(
        opcode, {ReadU16AsU32(inst.src0, sign), ReadU16AsU32(inst.src1, sign),
                 ReadU16AsU32(inst.src2, sign)}));
    Write16Bits(DestinationOperand(inst), result);
}

BUFFER_LOAD_FORMAT_D16_X and BUFFER_STORE_FORMAT_D16_X are MUBUF opcodes 0x80 and 0x84. The opcode is eight bits, including bit 25 of the first dword. These formatted operations carry a 16-bit value in a VGPR, but the descriptor’s format determines the memory representation. Translation therefore uses a 32-bit IR load/store container while retaining MemoryInfo::data_bits == 16 for conversion in spirvEmitterMemory.cpp. This is separate from unsupported D16 BVH intersection encodings.

The D16 load writes the low destination half and preserves the upper half. Descriptor swizzles, out-of-bounds values and EXEC masking remain part of the contract. Read BufferLoadFormatD16XCases(), BufferStoreFormatD16XCases() and Vop3Min3U16CapturedAndSelectors() in tests/ShaderRecompilerComputeTests.cpp; the formatted cases are included in --buffer-format-store-only, and the U16 minimum case is included in --med3-i16-only.

64-bit comparisons and EXEC

The frontend includes I64/U64 false/true comparisons and selected relational CMPX variants. EmitIntegerCompare() reads full-width register pairs and sign-extends signed 32-bit literals for I64 comparisons, including V_CMPX_EQ_I64. EmitCompareResult() masks the result with the incoming EXEC; CMPX replaces EXEC and its low/high halves without writing VCC. Ordinary comparisons write their mask destination. The compute test switch --integer64-compare-only exercises signed/unsigned edge values, literal extension and wave32/wave64 masks. Check the individual opcode table before claiming another comparison or modifier is supported.

Merged geometry stages become mesh shaders

RDNA 2 runs a merged ES/GS ("NGG") stage that has no fixed-function equivalent in Vulkan's classic pipeline. Kyty compiles that stage as a VK_EXT_mesh_shader mesh shader: DecodeFusedProgram() joins the front and back stage code into one instruction stream with an ordinary CFG edge between them, ShaderType::Mesh carries it through translation, and backend/spirv/spirvEmitterMesh.cpp emits the mesh entry point. The pipeline cache checks the guest's thread, vertex, primitive and LDS limits against the device's mesh-shader properties and stops with mesh shader exceeds host limits when they do not fit; if the extension is missing, that path is an EXIT_NOT_IMPLEMENTED.

Native tessellation and stage metadata

ShaderVertexInputInfo::logical_stage selects Vertex, Mesh, Local, TessellationControl or TessellationEvaluation. PrepareTessellationPrograms() prepares three programs; Tessellation.cpp analyses LS/HS memory dataflow to derive strides and lowers cross-stage memory. spirvEmitterTessellation.cpp emits the Vulkan tessellation stages. Enable this path with --tessellation; patch draws are skipped by default.

Wave size is stage metadata, not a universal 64-lane constant: vertex and pixel stages carry their own widths and the NGG prologue respects guest wave32. Mesh stages retain their own workgroup limits. Recent opcode work includes FP64 arithmetic/conversions, DPP8, packed integer and half-float operations, partial SDWA destinations, image float-maximum atomics and buffer-cache policy bits. Host FP64 denormal limitations are logged.

Full-float BVH32/BVH64 intersections compile through BvhIntersect and spirvEmitterBvh.cpp. The helper reads guest nodes through BDA and executes triangle/box math in SPIR-V; normal compute dispatch preserves surrounding shader work. A16, D16 and incomplete NSA forms remain unsupported. C11 traces the supported encodings, node layouts and GPU fixtures; this is not a Vulkan ray-query backend or a claim of universal game support.

One typed value IR

The current IR::Program derives from IR::ResourcePlan. The plan is the part the shader cache keeps for the lifetime of the run — stage, hash, descriptor sources, SRT reads, the values they depend on, the uniform-branch skeleton around them — while the program adds the blocks that exist only during translation and emission. Neither carries the old lane-mask mode or scalar-provenance structures:

// Retained resource analysis; evaluation scratch is mutable and renderer-serialized.
struct ResourcePlan {
    ShaderType stage;  uint64_t shader_hash;
    uint32_t user_data_base, user_data_count = 64;
    std::list<Inst>               value_storage;
    std::vector<MemoryInfo>       memory_info;
    std::vector<DescriptorSource> descriptor_sources;
    std::vector<ResourceBlock>    control_flow;
    mutable std::vector<uint8_t>         active_sources;
    std::vector<SrtRead>          srt_reads;
    std::vector<uint8_t>          clean_flat_slots;
    bool requires_specialization_memory, srt_plan_complete, resource_tracking_complete;
    ShaderInfo      info;
    UniformFillPlan uniform_fill;
};

struct Program : ResourcePlan {
    std::vector<std::unique_ptr<Block>> block_storage;
    BlockList        blocks;
    uint32_t         wave_size = 64, scratch_dwords = 0;
    bool             dispatcher_fallback = false;
    CFG::FailureKind cfg_failure_kind;
    std::string      fallback_reason;
    std::vector<BlockInfo>  block_info;
    std::vector<ExportInfo> export_info;
    BindingLayout    bindings;
};

Translation initially permits register-like identities and control-flow joins. RewriteToSsa() establishes SSA form, then propagation, identity removal, and dead-code elimination simplify it. EliminateReadLane() removes cross-lane reads when a direct value is provably safe; if it rewrites anything, the simplification sequence runs again. Once emission is done the renderer keeps only a CompiledShaderInfo (stage, hash, wave size, ShaderInfo, BindingLayout) next to the plan; the blocks are discarded.

SRT planning, resources, and bindings

TrackResources() walks typed values to identify reachable shader-resource-table reads. Constant reads receive flattened offsets while genuinely dynamic reads stay explicit. The same pass classifies buffers, images, samplers, access modes, formats, and sampled pairs.

Concrete resource data comes from MaterializeResources() (in ir/passes/ResourceMaterialization.cpp) reading current user data and guest memory into a ResourceSnapshot and a ResourceSpecialization. It runs in the renderer's pipeline cache on draws and dispatches, not inside the recompiler; CompileProgram() then calls ApplyResourceSpecialization() to bake those facts into the IR. It also skips resources that sit in uniform branches the current user data never takes, and detects whole-buffer fills from uniform stores so the renderer can replace a dispatch with a native clear. Compilation and per-draw resource state therefore remain separate concerns.

AllocateBindings() covers ordinary buffers/images/samplers plus emulator support bindings:

enum class DescriptorBindingKind : uint32_t {
    Buffers  = 0u,
    // … 48 typed image bindings; a binding can hold an array …
    Samplers = FirstImageBinding + ImageBindingCount,   // 49
    Gds,
    BdaPagetable, FaultBuffer,
    FlattenedSrt, ShaderData, SharedMemory,
    Count,                                              // 56
};

BdaPagetable and FaultBuffer support buffer-device-address translation and recoverable RDNA2 FLAT-access faults. FlattenedSrt and ShaderData expose data that the guest ABI expected in scalar registers or tables; PushData is 32 dwords of push constants, the first six of which a mesh draw uses for its own parameters. Finally, AnalyzeProgramRequirements() records the subgroup, derivative, LDS, scratch, and pixel-mask features emission must request.

The retained ResourcePlan owns immutable descriptor/SRT analysis and reusable evaluation scratch. TrackResources(ir, decoded, native_cfg) performs scalar-read and indirect-image planning; there is no separate SRT-plan call in the top-level pipeline. ExtractResourcePlan() derives stable clean-slot and specialization policy once. A shared memoized SrtWalker refreshes descriptors, active branches, flattened reads and specialization in place. Bounded indirect image tables, guarded Phis and GPU-selected raw buffers follow this same value graph.

ShaderStageRuntime borrows two pointers: compiled program info and the cache entry's current ResourceSnapshot. The renderer consumes them under its serialization contract; keeping that snapshot as an immutable per-draw copy would describe the former implementation.

SPIR-V emission

backend/spirv is split by responsibility:

Embedded vertex fetch

RDNA 2 has no conventional fixed-function vertex-input stage. A fetch prologue may load attributes into registers before the main vertex shader. When fetch_embedded is set, DetectEmbeddedVertexFetch() analyses the decoded instructions before translation and builds a Frontend::EmbeddedFetchPlan. That plan is passed through TranslateOptions::embedded_fetch, so translation emits the correct vertex-input operations directly; there is no later RewriteEmbeddedVertexFetches() pass.

Where the test weight is

ShaderRecompilerComputeTests.cpp and its included suites exercise executable shader behavior; shaderCfgTests.cpp focuses on control flow and translation. Resource tracking, materialization, scalar evaluation and vertex metadata have focused executables. Read the case that matches an operation and check whether it executed, skipped for missing host capabilities, or only validated compilation.

Typed LDS and instruction coverage

Compute LDS normally uses workgroup memory. If the requested LDS exceeds maxComputeSharedMemorySize, lds_storage selects the SharedMemory storage-buffer binding. BindSharedMemory() reserves one LDS region per workgroup and checks the total against maxStorageBufferRange; indirect dispatch dimensions must be read before sizing it. Typed LDS views share the same backing. Thread-dimension dispatches use ceiling division to obtain group counts and mask lanes outside the requested thread dimensions.

The recompiler adds V_CMPX_NLE_F16 and native 64-bit LDS atomic OR. Shared-memory storage exposes compatible typed views instead of separate unrelated 32-bit and 64-bit stores; shader-info collection and SPIR-V emission carry the required capabilities. Read spirvEmitterMemory.cpp, spirvEmitterMemoryHelpers.cpp and the compute/CFG regression tests together.

GPU-selected descriptors require a bounded set of compatible candidates. Resource tracking discovers buffer/image tables and nested selectors, materialization refreshes their current values, and binding/emission maps the selected candidate to native resources. Compatible image views can share a contiguous descriptor array. Unsupported shapes retain explicit guards: deleting a guard does not supply the missing candidate discovery or bindings.

Raw scalar/descriptor reads use the buffer backing path (TryReadBufferBacking()); formatted image synchronization is handled by the appropriate resource binding. Compute materialization also receives SrtRuntime::workgroup_counts. During compilation, host evaluation values are discarded while dependencies still used by GPU memory operations are retained.