glsl conditionally assigning out_color

David Carpenter 0
Viewed 14

I'm trying to achieve blending by using a shader to conditionally assign the fragment color.

Full shader code...

#version 450
#pragma shader_stage(fragment)

layout(binding = 1) uniform sampler2D texture_sampler;

layout(location = 0) in vec2 tex_coord;
layout(location = 1) in vec4 _rgba;
layout(location = 0) out vec4 out_color;

void main() {
    vec4 oc = textureLod(texture_sampler, tex_coord, 0);

    if (oc.r > _rgba.r)
        oc.r = _rgba.r;
    
    if (oc.g > _rgba.g)
        oc.g = _rgba.g;
    
    if (oc.b > _rgba.b)
        oc.b = _rgba.b;

    if (oc.a > _rgba.a)
        oc.a = _rgba.a;

    float threshhold = .5;

    if (oc.a > threshhold)
        out_color = oc;
}

(this flickers at run time btw...)

flickering garbage

changing the last bit of code to...

    float threshhold = .5;

    //if (oc.a > threshhold)
        out_color = oc;

does this...

rendered clean but with black background

If I remove the assignment completely, the text just dissappears. It seems that having the assignment there makes the driver expect it to always be assigned. Does anyone know why this is happening? Why can't I conditionally leave the pixel unmodified? Is there something I have to change in my Vulkan code for the pipeline?

shaderc vulkan glsl
1 Answers

This declaration in a shader

layout(location = 0) out vec4 out_color;

is basically telling the pipeline that whatever value out_color ends up with at the end of execution is what should go in the framebuffer. You're not allowed to just not assign a value. If you fail to write the value it's the same as having an uninitialized variable in C. The contents will be random based on whatever happened to be in that memory address at the start of execution.

From the GL wiki (but still applicable to GLSL shaders for Vulkan)

The shader must set all output variables at some point in its execution

The only two exceptions this. The first is if the output variable isn't being read by subsequent stages. In this case, there are no subsequent stages, and the variable is the output going to the framebuffer, so that's not applicable.

The second is in fragment shaders where you use the discard keyword. You're not using discard in your shader, but most likely that would fix your problem. Try changing your code to

    if (oc.a > threshhold)
        out_color = oc;
    else
        discard;

However, bear in mind that discard as a keyword can impact performance of the entire pipeline because it can no longer assume that every evaluation of the shader will produce output. See this other question

Jherico0
Related