OpenGL ES: Well defined way to discard primitive in vertex shader

user_1282931 0
Viewed 117

My vertex shader decides to discard some triangles. If a triangle should be discarded, what I currently do is set the output position to NaN:

gl_Position = vec4(intBitsToFloat(int(0xFFC00000u)), intBitsToFloat(int(0xFFC00000u)), intBitsToFloat(int(0xFFC00000u)), intBitsToFloat(int(0xFFC00000u)));

intBitsToFloat(int(0xFFC00000u)) is floating point NaN.

This works on the platforms I tested it on.

However, Section 2.3.4.1 of the OpenGL ES 3.2 Spec states that

The special values Inf and −Inf encode values with magnitudes too large to be represented; the special value NaN encodes “Not A Number” values resulting from undefined arithmetic operations such as 0/0. Implementations are permitted, but not required, to support Inf's and NaN's in their floating-point computations.

So my approach might actually yield undefined behavior. What's the proper, well defined OpenGL ES 3.x way to completely discard a triangle in the vertex shader?

vertex-shader undefined-behavior opengl-es nan shader
1 Answers

The result is required to be consistent, even if NaN is not generated. If you set all 3 vertices to NaN (or whatever the fallback value turns out to be) then they are going to be coincident locations. Any two coincident vertices in a triangle = zero area = culled. So I think this works in either case ...

user_5330370
Related