Procedural Geometry Shaders

Introduction

Welcome back to our Advanced Geometry and Model Loading course! We've reached the exciting conclusion of our journey together, having mastered face culling optimizations, sophisticated 3D model loading with tiny_obj_loader, and immersive environmental rendering through cubemap skyboxes. Each lesson has built upon the last, creating increasingly sophisticated rendering systems that bring professional-quality graphics within your reach. In this final lesson, we're unlocking a powerful stage of the graphics pipeline that has been working behind the scenes throughout our entire course: the geometry shader . While our previous work focused on transforming and rendering existing geometry, we now venture into the fascinating realm of procedural geometry generation, where we can dynamically create new geometric primitives directly on the GPU during rendering. We'll implement a stunning crystallization effect that transforms our familiar teapot model into a dynamic, crystal-covered surface with animated growth patterns. This technique demonstrates how geometry shaders enable effects impossible with traditional vertex and fragment processing alone: generating entirely new triangles, creating particle systems, producing wireframe overlays, and building complex procedural structures that respond to real-time parameters. By lesson's end, you'll command the complete OpenGL graphics pipeline, equipped with the knowledge to create sophisticated visual effects that rival those found in modern games and interactive applications.

Understanding the Graphics Pipeline's Hidden Stage

Throughout our previous lessons, we've worked extensively with vertex and fragment shaders, treating them as the primary programmable stages of the graphics pipeline. However, we've actually been missing one crucial step that OpenGL has been handling automatically behind the scenes: the geometry processing stage that occurs between vertex transformation and rasterization. Until now, OpenGL has been quietly performing a default geometry processing operation for us. When we submitted triangles through our vertex shaders, the graphics hardware automatically passed them through an invisible geometry stage that simply forwarded each triangle unchanged to the rasterizer. This automatic behavior worked perfectly for our previous lessons because we only needed to transform existing geometry, not create new primitives. The geometry shader gives us direct control over this previously hidden pipeline stage, allowing us to intercept primitives after vertex processing and before rasterization. This positioning grants us remarkable creative power: we can examine entire primitives (not just individual vertices), generate multiple output primitives from single inputs, or even discard primitives entirely based on dynamic criteria.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

glsl

#version 450 core
layout(triangles) in;
layout(triangle_strip, max_vertices = 18) out;

#version 450 core
layout(triangles) in;
layout(triangle_strip, max_vertices = 18) out;

glsl

in vec3 vPosition[3];
in vec3 vNormal[3];
in vec3 vWorldPos[3];
in vec2 vUV[3];

out vec3 gNormal;
out vec3 gWorldPos;
out vec2 gUV;
out float gCrystalFactor;

in vec3 vPosition[3];
in vec3 vNormal[3];
in vec3 vWorldPos[3];
in vec2 vUV[3];

out vec3 gNormal;
out vec3 gWorldPos;
out vec2 gUV;
out float gCrystalFactor;

glsl

// First, always emit the original triangle
for (int i = 0; i < 3; i++) {
    gl_Position = gl_in[i].gl_Position;
    gNormal = vNormal[i];
    gWorldPos = vWorldPos[i];
    gUV = vUV[i];
    gCrystalFactor = 0.0; // Mark as original geometry
    EmitVertex();
}
EndPrimitive();

// First, always emit the original triangle
for (int i = 0; i < 3; i++) {
    gl_Position = gl_in[i].gl_Position;
    gNormal = vNormal[i];
    gWorldPos = vWorldPos[i];
    gUV = vUV[i];
    gCrystalFactor = 0.0; // Mark as original geometry
    EmitVertex();
}
EndPrimitive();

glsl

// Calculate triangle center and average normal
vec3 center = (vWorldPos[0] + vWorldPos[1] + vWorldPos[2]) / 3.0;
vec3 avgNormal = normalize((vNormal[0] + vNormal[1] + vNormal[2]) / 3.0);

// Crystal parameters with time-based animation
float crystalHeight = 0.3;
float pulseFactor = 1.0 + sin(uTime * 6.28) * 0.2;
crystalHeight *= pulseFactor;

// Create crystal tip
vec3 crystalTip = center + avgNormal * crystalHeight;

glsl

// Create crystal facets - pyramid from base triangle to tip
for (int i = 0; i < 3; i++) {
    int next = (i + 1) % 3;
    
    // Calculate face normal for this crystal facet in world space
    vec3 worldPos1 = vWorldPos[i];
    vec3 worldPos2 = vWorldPos[next];
    vec3 worldTip = tipWorld.xyz;
    
    vec3 edge1 = worldPos2 - worldPos1;
    vec3 edge2 = worldTip - worldPos1;
    vec3 facetNormal = normalize(cross(edge1, edge2));
}

// Create crystal facets - pyramid from base triangle to tip
for (int i = 0; i < 3; i++) {
    int next = (i + 1) % 3;
    
    // Calculate face normal for this crystal facet in world space
    vec3 worldPos1 = vWorldPos[i];
    vec3 worldPos2 = vWorldPos[next];
    vec3 worldTip = tipWorld.xyz;
    
    vec3 edge1 = worldPos2 - worldPos1;
    vec3 edge2 = worldTip - worldPos1;
    vec3 facetNormal = normalize(cross(edge1, edge2));
}

C++

GLuint createShadersWithGeometry(const std::string &vertexFilePath, 
                                 const std::string &geometryFilePath, 
                                 const std::string &fragmentFilePath)
{
    // ... vertex and fragment compilation code ...
    
    // Create and compile geometry shader
    GLuint geometryShader = glCreateShader(GL_GEOMETRY_SHADER);
    glShaderSource(geometryShader, 1, &gSrc, nullptr);
    glCompileShader(geometryShader);
    if (!checkShader(geometryShader)) {
        // Clean up and exit on error
        return -1;
    }
    
    // Attach all three shaders to program
    glAttachShader(shaderProgram, vertexShader);
    glAttachShader(shaderProgram, geometryShader);
    glAttachShader(shaderProgram, fragmentShader);
}

GLuint createShadersWithGeometry(const std::string &vertexFilePath, 
                                 const std::string &geometryFilePath, 
                                 const std::string &fragmentFilePath)
{
    // ... vertex and fragment compilation code ...
    
    // Create and compile geometry shader
    GLuint geometryShader = glCreateShader(GL_GEOMETRY_SHADER);
    glShaderSource(geometryShader, 1, &gSrc, nullptr);
    glCompileShader(geometryShader);
    if (!checkShader(geometryShader)) {
        // Clean up and exit on error
        return -1;
    }
    
    // Attach all three shaders to program
    glAttachShader(shaderProgram, vertexShader);
    glAttachShader(shaderProgram, geometryShader);
    glAttachShader(shaderProgram, fragmentShader);
}

glsl

out vec3 vPosition;
out vec3 vNormal;
out vec3 vWorldPos;
out vec2 vUV;

void main() {
    vPosition = aPos;  // Model space position for geometry calculations
    vNormal = mat3(transpose(inverse(uModel))) * aNormal;
    vWorldPos = vec3(uModel * vec4(aPos, 1.0));
    vUV = aUV;
    gl_Position = uProjection * uView * uModel * vec4(aPos, 1.0);
}

out vec3 vPosition;
out vec3 vNormal;
out vec3 vWorldPos;
out vec2 vUV;

void main() {
    vPosition = aPos;  // Model space position for geometry calculations
    vNormal = mat3(transpose(inverse(uModel))) * aNormal;
    vWorldPos = vec3(uModel * vec4(aPos, 1.0));
    vUV = aUV;
    gl_Position = uProjection * uView * uModel * vec4(aPos, 1.0);
}

glsl

in float gCrystalFactor;

void main() {
    vec3 baseColor;
    float shininess;
    
    if (gCrystalFactor < 0.1) {
        // Original teapot surface - dark metallic
        baseColor = vec3(0.15, 0.15, 0.2);
        shininess = 32.0;
    } else {
        // Crystal - bright reflective blue
        baseColor = vec3(0.3, 0.5, 0.8);
        shininess = 64.0;
    }
    
    // Enhanced specular for crystals
    if (gCrystalFactor > 0.1) {
        spec *= (1.0 + gCrystalFactor * 2.0);
    }
}

in float gCrystalFactor;

void main() {
    vec3 baseColor;
    float shininess;
    
    if (gCrystalFactor < 0.1) {
        // Original teapot surface - dark metallic
        baseColor = vec3(0.15, 0.15, 0.2);
        shininess = 32.0;
    } else {
        // Crystal - bright reflective blue
        baseColor = vec3(0.3, 0.5, 0.8);
        shininess = 64.0;
    }
    
    // Enhanced specular for crystals
    if (gCrystalFactor > 0.1) {
        spec *= (1.0 + gCrystalFactor * 2.0);
    }
}

glsl

#version 450 core
layout(triangles) in;
layout(triangle_strip, max_vertices = 18) out;

glsl

in vec3 vPosition[3];
in vec3 vNormal[3];
in vec3 vWorldPos[3];
in vec2 vUV[3];

out vec3 gNormal;
out vec3 gWorldPos;
out vec2 gUV;
out float gCrystalFactor;

glsl

// First, always emit the original triangle
for (int i = 0; i < 3; i++) {
    gl_Position = gl_in[i].gl_Position;
    gNormal = vNormal[i];
    gWorldPos = vWorldPos[i];
    gUV = vUV[i];
    gCrystalFactor = 0.0; // Mark as original geometry
    EmitVertex();
}
EndPrimitive();

glsl

// Calculate triangle center and average normal
vec3 center = (vWorldPos[0] + vWorldPos[1] + vWorldPos[2]) / 3.0;
vec3 avgNormal = normalize((vNormal[0] + vNormal[1] + vNormal[2]) / 3.0);

// Crystal parameters with time-based animation
float crystalHeight = 0.3;
float pulseFactor = 1.0 + sin(uTime * 6.28) * 0.2;
crystalHeight *= pulseFactor;

// Create crystal tip
vec3 crystalTip = center + avgNormal * crystalHeight;

glsl

// Create crystal facets - pyramid from base triangle to tip
for (int i = 0; i < 3; i++) {
    int next = (i + 1) % 3;
    
    // Calculate face normal for this crystal facet in world space
    vec3 worldPos1 = vWorldPos[i];
    vec3 worldPos2 = vWorldPos[next];
    vec3 worldTip = tipWorld.xyz;
    
    vec3 edge1 = worldPos2 - worldPos1;
    vec3 edge2 = worldTip - worldPos1;
    vec3 facetNormal = normalize(cross(edge1, edge2));
}

C++

GLuint createShadersWithGeometry(const std::string &vertexFilePath, 
                                 const std::string &geometryFilePath, 
                                 const std::string &fragmentFilePath)
{
    // ... vertex and fragment compilation code ...
    
    // Create and compile geometry shader
    GLuint geometryShader = glCreateShader(GL_GEOMETRY_SHADER);
    glShaderSource(geometryShader, 1, &gSrc, nullptr);
    glCompileShader(geometryShader);
    if (!checkShader(geometryShader)) {
        // Clean up and exit on error
        return -1;
    }
    
    // Attach all three shaders to program
    glAttachShader(shaderProgram, vertexShader);
    glAttachShader(shaderProgram, geometryShader);
    glAttachShader(shaderProgram, fragmentShader);
}

glsl

out vec3 vPosition;
out vec3 vNormal;
out vec3 vWorldPos;
out vec2 vUV;

void main() {
    vPosition = aPos;  // Model space position for geometry calculations
    vNormal = mat3(transpose(inverse(uModel))) * aNormal;
    vWorldPos = vec3(uModel * vec4(aPos, 1.0));
    vUV = aUV;
    gl_Position = uProjection * uView * uModel * vec4(aPos, 1.0);
}

glsl

in float gCrystalFactor;

void main() {
    vec3 baseColor;
    float shininess;
    
    if (gCrystalFactor < 0.1) {
        // Original teapot surface - dark metallic
        baseColor = vec3(0.15, 0.15, 0.2);
        shininess = 32.0;
    } else {
        // Crystal - bright reflective blue
        baseColor = vec3(0.3, 0.5, 0.8);
        shininess = 64.0;
    }
    
    // Enhanced specular for crystals
    if (gCrystalFactor > 0.1) {
        spec *= (1.0 + gCrystalFactor * 2.0);
    }
}

Introduction

Understanding the Graphics Pipeline's Hidden Stage

Join the 1M+ learners on CodeSignal

The Power of Primitive-Level Processing

Configuring Input and Output Data Streams

Preserving Original Geometry

Procedural Crystal Generation Algorithm

Creating Crystal Facet Geometry

Enhanced Shader Compilation Pipeline

Vertex Shader Adaptations

Material Differentiation in Fragment Processing

Real-Time Crystal Animation Results

Conclusion and Next Steps