Project 5: Charles Wang

README.md

@@ -3,26 +3,54 @@ WebGL Clustered Deferred and Forward+ Shading
 
 **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+* Charles Wang
+* Tested on: Windows 10, i7-6700K @ 4.00GHz 16GB, GTX 1060 6GB (Personal Computer) Google Chrome 62 and Mozilla Firefox 56.0.1
 
-### Live Online
+![](img/example.gif)
+Clustered Forward+ Renderer with 2000 lights
 
-[![](img/thumb.png)](http://TODO.github.io/Project5B-WebGL-Deferred-Shading)
+## **Project Goals and Implementational Details**
 
-### Demo Video/GIF
+In this project I implemented: 
+* Light clustering based on camera frustum slices
+* Forward+ rendering
+* Deferred shading
+* G Buffer compacting normal x,y values into w slots
+* Blinn-Phong shading
 
-[![](img/video.png)](TODO)
+Clustering is a rendering prepass where lights in a scene are sorted into spacial clusters created from the camera frustum (in this implementation, this clustering is done on serially on the CPU). This way, a fragment shader for a specific fragment may query for the cluster and test illumination based on only the lights in the cluster.
 
-### (TODO: Your README)
+Forward shading is the simplest way of shading where each fragment tests against each light for illumination. For small scenes with few lights, this is fine. 
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+Clustered Forward+ shading follows generally the same algorithm as Forward shading, except we perform the clustering prepass to store light indices into a "cluster buffer" where each pixel column stores a list of lights for each cluster index.
 
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
+Clustered Deferred shading is like Clustered Forward+ in the sense that it utilizes the clustering prepass, but the fragment shading segment is slightly different. A Deferred shader uses two fragment shading passes: one pass where fragment values are stored in gbuffers (arbitrary textures), and then compiled for shading calculations at the very end using only the gbuffer information for attributes. Gbuffers can contain arbitrary values and thus, a gbuffer can contain a mismatch of information in the 4 value pixels. 
 
+## **Performance Analysis**
+
+### Comparing time for each rendering method with increasing light numbers
+
+All measurements are taken with the same scene with the same geometry and a constant light radius of 2 units across all lights. Larger light radii and varying cluster sizes may also affect runtime. However, first we want to test how magnitude of elements is reflected in each rendering method. Also, the max number of lights per cluster is increased with each number of lights so lights don't get clipped (so this analysis can be understood more practically). Clustering is especially useful for scenes with many lights with a defined radius of influence. This way, we can cull lights easily.
+
+The data was collected visually from the stat.js overlay, thus these measurements are more of an estimation than a tight measurement.
+
+![](img/lightschart.PNG)
+![](img/lightstable.PNG)
+
+The naive Forward renderer only outperforms the clustered renderers for scenes with very few lights. This is because there are few enough lights such that iterating over every light per fragment is not as costly as iterating over every light for clusters. We do notice that the clustered methods outperform the naive method very quickly and by a huge margin once we start introducing 1000s of lights. 
+
+### Comparing g_buffer packing normals vs dedicated g_buffer
+
+Actually, for a scene with 3000 lights, compacting the gbuffer lead to about a 3 ms slow down (33ms vs 36ms without compacting).
+
+I think this happened because I didn't gain much from compacting data and introduced another computation in converting the surface normal from world space 3d to screen space 2d (with some view matrix transformations). I only compacted from 3 gbuffers to 2 gbuffers. If there were more AOVs for my project, then perhaps I would gain more from compacting g buffers.
+
+### Cool AOVs/Debugging views
+Number of lights in each cluster
+![](img/numlightsstill.PNG)
+
+Cluster id (x,y,z) in (r,g,b)
+![](img/clusterid.PNG)
 
 ### Credits
 

src/init.js

@@ -1,5 +1,5 @@
 // TODO: Change this to enable / disable debug mode
-export const DEBUG = true && process.env.NODE_ENV === 'development';
+export const DEBUG = false && process.env.NODE_ENV === 'development';
 
 import DAT from 'dat-gui';
 import WebGLDebug from 'webgl-debug';
@@ -60,7 +60,7 @@ stats.domElement.style.top = '0px';
 document.body.appendChild(stats.domElement);
 
 // Initialize camera
-export const camera = new PerspectiveCamera(75, canvas.clientWidth / canvas.clientHeight, 0.1, 1000);
+export const camera = new PerspectiveCamera(75, canvas.clientWidth / canvas.clientHeight, 0.1, 50);
 
 // Initialize camera controls
 export const cameraControls = new OrbitControls(camera, canvas);

src/renderers/clustered.js

@@ -1,8 +1,8 @@
 import { mat4, vec4, vec3 } from 'gl-matrix';
-import { NUM_LIGHTS } from '../scene';
+import { NUM_LIGHTS, LIGHT_RADIUS } from '../scene';
 import TextureBuffer from './textureBuffer';
 
-export const MAX_LIGHTS_PER_CLUSTER = 100;
+export const MAX_LIGHTS_PER_CLUSTER = 5000;
 
 export default class ClusteredRenderer {
   constructor(xSlices, ySlices, zSlices) {
@@ -21,11 +21,183 @@ export default class ClusteredRenderer {
       for (let y = 0; y < this._ySlices; ++y) {
         for (let x = 0; x < this._xSlices; ++x) {
           let i = x + y * this._xSlices + z * this._xSlices * this._ySlices;
-          // Reset the light count to 0 for every cluster
           this._clusterTexture.buffer[this._clusterTexture.bufferIndex(i, 0)] = 0;
         }
       }
     }
+    // let h_fov = camera.fov;
+    // let v_fov = h_fov / camera.aspect;
+    let v_fov = camera.fov;
+    let h_fov = v_fov * camera.aspect;
+    let z_range = camera.far - camera.near;
+    let x_stride = h_fov/this._xSlices;
+    let y_stride = v_fov/this._ySlices;
+    let z_stride = z_range/this._zSlices;
+    let DEG2RAD = Math.PI / 180;
+    let z_vec = vec3.fromValues(0,0,1.0);
+    let y_vec = vec3.fromValues(0,1.0,0);
+    let x_vec = vec3.fromValues(1.0,0,0);
+    let origin = vec3.fromValues(0,0,0);
+    for(let l_idx = 0; l_idx < NUM_LIGHTS; ++l_idx) {
+      let pos = scene.lights[l_idx].position;
+      let light_w_pos = vec4.fromValues(pos[0],pos[1],pos[2],1);
+      let light_c_pos = vec4.create();
+      vec4.transformMat4(light_c_pos,light_w_pos,viewMatrix);
+
+
+      let z_start = Math.floor((-(light_c_pos[2]) - LIGHT_RADIUS - camera.near)/z_stride);
+      if(z_start < 0) {
+        z_start = 0;
+      } 
+      let z_end = Math.floor((-(light_c_pos[2]) + LIGHT_RADIUS - camera.near)/z_stride) + 1;
+      if(z_end > this._zSlices) {
+        z_end = this._zSlices;
+      }
+
+      // z_start = 0;
+      // z_end = this._zSlices;
+
+      let curr_z = (z_start * z_stride) + camera.near;
+
+    for (let z = z_start; z < z_end; ++z) {
+      let z_slice = -(z * z_stride) - camera.near;
+      let z_slice_p1 = -((z + 1) * z_stride) - camera.near;
+
+      // Z CLUSTER BACK BOUND
+      let zp_norm = vec3.clone(z_vec);
+      vec3.negate(zp_norm,zp_norm);
+      let zp_point = vec3.fromValues(0, 0, z_slice_p1);
+      let zp_plane = vec4.fromValues(0,0,zp_norm[2],-zp_point[2] * zp_norm[2]);
+      //vec4.normalize(zp_plane,zp_plane);
+
+      // Z CLUSTER FRONT BOUND
+      let zn_norm = vec3.clone(z_vec);
+      let zn_point = vec3.fromValues(0, 0, z_slice);
+      let zn_plane = vec4.fromValues(0,0,zn_norm[2],-zn_point[2] * zn_norm[2]);
+      //vec4.normalize(zn_plane,zn_plane);
+
+
+      let y_upperbound = Math.abs(curr_z * Math.tan(v_fov * DEG2RAD / 2.0));
+      let y_temp_stride = y_upperbound * 2.0 / this._ySlices;
+
+      let y_start = Math.floor(((light_c_pos[1] - LIGHT_RADIUS ) + y_upperbound) / y_temp_stride) - 9;
+      if(y_start < 0) {
+        y_start = 0;
+      }
+      let y_end = Math.floor(((light_c_pos[1] + LIGHT_RADIUS ) + y_upperbound) / y_temp_stride) + 1;
+      if(y_end > this._ySlices) {
+        y_end = this._ySlices;
+      }
+
+      y_start = 0;
+      y_end = this._ySlices;
+
+            //Z BACK
+            let dist_zp = vec4.dot(light_c_pos,zp_plane);
+            if(dist_zp > LIGHT_RADIUS) {continue;}
+            //Z FRONT
+            let dist_zn = vec4.dot(light_c_pos,zn_plane);
+            if(dist_zn > LIGHT_RADIUS) {continue;}
+
+      for (let y = y_start; y < y_end; ++y) {
+        let y_slice = (y * y_stride) - (v_fov/2.0);
+        let y_slice_p1 = ((y + 1) * y_stride) - (v_fov/2.0);
+
+          // Y CLUSTER UPPER BOUND
+          let yp_norm = vec3.create();
+          vec3.rotateX(yp_norm, y_vec, origin, y_slice_p1 * DEG2RAD);
+          let yp_plane = vec4.fromValues(yp_norm[0],yp_norm[1],yp_norm[2],0);
+          //vec4.normalize(yp_plane,yp_plane);
+
+          // Y CLUSTER LOWER BOUND
+          let yn_norm = vec3.fromValues(0,-1,0);
+          vec3.rotateX(yn_norm, yn_norm, origin, y_slice * DEG2RAD);
+          let yn_plane = vec4.fromValues(yn_norm[0],yn_norm[1],yn_norm[2],0);
+          //ec4.normalize(yn_plane,yn_plane);
+
+          let x_upperbound = curr_z * Math.tan(h_fov * DEG2RAD / 2.0);
+          let x_temp_stride = x_upperbound * 2.0 / this._xSlices;
+
+          let x_start = Math.floor(((light_c_pos[0] - LIGHT_RADIUS ) + x_upperbound) / x_temp_stride) - 1;
+          if(x_start < 0) {
+            x_start = 0;
+          }
+          let x_end = Math.floor(((light_c_pos[0] + LIGHT_RADIUS ) + x_upperbound) / x_temp_stride) + 4;
+          if(x_end > this._xSlices) {
+            x_end = this._xSlices;
+          }
+
+        //x_start = 0;
+        //x_end = this._xSlices;
+
+            //Y LOWER
+            let dist_yp = vec4.dot(light_c_pos,yp_plane);
+            if(dist_yp > LIGHT_RADIUS) {continue;}
+
+            //Y UPPER
+            let dist_yn = vec4.dot(light_c_pos,yn_plane);
+            if(dist_yn > LIGHT_RADIUS) {continue;}
+
+        for (let x = x_start; x < x_end; ++x) {
+          let i = x + y * this._xSlices + z * this._xSlices * this._ySlices;
+          //HI
+
+          //BASED ON THE X, Y, Z, FIGURE OUT WHAT PLANES WE NEED
+          let x_slice = (x * x_stride) - (h_fov/2.0);
+          let x_slice_p1 = ((x + 1) * x_stride) - (h_fov/2.0);
+
+          // X CLUSTER RIGHT BOUND
+          let xp_norm = vec3.create();
+          vec3.rotateY(xp_norm, x_vec, origin, -x_slice_p1 * DEG2RAD);
+          let xp_plane = vec4.fromValues(xp_norm[0],xp_norm[1],xp_norm[2],0);
+          //vec4.normalize(xp_plane,xp_plane);
+
+          // X CLUSTER LEFT BOUND
+          let xn_norm = vec3.fromValues(-1,0,0);
+          vec3.rotateY(xn_norm, xn_norm, origin, -x_slice * DEG2RAD);
+          let xn_plane = vec4.fromValues(xn_norm[0],xn_norm[1],xn_norm[2],0);
+          //vec4.normalize(xn_plane,xn_plane);
+
+            //X LEFT
+            //BOOLS FOR DEBUGGING PURPOSES
+            let bool_xp = true;
+            let bool_xn = true;
+            let bool_yp = true;
+            let bool_yn = true;
+            let bool_zp = true;
+            let bool_zn = true;
+            //BOOLS FOR DEBUGGING PURPOSES
+
+            let dist_xp = vec4.dot(light_c_pos,xp_plane);
+            if(dist_xp > LIGHT_RADIUS) {continue;}
+            //X RIGHT
+            let dist_xn = vec4.dot(light_c_pos,xn_plane);
+            if(dist_xn > LIGHT_RADIUS) {continue;}
+
+
+
+
+            let num_lights_in_cluster = this._clusterTexture.buffer[this._clusterTexture.bufferIndex(i, 0)]
+            //if(bool_xp && bool_xn && bool_yp && bool_yn && bool_zp && bool_zn) {
+              if(num_lights_in_cluster < MAX_LIGHTS_PER_CLUSTER){
+                let texel = Math.floor((num_lights_in_cluster + 1)/4.0);
+                let texel_idx = this._clusterTexture.bufferIndex(i, texel);
+                let float_idx = (num_lights_in_cluster + 1) - (texel * 4);
+                this._clusterTexture.buffer[texel_idx + float_idx] = l_idx;
+                ++num_lights_in_cluster;
+              }
+            //} 
+
+
+          //GET THE TOTAL NUMBER OF LIGHTS IN THIS CLUSTER
+
+          // Reset the light count to 0 for every cluster
+          this._clusterTexture.buffer[this._clusterTexture.bufferIndex(i, 0)] = num_lights_in_cluster;
+        }
+      }
+    }
+    //FOR EACH OF THE FOUR PLANES, CHECK IF LIGHT IS "IN"
+  }
 
     this._clusterTexture.update();
   }

src/renderers/clusteredDeferred.js

@@ -8,6 +8,7 @@ import QuadVertSource from '../shaders/quad.vert.glsl';
 import fsSource from '../shaders/deferred.frag.glsl.js';
 import TextureBuffer from './textureBuffer';
 import ClusteredRenderer from './clustered';
+import { MAX_LIGHTS_PER_CLUSTER } from './clustered';
 
 export const NUM_GBUFFERS = 4;
 
@@ -21,20 +22,23 @@ export default class ClusteredDeferredRenderer extends ClusteredRenderer {
     this._lightTexture = new TextureBuffer(NUM_LIGHTS, 8);
 
     this._progCopy = loadShaderProgram(toTextureVert, toTextureFrag, {
-      uniforms: ['u_viewProjectionMatrix', 'u_colmap', 'u_normap'],
+      uniforms: ['u_viewMatrix','u_viewProjectionMatrix', 'u_colmap', 'u_normap'],
       attribs: ['a_position', 'a_normal', 'a_uv'],
     });
 
     this._progShade = loadShaderProgram(QuadVertSource, fsSource({
       numLights: NUM_LIGHTS,
       numGBuffers: NUM_GBUFFERS,
+      xSlices: xSlices, ySlices: ySlices, zSlices: zSlices,
+      maxLights : MAX_LIGHTS_PER_CLUSTER
     }), {
-      uniforms: ['u_gbuffers[0]', 'u_gbuffers[1]', 'u_gbuffers[2]', 'u_gbuffers[3]'],
+      uniforms: ['u_gbuffers[0]', 'u_gbuffers[1]', 'u_gbuffers[2]', 'u_gbuffers[3]','u_viewMatrix','u_invViewMatrix','u_lightbuffer', 'u_clusterbuffer', 'u_height', 'u_width', 'u_near', 'u_far'],
       attribs: ['a_uv'],
     });
 
     this._projectionMatrix = mat4.create();
     this._viewMatrix = mat4.create();
+    this._invViewMatrix = mat4.create();
     this._viewProjectionMatrix = mat4.create();
   }
 
@@ -106,6 +110,7 @@ export default class ClusteredDeferredRenderer extends ClusteredRenderer {
     // Update the camera matrices
     camera.updateMatrixWorld();
     mat4.invert(this._viewMatrix, camera.matrixWorld.elements);
+    mat4.invert(this._invViewMatrix, this._viewMatrix);
     mat4.copy(this._projectionMatrix, camera.projectionMatrix.elements);
     mat4.multiply(this._viewProjectionMatrix, this._projectionMatrix, this._viewMatrix);
 
@@ -123,6 +128,7 @@ export default class ClusteredDeferredRenderer extends ClusteredRenderer {
 
     // Upload the camera matrix
     gl.uniformMatrix4fv(this._progCopy.u_viewProjectionMatrix, false, this._viewProjectionMatrix);
+    gl.uniformMatrix4fv(this._progCopy.u_viewMatrix, false, this._viewMatrix);
 
     // Draw the scene. This function takes the shader program so that the model's textures can be bound to the right inputs
     scene.draw(this._progCopy);
@@ -163,6 +169,24 @@ export default class ClusteredDeferredRenderer extends ClusteredRenderer {
       gl.uniform1i(this._progShade[`u_gbuffers[${i}]`], i + firstGBufferBinding);
     }
 
+    gl.activeTexture(gl.TEXTURE5);
+    gl.bindTexture(gl.TEXTURE_2D, this._lightTexture.glTexture);
+    gl.uniform1i(this._progShade.u_lightbuffer, 5);
+
+    // Set the cluster texture as a uniform input to the shader
+    gl.activeTexture(gl.TEXTURE3);
+    gl.bindTexture(gl.TEXTURE_2D, this._clusterTexture.glTexture);
+    gl.uniform1i(this._progShade.u_clusterbuffer, 3);
+
+    gl.uniformMatrix4fv(this._progShade.u_viewMatrix, false, this._viewMatrix);
+    gl.uniformMatrix4fv(this._progShade.u_invViewMatrix, false, this._invViewMatrix);
+
+    gl.uniform1f(this._progShade.u_width, canvas.width);
+    gl.uniform1f(this._progShade.u_height, canvas.height);
+    gl.uniform1f(this._progShade.u_near, camera.near);
+    gl.uniform1f(this._progShade.u_far, camera.far);
+
+
     renderFullscreenQuad(this._progShade);
   }
 };

src/renderers/clusteredForwardPlus.js

@@ -6,6 +6,7 @@ import vsSource from '../shaders/clusteredForward.vert.glsl';
 import fsSource from '../shaders/clusteredForward.frag.glsl.js';
 import TextureBuffer from './textureBuffer';
 import ClusteredRenderer from './clustered';
+import { MAX_LIGHTS_PER_CLUSTER } from './clustered';
 
 export default class ClusteredForwardPlusRenderer extends ClusteredRenderer {
   constructor(xSlices, ySlices, zSlices) {
@@ -15,9 +16,9 @@ export default class ClusteredForwardPlusRenderer extends ClusteredRenderer {
     this._lightTexture = new TextureBuffer(NUM_LIGHTS, 8);
 
     this._shaderProgram = loadShaderProgram(vsSource, fsSource({
-      numLights: NUM_LIGHTS,
+      numLights: NUM_LIGHTS, xSlices: xSlices, ySlices: ySlices, zSlices: zSlices, maxLights : MAX_LIGHTS_PER_CLUSTER
     }), {
-      uniforms: ['u_viewProjectionMatrix', 'u_colmap', 'u_normap', 'u_lightbuffer', 'u_clusterbuffer'],
+      uniforms: ['u_viewMatrix','u_viewProjectionMatrix', 'u_colmap', 'u_normap', 'u_lightbuffer', 'u_clusterbuffer', 'u_height', 'u_width', 'u_near', 'u_far'],
       attribs: ['a_position', 'a_normal', 'a_uv'],
     });
 
@@ -64,7 +65,8 @@ export default class ClusteredForwardPlusRenderer extends ClusteredRenderer {
 
     // Upload the camera matrix
     gl.uniformMatrix4fv(this._shaderProgram.u_viewProjectionMatrix, false, this._viewProjectionMatrix);
-
+    gl.uniformMatrix4fv(this._shaderProgram.u_viewMatrix, false, this._viewMatrix);
+
     // Set the light texture as a uniform input to the shader
     gl.activeTexture(gl.TEXTURE2);
     gl.bindTexture(gl.TEXTURE_2D, this._lightTexture.glTexture);
@@ -75,6 +77,12 @@ export default class ClusteredForwardPlusRenderer extends ClusteredRenderer {
     gl.bindTexture(gl.TEXTURE_2D, this._clusterTexture.glTexture);
     gl.uniform1i(this._shaderProgram.u_clusterbuffer, 3);
 
+
+    gl.uniform1f(this._shaderProgram.u_width, canvas.width);
+    gl.uniform1f(this._shaderProgram.u_height, canvas.height);
+    gl.uniform1f(this._shaderProgram.u_near, camera.near);
+    gl.uniform1f(this._shaderProgram.u_far, camera.far);
+
     // TODO: Bind any other shader inputs
 
     // Draw the scene. This function takes the shader program so that the model's textures can be bound to the right inputs