I definitely have a big interest in this (It sounds like it'd be the more advanced-bits mainly (I've worked in VFX for 23 years)) but was a bit disheartened to learn that it was a paid service.
I also searched the 'try it for free' version and the word 'aliasing' only seemed-to-be-mentioned once!? I would've thought that the arch-enemy of computer graphics (at least the raster-display kind) might have gotten more of a mention!?
It also seemed odd to not really talk about 'filtering' and (at least from a brief investigation) it doesn't seem to really concern itself much with pixels at all! Maybe that's why there's a section titled "Rendering is Very Hard"?) Edit: I read-on from there a bit and this turns out to be about algorithmic complexity side of the rendering equation and not that it's hard to talk about pixels! fair enough! this document still seems more about CG in-the-abstract than the practical to me!
At least 'smootherstep()' got a mention, although what about 'smootheststep()'!? (easier to read if u split the words up: "smoothest step" example here: https://www.shadertoy.com/view/ltSfWV)
This is a book mostly about writing a more modern Monte Carlo global illumination path tracing style of renderer, rather than, say, a rasterizer. This is why ‘antialiasing’ is replaced with ‘sampling’ and ‘integration’. When you start with Monte-Carlo and the assumption that you’re multi-sampling, aliasing and filtering are generally small problems, not big ones. The code provided does loop over pixels. You’re right that the book doesn’t contain a chapter on signal processing, but graphics is now far too big a topic to be covered in one book, and it’s certainly questionable whether one needs to rabbit-hole on filtering in a college intro course; Monte-Carlo integration of multiple rays over a pixel is okayish at antialiasing & filtering, and pretty good for a first renderer. Antialiasing and filtering aren’t solved problems anyway, especially for HDR, so it makes sense to leave that topic for advanced courses.
Thanks! I see what you're getting-at and mostly agree but I think it really would be helpful to mention a bit more about the actual pixel-production side of things in a document like this, to help make it a more complete picture of the whole rendering process.
For instance, just shooting N-samples-per-pixel and then averaging them, is not what most 'Production'-renderers used in VFX actually get up to. There's also the 'Pixel Filter' (Box, Gaussian, Catmull-rom, Sinc, Nearest, Farthest, Highest-coverage etc) that the samples have to go through too! And I think it's useful and interesting to cover some aspects of that, even if it's not a full-on signal-processing section.
It is a very useful topic for a graphics degree or someone writing a production renderer, I agree with that, but it’s obvious that Morgan made a conscious decision here, and obvious that the target audience of this book is not people writing production renderers (yet). Morgan knows all about pixel filters and antialiasing techniques. He’s an insanely knowledgeable and smart person, and he gathered a ton of feedback from a lot of other insanely knowledgeable and smart people. (FWIW I also know about filtering in production, and have worked in VFX and CG films, written parts of production renderers, and I still agree with Morgan’s choice.)
It’s not possible to get a complete picture of production renderers in one book, that’s a big broad topic that takes years. This is especially true for an introductory book. PBRT is the closest thing we have, and it’s not an intro book, and it still leaves out lots of things production people deal with.
What you can’t see easily is what tradeoffs Morgan made in the Graphics Codex. What subjects here might have been dropped if he’d included signal processing? The truth is that everything covered in this book has an advanced side to it that wasn’t included. While filtering might be your favorite topic, someone else might say that Bidirectional & Metropolis path tracing really should have been included, someone else might insist it should cover rasterizing, or Radiosity, or CIE XYZ color response functions. There’s another thing production renderers do that isn’t covered here: color management. But, simply using sRGB gets you 90% of the way there.
Box filtering via averaging N samples over a pixel gets you at least 80% or 90% of the way there, and it’s absolutely trivial to explain and implement. That other ten or twenty percent only comes up in relatively rare cases - if you don’t have any high frequency geometry or textures, Box filtering is “good enough”. Adding a Gaussian or any other pixel filter takes a lot more time to explain, requires an invasive refactor of the main render loop, still only gets you to 95% or 98%, and then leads you into a place where it’s not obvious what the best choice is or even how to find it… even production studios don’t know, and they spend time evaluating and re-evaluating pixel and texture filters.
Does that make sense? I think it’s fair to ask the question and think about why pixel filters weren’t included here. I bet Morgan would happily answer the question via email. But I also think coming to a conclusion this particular book should cover signal processing is making premature assumptions. The book is meant to give a solid, practical, and easy introduction, to college students, not to ensure the reader is an expert and production ready after 1 book, right?
These days it’s worth noting that production renderers are all introducing denoisers, which is eating into pixel filter territory. It’s not obvious or a given that production renderers are going to need to continue to worry about pixel filtering in the future. On the real-time side, DLSS for example, is amazing at producing nicely filtered antialiased images, both pixels and texels, without needing to implement either.
Thanks heaps dahart for your extremely thoughtful reply! I totally see what you're getting at and I feel silly for having even accidentally suggested that Morgan was not across anything tbh!
I still think that even an extremely cursory mention about how to make pixels like 'We will average samples from N rays within each pixel as our antialiasing strategy, this is equivalent to box-filtering (see citations [X][Y][Z])" I think something like that would be pretty helpful just to connect the dots between the amazing ideas in the book and actual-pixel-production. (Filtering/signal-processing is not totally a pet-favorite idea btw, I just think it's good to have a mention of it so that it's at least connected with the rest of a graphics-HOWTO in a non-abstract way)
Very interesting point about AI-based denoisers too.. definitely changes the pixel-filtering situation in an interesting way! Although I'm super interested in it I'm sticking my fingers in my ears about that for the moment!
Fair point, you’re right it could/should mention box filters somewhere. It has the let’s average samples over a pixel part, and it has decent citations (I noticed Alvy Ray Smith’s “A Pixel is Not a Little Square”). Connecting the averaging to basic terminology is a completely reasonable thing to do.
Apologies I didn’t mean to suggest filtering is your pet topic, filtering is very important (and fun & fascinating), mandatory for a graphics education, I only wanted to point out there are other equally deserving topics for an intro book/course. I was lucky enough to take graphics from Peter Shirley, and the first two topics he covered were filtering and colors, in depth. I’d never truly understood the shortcomings of box filtering until he had us rendering a 2d sinc function and try to get rid of any hint of aliasing with a million samples per pixel. That assignment made it really tangible and visceral, gave real practical intuition for filtering that stuck with me.
Definitely start playing with denoisers. There’s a lot to do and it needs people who care about filtering to scrutinize it.
I'm writing a book on computer graphics, I would love to have a list of in depth topics people are interested in :) let me know if you have some requirements
These are not requirements (it's your book!) but here are some suggestions for things that I'd find interesting:
Convolution (including via-FFT where it's mainly just complex-multiplication). Different neighborhoods and operations on them: Laplacian, curl/divergence, derivatives/slopes, completely-custom-weights/operations etc. Non-raster (ie analytical) convolution is something I find fascinating and haven't experimented with.
Noises both in the Perlin sense and the Blue-noise sense. Such a fundamentally useful building block!
Dealing with anisotropy both in shading, and texture filtering.
Color spaces. Hardly need to say more than that (apart from maybe 'HSV' should get a solid go). I think getting some solid clarity on what 'linear-to-light- color spaces are and how and when to work with them, is a critically important thing that is very often overlooked.
Acceleration structures - you often see stuff written about these for 3d (to speed up ray-intersection-tests for example) but I think there's probably lots of interesting acceleration stuff for 2d graphics too!
Feedback (ie 'solvers') in the 2D pixel realm - Game of Life (I have HashLife in mind) is just the tiniest-tip of the iceberg there, and Reaction Diffusion takes some solid steps but there's also repeated-warping (ie cheap 'advection') and repeated neighborhood-based pixel/cell operations that I think are an under-explored area.
Complex numbers and Orbits-based fractal-rendering. Mainly because I think it's fascinating, but also very useful for getting more comfortable with complex numbers and antialiasing these beasts that can indeed cause a LOT of aliasing!
Lots-of-points style fractal/etc rendering (eg Flame fractals, IFS, Buddhabrot, that kind of thing, or just HEAPS of particles!) Including how to cope with zooming-in when using that rendering scheme (I have not used it myself but I heard-about and saw great things that used 'Metropolis sampling' in order to make zooming practical.)
I think it would be great to include a section covering all the amazing work that originates in the demoscene and all the great techniques that have come from it. ShaderToy is one example of this, and there are many more.
That's enough for now! Best of luck with your book! Happy to talk further if u want! Email is on my profile.
Nice idea, but I'm also planning a book on advanced cuda programming, and I will cover convolutions there, I'll think if it is interesting to have some kind of cross chapter that is valid for both books.
> Noises
I cover those in normal mapping
> anisotropy
I have a full chapter on brdf that includes anysotropy for raster and raytracing, I use it also to derive anisotropic filtering
> Color spaces
I have to make choices, and I don't think I'll cover in depth color spaces, it's a topic I know, but I find it very annoying to write about, in particular, making it fun to read is difficult
> acceleration structures
I will have a short section on them as I use them for raytracing, at the moment I use only a basic octree, but I could expand to include other partitioning structures if that does not make the book too heavy to read
> diffusion processes and 2d/3d solvers
Will be better in the cuda book
> complex numbers
I derive complex, quaternions, and geometric algebra for rotations
> orbit based fractals
I'm not familiar with this topic, apart from basic fractal rendering, I'll look into it
You may find this paper fun to read https://arxiv.org/abs/2402.06184 even if it's not exactly about their rendering
> metropolis sampling
I do not expect to write about mcmc and metropolis in this book, but I keep the idea
> Demoscene
I need to ask a friend that worked in the Demoscene if he would be interested in write something, but I'm not the right person to write about it as I only have a summary knowledge of it, I'm afraid that will be out of scope
some really in-depth coverage of debugging and troubleshooting tools, e.g. usage of RenderDoc, but not just fluffy introductions, but some really deep down guides.
and something similar for performance profiling also!
it looks alright -- 10 bucks is not much for a book. but there are already many great graphics books out there, and resources like inigo quilez's blog are absolutely incredible for the burgeoning graphics programmer
My current project requires better solutions for conic sections than I have thus far been able to find --- the last time I tried solving the problem I was chaining trigonometric triangles and got 5 or 7 deep (plus calculating the chord of a circle) and performance bogged down --- this was in OpenSCAD, and I think before Manifold, so maybe I need to revisit my code in Python in (Open)PythonSCAD w/ Manifold enabled.
It doesn’t really compare, very different topics and very different content. The Graphics Codex is about rendering, and Projective Geometric Algebra is pure geometry presented in a subset of Clifford Algebra. I think the only ‘conics’ covered in the Graphics Codex might be sphere-line intersection, and calling that conics is a stretch. ;)
Out of curiosity, what kind of conics problem are you trying to solve?
This seems quite interesting, I'll probably read it later.
"Computer graphics from scratch" by Gabriel Gambetta is also a good resource if you want to write a renderer
I definitely have a big interest in this (It sounds like it'd be the more advanced-bits mainly (I've worked in VFX for 23 years)) but was a bit disheartened to learn that it was a paid service.
I also searched the 'try it for free' version and the word 'aliasing' only seemed-to-be-mentioned once!? I would've thought that the arch-enemy of computer graphics (at least the raster-display kind) might have gotten more of a mention!?
It also seemed odd to not really talk about 'filtering' and (at least from a brief investigation) it doesn't seem to really concern itself much with pixels at all! Maybe that's why there's a section titled "Rendering is Very Hard"?) Edit: I read-on from there a bit and this turns out to be about algorithmic complexity side of the rendering equation and not that it's hard to talk about pixels! fair enough! this document still seems more about CG in-the-abstract than the practical to me!
At least 'smootherstep()' got a mention, although what about 'smootheststep()'!? (easier to read if u split the words up: "smoothest step" example here: https://www.shadertoy.com/view/ltSfWV)
This is a book mostly about writing a more modern Monte Carlo global illumination path tracing style of renderer, rather than, say, a rasterizer. This is why ‘antialiasing’ is replaced with ‘sampling’ and ‘integration’. When you start with Monte-Carlo and the assumption that you’re multi-sampling, aliasing and filtering are generally small problems, not big ones. The code provided does loop over pixels. You’re right that the book doesn’t contain a chapter on signal processing, but graphics is now far too big a topic to be covered in one book, and it’s certainly questionable whether one needs to rabbit-hole on filtering in a college intro course; Monte-Carlo integration of multiple rays over a pixel is okayish at antialiasing & filtering, and pretty good for a first renderer. Antialiasing and filtering aren’t solved problems anyway, especially for HDR, so it makes sense to leave that topic for advanced courses.
Thanks! I see what you're getting-at and mostly agree but I think it really would be helpful to mention a bit more about the actual pixel-production side of things in a document like this, to help make it a more complete picture of the whole rendering process.
For instance, just shooting N-samples-per-pixel and then averaging them, is not what most 'Production'-renderers used in VFX actually get up to. There's also the 'Pixel Filter' (Box, Gaussian, Catmull-rom, Sinc, Nearest, Farthest, Highest-coverage etc) that the samples have to go through too! And I think it's useful and interesting to cover some aspects of that, even if it's not a full-on signal-processing section.
It is a very useful topic for a graphics degree or someone writing a production renderer, I agree with that, but it’s obvious that Morgan made a conscious decision here, and obvious that the target audience of this book is not people writing production renderers (yet). Morgan knows all about pixel filters and antialiasing techniques. He’s an insanely knowledgeable and smart person, and he gathered a ton of feedback from a lot of other insanely knowledgeable and smart people. (FWIW I also know about filtering in production, and have worked in VFX and CG films, written parts of production renderers, and I still agree with Morgan’s choice.)
It’s not possible to get a complete picture of production renderers in one book, that’s a big broad topic that takes years. This is especially true for an introductory book. PBRT is the closest thing we have, and it’s not an intro book, and it still leaves out lots of things production people deal with.
What you can’t see easily is what tradeoffs Morgan made in the Graphics Codex. What subjects here might have been dropped if he’d included signal processing? The truth is that everything covered in this book has an advanced side to it that wasn’t included. While filtering might be your favorite topic, someone else might say that Bidirectional & Metropolis path tracing really should have been included, someone else might insist it should cover rasterizing, or Radiosity, or CIE XYZ color response functions. There’s another thing production renderers do that isn’t covered here: color management. But, simply using sRGB gets you 90% of the way there.
Box filtering via averaging N samples over a pixel gets you at least 80% or 90% of the way there, and it’s absolutely trivial to explain and implement. That other ten or twenty percent only comes up in relatively rare cases - if you don’t have any high frequency geometry or textures, Box filtering is “good enough”. Adding a Gaussian or any other pixel filter takes a lot more time to explain, requires an invasive refactor of the main render loop, still only gets you to 95% or 98%, and then leads you into a place where it’s not obvious what the best choice is or even how to find it… even production studios don’t know, and they spend time evaluating and re-evaluating pixel and texture filters.
Does that make sense? I think it’s fair to ask the question and think about why pixel filters weren’t included here. I bet Morgan would happily answer the question via email. But I also think coming to a conclusion this particular book should cover signal processing is making premature assumptions. The book is meant to give a solid, practical, and easy introduction, to college students, not to ensure the reader is an expert and production ready after 1 book, right?
These days it’s worth noting that production renderers are all introducing denoisers, which is eating into pixel filter territory. It’s not obvious or a given that production renderers are going to need to continue to worry about pixel filtering in the future. On the real-time side, DLSS for example, is amazing at producing nicely filtered antialiased images, both pixels and texels, without needing to implement either.
Thanks heaps dahart for your extremely thoughtful reply! I totally see what you're getting at and I feel silly for having even accidentally suggested that Morgan was not across anything tbh!
I still think that even an extremely cursory mention about how to make pixels like 'We will average samples from N rays within each pixel as our antialiasing strategy, this is equivalent to box-filtering (see citations [X][Y][Z])" I think something like that would be pretty helpful just to connect the dots between the amazing ideas in the book and actual-pixel-production. (Filtering/signal-processing is not totally a pet-favorite idea btw, I just think it's good to have a mention of it so that it's at least connected with the rest of a graphics-HOWTO in a non-abstract way)
Very interesting point about AI-based denoisers too.. definitely changes the pixel-filtering situation in an interesting way! Although I'm super interested in it I'm sticking my fingers in my ears about that for the moment!
Fair point, you’re right it could/should mention box filters somewhere. It has the let’s average samples over a pixel part, and it has decent citations (I noticed Alvy Ray Smith’s “A Pixel is Not a Little Square”). Connecting the averaging to basic terminology is a completely reasonable thing to do.
Apologies I didn’t mean to suggest filtering is your pet topic, filtering is very important (and fun & fascinating), mandatory for a graphics education, I only wanted to point out there are other equally deserving topics for an intro book/course. I was lucky enough to take graphics from Peter Shirley, and the first two topics he covered were filtering and colors, in depth. I’d never truly understood the shortcomings of box filtering until he had us rendering a 2d sinc function and try to get rid of any hint of aliasing with a million samples per pixel. That assignment made it really tangible and visceral, gave real practical intuition for filtering that stuck with me.
Definitely start playing with denoisers. There’s a lot to do and it needs people who care about filtering to scrutinize it.
> paid service
What's new section says:
App Changes
Ah nice! That is very cool thanks for the heads-up!
I'm writing a book on computer graphics, I would love to have a list of in depth topics people are interested in :) let me know if you have some requirements
These are not requirements (it's your book!) but here are some suggestions for things that I'd find interesting:
Convolution (including via-FFT where it's mainly just complex-multiplication). Different neighborhoods and operations on them: Laplacian, curl/divergence, derivatives/slopes, completely-custom-weights/operations etc. Non-raster (ie analytical) convolution is something I find fascinating and haven't experimented with.
Noises both in the Perlin sense and the Blue-noise sense. Such a fundamentally useful building block!
Dealing with anisotropy both in shading, and texture filtering.
Color spaces. Hardly need to say more than that (apart from maybe 'HSV' should get a solid go). I think getting some solid clarity on what 'linear-to-light- color spaces are and how and when to work with them, is a critically important thing that is very often overlooked.
Acceleration structures - you often see stuff written about these for 3d (to speed up ray-intersection-tests for example) but I think there's probably lots of interesting acceleration stuff for 2d graphics too!
Feedback (ie 'solvers') in the 2D pixel realm - Game of Life (I have HashLife in mind) is just the tiniest-tip of the iceberg there, and Reaction Diffusion takes some solid steps but there's also repeated-warping (ie cheap 'advection') and repeated neighborhood-based pixel/cell operations that I think are an under-explored area.
Complex numbers and Orbits-based fractal-rendering. Mainly because I think it's fascinating, but also very useful for getting more comfortable with complex numbers and antialiasing these beasts that can indeed cause a LOT of aliasing!
Lots-of-points style fractal/etc rendering (eg Flame fractals, IFS, Buddhabrot, that kind of thing, or just HEAPS of particles!) Including how to cope with zooming-in when using that rendering scheme (I have not used it myself but I heard-about and saw great things that used 'Metropolis sampling' in order to make zooming practical.)
I think it would be great to include a section covering all the amazing work that originates in the demoscene and all the great techniques that have come from it. ShaderToy is one example of this, and there are many more.
That's enough for now! Best of luck with your book! Happy to talk further if u want! Email is on my profile.
> Convolution
Nice idea, but I'm also planning a book on advanced cuda programming, and I will cover convolutions there, I'll think if it is interesting to have some kind of cross chapter that is valid for both books.
> Noises I cover those in normal mapping
> anisotropy I have a full chapter on brdf that includes anysotropy for raster and raytracing, I use it also to derive anisotropic filtering
> Color spaces I have to make choices, and I don't think I'll cover in depth color spaces, it's a topic I know, but I find it very annoying to write about, in particular, making it fun to read is difficult
> acceleration structures I will have a short section on them as I use them for raytracing, at the moment I use only a basic octree, but I could expand to include other partitioning structures if that does not make the book too heavy to read
> diffusion processes and 2d/3d solvers Will be better in the cuda book
> complex numbers I derive complex, quaternions, and geometric algebra for rotations
> orbit based fractals I'm not familiar with this topic, apart from basic fractal rendering, I'll look into it You may find this paper fun to read https://arxiv.org/abs/2402.06184 even if it's not exactly about their rendering
> metropolis sampling I do not expect to write about mcmc and metropolis in this book, but I keep the idea
> Demoscene I need to ask a friend that worked in the Demoscene if he would be interested in write something, but I'm not the right person to write about it as I only have a summary knowledge of it, I'm afraid that will be out of scope
Thanks very much for all the ideas!
some really in-depth coverage of debugging and troubleshooting tools, e.g. usage of RenderDoc, but not just fluffy introductions, but some really deep down guides.
and something similar for performance profiling also!
OK thanks! The book will cover most of the math's in an intuitive way but rigorously nonetheless. It aims to build the theory from ground up.
I already have a full chapter on aliasing.
The exercices use renderdoc and nvidia tools ;)
Compute shaders from a high-level pipeline perspective. How data moves from CPU and between shaders to accomplish a task.
> a paid service
It's a book you purchase. I've never had a book charge me a recurring fee, and this one is no exception.
it looks alright -- 10 bucks is not much for a book. but there are already many great graphics books out there, and resources like inigo quilez's blog are absolutely incredible for the burgeoning graphics programmer
> but there are already many great graphics books out there
Would you mind listing your personal top n books?
Where is the $10 purchase link?
It used to be on the iOS App Store. But, apparently it is “No longer available”?
Anyone know how to buy it today?
Apparently it is now free as noted elsethread.
How does this compare to:
https://projectivegeometricalgebra.org/
My current project requires better solutions for conic sections than I have thus far been able to find --- the last time I tried solving the problem I was chaining trigonometric triangles and got 5 or 7 deep (plus calculating the chord of a circle) and performance bogged down --- this was in OpenSCAD, and I think before Manifold, so maybe I need to revisit my code in Python in (Open)PythonSCAD w/ Manifold enabled.
It doesn’t really compare, very different topics and very different content. The Graphics Codex is about rendering, and Projective Geometric Algebra is pure geometry presented in a subset of Clifford Algebra. I think the only ‘conics’ covered in the Graphics Codex might be sphere-line intersection, and calling that conics is a stretch. ;)
Out of curiosity, what kind of conics problem are you trying to solve?
I am trying to work out a toolpath problem for tapered ball nose tooling on CNC machines as a part of:
https://github.com/WillAdams/gcodepreview
This seems quite interesting, I'll probably read it later. "Computer graphics from scratch" by Gabriel Gambetta is also a good resource if you want to write a renderer
This looks like it focuses on path tracing.
In which case, https://pbr-book.org/ is a great, completely free book.
It’s a great book!