Recently, I have been doing some research about spherical harmonics for a monthly graphics round-table that we run at Snowed In Studios. In the process of solidifying my understanding of this peculiar tool, I figured it may be valuable to document the intuitions that I've developed in the process.
Before moving forward with the rest of this post, I will note that there are some truly excellent resources on spherical harmonics.
- Spherical Harmonic Lighting: The Gritty Details By Robin Green
- Spherical Harmonics By Patapom
- Stupid Spherical Harmonics (SH) Tricks By Peter-Pike Sloan
For the interested reader, I would suggest perhaps reading these in the order presented here. I read them in the reverse order and suspect that it may have been easier the other way around.
Since these excellent resources are already available, I will use that as a bit of a shortcut to skip portions of the topic if I feel that my particular lens does not add any new insights/perspectives.
As a quick whirlwind introduction, spherical harmonics can be viewed as a series of functions defined on the sphere with which we can construct any other function.
As an example, if we have a series of functions:
We could describe a target function
These types of functions are known as basis functions.
Robin Green [1] has an excellent description of these, and as a result, I think it would be to both our benefits that we simply reiterate the excellent description and visuals:
Basis functions are small pieces of signal that can be scaled and combined to produce an approximation to an original function, and the process of working out how much of each basis function to sum is called projection. To approximate a function using basis functions we must work out a scalar value that represents how much the original function
$f(x)$ is like the each basis function$B_i(x)$ . We do this by integrating the product$f(x)B_i(x)$ over the full domain of$f$ .Using this projection process over all our basis functions returns a vector of approximation coefficients. If we scale the corresponding basis function by the coefficients...
... and sum the results we obtain our approximated function.
Spherical harmonics have the additional property that they form an orthogonal basis. This means that if we integrate the product of one function multiplied by any other function in the set for a particular input range
Additionally, the result of an integral of one of our functions multiplied by itself gives us some constant
An example of a set of functions that meet these criteria are
and
Notice how in the case of
We will soon touch on why this is a valuable property for our spherical harmonics. But you can imagine that this implies that our functions do not "interfere" with each other.
An example of interference can be seen with the simple linear equations:
If we were to try to use these as a basis function for the function
In the range
If we then try to reconstruct our function with
Oops! That's not our source function at all...
(Blue is our original
If instead, we use the first 2 functions from the fundamental function of our spherical harmonics the Associated Legendre Polynomials we get:
If we use these as our basis functions, we also need to normalize our coefficients [2] with
Combining them
Tada! We've moved from our function to constants for our basis and back.
Question: Unfortunately, I'm not familiar with where this normalization constant comes from and why we have to use it here. If you happen to know, please reach out. I would be very happy to be enlightened and will happily amend this section with that information.
Edit: It seems likely that the normalization constant can be derived by following the steps outlined in at this website. I haven't done the derivation myself - but I will run with this theory for now.
You'll notice that with our original functions above
Now that we've defined orthogonal basis functions, here is a sample of the first few basis functions for our spherical harmonics
The Associated Legendre Polynomials are only the first part of the picture for our spherical harmonics. The set of spherical harmonic functions are fully defined as:
Where
Question: I'm not quite sure why we use the Associated Legendre Polynomials for spherical harmonics. Could we perhaps have used a Fourier Series instead? Is there any benefit to the use of this basis instead of another in graphics programming?
Robin Green [1] once again explains the meaning of
The two arguments
$l$ and$m$ break the family of polynomials into bands of functions where the argument$l$ is the band index and takes any positive integer value starting from 0, and the argument$m$ takes any integer value in the range [0,l]. Inside a band the polynomials are orthogonal w.r.t. a constant term and between bands they are orthogonal with a different constant.
We can break these functions into 3 distinct parts. If we look at the case where
We have
Which is our normalization constant (We will be touching on the meaning of this constant soon$^{tm}$)
Which is our Associated Legendre Polynomial applied along our vertical axis.
And
Which is our function applied along the horizon of our sphere. (This would be
If we visualize the vertical cross section of our function with
And the horizontal cross section we get:
If we combine both of these in a 3D visualization, where the horizontal cross section is the view from the top down and the vertical cross section is the view from the front we get:
from the front and
from the top. Identical to our original functions!
Additionally, here is one of my favourite basis functions
Vertical cross section:
Horizontal section:
Visualizing the function in 3D from the front:
From the top:
Hopefully that provides some insight into the primary pieces of the spherical harmonic functions!
It's important to note, that
In practice, you'll only see a fixed number of coefficients (likely a pretty small number). In game rendering, you are likely to see 9 coefficients as a common number of terms used to represent irradiance (using spherical harmonics to represent irradiance is a very common use).
Using only 9 terms means you get a very low frequency representation of your original function. As a result, you likely don't want to use a spherical harmonics representation for a high frequency function. (Something like a noise function would be very poorly approximated by spherical harmonics, see [3] for an excellent visualization of this property.).
That's why you'll likely see it being used to represent values like irradiance which vary much more slowly across our sphere.
Here is an example of the irradiance of the cubemap in the background where the sphere's radius is scaled by the magnitude of the red channel of the irradiance. Notice how this is a very smooth shape with no sharp changes.
Here is the same irradiance, but represented using 4 spherical harmonic coefficients. Notice how, although not a perfect approximation, it is definitely similar to our original function.
This is the premise underpinning the use of spherical harmonics to represent irradiance in games.
If you'd like to play around with this scene you can head over to the shadertoy here: https://www.shadertoy.com/view/7sKfzR
Given everything that we've touched on, we finally have all the pieces needed to talk about, and derive, some valuable properties of spherical harmonics.
One of the most valuable properties of spherical harmonics is that the integral of the product of two functions defined as spherical harmonic coefficients is equivalent to the dot product of those coefficients.
In essence, given two functions defined as a set of spherical harmonic coefficients:
This property, if true, could be very valuable!
Imagine, we have a function
If we were then to also define
to
This would be quite useful! We could make very interesting use of this property to transform other functions into spherical harmonic coefficients to then apply to our radiance. See [3] for an interesting combination of spherical harmonics and ambient occlusion.
I will note that it is indeed possible to turn both our radiance function and our cosine term into spherical harmonic coefficients. Patapom [3] has an excellent breakdown of the topic. I highly recommend the read.
Unfortunately, I don't plan on explaining these particular pieces of the puzzle. I find all of the three linked resources above do a wonderful job of breaking down this topic.
What I want to touch on, is actually showing why this property:
is true.
I think this is very clever and very interesting. I hope you find it even slightly as interesting as I do!
For illustration, we will start with two functions that are only represented using 2 coefficients.
As we've seen before, with these coefficients, we can reconstruct an approximation of the functions by applying the coefficients to our basis functions:
With this approximation, we can substitute our two functions with our new formulation.
Expanding our new polynomial
Distributing our integral
You may now be thinking that we haven't really simplified anything, but if you remember that our functions form an orthogonal basis. As a result
Which in this case means
And
With these identities, our middle term
And our outermost terms become
But now we're stuck with some pesky constants that we don't know...
Fear not! Some very smart people have already figured this out and we finally get to revisit an important part of our spherical harmonic basis functions.
What is it?
Our normalization constant!
From before!
This normalization constant being applied to our spherical harmonics actually changes this integral from
To
Where
As we are canceling out our constant term by dividing by our root such that
Looking at our spherical harmonic functions again
If we assume that this theory about our normalization constants is true, then we should be able to see that the first part of our function (when
When integrated across our sphere, should give us a value of
From now on - we will expand
Let's look at a simple case where
To calculate our orthogonal basis constant we need to integrate:
Since we're integrating on a sphere using our spherical harmonic function we get
If
Then we get
And if we solve
We get
Matching our derived normalization constant!
Why do we need $\sqrt{2}$ in our normalization constant when $m \neq 0$ ?
The reason we need$\sqrt{2}$ when$m \neq 0$ is relatively simple.Above, we used the property that
$$ \int_0^{2\pi}d\phi=2\pi $$
As a part of our normalization constant derivation.
However, when
$m \neq 0$ we have the additional terms$$ \displaylines{ sin(-m\phi) & m < 0 \newline cos(m\phi) & m > 0 } $$
You'll note that this changes our integration of
$\phi$ from$$ \int_0^{2\pi}d\phi=2\pi $$
to
$$ \int_0^{2\pi}cos(m\phi)^2d\phi=y $$
(Using
$m > 0$ as an example here and squared due to how we're calculating our basis constant)If we calculate the result of this integral, we get
$$ \displaylines{ \int_0^{2\pi}sin(-m\phi)^2d\phi=\pi & m < 0 \newline \int_0^{2\pi}cos(m\phi)^2d\phi=\pi & m > 0 } $$
You'll notice that we're missing a factor of 2 from this integral as opposed to the version without a
$cos$ or$sin$ term!However, our original normalization constant was written with the expectation that we would have a factor of
$2\pi$ . As a result, we need to reintroduce this factor as a new constant. (As a square root here because of the formulation of our normalization constant as explained above)
To recap these results, we can show that
By using the fact that
But not only that the constant of the product of a single function is
This allows us to finally finish our derivation of:
Since
And this property continues for any number of coefficients. (If unconvinced, I suggest running through the exercise for
Another property of our coefficients, is that interpolating the result of our functions at any
First, we will use lerp in the form
As we saw before, we can define our function in terms of our basis coefficients and our basis functions
If we plug these into our lerp equation (using the 2 coefficient form for simplicity)
If we look at the final form of our equation
We can package the terms before our basis functions as new basis coefficients
Which as we've seen above, is the form of our
Finally, for a few additional simple derivations before we wrap things up.
Proof that addition of our functions is equivalent to addition of our coefficients.
Replacing our functions with our coefficients and basis functions we get:
Proof that multiplying our function by a scalar is equivalent to multiplying our coefficients.
Replacing our functions with our coefficients and basis functions we get:
And we're done! The math is complete! Thanks for reading all the way to the end of this part!
Hopefully, this has provided some interesting insights into the magical world of spherical harmonics!
I know that I find myself far more comfortable in the meaning of these functions and their implementations after writing this.
If you haven't read the 3 links shared above, I highly recommend them. I found them incredibly insightful and they touch on a variety of practical and theoretical aspects of spherical harmonics that I did not touch on in this post. (Especially in terms of their practical usage in graphics programming)
If you're interested in visualizing and playing with the various pieces described here. I've created a few shadertoys that you can explore and modify to your hearts content (These were used for the various graphs and images shown above).
- https://www.shadertoy.com/view/ssVfRR - Visualize the Associated Legendre Polynomials
- https://www.shadertoy.com/view/sdGBRh - Visualize how the spherical harmonics are applied to a circle for both the vertical and horizontal cross sections.
- https://www.shadertoy.com/view/7sKBzR - Visualize spherical harmonics in 3D where the value of the harmonic is used to scale a unit sphere.
- https://www.shadertoy.com/view/7sKfzR - Visualize a representation of irradiance using spherical harmonics along with a ground truth irradiance calculation for comparison.
As presented throughout the post, I am left with some lingering questions. If you happen to have insights about them, please reach out!
- I'm not quite sure why we use the Associated Legendre Polynomials for spherical harmonics. Could we perhaps have used a Fourier Series instead? Is there any benefit to the use of this basis instead of another in graphics programming?
This is a bit of an experimental approach to a blog post. Hopefully this was easy to read and navigate.
Bye!
[1] https://3dvar.com/Green2003Spherical.pdf
[2] https://en.wikipedia.org/wiki/Legendre_polynomials#Orthogonality_and_completeness
[3] https://patapom.com/blog/SHPortal/