I’d like to share my problem and solution in the hopes of either just spreading some minor knowledge, or hearing other wise thoughts.
I’m building some software similar to standard video composition tools. Meaning, I have multiple visual elements that I tweak individually, before eventually stacking them all together. Kind of like multiple layers of a sandwich. I can tweak the bread, ham, cheese individually, and eventually stack them all together.
Recently, I’ve started fiddling with transparency and am having trouble properly compositing these individual element. Take, as an example the following two elements:
Layer 1, a spaceship with @paul.houx cool bloom effect. I hope you won’t mind me leveraging your sweet art for the purposes of this explanation.
Layer 2, a colorful gradient background.
Currently, each layer has its own FBO. Complete spaceship with bloom in fbo 1, and gradient background in fbo 2. Simple.
Unfortunately, compositing these fully complete layers isn’t as straight-forward. The background layer is easy, that’s just opaque. But putting the complete spaceship on top afterwards isn’t straight-forward. We’d like the actual spaceship to be fully opaque, but its bloom to properly bleed out and on to the colorful gradient background.
At first I thought this was a pre-multiplied alpha issue, but now I’m thinking it’s actually more about information loss.
I use @paul.houx’ awesome blur shader for the bloom effect. It keeps the alpha channel at 1.0, and uses additive blending to apply the bloom. The problem, I believe, stems from having these two layers finish rendering individually into their own FBO’s. What I mean is, the moment the bloom effect is added on to the actual spaceship, we can no longer properly tell which pixels are to be 100% opaque, and which should be a little see-through. There’s no way to differentiate between the two.
It’s likely possible to have the blur shader calculate a color texture where the alpha value represents how much to add, and keeping this alpha channel in-tact to then later properly add this on top of the gradient background. But even if I figured out how to do properly do the math in such a shader, I feel I’m just working towards a much more complex solution, when the far easier approach is probably to instead just ensure that the individual layers are actively rendered into the same FBO as they’re constructed.
So, for example, there’d exist one composite FBO, which would first have the gradient rendered into it, then the opaque spaceship on top, and finally the bloom effect on top of that. Of course, you have to temporarily calculate the bloom effect into a separate fbo, but it’d get added to a composition existing of both the gradient background and the space-ship.
So I feel the lesson here is: If your graphics output has to composite multiple layers of visuals, and some of them will use transparency, it’s probably less painful to aim for a method that actively stacks them during rendering, rather than finish them all individually and then composite them together.
I’d much welcome any thoughts on the subject.
P.s. As a side note - two great links discussing pre-multiplied alpha which I proceeded to read up on, but have come to the conclusion that my woe’s are only tangentially related to.