Multi-layer compositing and the use of bloom (or other transparent effects)

Using Cinder
1

Hey embers,

I’d like to share my problem and solution in the hopes of either just spreading some minor knowledge, or hearing other wise thoughts.

I’m building some software similar to standard video composition tools. Meaning, I have multiple visual elements that I tweak individually, before eventually stacking them all together. Kind of like multiple layers of a sandwich. I can tweak the bread, ham, cheese individually, and eventually stack them all together.

Recently, I’ve started fiddling with transparency and am having trouble properly compositing these individual element. Take, as an example the following two elements:

Untitled Diagram
Untitled Diagram921×391 47.5 KB

Layer 1, a spaceship with @paul.houx cool bloom effect. I hope you won’t mind me leveraging your sweet art for the purposes of this explanation.

Layer 2, a colorful gradient background.

Currently, each layer has its own FBO. Complete spaceship with bloom in fbo 1, and gradient background in fbo 2. Simple.

Unfortunately, compositing these fully complete layers isn’t as straight-forward. The background layer is easy, that’s just opaque. But putting the complete spaceship on top afterwards isn’t straight-forward. We’d like the actual spaceship to be fully opaque, but its bloom to properly bleed out and on to the colorful gradient background.

At first I thought this was a pre-multiplied alpha issue, but now I’m thinking it’s actually more about information loss.

I use @paul.houx’ awesome blur shader for the bloom effect. It keeps the alpha channel at 1.0, and uses additive blending to apply the bloom. The problem, I believe, stems from having these two layers finish rendering individually into their own FBO’s. What I mean is, the moment the bloom effect is added on to the actual spaceship, we can no longer properly tell which pixels are to be 100% opaque, and which should be a little see-through. There’s no way to differentiate between the two.

It’s likely possible to have the blur shader calculate a color texture where the alpha value represents how much to add, and keeping this alpha channel in-tact to then later properly add this on top of the gradient background. But even if I figured out how to do properly do the math in such a shader, I feel I’m just working towards a much more complex solution, when the far easier approach is probably to instead just ensure that the individual layers are actively rendered into the same FBO as they’re constructed.

So, for example, there’d exist one composite FBO, which would first have the gradient rendered into it, then the opaque spaceship on top, and finally the bloom effect on top of that. Of course, you have to temporarily calculate the bloom effect into a separate fbo, but it’d get added to a composition existing of both the gradient background and the space-ship.

So I feel the lesson here is: If your graphics output has to composite multiple layers of visuals, and some of them will use transparency, it’s probably less painful to aim for a method that actively stacks them during rendering, rather than finish them all individually and then composite them together.

I’d much welcome any thoughts on the subject.

Cheers,

Gazoo

P.s. As a side note - two great links discussing pre-multiplied alpha which I proceeded to read up on, but have come to the conclusion that my woe’s are only tangentially related to.

2

Hi,

the bloom sample renders the scene to a black background (with alpha set to 1). The subsequent blur shaders also sample from the background, so it becomes part of the blurred result. Blooming then samples the blurred result to produce the bloom effect. In the final image, the black background has become part of the final image and there is no way to just use the space ship and the bloom effect separately.

You could try to clear the scene buffer with ColorA(0, 0, 0, 0) instead, so you start with a transparent background. The subsequent passes then don’t have a background and compositing may be easier.

I’d recommend to use premultiplied alpha for your passes, as this may make compositing multiple Fbo’s easier.

~Paul

1 Like
3

Hey Paul,

Thanks for chiming in, and again, thanks for the awesome sample code you’ve thrown together. It’s been very helpful in numerous ways.

Two points on your response

  1. When you state

I feel like I’m overlooking something in the sample code. Your statement makes it sound like the blooming is a separate effect from the blurring where as I would say the bloom is effectively the blur, albeit applied to a specific subset of the scene. Or is that not correct?

  1. The problem I face with having the alpha channel be set to 0.0, is I’m not sure how to alter the blur shader so it produces a useful result. It currently calculates rgb without and alpha channel and then uses additive blending. My current graphics pipeline does not have the full background (i.e. the colorful gradient) available at that time, so once the bloom/blur is applied to the spaceship, I’m toast. I can no longer tell regular opaque color contributions from the more transparent ones the bloom/blur applies.

This is where I’m not sure this is a decent alternative available to rewrite the pipeline have the actual background available at that time.

Cheers,
Gazoo

4

Ah yeah, my bad: the bloom effect is indeed created by additively blending the blurred image on top of the original image. So ignore my ramblings.

Regarding blurring with alpha: my gut feeling is that if your original image (the spaceship on a transparent background) is using premultiplied alpha, you can use the full RGBA vec4 when blurring, not just the RGB vec3. Could you try that?

~Paul

5

No worries. Appreciate any reply tbh @paul.houx.

I have attempted this simple tweak, unfortunately, it results in a very different visual impression.

Blurring of grid, where alpha is fixed to 1.0.

frame117
frame1171280×800 172 KB

Blurring of grid where alpha is mixed just like the rgb channels.

frame229
frame2291280×800 108 KB

When the alpha channel is blurred and the results are composited using additive blending, the bloom comes up so meekly, so when it’s eventually added in it’s either invisible or completely overblown.

I’ve googled ‘blur using alpha channel’ and a few other keywords, but not been able to figure out a way to achieve the same look with the alpha channel involved.

Note the composited background is currently dark-grey, but that’s sort of beside the point. Just wanted to note that the 2nd render does indeed let a background layer through due to the composited grid and bloom effect resulting in a transparent image.

Cheers,
Gazoo