Really nice article! Succinctly demonstrates the problem with not using premultiplied alpha.
> As an Artist: Make it Bleed!
> If you’re in charge of producing the asset, be defensive and don’t trust the programmers or the engine down the line.
If you are an artist working with programmers that can fix the engine, your absolute first choice should be to ask them to fix the blending so they convert your non-premultiplied images into premultiplied images before rendering them!
Do not start bleeding your mattes manually if you have any say in the matter at all, that doesn't solve the whole problem, and it sets you up for future pain. The only right answer is for the programmers to use premultiplied images. What if someone decides to blur your bled transparent image? It will break. (And there are multiple valid reasons this might happen without your input.)
Even if you have no control over the engine, file a bug report. But in that case, go ahead and bleed your transparent images manually & do whatever you have to, to get your work done.
Eric Haines wrote a more technical piece on this problem that elaborates on the other issues besides halo-ing:
I'm not sure I understand your concern. If the software converts all your assets into a premultiplied form, the bleeding you applied won't hurt anything even if it doesn't help. Yes, it's extra work that shouldn't be necessary - but we often find ourselves living in an imperfect world.
I completely agree that premultiplied alpha should be used everywhere. I'd even go a step farther and say that you should use high bit depth linear values too, but that's a topic for another day.
Just pointing out that attempting to solve the real problem should be tried first, before jumping to work-arounds.
If the software converts to pre-multiplied, then there'd be no halo problem and no bleeding necessary, right?
You're right that bleeding won't hurt assets if you're comping, but it will hurt assets if you're not converting to premult and then do texture filtering or mipmapping or blurring.
My concern is with using bleeding is the article's suggestion to use bleeding as a first resort, rather than a last resort (as an artist). It's a hack that totally works in a lot of cases, but it's still a hack. I've watched artists in film and games use random combinations of bleeding, (un)pre-multiply, gamma, and other stuff whenever something goes wrong with matting, and often it's not the right solution. A lot of people are scared of understanding premultiplied alpha - and the technical name isn't doing anyone any favors - and instead of figuring out the right solution they try every combination of hacks until it works. General misunderstanding and superstition about premultiplied alpha is the most common reason I've seen for people using un-premultiply nodes in production.
A 16bpp linear pre-multiplied format would be awesome. sRGB is a pain to use in practice:
- Slow to convert to linear colour space (requires a pow).. except
- GPUs use an 8-bit lookup table to convert input sRGB to output linear value.
- This doesn't work for more than 8 bits as the table gets excessively large very quickly.
It's a pity PNG doesn't have flag to mark the image data as being pre-multiplied.
That little ditty is also very likely why PNG has zero uptake in visual effects and CGI.
Associated (aka premultiplied) alpha is the _sole_ means to embody both occlusion and emission. Unassociated (aka straight or key) alpha cannot represent these facets.
Consider a candle flame that exists as mostly emission and low to no occlusion. With associated alpha, you can use zero alpha triplets with non-zero emission RGB to represent this real-world scenario. With unassociated alpha? Impossible.
> That little ditty is also very likely why PNG has zero uptake in visual effects and CGI.
The main reason why CG studios can't use PNG is because both renderers and compositors always output pre-mult images. Yes you can choose to un-pre-mult them after rendering, but the native result of a blending operation is always a pre-mult color, regardless of the input sources. Unpremultiplied blending still results in premultiplied colors.
Since everyone knows that un-premultiplying (dividing) is to be avoided at all costs, it means you can't render or comp something and then save the file in PNG.
> Associated (aka premultiplied) alpha is the _sole_ means to embody both occlusion and emission.
I would suggest avoiding thinking of image colors as emissive. That's a material property, and using RGBA to encode material properties is only something you'd do if you were stuck in a weird fixed-function pipeline with no choice, or if you were really really low on disk space. Otherwise, emission colors go in their own separate emission channel that doesn't have an alpha value.
Feel free to reference the original Porter Duff paper regarding "luminescent pixels" or Alvy Smith's opinion on the matter as relayed by Zap Andersson in the legendary Adobe thread.
Remember that a ray tracing engine uses associated alpha as that is the sole format it can generate. Only associated alpha models emission and occlusion.
Yes you're right; I wasn't arguing with you about that. I always feel like calling it additive rather than emissive. But just because you can represent additive colors in premult images doesn't mean you should, and I'd speculate wildly that it occurs less often in production than halo problems. Someone who writes lens flare and rainbow shaders is going to scold me for saying that though...
In the context of the OP's article, and of artists who paint images with transparency in them, worrying about emissive colors isn't really an issue. Artists very rarely paint pre-mult images, they can't work with premult images, generally speaking. No doubt a few people who know what they're doing do it, but I can't personally say I've ever seen an artist painted premult image with emissive colors, nor do I recall ever seeing a software rendered layer with emissive colors either. Is this common now? I've been out of film & games for a few years now.
I'm already familiar with everything you referenced; and I can vouch that it's all very good stuff so thanks for sharing, especially the Adobe thread. I hope others here benefit. It's amusing that an entire industry knows who Chris Cox is because of this thread, right? :P
> Consider a candle flame that exists as mostly emission and low to no occlusion.
I don't think associated alpha helps much here. You can special case pixels that are doing pure emission, but when a pixel is doing both you need the lighting to affect the color of the occlusion but not affect the color of the emission.
But that means you need to dedicate specific objects to being purely emissive, to avoid blurring at the boundaries.
And once you've separated the objects, you don't really need to have purely-emissive textures and non-emissive textures in the same file, with the same exact format. You might as well store emissive textures as RGB and save on memory.
Using a different format can even benefit you. You're less likely to accidentally blend emissive and non-emissive pixels, and you're less likely to accidentally apply lighting calculations to emissions.
Except associated alpha models emission and occlusion via the operation. You don't need zero alpha either, as any ratio can occlude partially and emit partially as well.
You can't do both in a single pixel, unless you turn off lighting entirely and make everything fullbright.
Let's have blue-tinted pane of glass, (0, 0, .5, .5). And a red tinted pane, (.5, 0, 0, .5).
Then a blue glow, (0, 0, .5, 0). And a red glow, (.5, 0, 0, 0).
If you have your blue glass glow red, and your red glass glow blue, both combinations come out as (.5, 0, .5, .5).
Under 99% of lighting conditions, it will look wrong. If you put it in darkness, it will look overwhelmingly wrong.
Even if occlusion and emission are the same color it doesn't work. A dark blue object that glows brightly, and a bright blue object that glows dimly, both will have the same RGBA.
Objects that block a certain amount of light, and then emit a certain amount of light: You can only make that simplification if the entire world is evenly lit by white light.
Blur the entire matchstick into one pixel. Under white light it has to emit brown plus yellow. In a dark room it has to emit just yellow.
It's a clever technique but I can't figure out any way it's not fundamentally incompatible with having lighting. If one pixel has to both be lit and emit extra light, you need two RGB values.
RGBA in associated is unique in that it represents distilled geometry. If you were to zoom in on a pixel that had a degree of geometry occluding it, the coverage is represented by the alpha ratio, while the RGB is purely emission.
It is complimentary to lighting.
The math works because of the differing forms of the alpha over formula. FG.RGB + ((1.0 - FG.Alpha) * BG.RGB), resulting in a pure add at the extreme case of alpha being zero, or ratios of addition when non-zero.
Within the limitations of the RGB model, it works extremely well.
The folks cited are extremely adept imaging people, covering years of experience and several Academy Achievement Awards.
They're plenty smart but they're talking about a different use case.
You don't know what color will be emitted unless you know what light is hitting the surface.
A dark glowing surface and a bright non-glowing surface have the same emissions under white light, but different emissions under other kinds of light. The method you're talking about requires the emissions be precalculated, which means you can't apply lighting at runtime.
I agree with you, to do it properly you need two different RGBA objects, the emissive one will have A=0. But the same format suffices to represent both.
I've been throwing around the idea of making a new intermediate file format which can contain only pre-multiplied transparency. But, do you know any file format already supports this?
TIFF can do premultiplied alpha. The problem with TIFF is that it's such a grab bag of options, it makes it difficult to know what you're dealing with.
> As an Artist: Make it Bleed!
> If you’re in charge of producing the asset, be defensive and don’t trust the programmers or the engine down the line.
If you are an artist working with programmers that can fix the engine, your absolute first choice should be to ask them to fix the blending so they convert your non-premultiplied images into premultiplied images before rendering them!
Do not start bleeding your mattes manually if you have any say in the matter at all, that doesn't solve the whole problem, and it sets you up for future pain. The only right answer is for the programmers to use premultiplied images. What if someone decides to blur your bled transparent image? It will break. (And there are multiple valid reasons this might happen without your input.)
Even if you have no control over the engine, file a bug report. But in that case, go ahead and bleed your transparent images manually & do whatever you have to, to get your work done.
Eric Haines wrote a more technical piece on this problem that elaborates on the other issues besides halo-ing:
http://www.realtimerendering.com/blog/gpus-prefer-premultipl...