Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Path Tracing Quake in Blender (matthewearl.github.io)
155 points by kipi on June 21, 2021 | hide | past | favorite | 39 comments


A lot of valuable work on Blender side, but the main goal is questionable, and author explains why.

Pre-calculated lighting had very little to do with physical correctness, it was purely an artistic placement of light sources in a specific map processed with a specific tool (official or third party) with specific defaults. Two adjacent rooms could be lit in a very different manner because map maker decided it looked good; two maps coming from different sources could not be expected to share any common lighting principles. Quirks like overbright values were not even consistent among officially supported renderers, and were inherited by later Quake and Source engines (which would add their own quirks and light processing options). To put it shortly, there was no reference for the final result except the final result itself, and it often was unstable (changing textures, geometry, or processing constants would shift the look too much, and require fixes).

To make the game look as intended, you have to somehow rely on original lightmaps that are tightly coupled with original low resolution geometry and textures. Given that people still argue which of the original renderers gives the “true” picture, I have my doubts about making a new one that pretends some average light model is good enough for everything. Even for episode 1, hand-placed lights had to be reintroduced into the system, and ad-hoc fixes had to be done, but manual fixes are not an option for all the existing maps.


> but the main goal is questionable, and author explains why.

Kind of, but on the other hand:

> people still argue which of the original renderers

Now you have N+1 renderers - I'm not sure why the new one would be any more questionable than the last N-1. Obviously you shouldn't pretend this is somehow "the way it was meant to be rendered", but they don't seem to claim that; rather, this is one additional option of how to render it, with novel upsides not shared by the previous ones (but also with it's own set of downsides).


This isn't new. In the very beginning, people made custom patches for LIGHT.EXE to fix bugs and introduce exciting new options, then made their own tools. After the source release, ports started adding everything their authors considered cool (bump mapping! every cool game has it!). Modern rendering technologies are used, too — though I really doubt something as simplified and optimized for the goal of being playable on contemporary hardware as Quake is a good test case. It feels like using a rotating cube to showcase ambient occlusion, visual effect shaders, etc. You'd better use more complex data.

I'm saying that all of that doesn't converge. For lighting, we have original map data with original lightmaps, and that's it. How to interpret these? We can often reason about the port author used to check how the map looks, and sometimes guess whether author was lazy (guys, any map that is not pitch black is playable) or made very specific choices that have to be presented more or less intact. This makes any decision, even purely technical, an artistic choice. Should we render it this way? Sure, why not. Should we add motion blur? Sure, why not. Should we add lens flares? Sure, why not. And so on, and so on.

This might look like rambling, but there is an example of nonsense that is so ubiquitous most people don't even understand there is a problem: rendering resolution. To run software Quake in “mere” 640×480 with 25+ fps in 1996, you had to use the top home PC configuration available. And, contrary to modern expectations, first generation of hardware accelerators did not perform much better, there were no 60 fps 1280×960 crazy rides. They allowed to maybe play at the same ~25 fps (often in puke HiColor) if you did not have a monster 200 MHz Pentium, and be happy about it. Professional OpenGL accelerators are often mentioned, but their fillrate was actually less than that of a Voodoo, and they targeted the same typical resolutions (the difference was that they implemented crazy realtime pixel shading like reflection mapping in hardware, and offered the same fillrate in scenes using it, hence the crazy price tags). So 640×480 was Ultra Super High Definition in which people would realistically play Quake, and most would definitely use a lower one. Consequently, the complexity of 3D objects and textures match that specific range of screen resolutions.

What does anyone playing an old 3D game, streaming, or recording videos do today instantly on the first run? Setting game resolution to native, FullHD and higher. And then the game starts to look like empty amateurish junk from mid-2000 because neither geometry nor textures are matching the rasterizer resolution. Those giant flat polygons with sharpest edges never were the intended look of the game. People feel that something is wrong, add lots of common and uncommon texture filtering, make high resolution substitutes, but it all looks like a makeup on a corpse. The fix is not using the improper resolution in the first place. But this is so common that a lot of young viewers believe that this is how “retro games” looked like. They even make their own works based on that assumption.


My rose colored glasses of sentimentality says that going from p166 to voodoo1 was going from 320x240 to 640x480, where you could find people much better.

But yeah, you never saw the blockheads of https://blog.habets.se/static/2015-03-27-e1m1-2c2b74e-0102.p...

Triangle count in models were just never a thing I remember seeing.

At the same time, though, we had nothing to compare to. It was perfectly normal that the legs move the same way when you ran forward as when you ran sideways.

So it's not just "what did we see then with eyes from 2021?", but "what did we see then with eyes from 1996?".


> Texture coordinates can be encoded as Blender UV maps.

Will note that one minor detail about the Quake map format you may find interesting... Quake does not encode the texture coordinates for vertexes in the map. Instead, Quake encodes the coordinate system for textures of each face. This is transformed to screen-space during rendering.

This is different from modern renderers, which record the texture coordinates of vertexes.

Quake's system is less flexible, can't be used for character models, and can be inconvenient during asset creation, but it is a bit faster and more convenient at run-time. When you're rendering, you want to know the gradient of texture coordinates in the X and Y direction on screen, which is easy to calculate using Quake's system.

(Obviously the author knows this, but it wasn't spelled out in the article.)


This approach is actually more convenient during asset creation as it gives seamless texture alignment. The original Quake 1 MAP format used by the editing tools were a bit too restrictive, though the BSP format was more flexible so editors over the year improved on that while remaining compatible.

Also AFAIK the texture coordinates are converted to regular U/V pairs when rendering.


As someone who’s used Quake level editors… no, it’s not more convenient. You can align textures to each other but not to the geometry, not easily. One of the most annoying things to do in old Quake level editors is to try and take a standard "crate" and rotate it by some amount. It's just a cube with square textures, and yet it's a pain.

The texture coordinates are not converted to U/V pairs. You may be thinking of one of other Quake-derived engines like GLQuake.


I have also done Quake mapping and even done an editor for fun[0] some time ago :-P.

You are supposed to stick to the grid, the crate texture is sized so that it fits perfectly on the grid. If you go off the grid things become a bit harder, but this is why pretty much everything on Quake is axis aligned :-P.

It is ridiculously simple to make stuff actually.

About rotation, most editors that people used even in the 90s had support for texture locking for both translation and rotation. id's original editor was very primitive though, but later editors like Worldcraft and Radiant had those features.

WRT. texture coordinates, they are converted to U/V (or actually S/T) pairs, but it happens quite late during the edge span rasterization. You can see the inner rasterizer loop here[1] where it draws an edge span and performs a linear interpolation of the S/T values calculated at [2] above and [3] (during the surface draw setup).

Having said that i think what you described is closer to what Ken Silverman was working on at the time as a successor to the Build engine (at least based on my understanding from what he wrote years ago).

[0] https://i.imgur.com/5xWOw3K.jpg

[1] https://github.com/id-Software/Quake/blob/master/WinQuake/d_...

[2] https://github.com/id-Software/Quake/blob/master/WinQuake/d_...

[3] https://github.com/id-Software/Quake/blob/master/WinQuake/d_...


> WRT. texture coordinates, they are converted to U/V (or actually S/T) pairs, but it happens quite late during the edge span rasterization. You can see the inner rasterizer loop here[1] where it draws an edge span and performs a linear interpolation of the S/T values calculated at [2] above and [3] (during the surface draw setup).

The whole distinction I was making was between calculating UV coordinates for the vertexes versus directly calculating the gradients.

Unless I’m reading this wrong, the UV coordinates for vertexes are not calculated. Instead, the gradients are converted to screen space and used to calculate pixel UV coordinates, exactly as I had said:

> …Quake does not encode the texture coordinates for vertexes in the map…

I have used Radiant and the “texture lock” was a bit of a joke for a long time. If a game is released in 1996 but it takes many years before something otherwise simple is easy to achivee in editors… the only natural conclusion is that the map format is optimized for renderer simplicity rather than map-making convenience.

This problem can be solved but the fact that the problem exists is just a curiosity in an otherwise obsolete engine. There is no reason why you would encode the texture coordinates that way in modern engines.


They are calculated, you can see that in lines 290 and 296 of d_scan.c where the S and T coordinates (these are the texture coordinates, just another name for UV coordinates[0]) are calculated.

But yeah, they are not stored in the BSP file and are derived from the surface.

As for why it is done like that, it isn't for rendering simplicity but for storage. It takes less memory and disk space to store only the surface plane and texture axes and calculate the texture coordinates when you need them than store them on memory/disk for every vertex. Not only you store a few less bytes even for a triangle (with additional memory for each vertex) but also allows sharing of vertex and edge data which are common between surfaces even if their texture coordinates would differ.

After all the game had to run in systems with just 8MB of RAM.

> I have used Radiant and the “texture lock” was a bit of a joke for a long time.

Well, i just tried it in QERadiant and seems to work fine.

In any case, yes there are limitations, but when you are sticking to the grid - which is what most of Quake maps do, going off the grid isn't that common - the automatic texture alignment is also very convenient. This was also a big improvement over Doom where every linedef had to be manually adjusted and misaligned textures were commonplace.

[0] Traditionally S,T as convention are supposed to refer to coordinates in surface space and U,V in texture space but in practice everyone uses them interchangeably, especially with 3D APIs that use both to mean the same thing. Some older 3D graphics books do make a distinction though.


For anyone interested in seeing a side-by-side comparison: https://viewsync.net/watch?v=uX0Ye7qhRd4&t=4&v=hcareEsIXHM&t...


I did something similar here: https://blog.habets.se/2015/03/Raytracing-Quake-demos.html [2015]

The author's write-up is better, and his target (Blender) enables some things mine (POV-Ray) doesn't.

I also really like the idea to use emissive textures. I just used point light sources.

I'm still rendering, and you can join if you want: https://qpov.retrofitta.se/


Love this. There are high-res textures for Quake which would have made this look even better. The Quake Revitalization Project[0] has created new textures "without changing the mood, theme, or atmosphere", and I believe they're packaged by default with projects like nQuake.

[0]: http://qrp.quakeone.com/retexture/


I never like projects like this. They always claim that they don't change the mood, theme, or atmosphere but maybe my definition of that is different. They definitely feel like they change the mood and atmosphere to me.


If lighting and texture swaps are tasteful to a large crowd, I wonder if model swaps would be. Model swaps are more effort, but they were popular in the case of the modded FF7 community. I'd be interested in seeing a fully overhauled version of Quake.


The motion blur is quite good here. Here is some light reading/watching on the subject.

https://docs.blender.org/manual/en/latest/render/cycles/rend...

https://youtu.be/VXIrSTMgJ9s

In my estimation of things: motion blur is only beneficial to try to work around sample and hold displays. If there were fully continuous (1000 Hz is close enough) or sub 1 ms strobed displays then motion blur would add nothing.


To take advantage of a 1000 Hz monitor, you'd need to render 1000 different images per seconds!

And if you can renderer 1000 images per seconds, then generating motion blur that work on a 100 fps monitor is as simple as displaying an average of 10 frames.

The various motion blur techniques allow simulating higher frame rate without paying the full cost of rendering all the extra frames.


Yes. Motion blur is temporal interpolation, this is, it's a way of representing what happens "in between frames". The shorter the time between frames, the more subtle the effects of interpolation (like the higher the resolution of an image, the less blurry it becomes in between pixels).


Counter point: Motion blur is a simple form of low-pass filtering (in the time direction). You need some kind of filtering to prevent shutter artifacts (think video of a spinning wheel), and that stays true even at a million fps.


I don't think this is how it works. We have a discrete number of rods and cones which work as a well behaved spatial sampler. Human visual system temporal sampling is smeared stochastically across the rods and cones rather than being clocked. If you truly displayed 1 million fps and there were no shutter artifacts (as there are none in any fixed-pixel displays that we are currently looking at), then the motion would be life-like. The human visual system doesn't take a temporally clocked set of frames then apply a low-pass filter to it and doing it as an approximation of actual perceived motion blur would look off (as many gamers lament).

Blurbusters has a fair amount of literature on this topic.

https://blurbusters.com/blur-busters-law-amazing-journey-to-...


This has nothing to do with biology, it's an argument from signal processing, which is well-understood theory (Nyquist's theorem and so on). If an object oscillates at 1 MHz, and you take 1 million still frames per second, it will rest in the same place in every frame, and thus look static. In reality, such an object would look blurred to the human eye.(+) It's this kind of artifact that motion blur (to be more precise, low-pass filtering) can avoid.

Edit: The article you linked to is very confused about some basic terminology. It equates response time artifacts of an LCD monitor that display sharp, digital images with motion blur. That's so wildly wrong I'm not even sure where to start. Maybe here: When displaying video, motion blur is a property of the source, response time one of the output device.

(+) Edit 2: To expand on this, the human vision system integrates arriving photons over time, and this way implicitly behaves a lot like a low-pass filter. A low-pass filter is different from a fixed-rate shutter, which is what people mean when they say the eye doesn't have a fixed framerate. However, there is a limited temporal resolution.

A more everyday example of this effect would be a dimmed LED. You can pulse an LED at 1 MHz, it will look dimmed, not blinking. But when filming/rendering this LED at 1 million still images per second, it will either be all on or all off, both of which are wrong (i.e., an artifact of your chosen shutter speed).


>This has nothing to do with biology

>look blurred to the human eye

Ah but it has everything to do with biology. You are proposing a far too simple model for the signal processing actually at play. Unfortunately there is no clock going to the rods and cones, they simply fire signals off asynchronously and the timebase is reconstructed in your noggin. How would you go about filtering a few million samples all on their own timebases that are themselves not uniformly (or even periodically) sampled? It would be a truly awful approximation.


I know there are lots of funny theories about vision on the internet, but as a simple empirical fact that everybody can verify, dimming an LED works. Whatever idea about vision someone might have, if it doesn’t account for that fact, it’s wrong, disproved by everyday experience. (More generally, photons hitting cones are actually also discrete events, and we don’t see everything flickering all the time. But that’s a more complex argument that depends on a prior understanding of quantum physics. In contrast, the LED example is simple and deterministic.)

However, the complexity of the biology behind this doesn’t actually matter for my argument. My point is that we need proper signal processing in order to not irreparably damage the signal before it reaches the eye, when we’re still in the realm of precise technology. I’m not sure if I explain this poorly or if you’re a tiny bit motivated to misunderstand me, but of course, it is a complex subject.

As a last honest attempt, can we agree in the simplest possible case — that an LED blinking at 1 MHz filmed at 1 million still frames per second can’t ever reproduce a signal that can be interpreted as being dimmed? If so, we already agree that we have an artifact. Then the question is only how to remove it, and signal processing theory gives an answer.

If not, I’d recommend that you work through it with pencil and paper: what is the LED doing, what are the still images showing, what is the monitor showing to the eye? You can’t miss the conclusion if you do this carefully.


I find the motion blur distracting :(


If you notice it, it’s not done correctly (which in games unfortunately seems to be always; presumably because it’s faked with a blur in post-processing). Few people complain about motion blur in movies, where cameras produce it through physics alone. It’s artificially added to CGI scenes so that they don’t stand out and distract the viewer.


What do you mean "if" you notice. I complain about it every single time. Of course, it's much worse in games. Only vsync is worse


The author mentions using frustum culling for performance. Won't this lead to lighting artifacts with indirect illumination when path tracing? Even when an object is behind the camera it could still affect the lighting of what's in front of it.


Author here, I intersect the view frustum with the light's sphere of influence (and also take into account PVS info) so it still includes lights that are reachable by one bounce. Technically this means I might omit two bounce reflections but in practice this doesn't appear to be a problem.


Since it's coarse as the BSP, you won't lose objects behind you. The BSP data structure doesn't take into account the camera's looking direction, just the camera's position. Thus, it includes every object visible from all possible looking directions visible at a given point, so it will always include objects behind you.


This was actually one of the first things I did with Blender when I got it! When I was a teen and Blender was at 2.5 or 2.6, I booted it up and tried to recreate the lighting in some Quake 2 maps that I had downloaded. Then I used the Shift + ` shortcut to go into first-person and re-explore my childhood in 15-fps ray-traced perfection.


All this work and then Youtube turns it into a smeary, illegible mess.


Even the screenshots look smeared compared to the game ones though.


I suspect it is the added motion blur :(


Please can you do the same for Thief? :P


The Thief engine and Dromed Editor were interesting - you essentially carved out the level, and sometimes it wouldn't get the lighting maps right - which were very costly to generate.

I had a lot of fun making Thief levels though.


I'd love to see if the fake water surface texture could be replaced with a procedural deformation faking waves so that the path tracing could render caustics.


Unfortunately, Blender not very good at caustics. In theory, any path tracer can do them, but in practice, the sampling needs to prioritize caustic-generating regions to be even remotely efficient.

There might be add-ons that help, but vanilla Cycles will take orders of magnitude more samples to create a good image when there's caustics involved than when there's not. For that reason, it's still common to fake caustics in Blender using a texture or projection effect.


Dungeon keeper (I & II) next!


[flagged]


Quake demos can be "recammed", though. See https://www.youtube.com/watch?v=jzcevsd5SGE.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: