Weird SGI VPro bug

John Tsiombikas

20 April 2024

Story time

Last year, I released deeprunner, a 3D shooter game for the SGI O2 (and other computers, the O2 was the performance target). After the frantic jam release (I live-streamed the last 12 hours of hacking for the jam, and the resulting video is available on youtube as a cure for insomnia), I also tried it on my more powerful SGI Octane2 with VPro V6 graphics, and discovered that the laser weapon beams were not being drawn at all!


The laser beams were drawn as long thin quads, starting from Z=0 in view space, and extending to the current draw distance, which coincides with the far clipping plane. Through trial and error, I managed to make the beams appear properly on the Octane2, by tweaking them to start slightly further away from 0, at Z=-0.01 or something like that. (relevant commit)

Thinking about this, didn't really make a lot of sense. Is that a divide by zero somewhere? But it couldn't be. Polygons are clipped against the view volume in homogeneous clip space, and only then division by W (which is a function of Z) takes place. What gives? That's clearly a hardware bug, because all the vertex processing on VPro happens on the hardware, but it doesn't make any sense.

A year passed and I didn't give this much thought, until suddenly a couple of days ago I mentioned this bug again as a curiosity in the SGI user group discord channel, and thought it would be nice to write a small self-contained program which reproduces and demonstrates the bug. Also it would help to factor out any weird side-effects and interactions with a larger hastily written codebase.

Reproducing the bug

So I did; the bug was extremely easy to reproduce. I made a tunnel out of a few thin long quads, starting either from Z=0 or Z=-0.01 (toggling between those two values by hitting 'z' on the keyboard), and there it is, in all it's glitchy brokenness.

bug screenshots

(sorry for the shitty screenshots, they're frames from a video I captured, and I'm too lazy to go back and capture proper shots right now. I may replace them later)

The truth is out there

This sparked some conversation, and a user going by the name "stormy" recalled the following post by an ex-SGI hardware engineer going by the name "oppositelock", which might be relevant:

At the time, I was a fairly junior employee doing research in AGD, the advanced graphics division ... and the "VPro" graphics for Octane were fatally broken on a fundamental level, where fixing it would mean going back to the algorithmic drawing board, not just some Verilog tweak, basically, taping out a new chip. Why was it so broken? Because some engineers decided to implement a cool theory and were allowed to do it (no clipping, recursive rasterization, hilbert space memory organization).

Further down he explains:

the framebuffer had a recursive rasterizer which followed a hilbert curve through memory, the thinking being that you bottom out the recursion instead performing triangle clipping, which was really expensive for the hardware at the time.

The problem was that when you take some polygons which come close to W=0 after perspective correction, their unclipped coordinates get humongous and you run out of interpolator precision. So, imagine you draw one polygon for the sky, another for the ground, and the damn things Z-fight each other!

SGI even came out with an extension to "hint" to the driver whether you want fast or accurate clipping on Octane. When set to fast, it was fast and wrong. When set to accurate, we did it on the CPU.

He's referring to the SGIX_vertex_preclip extension.

How intriguing! The rasterization approach he describes sounds very interesting, and apparently it mostly works, since only very few corner cases are broken like this. But on the other hand, this was the graphics hardware for "serious" (and very expensive) graphics workstations, the lifeblood of SGI, and something they've been doing for almost 2 decades at that point. It's not excusable like Sony bodging a shitty rasterizer for the playstation 1. Even slightly broken rendering should have stayed as experiments on the prototyping stage and not made it into the final hardware.

In any case, the bug now makes sense, and the described "fix" also would probably work, in expense of performance, if it's just doing transformations and clipping in software the correct way.

Trying the fix

So I tried the SGIX_vertex_preclip extension, and indeed it works. As soon as I add the following two lines, the glitch disappears, and everything is rendered correctly:


In fact it doesn't seem to matter if the hint is set to GL_FASTEST or GL_NICEST, as long as it's not GL_DONT_CARE, it seems to work. According to the extension specs, the hint only affects the routines detecting which polygons need to be pre-clipped in software. So, in some cases it might make a difference, but for my tunnel it doesn't.

Here's the final source code of the simple OpenGL program reproducing the VPro hardware bug. Hitting 'z' toggles between Z=0 and Z=-0.01, and hitting 'c' enables or disables the vertex pre-clip extension.


Final thoughts

Obviously, if the issue can be worked around by slightly changing the geometry, as I did with my laser beams, that's a much preferable solution, rather than switching over to software vertex processing, and turning the VPro into a glorified 3dfx voodoo. But it's good to know that an easy fix exists, in case this issue crops up while porting something else where such changes might prove non-trivial.

In fairness the driver does make an effort to only switch on software vertex processing for a class of polygons which are suspect to glitch out, not for everything in the scene. But on the other hand even determining that is bound to add some overhead.

During the discussion on the SGI user group discord, some people mentioned that the SGI port of Quake III also probably suffers from the same issue while rendering the skybox. It could be interesting to try and enable the pre-clip extension in the game, and see if the issue goes away. I'll add a note about it here if I ever find the time and inclination to try it.

Discuss this post

Back to my blog