John Tsiombikas nuclear@mutantstargoat.com
3 October 2021
Octal constants in C, is one of those rarely used things, most people find out about when they accidentally add a leading 0 to what they think is a decimal number. The use of decimal is obvious. Binary is too verbose (and there's no such thing as binary constants in C89). Hexadecimal is the go-to shorthand for controlling bits in a binary value, due to how nicely it aligns with nibbles, bytes, and so on. But what the hell is octal useful for?
Any UNIX user and systems programmer has been exposed to the notion of using octal to represent permission bits, but it's very hard to think of another example of what would one use octal for.
Well, I feel this is a momentous occasion for me, because after about a quarter
century of programming, I finally, for the first time outside of arguments to
open and chmod, of my own volition, used octal in my code. Exciting I know!
Here's how it happened...
The use of octal in UNIX permissions is not arbitrary. It's useful because there are three permission bits (read, write, execute) for each category of user (owner, group, and others). By using octal we can represent each one of these groupings with a single octal digit (2^3 = 8), making it a succinct way of encapsulating the permissions of a filesystem object in a single 3-digit octal number.
I happened to stumble onto another case where grouping bits by 3 was very convenient.
I'm currently hacking away on a grid-based dungeon crawler for the Dungeon Cralwer VR Game Jam 2021.
During loading, I have a grid with solid and open cells, and I'm trying to determine which one of the multiple different "tiles" I want to use as the walls geometry for every one of the open cells (at least those not explicitly forced to a specific tile by the level data files). I have tiles for "straight corridor", "left turn", "right turn", "cross-section", "T-section", and so on:
To decide what to use on each cell, I need to examine its neighbors. For instance if it's open above and below, but solid to the sides, the straight corridor tile is appropriate. If it's open below and to the left, but solid elsewhere, it should be a left corner, and so on. Now trying to do that with a series of conditions each one checking up to 8 different surrounding cells like so, is exhausting and ugly:
if(cell_solid(x-1, y-1) && cell_solid(x, y-1) && cell_solid(x+1, y-1) &&
!cell_solid(x-1, y) && cell_solid(x+1, y) && cell_solid(x-1, y+1) &&
!cell_solid(x, y+1) && cell_solid(x+1, y+1) {
cell->tile = TILE_LTURN;
}
/* ... and so on for EVERY CASE */
Obviously there are many different ways to make this less horrible. But here's a simple one: what if for each tile we first loop in its 3x3 neighborhood, and for every row we set a bit to 1 if the corresponding cell is solid, and to 0 if it's open. That leaves us with a 9-bit binary number (3 bits per row), which completely describes the neighborhood of any given cell.
+---+---+---+
| X | X | X | 1 1 1 7
+---+---+---+
| | | X | -> 0 0 1 -> 1 -> 0715 in octal
+---+---+---+
| X | | X | 1 0 1 5
+---+---+---+
This way we can pack the whole block into a self-describing octal number, and be able to tell at a glance what's in every row! Then we can use that in a large, but much more manageable and readable switch, to assign tiles to cells:
switch(desc) {
...
case 0715:
cell->tile = TILE_LTURN;
break;
case 0555:
cell->tile = TILE_STRAIGHT;
break;
...
}
Sometimes more than one configurations may need the same tile, which can be handled by falling through multiple cases, and some cases might be handled better in the default case with a few ifs masking bits we don't care about, to make the switch shorter.
Of course I'm glossing over some details. Like some cases are best handled by the same tile with different orientations, or some times multiple similar tiles will have to be defined just to make the wall and ceiling geometry line up better. Or overlay tiles will have to be used, to cover up nasty seams. But that's a story for another time.
I think it's beautiful how just using octal, turned this hellish surrounding cell lookup, into a simple number which encapsulates the whole thing so neatly. Nobody shall complain about octal being useless again in my presence.