I always planned to come back to Credence, compile theory is an area of computer science and programming I enjoy so much I'd rather not use any AI.

I learned a lot working on Pulse2D, the Teensy 4.1 2D game engine that is now complete, with a pilot space shooter game.

A few things that changed how I think about programming:

• You can do a lot without any heap at all. The Teensy 4.1 has no heap and a very small stack. A single variable definition inside a function frame would cause a crash. Yet I was able to build a whole game.
• The biggest latency hit in modern computing is waiting for RAM.
• And lastly: The importance of memory alignment, the cache line, and Array of Structs vs. Structs of Arrays

The Pulse2D engine required me to align a few assignments in a way I had never seen before. I realized I really didn't know much about alignment and its importance. Now that I do, I'm going to talk about what I learned and what caused the crash in Pulse2D:

If you have a 64-byte struct, and it accidentally starts at address 0x0032, it crosses the boundary between two hardware cache lines. To read that one object, the CPU now has to fetch two separate cache lines from RAM, doubling the latency penalty. alignas(64) forces the object to snap to the grid.

That "boundary" on the Teensy 4.1 is non-negotiable. Large objects needed at runtime were built with a static placement-new pattern where their storage alignment and placement were necessary. You can see the pattern here, and how it's used in game development.

Moreover, in modern systems development, instead of hectic, random malloc or similar calls that can take up to 300 cycles each and lead to fragmentation, developers have moved to "arenas." One single large chunk of memory with a pointer at the start and a virtual "push down" or way to increment and decrement for the entire application.

Next, we need to take a moment to talk about Cache Hierarchy.

To stop the CPU from spending its entire life waiting, hardware engineers made tiny pools of ultra-fast memory directly onto the processor die. This is the Cache Hierarchy.

Using a modern Zen 4 chip as a baseline:

• L1 Cache (The Workbench): Every single core gets its own private L1 cache. It is tiny (usually 32 KB for data). It operates at the exact same speed as the core, and a read takes ∼4 clock cycles.
  
• L2 Cache (The Closet): Also private to each core, but slightly larger (e.g.,
  1 MB). A read takes ∼14 clock cycles.
  
• L3 Cache (The Warehouse): A massive pool of memory (32 MB to 128 MB+)
  shared by all the cores. A read takes ∼40 to 50 clock cycles.
  When your compiler emits a LOAD instruction, the CPU checks L1. If it’s not there (a cache miss), it checks L2, then L3. If it misses completely, the entire pipeline stalls, and it waits for 300 cycles.

Now, understanding this can fundamentally change how you design data structures. The standard Object-Oriented approach is an Array of Structs (AoS):

struct Entity {
  Vector3 position;
  Vector3 velocity;
  Color color;
  String name;
};
std::vector<Entity> entities;

If you write a physics loop that only updates position based on velocity, the CPU pulls the cache line containing the Entity. But that cache line is clogged with color and name data that the physics math doesn’t even look at. You just filled the L1 cache with useless bytes.

Behold, the Data-Oriented approach - A Struct of Arrays (SoA):

struct EntitySystem {
  std::vector<Vector3> positions;
  std::vector<Vector3> velocities;
  std::vector<Color> colors;
  std::vector<String> names;
};

Now, when the physics loop runs, it iterates through a contiguous array of only positions and velocities. Every byte pulled into that 64-byte cache line is mathematically useful and fits in the cache line. The hardware prefetcher maxes out the pipeline, and performance skyrockets.

I've come back and applied these things to Credence in many ways. I am also working on adding a union data type - where alignment can come into play for real.

I'll talk about that next.