I was initially going to have the same answer as everyone else had and chalk this up to the issues with rand(). However, I thought better of doing so and instead analyzed the distribution your math is actually producing.

TL;DR: The pattern you see has nothing to do with the underlying random number generator and instead is simply due to the way your program is manipulating the numbers.

I'll stick to your blue function since they're all similar.

uint8_t blue(uint32_t x, uint32_t y) {
    return (rand() % 2)                  ? (x + y) % rand() :
           ((x * y % 1024) % rand()) % 2 ? (x - y) % rand() :
                                           rand();
}

Each pixel value is selected from one of three functions: (x + y) % rand()

Let's look at images produced by each of these alone.

• rand()

This is what you would expect, just noise. Call this "Image C"

• (x + y) % rand()

Here you're adding the pixel coordinates together and taking the remainder from dividing by a random number. If the image is 1024x1024 then the sum is in the range [0-2046]. The random number you're diving by is in the range [0,RAND_MAX], where RAND_MAX is at least 32k and on some systems is 2 billion. In other words there's at best a 1 in 16 chance that the remainder isn't just (x + y)

However you're only using the lowest 8 bits, because you return a uint8_t, so you'll have stripes of gradients 256 pixels wide.

Call this "Image A"

• (x - y) % rand()

Here you do something similar, but with subtraction. As long as x is greater than y you'll have something similar to the previous image. But where y is greater, the result is a very large number because x and y are unsigned (negative results wrap around to the top of the unsigned type's range), and then the the % rand() kicks in and you actually get noise.

Call this "Image B"

Each pixel in your final image is taken from one of these three images using functions rand() % 2 and ((x * y % 1024) % rand()) % 2. The first of these can be read as choosing with 50% probability (ignoring issues with rand() and its low order bits.)

Here's a closeup of where rand() % 2 is true (white pixels) so Image A is selected.

The second function ((x * y % 1024) % rand()) % 2 again has the issue where rand() is usually greater than the thing you're dividing, (x * y % 1024), which is at most 1023. Then (x*y%1024)%2 doesn't produce 0 and 1 equally often. Any odd number multiplied by any even number is even. Any even number multiplied by any even number is also even. Only an odd number multiplied by an odd number is odd, and so %2 on values that are even three quarters of the time will produce 0 three quarters of the time.

Here's a closeup of where ((x * y % 1024) % rand()) % 2 is true so that Image B could be selected. It's selecting exactly where both coordinates are odd.

And here's a closeup of where Image C could be selected:

Finally combining the conditions here's where Image B is selected:

And where Image C is selected:

The resulting combination can be read as:

With 50% probability use the pixel from Image A. The rest of the time pick between Image B and Image C, B where both coordinates are odd, C where either one is even.

Finally, since you're doing the same for three different colors, but with different orientations the patterns are oriented differently in each color and produce the crossing strips or grid pattern you're seeing.