How do 2d arrays work in C when using typedef?

It's b, and MxN array of doubles.

typedef double X_t[N];

defines X_t as an array of N doubles.

When you make an array of these, each X_t is a row. So

X_t A[M];

creates M rows of this.

double X_t[N] denotes the type of an array with N elements of the type double.

X_t A[M]; is a declaration of an array with M elements that have the type double[N].

So this declaration is equivalent to

double A[M][N];

Here is a demonstrative program.

#include <stdio.h>

int main(void) 
{
    enum { M = 2, N = 5 };
    typedef double X_t[N];
    
    printf( "sizeof( X_t ) = %zu\n", sizeof( X_t ) );
    
    putchar( '\n' );
    
    X_t A[M];
    
    printf( "sizeof( A ) = %zu\n", sizeof( A ) );
    printf( "sizeof( A[0] ) = %zu\n", sizeof( A[0] ) );
    printf( "sizeof( A ) / sizeof( A[0] ) = %zu\n", sizeof( A ) / sizeof( A[0] ) );
    

    return 0;
}

The program output is

sizeof( X_t ) = 40

sizeof( A ) = 80
sizeof( A[0] ) = 40
sizeof( A ) / sizeof( A[0] ) = 2

So it is seen that the array A has two elements with the size 40 bytes that is equivalent to N * sizeof( double ).

Here's the way to think it through:

X_t A[M];

A is an M-element array of something - that something is X_t. So right away that eliminates c as a possible answer.

Now you look at the definition of X_t. For the moment, pretend that typedef isn't there:

double X_t[N];

This declares X_t as an N-element array of double. When we add the typedef keyword:

typedef double X_t[N];

this means X_t is an alias for the type "N-element array of double". So, if A is an M-element array of X_t, and X_t is an alias for "N-element array of double", then the type of A is "M-element array of N-element array of double", or "MxN array of double." It's equivalent to writing

double A[M][N];

Since the [] operator is postfix, anytime you substitute an array typedef with its "naked" equivalent you put the substituted array size on the right. Assume the following typedefs:

typedef double A_type[M];  // A_type is an array of double
typedef A_type B_type[N];  // B_type is an array of A_type
typedef B_type C_type[O];  // C_type is an array of B_type

and this declaration

C_type x;

We want to know what the actual type of x is, so we do the following:

1. x is an object of type C_type. C_type is a typedef name for the type "O-element array of B_type", so we replace C_type with B_type [O]. Since the [] operator is postfix, the [O] goes to the rightmost side of the declarator:

   B_type x[O];
   

2. x is an O-element array of type B_type. B_type is a typedef name for the type "N-element array of A_type". As in the previous step, we replace B_type with A_type [N] and put the [N] on the rightmost side of the declarator:

   A_type x[O][N];
   

3. x is an O-element array of N-element arrays of A_type. A_type is a typedef name for the type "M-element array of double". We replace A_type with double [M], and again the [M] goes to the rightmost side of the declarator, leaving us with:

   double x[O][N][M];