Why would we want to reference a variable address? What are its advantages?

To give other software references to variables:

double x0, y0;
double x1, y1;
double x2, y2;

/*  Convert polar coordinates to rectangular coordinates.
    This code needs two results from each call, but a function can only
    return one thing.  And we do not want the function to set the same
    variables each time, so we cannot use fixed global variables.  In
    each call, we pass the function pointers to two variables we want
    it to put the results in.
*/
ConvertPolarToRectangular(&x0, &y0, 3,  .17);
ConvertPolarToRectangular(&x1, &y1, 4,  .23);
ConvertPolarToRectangular(&x2, &y2, 5, -.03);

To work with data when we do not know how much we need when we compile the program:

//  Get number of items needed.
scanf("%d", &N);

/*  Allocate memory for records.  This makes Records point to enough
    memory for N records (if the malloc succeeds).
*/
struct Record *Records = malloc(N * sizeof *Records);

To create linked lists, trees, and other data structures that connect to each other:

struct ListItem
{
    struct ListItem *Next;
    int Data;
}

struct ListItem *FirstItem = malloc(sizeof *FirstItem);
struct ListItem *SecondItem = malloc(sizeof *SecondItem);
FirstItem->Next = SecondItem;
SecondItem->Next = NULL;

/*  Now we can keep allocating memory for as many items as we need
    and keep linking them together into a list.  When we want to
    delete one item, we can take it out of the chain without moving
    the other items in memory:
*/
FirstItem->Next = ThirdItem;
free(SecondItem);

Besides performance, there is the ability to change the referenced variable.. If pass a variable to a function as an argument, the value of the variable is copied. If the function changes its argument, the original variable does not change. If you pass the address of a variable, however, then the function has the ability to change the original variable’s value. Sometimes that’s bad; sometimes that’s great. If you need that ability, then this is a crucial feature.

One common reason to take the address of a variable would be to allow a variable defined in one function to be changed in another.

In C, all parameters to functions are passed by value. So if you want a function to change the value of a variable defined in some other function, you need to pass its address.

For example:

void increment(int *x)
{
    *x = *x + 1;
}

int main()
{
    int num = 4;
    increment(&num);
    printf("num=%d\n", num);
}

Here, the address of num is passed to the increment function. This function can then dereference this pointer to read and write the value stored at that address. This code will output "num=5".

I assume that by "reference a variable address" you mean "using a pointer to pointer".
That is usually the solution for the need to manipulate a pointer variable (e.g. the head pointer for inserting something into a linked list) inside a function and have the effect visible outside.
I.e. you use a pointer to the variable you want to manipulate in an outside-visible way as a parameter to the function. If the variable is a pointer you end up with a pointer to pointer and inside the function you "reference the address of a variable".

The part "is it faster to reference a variable than to access the variable directly" is a little tricky. I think the true part of that statement is the situation in which the variable is large (e.g. a complex struct). Giving that as a copy-by-value-parameter to a function takes copying. Only giving the address (more or less one number) is faster. But the actual accessing then takes to first read from the address before using the value and is not faster.