I'm having a hard time using pass by reference

As someone said before, the largest_digit function is void, so it can't be assinged to a variable when is called. What you can do instead, is modifying *digit before the call, and then assign the value of *digit to what you want, in this case hold1.

Other thing is that you need to save the value into *digit before returning, for example, instead of doing hold = *digit, you should do *digit = hold.

Here are the suggested modifications:

void largest_digit(int* digit) {
   int hold, hold1;

    if(*digit == 0 || *digit < 0) {
        *digit = 0;
        return;
    }

    // If-statement with Recursion.
    if(*digit > 0){
        hold = (*digit) % 10;
        (*digit) /= 10;
        largest_digit(digit);
        hold1 = *digit;

        if(hold > hold1) {
            *digit = hold;
            return;
        } else {
            *digit = hold1;
            return;
        }
    }
}

With this main,

int main() {

   int a=123, b=652, c=3274;
   largest_digit(&a);
   largest_digit(&b);
   largest_digit(&c);
   printf("%d\n",a);
   printf("%d\n",b);
   printf("%d\n",c);

   return 0;
}

the output is

3
6
7

You said you were passing it by reference, but you just tried to pass it by value here

hold1 = largest_digit(*digit/10);

Create a new int with *digit/10 and pass the address to largest_digit

int hold1Temp = *digit/10;
hold1 = largest_digit(&hold1Temp);

EDIT: Your function should be something like this:

void largest_digit (int* digit)
{
  if (*digit <= 0) return; // if there's no more digit to compare, stop
  int currentDigit = *digit % 10; // if you receive 982, it gets the 2
  int nextDigit = *digit/10; // if you receive 982, it turns into 98
  largest_digit(&nextDigit); // does the same thing again
  *digit = currentDigit > nextDigit ? currentDigit : nextDigit; // chooses the biggest digit
}

A couple of things first:

1. The unary indirection operator (*) used on a pointer means "look at what is this pointing to". Therefore, the statement *digit; alone is not useful to anything. You can very well remove it from your code (I see you use it multiple times), perhaps you meant to do an assignment? The statement *digit = X; is an assignment and modifies the data pointed by the pointer.

2. "Passing by reference" does not exist in C. You can only pass by value. That value though can be a pointer to another value, that is how you "simulate" passing something by reference.

3. A function declared as void f(...) does not return any value. Therefore, assigning the "return value" of such a function to a variable does not make sense.

Now, considered the above:

Your call largest_digit(*digit/10) is not passing a pointer, but dereferencing the pointer digit, dividing the value by 10, and then passing that as parameter. As you already figured, this is wrong. To correctly pass by reference in your case, you would need to either modify the original value pointed to by digit, or create a new one and pass its address.

In any case, passing a pointer around (instead of the value directly) for this kind of recursive function does not make much sense and is only a complicated twist that does not accomplish much other than making your life harder. Use a plain value as argument.

int largest_digit(int num) {
    if (num < 0)
        return largest_digit(-num);

    if (num == 0)
        return 0;

    int cur = num % 10;
    int next = largest_digit(num / 10);

    if (cur > next)
        return cur;

    return next;
}

int main(void) {
    printf("%d\n", largest_digit(1923)); // 9
    printf("%d\n", largest_digit(4478)); // 8
    printf("%d\n", largest_digit(-123)); // 3
}

NOTE: for simplicity, the above function also handles negative numbers by calling largest_digit(-num) if the number is negative, therefore it only supports negative digits down to INT_MIN+1 (that is, it does not correctly handle INT_MIN).

Your trouble is that other than the case where *digit is negative, you never actually set *digit. Each time you do this:

*digit;

The above only dereferences the pointer and looks up the value, but it doesn't actually change it. What you need to do on each of your return routes is to actually set the value to something:

*digit = ...something...;

Without setting this value anywhere, the value of num in your main() function is never actually going to change.

In addition, you are treating largest_digit as if it has a return value, which it does not:

hold1 = largest_digit(*digit/10); // <- assigning the return value does not make sense