How can I encapsulate a function like this in C?

Function calls cannot be dynamically created in this way. If you want a way to generically call functions, you need to give them a common interface.

If you look at how the pthread_create function is defined:

   int pthread_create(pthread_t *thread, const pthread_attr_t *attr,
                      void *(*start_routine) (void *), void *arg);

It takes a function pointer for the thread start function which takes a single void * argument and returns a void *, along with a void * representing the arguments. Your traverseTree code should do the same:

void traverseTree(TreeNode tree_root, void *(*func)(TreeNode, void *), void *args)
{ 
    for(int i = 0; i < tree_root->n_children; i++) {
      traverseTree(tree_root->children[i], func, args); 
    }
    func(tree_root, args);
}

For a given function's arguments, you can create a struct containing those arguments and pass a pointer to it. Then in the function, you would cast the void * argument to a pointer to the proper struct type.

You should be able to recast member op_fun to the desired function type. So for example, if you have type defined:

result_type (op_fun_2_t)(type_1 arg_1, type_2 arg_2);

and object defined:

Operate my_obj = malloc ...;

and function defined:

result_type my_func(type_1 arg_1, type_2 arg_2);

and member op_fun set:

my_obj->op_fun = (void *)my_func;

Then you should be able to call my_func using an explicit cast:

((op_fun_2_t)(my_obj->op_fun))(type_1 arg_1, type_2 arg_2);

You will need to define a type associated with each possible combination of argument types and counts and make sure that you cast the function to the type assigned to op_fun in each case. If it is cast to the wrong type, you will have undefined behavior.

You have to know a priori the possible number of arguments and the type of each one.

If the return type differs, you define a union-object to make a common object of all the types. So, instead of defining result type of type size_t you will define it as union OBJECT and make a dispatcher for each possible case.

If the arguments types can vary, this is more complicated and there are lots of ways to implement generic functions. For example, use a trie which dispaches on ith level depending on the ith type.

Anyway, what you try to do is to implement generic functions and you can find literature on that to see how the systems do it.

This would be a so-called "XY problem" where you need to solve X and think Y is the right solution, so you ask about your problems implementing Y...

Ok so after some clarification, the actual problem you seem to want a solution to is how to do generic programming with a "traverse" function applied to some container. This is not something you need to re-invent the wheel for, there's well-known design patterns for this in C that have been used since the dawn of time.

For reference I'll give two different solutions, the "old school" solution and the modern C solution.

"Old school" generic programming

The old school version would involve using void* which is the generic object pointer (but cannot be used with function pointers). You'd design a callback function of a specified format taking void* as parameters. If you look at well-known libraries, bsearch, pthreads among others are using this method. Something along the lines of:

typedef void callback_t (void*);

Then we pass this to a generic "traverse" function, like this:

void traverse (void*       array, 
               size_t      item_size, 
               size_t      items, 
               callback_t* callback)

Depending on which callback and items we pass along, the algorithm is executed for that specific type. Full example:

typedef void callback_t (void*);

void traverse (void*       array, 
               size_t      item_size, 
               size_t      items, 
               callback_t* callback)
{
  for(size_t i=0; i<items; i++)
  {
    unsigned char* cptr = array;
    callback( &cptr[i*item_size] );
  }
}

void print_int (void* arg)
{
  int* iptr = arg;
  printf("%d ", *iptr);
}

void print_string (void* arg)
{
  char* str = *(char**)arg;
  printf("%s ", str);
}


int main (void)
{
  int iarray[10] = {0,1,2,3,4,5,6,7,8,9};
  traverse(iarray, sizeof(int), 10, print_int);
  puts("");
  
  char* strarray[2] = {"hello", "world"};
  traverse(strarray, sizeof(char*), 2, print_string);
}

Output:

0 1 2 3 4 5 6 7 8 9
hello world

Modern C generic programming

Less kind and grumpy people like me might call the previous version "old fart" generic programming. There's lots of obvious problems with it: we have to pass lots of arguments, it's not type safe at all, the API is cumbersome. Slip a key when implementing it and it crashes ungracefully somewhere unexpected - so it is hard to debug too.

Modern C features type-generic programming, which can detect types at compile time. We'd cook up an admittedly cryptic-looking macro like this:

#define print(array, arg)             \
  _Generic( (&array),                 \
            int(*)[]: print_int,      \
            char*(*)[]: print_string ) (arg)

This one takes an array as parameter, then checks if we can form a pointer to that array matching any of the supported forms. If not, compiler error. Type safe.

It then calls the appropriate callback function, which too as it turns out can be declared with the correct parameter type and not void*, so those turn type safe too. They'll look simple, like this:

void print_int (int* iptr)
{
  printf("%d ", *iptr);
}

Below follows an example with two different forms of generic tasks, a print task and a "increase array item by 1 task". This is 100% standard C as well:

#include <stdio.h>

void print_int (int* iptr)
{
  printf("%d ", *iptr);
}

void print_string (char** str)
{
  printf("%s ", *str);
}

#define print(array, arg)             \
  _Generic( (&array),                 \
            int(*)[]: print_int,      \
            char*(*)[]: print_string ) (arg)

void inc_int (int* iptr)
{
  (*iptr)++;
}

void inc_string (char** str)
{
  // some nonsense increasing each character value
  for(char* s=*str; *s!='\0'; s++)
  {
    (*s)++;
  }
}

#define inc(array, arg)               \
  _Generic( (&array),                 \
            int(*)[]: inc_int,        \
            char*(*)[]: inc_string ) (arg)

#define traverse(array, action)       \
  for(size_t i=0; i<sizeof(array)/sizeof(*array); i++) { action(array, &array[i]); }

int main (void)
{
  int iarray[10] = {0,1,2,3,4,5,6,7,8,9};

  // making this point to read/write compound literals instead of read-only string literals:
  char* strarray[2] = { (char[]){"hello"}, (char[]){"world"} };
  
  traverse(iarray, print); puts("");
  traverse(strarray, print); puts("");
  
  traverse(iarray, inc);
  traverse(iarray, print); puts("");
  
  traverse(strarray, inc);
  traverse(strarray, print); puts("");
}

Output:

0 1 2 3 4 5 6 7 8 9
hello world
1 2 3 4 5 6 7 8 9 10
ifmmp xpsme

To avoid code repetition, one can also really get crazy and define X macros which also creates the actual functions. Generally not a good idea because the code turns hard to read, if easy to maintain.