Is using malloc to allocate a buffer faster than using a statically allocated buffer when making large buffers?

user_15014525 0
Viewed 332

I am trying to make an array like this:

char tmp[4][32768];
//code

This is in a function and I discovered that making this array global will increase the speed of the program (unfortunately I could not keep it like that as the function calls itself recursively when encountering certain data). I would like to make this function faster but I have read that malloc can be pretty slow. So would it be faster to leave it like that or do this?:

char** tmp;
tmp = malloc(4 * sizeof(char*));
tmp[0] = malloc(32768);
tmp[1] = malloc(32768);
tmp[2] = malloc(32768);
tmp[3] = malloc(32768);
//code
free(tmp[0]);
free(tmp[1]);
free(tmp[2]);
free(tmp[3]);
free(tmp);
memory c performance
2 Answers

It's not a matter of being faster or slower. It's a matter of whether or not you blow up the stack.

You're creating an array 128K in size which is fairly large for a local variable, and locals typically live on the stack. If you call this function recursively, that's another 128K on the stack for each call. Just a few recursive calls would be enough to cause a stack overflow which would likely result in a crash.

Allocating memory dynamically is pretty much your only option. You can however reduce this to a single allocation:

char (*tmp)[32768] = malloc(4 * sizeof *tmp);

This allocation works because a 2D array of type char [4][32768] decays to a pointer of type char (*)[32768] which matches the type of tmp above. The memory returned by malloc is enough for 4 objects of type char[32768]. This memory is contiguous, unlike doing 4 separate calls to malloc for each subarray.

user_16871190

All things being equal, allocating memory through malloc (or calloc or realloc) will be both slower and more labor-intensive to manage than using an auto (local) array, which will be slower than using a static array.

All things are typically not equal, though. As you've discovered, a static array doesn't work well when dealing with recursion (or when you just need to preserve multiple states, as anyone who's tried to use strtok to tokenize a string like "a=b&c=d&e=f" knows all too well). A 128K auto array is somewhat on the large side, and if the function is recursive that can be an issue since stack space is typically limited for an individual process.

This is kind of an ideal use case for dynamic memory, which typically isn't as limited as memory on the stack. The trade-off is that it will be slower that the alternatives and will require more work on your part.

Exactly how much slower will depend on how you allocate and use it - allocating it as a single contiguous chunk like so only requires a single malloc (and corresponding free) call and should be better for locality:

char (*tmp)[32768] = malloc( 4 * sizeof *tmp );
if ( tmp )
{
  for ( size_t i = 0; i < 4; i++ )
    strcpy( tmp[i], some_string ); // or however you intend to use tmp;
  ...
  free( tmp );
}

but if your heap is sufficiently fragmented you may not be able to allocate it as a single contiguous chunk and have to do it piecemeal, using multiple malloc (and free) calls:

char **tmp = malloc( 4 * sizeof *tmp );
if ( tmp )
{
  for ( size_t i = 0; i < 4; i++ )
  {
    tmp[i] = malloc( sizeof *tmp[i] * 32768 );
    if ( tmp[i] )
    {
      strcpy( tmp[i], some_string );
    }
  }
  ...
  for ( size_t i = 0; i < 4; i++ )
    free( tmp[i] );
  free( tmp );
}

The only way to know for sure which is faster and works better for your purposes is to code up multiple versions and to measure their performance.

Having said all that...

It doesn't matter how fast your code is if it does the wrong thing, or gives the wrong answer, or vomits all over the floor at the first hint of bad input, or exposes sensitive data, or provides an entry point for malware, or cannot be modified or maintained. Code for correctness and maintainability first, without consideration for raw execution speed, then for all the other issues listed above. Unless you have a hard performance requirement ("This operation must complete within X msec"), don't worry about speed. Get it right first, then if you aren't satisfied you can always tune the performance later.

Also, depending on what the rest of your code does, the deltas between the different methods of allocating and managing that memory may be insignificant - if your code is spending the bulk of its time waiting for someone to type in data, then the extra few clocks spent using malloc as opposed to an auto array really don't matter. On the other hand, if you're calling this code thousands of times in a tight loop, those extra clocks will add up to a significant difference.

Make your decisions based on analysis and measurement, not guesswork.

user_1345540
Related