How check apache & php-fpm config is appropriate (not too high or too low)

As you noted, this depends on your script(s). We have this dynamically adjusted in our deploy scripts based on the server(s) being rolled up.

The following script is based on running Apache, on Centos, on AWS infrastructure but could easily be adapted to what you are using.

Basically:

• set the size of apache processes
• set the size of php process
• scripts gets available memory, cores and does some crunching and then modifies the config.
• we run this as part of stack roll up

Primary Source / Based on:

• https://medium.com/@sbuckpesch/apache2-and-php-fpm-performance-optimization-step-by-step-guide-1bfecf161534

Steps:

1. Calculate process size

You need to know how many processes can run on your machine. So calculate the process size of your main CPU/memory drivers is necessary.

cd /tmp
curl https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py --output ps_mem.py
chmod a+x ps_mem.py
sudo python ps_mem.py
# Sample numbers:
# 28.4 MiB + 103.0 KiB = 28.5 MiB memcached
# 34.7 MiB + 9.5 KiB = 34.7 MiB amazon-cloudwatch-agent
# 24.8 MiB + 18.0 MiB = 42.8 MiB httpd (15)
# 69.1 MiB + 7.0 MiB = 76.0 MiB php (2)
# 228.2 MiB + 46.0 MiB = 274.3 MiB php-fpm (36)

Here you can see that there are 15 httpd processes, consuming a total of 43MiB, so each Apache process is using roughly 3MiB of RAM. The php-fpm process will use about 7.6MiB.

2. Calculate Apache MaxRequestWorkers

To be safe though, reserve 15% of memory for all other processes (in my case ~1.2GiB) and round up apache process size to 3MiB.

MaxRequestWorkers = (Total RAM - Memory used for Linux, DB, etc.) / process size
MaxRequestWorkers = (8000MB - 1200MB) / 3MB = 2,266

3. Calculate php-fpm max-children

To be safe though, reserve 1 GiB for all other processes and round up php process size to 8MiB.

max_children = (Total RAM - Memory used for Linux, DB, etc.) / process size
max_children = (8000MB - 1200MB) / 8MB = 850

4. Here is the script we use, on roll up.

#!/bin/bash

# Creates a configuration script to run once final servers are up.
PROCESS_SIZE_APACHE_MB=3
PROCESS_SIZE_PHP_MB=8

# Get some values from the server
MEMORY_KB=`grep MemTotal /proc/meminfo | awk '"'"'{print $2}'"'"'`
MEMORY_MB=$(($MEMORY_KB / 1024))
MEMORY_AVAILABLE_MB=$(($MEMORY_KB / 1178))
NUM_CORES=`nproc --all`
echo "Memory: $MEMORY_MB MB"
echo "Memory Available: $MEMORY_AVAILABLE_MB MB"
echo "Num Cores $NUM_CORES"

#Now do some calculations
SERVER_LIMIT=$(($MEMORY_AVAILABLE_MB / $PROCESS_SIZE_APACHE_MB))
echo "HTTP MPM Server Limit: $SERVER_LIMIT"

#Convert Apache from mpm-prefork to mpm-worker
#Set params
#<IfModule mpm_*_module>
#  ServerLimit           (Total RAM - Memory used for Linux, DB, etc.) / process size
#   StartServers          (Number of Cores)
#   MinSpareThreads       25
#   MaxSpareThreads       75
#   ThreadLimit           64
#   ThreadsPerChild       25
#   MaxRequestWorkers     (Total RAM - Memory used for Linux, DB, etc.) / process  size
#   MaxConnectionsPerChild   1000
# </IfModule>
# /etc/httpd/conf.modules.d/00-mpm.conf

echo "
# LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
# LoadModule mpm_worker_module modules/mod_mpm_worker.so
LoadModule mpm_event_module modules/mod_mpm_event.so

<IfModule mpm_*_module>
  ServerLimit           $SERVER_LIMIT
  StartServers          $NUM_CORES
  MinSpareThreads       25
  MaxSpareThreads       75
  ThreadLimit           64
  ThreadsPerChild       25
  MaxRequestWorkers     $SERVER_LIMIT
  MaxConnectionsPerChild   1000
</IfModule>
" > /etc/httpd/conf.modules.d/00-mpm.conf

# Configure the workers
# pm = dynamic
# pm.max_children         (total RAM - (DB etc) / process size) = 850
# pm.start_servers        (cpu cores * 4)
# pm.min_spare_servers    (cpu cores * 2)
# pm.max_spare_servers    (cpu cores * 4)
# pm.max_requests         1000
MAX_CHILDREN=$(($MEMORY_AVAILABLE_MB / $PROCESS_SIZE_PHP_MB))
echo "Max Children: $MAX_CHILDREN"
NUM_START_SERVERS=$(($NUM_CORES * 4))
NUM_MIN_SPARE_SERVERS=$(($NUM_CORES * 2))
NUM_MAX_SPARE_SERVERS=$(($NUM_CORES * 4))

sed -c -i "s/^;*pm.max_children.*/pm.max_children = $MAX_CHILDREN/" /etc/php- fpm.d/www.conf
sed -c -i "s/^;*pm.start_servers.*/pm.start_servers = $NUM_START_SERVERS/" /etc/php- fpm.d/www.conf
sed -c -i "s/^;*pm.min_spare_servers.*/pm.min_spare_servers =  $NUM_MIN_SPARE_SERVERS/" /etc/php-fpm.d/www.conf
sed -c -i "s/^;*pm.max_spare_servers.*/pm.max_spare_servers =  $NUM_MAX_SPARE_SERVERS/" /etc/php-fpm.d/www.conf
sed -c -i "s/^;*pm.max_requests = 500.*/pm.max_requests = 1000/" /etc/php-> fpm.d/www.conf

No tool will give you that kind of metric because the best configuration depends greatly on your php scripts. If you have 4 cores and each request consumes 100% of one core for 1 second, the server will handle 4 request per second in the best case regardless of your mpm and php configuration. The type of hardware you have is also important. Some CPUs perform multiple times better than others.

Since you are using php_fpm, the apache mpm configuration will have little effect on performance. You just need to make sure the server doesn't crash with too many requests and have more apache threads than php processes. Note that the RAM is not the only thing that can make a server unreachable. Trying to execute more process than the CPU can handle will increase the load and the number of context switches, decrease the CPU cache efficiency and result in even lower performance.

The ideal number of php processes depends on how your scripts use CPU and other resources. If each script uses 50% of the time with I/O operations for example, 2 processes per core may be ideal. Assuming that those I/O operations can be done in parallel without blocking each other.

You'll also need to take into account the amount of resources used by other processes such as the DB. SQL databases can easily use more resources than the php scripts themselves.

Spare Servers and Spare Threads are the number of processes/threads that can be idle waiting for work. Creating threads takes time, so it's better to have them ready when a request arrives. The downside is that those threads will consume resources such as RAM even when idle, so you'll want to keep just enough of them alive. Both apache and php_fpm will handle this automatically. The number of idle threads will be reduced and increased as needed, but remain between the minimum and maximum values set in the configuration. Note that not all apache threads will serve php files as some requests may be fetching static files, therefore you should always have more apache threads than php processes.

Start Server and Start Threads represents just the number of processes/threads created during the startup. This have almost no effect on performance since the number of threads will be immediately increased or reduced to fit the values of Spare Threads.

MaxConnectionsPerChild and max_requests are just the maximum amount of requests executed during the process/thread life. Unless you have memory leaks, you won't need to tune those values.