Friday, November 13, 2015

How to optimize Apache concurrency for running WordPress

To a technical savvy person, hosting your own website on a VPS server can bring a tremendous sense of accomplishment and a wealth of learning opportunities. If WordPress is what you fancy, with a minimal monthly financial commitment, you can host a WordPress website on the LAMP platform (Linux, Apache, MySQL, PHP). For example, the entry-level, $5 per month plan offered by DigitalOcean, of which I am an affiliate, will give you a 512MB RAM, single-core VPS.

With such a small RAM capacity, you will need to optimize how your Apache webserver is configured to run PHP applications such as WordPress and Drupal. The goal is to maximize the number of concurrent web connections.

This tutorial details the Apache/PHP setup procedure on Debian 8.2, aka Jessie. The procedure assumes Apache is yet to be installed. However, if Apache2 is already installed, you will find practical information below on how to reconfigure Apache2 to run a different multi-processing module.

Background knowledge

According to a recent Netcraft webserver survey, Apache powers 46.91% of the top million busiest websites on the Internet. Busy websites mean many concurrent web connections.

Concurrent connection requests to Apache are handled by its Multi-Processing Modules. MPMs can be loosely classified as threaded or non-threaded. Older Apache releases default to a MPM named Prefork. This MPM is non-threaded. Each connection request is handled by a dedicated, self-contained Apache process.

Newer Apache releases default to a threaded MPM, either Worker or Event. The Worker MPM uses one worker thread per connection. One issue with this approach is that a thread is tied up if the connection is kept alive despite it being inactive.

The Event MPM, a variant of Worker, addresses the aforesaid keep-alive issue. A main thread is used as the traffic controller that listens for requests and passes requests to worker threads on demand. In this scenario, an inactive but kept-alive connection does not tie up a worker thread.

Note that MPMs are mutually exclusive: only 1 MPM can be active at any given time.

Traditionally, the Apache core functionality serves static web content (e.g., HTML text and images). To serve dynamic content, such as PHP pages from WordPress, Apache requires special modules that execute PHP code.

For the Prefork MPM, each spawned Apache process embeds its own copy of the PHP handler (mod_php). Concurrency in this model is limited by the number of processes that Apache can spawn given the available memory.

For both Worker and Event MPMs, PHP requests are passed to an external FastCGI process, PHP5-FPM. PHP-FPM stands for PHP-FastCGI Process Manager. Essentially, the webserver and the PHP handler are split to separate processes. Apache communicates with PHP-FPM through an Apache module, either mod_fastcgi or mod_fcgid. Optimizing concurrency in this model means configuring both the MPM and the PHP handler (PHP-FPM) to have pools of processes and threads to handle requests.

The rest of this tutorial covers the cases of installing the Event MPM from scratch as well as migrating to Event from the Prefork MPM.

Installing Apache2

This tutorial starts with the installation of Apache2. If Apache is already installed, you should find out which MPM is currently running using the command apache2ctl -V, and proceed to the next section.

$ sudo apt-get update && sudo apt-get upgrade
$ sudo apt-get install apache2

Next, note the Apache version you just installed and the MPM that is loaded.

$ sudo apache2ctl -V
Server version: Apache/2.4.10 (Debian)
...
Architecture: 64-bit
Server MPM: event
threaded: yes (fixed thread count)
forked: yes (variable process count)
...

The above output tells us that we are running Apache release 2.4. Beginning with 2.4, Apache runs the Event MPM by default. If you are running an older version of Apache, the default MPM is either Prefork or Worker.

Configuring Apache2

  1. Back up the Apache configuration file, /etc/apache2/apache2.conf.

    $ sudo cp /etc/apache2/apache2.conf{,.orig}

  2. Edit the configuration file.

    Below is a subset of configuration parameters belonging to the Apache core module. You should adjust their values in order to optimize concurrency. The corresponding values are what I use for an entry-level VPS.

    Timeout 100
    KeepAlive On
    MaxKeepAliveRequests 1000
    KeepAliveTimeout 5

    For an in-depth explanation of the above parameters, please refer to Apache on-line documentation.

  3. Enable mod_rewrite.

    While the mod_rewrite module is not strictly relevant to optimizing concurrency, I've included it here as a reminder to install the module. It is an important module for running WordPress.

    $ sudo a2enmod rewrite

Installing Event MPM

If you are already running the Event MPM, skip to the next section, 'Configuring Event MPM'. Otherwise, follow the procedure below.

  1. Install the Event MPM.

    $ sudo apt-get install apache2-mpm-event

  2. Disable existing MPM.

    Recall that only 1 of Prefork, Worker or Event MPM can be running at any given time. Therefore, if you were previously running Prefork or Worker, you must first disable it, and then enable Event.

    To disable the Prefork MPM, run this command:

    $ sudo a2dismod mpm_prefork

    To disable the Worker MPM, run this:

    $ sudo a2dismod mpm_worker

  3. Enable the Event MPM.

    $ sudo a2enmod mpm_event

    Note that the above enable and disable commands are quite 'forgiving'. If you attempt to enable an MPM that is already enabled, or disable an MPM that is already disabled, it will simply return a harmless informational message.

Configuring Event MPM

To configure the Event MPM, modify its configuration file, /etc/apache2/mods-available/mpm_event.conf. Before making any changes, back it up using the following command:

$ sudo cp /etc/apache2/mods-available/mpm_event.conf{,.orig}

Edit the file to specify the following parameters:


<IfModule mpm_event_module>   
  StartServers 2  
  MinSpareThreads 25  
  MaxSpareThreads 75  
  ThreadLimit 25  
  ThreadsPerChild 25  
  MaxRequestWorkers 250  
  MaxConnectionsPerChild 10000  
  ServerLimit 12  
</IfModule>  

The above configuration is what I recommend for an entry-level VPS (512MB RAM, single-core). You need to adjust the parameters to satisfy your own system requirements. For a detailed explanation of the above parameters, click here. Note that the Event MPM shares the same parameters as the Worker MPM.

Installing PHP5 handler

To execute PHP code, Apache requires a PHP handler. PHP5-FPM is the PHP handler to use with the Event MPM.

For a new PHP installation, install php5-fpm followed by the meta-package php5.

$ sudo apt-get install php5-fpm php5

In addition to the above packages, I also installed other PHP5 packages which WordPress requires. While they are not strictly relevant to optimizing concurrency, I've included them here for completeness.

$ sudo apt-get install php5-mysql php5-gd php5-curl

Configuring virtual host

Suppose your WordPress website has the domain name example.com. To set up a virtual host with that domain name, follow the steps below:

  1. Create the Apache configuration file for example.com.

    Instead of creating the file from scratch, use the default site as a template.

    $ sudo cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/example.com.conf

  2. Edit the configuration file.

    Customize the following site-specific parameters:

    ServerName example.com
    ServerAlias www.example.com
    ServerAdmin info@example.com
    DocumentRoot /var/www/example.com
    
  3. Create DocumentRoot directory.

    $ sudo mkdir /var/www/example.com
    $ sudo chown -R <webuser>:<webuser> /var/www/example.com

    Notes:

    • WordPress should be installed in the directory /var/www/example.com. For instructions on how to install WordPress, refer to my earlier post.

    • The DocumentRoot directory should be owned by a non-root user.

  4. Enable the new site.

    $ sudo a2ensite example.com.conf

  5. Disable the default site.

    $ sudo a2dissite 000-default.conf

Configuring PHP handler

Follow the procedure below to configure PHP5-FPM.

  1. Create a custom PHP configuration file for example.com by copying the template from the default site.

    $ sudo cp /etc/php5/fpm/pool.d/www.conf /etc/php5/fpm/pool.d/example.com.conf

  2. Edit the configuration file.

    Customize the following parameters.

    [example.com]
    user = <webuser>
    group = <webuser>
    listen = /var/run/php5-fpm_example.com.sock
    pm = dynamic
    pm.max_children = 5
    pm.start_servers = 2
    pm.min_spare_servers = 1
    pm.max_spare_servers = 3
    pm.max_requests = 2000

    Notes:

    • The user and group parameters specify respectively the Unix user and group names under which the FPM processes will run. You should specify a non-root user for both.

    • The listen parameter specifies the source address that the FPM will listen to for receiving PHP requests. In this case, it will listen to the Unix socket /var/run/php5-fpm_example.com.sock.

    • The rest of the parameters are for an entry-level VPS system. You should adjust their values to satisfy your system requirements.

    • Click here for more details about the above parameters.

  3. Restart PHP5-FPM.

    $ sudo systemctl restart php5-fpm

Installing FastCGI

Apache requires a FastCGI module to interface with the external PHP5-FPM processes. You can use 1 of 2 FastCGI modules: mod_fastcgi and mod_fcgid. Click here for a discussion of their differences. This tutorial uses mod_fastcgi.

Before you install mod_fastcgi, you must:

  1. Enable non-free.

    Debian pre-packages the mod_fastcgi module in the non-free archive area of its repositories. Make sure that non-free is included in the /etc/apt/sources.list file.

  2. Disable mod_php.

    If Apache2 was previously installed with the Prefork MPM, most likely, it is configured to execute PHP using the mod_php module. In this case, you must disable the mod-php module before you install mod_fastcgi. Otherwise, the install will fail with the error message, 'Apache is running a threaded MPM, but your PHP Module is not compiled to be threadsafe. You need to recompile PHP.'

    To disable mod_php, run this command:

    $ sudo a2dismod php5

To install mod_fastcgi, execute the following command:

$ sudo apt-get install libapache2-mod-fastcgi

Configuring FastCGI

  1. Back up configuration file.

    Before you edit the configuration file /etc/apache2/mods-available/fastcgi.conf, back it up using the following command.

    $ sudo cp /etc/apache2/mods-available/fastcgi.conf{,.orig}

  2. Edit the file.

    Insert the following lines:

    
    <IfModule mod_fastcgi.c> 
      AddHandler php5-fcgi .php 
      Action php5-fcgi /php5-fcgi 
      Alias /php5-fcgi /usr/lib/cgi-bin/php5-fcgi_example.com 
      FastCgiExternalServer /usr/lib/cgi-bin/php5-fcgi_example.com -socket /var/run/php5-fpm_example.com.sock -pass-header Authorization 
      <Directory /usr/lib/cgi-bin> 
          Require all granted 
      </Directory> 
    </IfModule>
    

    Notes:

    • example.com should be replaced with your own domain name.

    • To access the website, you need to grant the proper permission explicitly using the Require all granted statement. Without it, access to the website will be denied with the error message 'You don't have permission to access /php5-fcgi/index.php on this server.'

  3. Enable additional modules.

    $ sudo a2enmod actions fastcgi alias

  4. Restart Apache.

    The final step is to restart Apache to make all the above changes go live.

    $ sudo systemctl restart apache2

Threads in action

Concurrency for WordPress occurs at both the webserver (Apache2) and the PHP handler (PHP-FPM) levels. You can use the ps -efL command to monitor the processes and threads at either level.

To monitor Apache processes and threads, execute the following ps command.

$ ps -efL |grep apach[e]
www-data 31681 24441 31681 0 27 03:25 ? 00:00:00 /usr/sbin/apache2 -k start
www-data 31681 24441 31684 0 27 03:25 ? 00:00:00 /usr/sbin/apache2 -k start
...

The second and the fourth columns are the process ID (PID) and the thread ID respectively. Note that the above output reports 2 different threads (31681 and 31684) of the same process (31681).

Execute the following command to monitor PHP.

$ ps -efL |grep ph[p]
root 24398 1 24398 0 1 Nov10 ? 00:00:17 php-fpm: master process (/etc/php5/fpm/php-fpm.conf)
peter 31519 24398 31519 0 1 03:14 ? 00:00:17 php-fpm: pool example.com
peter 31520 24398 31520 0 1 03:14 ? 00:00:16 php-fpm: pool example.com
peter 31827 24398 31827 0 1 04:15 ? 00:00:15 php-fpm: pool example.com

Conclusion

When traffic to your website increases over time, your webserver must scale up to handle the increase in traffic. This tutorial explains how to configure Apache2 and PHP to optimize the number of concurrent connections. After you try it out, if you still find that your website cannot keep up with the traffic, you should consider upgrading your VPS plan to have more RAM.

If you are interested in WordPress, please refer to my earlier posts.