Call PHP from virtual/custom “web server”

Question 1

Basically, I'm trying to figure out how PHP can be called from a "web server".

I've read the documentation, but it didn't help much.

As far as I can tell, there are three ways to invoke PHP:

via command line (eg: php -f "/path/to/script.php")
via CGI(??) / via FastCGI (???)
via a web server (eg: Apache) module

So let's start with CGI. Maybe I'm just blind, but the spec doesn't mention how on Earth the web server passes data (headers & callbacks) to the thing implementing CGI. The situation is even worse with FastCGI.

Next up, we have server-specific modules, which, I don't even know what to search for since all leads end up nowhere.

Question 2

Invoking a CGI script is pretty simple. PHP has a few peculiarities, but you basically only need to setup a list of environment variables, then call the PHP-CGI binary:

setenv GATEWAY_INTERFACE="CGI/1.1"
setenv SCRIPT_FILENAME=/path/to/script.php
setenv QUERY_STRING="id=123&name=title&parm=333"
setenv REQUEST_METHOD="GET"
...

exec /usr/bin/php-cgi

Most of them are boilerplate. SCRIPT_FILENAME is how you pass the actual php filename to the PHP interpreter, not as exec parameter. Crucial for PHP is also the non-standard variable REDIRECT_STATUS=200.

For a GET request you only need the environment variables. For a POST request, you simply pipe the HTTP request body as stdin to the executed php-cgi binary. The returned stdout is the CGI response consisting of an incomplete HTTP header, rnrn, and the page body.

(Just from memory. There maybe a few more gotchas.)

Question 3

Which method of running PHP consumes less memory?

I assume that per PHP-processed request they are more or less the same. But if you have mod_php loaded into apache serving images too, then I assume your memory footprint will be higher due to serving static files.

Also which method consumes memory nearly constant. I see with mod_php my servers memory usage fluctuating between 300MB and 800MB, every few seconds.

You can make both pretty constant. If you carefully set MaxClients, MinSpareServers and MaxSpareServers, you pretty much can tell how many processes are running. If you also set memory_limit within your PHP config, you can calculate how much memory you need. You can do the same for fcgi too, since you can decide how many processes are running.

But with FastCGI, first response from server comes very late. I see with FastCGI there is an initial delay per webpage request. Once first response from server arrives, other items like images, css, js loads pretty faster.

This doesn't make sense. I am not sure why it happens in your case.

Is it OK to run mix of both? I have 5 sites on dedicated server. Is it ok if I run few with mod_php and rest with FastCGI?

I guess, but it will both be a nightmare to maintain and will probably be harder to configure for saving memory. Quite the contrary I believe.

I am sure that my server goes down mostly because of improper memory usage by mod_php. I checked all scripts. Is there any way to make sure memory consumption on server remains nearly constant?

Configure memory and processes as I outlined above, and keep monitoring.

Does complexity of .htaccess affects memory usage significantly? If yes, can it be a single reason to make server run out of memory?

I don't think so. per-directory .htaccess can slow things down, but unless there is some serious bug in Apache, it should not cause mass memory consumption.

Does apache MPM prefork/worker settings affect memory consumption? Do they affect mod_php and FastCGI mode equally?

It might, but I recommend to stay away from worker, as PHP is mostly not thread safe.

When I run "top" command, I see apache (httpd) consuming memory around 40MB. There are many instances of httpd running. Also in addition to that FastCGI forks some processes of similar size. What is normal memory size for httpd process?

30MB is the min. The upper limit depends on your application (I have seen cases where it was ~1GB)

As I am running Wordpress on all of our sites, which will be good way in that context?

It is probably a matter of taste. I have recently moved away from apache towards nginx+fastcgi. it takes a bit of time to get used to, but it does work well. No problems whatsoever with wordpress (even not with supercache, which is rather involved in terms of web server).

Does FastCGI/SuExec works fine with APC? Do I need to reconfigure APC to work with SuEXEC and FastCGI.

I am not using suExec, but fastcgi works well with APC. No need to configure anything.

Question 4

Looks a bit like this question https://superuser.com/questions/646062/granting-write-permissions-to-www-data-group

I wanted to suggest using the PYTHONPATH environment variable to point to a library install location readable by the www-data user where you'd copy the python modules it needs to acces, but I think this is considered bad form nowadays.

Probably best is to create a Python Virtualenv that is accessible for the www-data user and install all required modules into that, using the pip command from that virtualenv. You may have to use some sudo/chown dance to get this right still.

Another way perhaps is to not bother with calling a python subprocess at all, but use Pyro's HTTP gateway. That way you can simply do a HTTP request from PHP to a locally running Pyro http gateway process, which will translate it into a proper Pyro call. I don't know PHP but it seems to me that it should be easy to make a custom http request to a server running on some localhost port. This may be faster as well because you're not starting up python processes for every call.

(edit): another succesfully working solution seemed to be the following, where sudo is used to invoke pip under the appropriate user, letting it install the library into www-data's .local library folder:

create /var/www/.local and /var/www/.cache folders, giving www-data permissons to these folders only (and not /var/www to avoid security issues)
invoke sudo -H -u www-data pip3 install pyro4 You may still need to add --user to the pip command if it's an older version, because I think that only recent pip versions install to the user's lib folder by default instead of to the global system python's lib folder.

Question 5

When using CGI : a PHP process is launched by Apache, and it is that PHP process that interprets PHP code -- not Apache itself.

In theory, a distinct PHP process has to be created for each request -- which makes things slower : Apache has more work to do to answer a request.
(Well, as pointed out by @AlReece45 in a comment, this can be made better using FastCGI)

When using PHP as an Apache module (mod_php, or mod_php5), the PHP interpreter is kind of "embedded" inside the Apache process : there is no external PHP process.

Which means :

No forking to answer a request (faster)
Better communication between Apache and PHP

Generally speaking, I would say that mod_php is the solution that's used the most.

Question 6

You have two options.

Start a separate JVM instance by doing a exec and using a java command.
Spawn a new thread - this will use server's resources though.

You can also combine options 1 and 2 and create a thread that does the exec call.

Call PHP from virtual/custom “web server”

Answers

php,cgi,webserver,fastcgi,mod-php