Remember FastCGI? (2021)
I went down this rabbit hole a while ago[1], it's a fun experience to try out technologies that nowadays are considered unconventional or outdated.
[1]: https://deadlime.hu/en/2023/11/24/technologies-left-behind/
If you find yourself with a scripted language where processing HTTP requests might be too slow or unsafe, I can still see some utility for FastCGI. For most of the rest of us, HTTP won, just write little HTTP webservers.
Those of us with an interest in Gemini (the small web protocol, not the AI thing) will have become familiar with CGI and its variants again, as they are commonly used to serve dynamic content over that protocol. Another variant is SCGI[0]. It doesn't seem to have ever been nearly as popular as FastCGI, but seems to be more commonly supported by Gemini servers due to its simplicity.
It's baffling how FastCGI ever was popular, considering how weird it's premise is: let's take this well-known standard simple text based protocol and convert it to our own custom binary protocol, that will make writing services somehow simpler?! You are still dealing with all the complexity of writing a daemon, only difference is that instead of parsing HTTP you parse FastCGI.
Maybe today with all the complexity that is with http2/http3 etc it would make slightly more sense. But FastCGI was popular when all the world was just good old plain http1.
Rails, PHP, a lot of Python stuff (WSGI is CGI wrapped in a dict).
It's not only just about separation of concerns, but also separation of crashes/bugs/issues. FastCGI servers can run for years without restarts.
Thread creation/teardown/sleeping has gotten a lot faster as well in linux.
https://php-fpm.org/about/ it may be old but PHP-FPM is still one of the best FastCGI servers from a pragmatic point of view... ex: the ability to gracefully hot reload code, stop and start workers without losing any queries... all in production.
Similar protocols like ASGI for Python are still ubiquitous. I assume because they offer separation of concerns and as a result good performance.
You would think the separation argument would still hold true for compiled languages, even if performance is no longer as relevant.
There are also other modes/roles of FastCGI: Authorizer and Filter which may be interesting: https://fastcgi-archives.github.io/FastCGI_Specification.htm...
Not sure I've ever seen Filter in real life.
Lots of people are forgetting the context in which FastCGI was conceived: servers were expensive uniprocesser machines and enterprise servers with 2 or 4 sockets were relatively extremely expensive. All processors at the time paid heavier penalties for context switching and/or creating between processes. So the optimization folks building web servers were things like preforking the front end in Apache and/or preforking the middle end (FastCGI, WSGI, SCGI, Java App Servers, et al). The database was Oracle, or MySQL or SQLServer which was already multi client based. Also, in *nix land, the popular scripting languages (Perl, PHP, bash) of the time had a ways to go in reducing their startup times.
The thing (Fast)CGI had, that http proxying doesn't (and lots of web frameworks/libraries a bit too tied to http, like go net/http don't) have is the SCRIPT_NAME (path processed so far) / PATH_INFO (path left to handle) distinction
Python has cgi-bin support, I made a proxy of the poor like this:
$ mkdir cgi-bin $ echo -e "#!/bin/bash\necho -e \"Content-type: text/html\n"curl -s -k http://www.zoobab.com -o -" > cgi-bin/proxy.sh $ python -m http.server --cgi 8000 $ curl http://localhost:8000/cgi-bin/proxy.sh
You should get the html page of http://www.zoobab.com
Sure, but does anyone remember SCGI?
I never really understood the rational for using fastcgi. For everything I wanted to do it always made more sense to just use custom http servers and proxys.
CGI makes sense, it fills a specific environment, one request, one process, but fastCGI? why? its a server with a different incompatible HTTP. did everyone jump on it just because it had CGI in the name and the associativity CGI == web process was too firmly entrenched in peoples minds.
But I am not really in a web ops role, so who knows, perhaps fastcgi does have some sort of advantage when run at scale.
In the pre-days before fastcgi, I knew a guy who compiled his c++ "cgi-bin" programs directly into apache, so running them was just a function call, not a fork.
Was FCGI ever common outside of PHP?
Why "Remember"? php-fpm is probably powering millions of sites right now.
My biggest annoyance with FastCGI is the ambiguity between FastCGI Server, FastCGI Application and FastCGI Proxy in documentation and articles.
To give some examples what I mean:
- Apache can be a FastCGI server (mod_fcgid) and proxy (mod_proxy_fcgi) - Nginx (and most other Webservers I checked) is only a FastCGI proxy
Wikipedia [1] lists both as "Web servers that implement FastCGI". For me it took some time to recognize the difference whether a server speaks the FastCGI protocol or can also host/spawn FastCgi supporting applications which do not daemonize by themself.
For the (probably) most used FastCGI application PHP this is easy because php-fpm is both a FastCGI server and application.
An example for a (Fast)CGI application which is not also a server is MapServer [2] which is listed by Wikipedia as CGI program and by its own documentation with "FastCGI support" or "FastCGI-enabled".
The fact that it is only a FastCGI application and needs an additional FastCGI server to host it is (in my opinion) not clearly communicated.
A common tutorial to combine MapServer with Nginx is to use fcgiwrap or spawn-fcgi as FastCGI server.
Since a FastCGI application can usually also act as CGI application it is easy to miss that the first option is running the application as CGI and only the second as FastCGI where one instance serves more than one request.
For me the main difference is that a FastCGI server will open a socket and can spawn worker processes (which php-fpm does and MapServer does not) and a FastCGI application (the worker process) will accept and handle connections/requests.
A FastCGI proxy translates requests (usually from HTTP to FastCGI protocol) and redirects them to a FastCGI server. It does not spawn processes like Apaches mod_fcgid does.
Due to the simplicity it should be even possible to use systemd as simple FastCGI server. [3] (I have never tried it)
My recommendation if you ever come across an FastCGI application which cannot open a socket / daemonize by itself use Apache mod_fcgid. Even if you already use another Webserver to handle HTTP. From all options I checked it has by far the most useful options to dynamically limit the number of worker processes and respawn them from time to time.
[1] https://en.wikipedia.org/wiki/FastCGI [2] https://en.wikipedia.org/wiki/MapServer [3] https://nileshgr.com/2016/07/09/systemd-fastcgi-multiple-pro...
I still see it today
RIP, FastCGI.
FastCGI is a protocol you can use between a reverse proxy and a back end. It's better for this than HTTP because it passes proxy-generated metadata out of band from client-generated metadata, so you can't have vulnerabilities like fake X-Forwarded-For. It's also strictly defined so you can't have request parsing differences or request smuggling.
If you're currently using a reverse proxy, did you remember to make sure that your proxy always deletes X-Forwarded-For from the client, always adds its own, OR that the backend always ignores it? And you have to do this for each piece of metadata you expect to come from the proxy. With FastCGI this is not needed.
I chose SCGI instead of FastCGI, though, since nginx doesn't support multiplexing and I don't use large request bodies. SCGI not supporting multiplexing makes it much simpler to write a back end. You just accept a new socket and fork for each request.
By the way, FastCGI wasn't designed as "binary HTTP" as implied by some sibling comments, but rather "CGI over a socket". It passes the environment variables the CGI program would have had, and multiplexes its stdin, stdout and even stderr. SCGI is the same but without multiplexing or stderr.
Author complains about having to use a reverse proxy at all, which is fine for prototyping, but I have about 5 domains pointed at the same server, and multiple apps on some domains, so why wouldn't I use a reverse proxy to route those requests? And yes, I run the same nginx reverse proxy on my development machine for testing.