Freezing Layers (static vs dynamic)

Let me warn you right up front: this is a philosophical and rambling post.  I'm trying to pull together a series of observations into a coherent structure and it's quite possible that I failed at that.  In addition, because of its lack of quality, “PHP” throughout refers to the 5.x and earlier lines.  Version 7.0 basically addresses the error complaint.  Now without further ado, here is your ancient, rambling post...

I've written some Perl code using FCGI managed by mod_fcgid, in which I constructed my own request loop and application pre-loader.  (And complained about the way mod_fcgid is set up, since it doesn't pre-fork fastcgi children—whenever concurrency is exceeded, starting from zero after a fresh startup, someone has to wait for the world to load just like the bad old CGI days.)

I've written plenty of PHP as well, where you get to generate a response in response to routing once the server environment gets around to delivering the request to you.  The only things that may persist from request to request are allocated on the C level, not by the PHP script directly: most famously, opcode caches and persistent database connections.


It's impossible to make PHP reliable in the face of bugs, because you can't catch all runtime fatal errors.  Even when a global error handler is set via set_error_handler, trying to create objects of undefined classes, call undefined methods, etc. will stop the interpreter without giving your script a chance to send a response (except via register_shutdown_function, which doesn't seem to have access to the error and therefore can't log anything useful).

Taken together, this means PHP spends more time on every request exploring a generally unchanged environment. It means that some bugs can't be handled within the script. PHP dumps some info to its error log and exits without giving the script a chance to do anything.

In contrast, an FCGI or a PSGI/WSGI/Rack based app gets to persist whatever global state it chooses.  It can explore its environment once at startup, then keep all that in memory until the server is reloaded.  Even if it doesn't store anything explicitly, it keeps all the loaded code around, even saving the need to retrieve it from the opcode cache on every request.  The obvious tradeoff here is that these non-PHP apps also don't notice if their environment does change unless they are explicitly notified, or unless they check themselves via middleware or Module::Refresh.

Inside the non-PHP languages, "request processing" exists as a concept.  There's a division between code running before the request loop has started, and code running as part of the request loop.  PHP always executes in the context of handling a request, because its code can't be found until a request happens.  That's just the way PHP itself is defined.

Some other languages squeeze out performance by separating compile and run time: Go for instance.  Go is static enough to take advantage of knowing what functions are available and compiling those exact addresses into the resulting binary.  Part of the environment, in this case, gets explored by the compiler and saved implicitly into the binary.  There is no "function call" opcode being interpreted at runtime by "the language", because the language is gone by then.  All that's left is the machine-level address of the function and the hardware opcode to call it.

At the same time, this removes some flexibility from the language.  A language like PHP can obviously define fresh code by way of eval, but C requires a bit more careful effort to support shared libraries and plugins.  In both of those cases, the final symbol addresses can't be resolved until run time on the target machine, and the compiler/linker must have faith that the symbols will be present with a compatible interface.

Loading code based on some configuration, as commonly done in Java and the whole world of dynamic languages (PHP's include_path can be modified by running code), represents a lot of flexibility in choosing which code gets loaded.  It's not infeasible to write environment-specific files and then include "$env.php" to load code or settings specific to the environment.  Such as an exception handler that emails uncaught exceptions in production, with stack trace, to the developer team.

Go's "single static binary" deployment mechanism is pretty much as far as you can go toward the static end of this spectrum.  C, Java, and most dynamic languages fall in between, and finally Lisp is about the most-dynamic language I can think of, by virtue of letting code running in the image redefine how lexing and compiling even happens.

Some people mock "design patterns" as an indication you're not using their favorite language—one that's more expressive, and generally more functional and more dynamic.  But it turns out those patterns are serving an important purpose: they're a type-checked implementation of that other language.  If you're using a Strategy then someone has to pass in a Strategy and to get that type, the implementation must really exist.  You can't pass just any old String and bring down the whole runtime with an unstoppable undefined function blorf error.

You can look at use strict as a lexical switch meaning "I don't want to do wild and crazy things," which limits some constructs and makes such code… a little more static.  The same thing happens with advice to avoid the more-dynamic features of PHP: eval, and variable variables/functions/methods.  At one point in my career, I ran into a fairly static PHP codebase, and it turned out to be exceedingly handy that I could just grep around it and be pretty sure I found every usage of something.  It couldn't be hiding in a string or variable variable.

This isn't strictly about programming languages, either.  To launch an auto scaling instance in AWS EC2, I need to specify a base image.  The generic images don't have any of my code or configuration on them, so I roll my own image.  This means the typical image boots twice: once during the image-building process, where it starts with a generic image and becomes my image, and once when auto scaling launches it.

When I was new at this, I wrote a set of "build me an instance!" scripts that did everything I needed.  Then they collected a pile of exceptions, because a number of them would fail if they were run twice, such as the one that called useradd.  It turned out that I really wanted two sets of scripts: one specifically to build the instance for packaging, and one to update things on the live instance.  Most of what needs to be done can be frozen into the base image ahead of time; but others (such as how the final DNS address needs to be known before it may be added to an SNS subscription for code updates) need to be handled later.

Overall, then, dividing work into "phases" is both "the thing that makes code more static" and the important optimization of "skipping redundant work."  For a production server that's going to run the same code for 24-72 hours at a time (unless there's a hot-fix to deploy), there's no real benefit to recompiling the world for each of the thousands of PHP requests that will hit the system over that period of time.  Facebook decided that simply caching the oparrays like APC or OpCache wasn't enough, and built HipHop (now HHVM), which acts in effect as both an opcode cache and optimizer.

Dynamic languages end up feeling more open and permissive because more things are deferred until runtime, and can thus be interacted with there.  PHP's extract() function has no C equivalent, because the variable names are gone by the time the C code is running.  Everything is accessed instead by the machine-level address and offset.  It's faster, but it's also not exposed to the program for any sort of access via a string containing the variable name.

In more static languages, you can build anything you need yourself, including interpreters.  Thus, Greenspun's Tenth Rule.  The Python community takes the "built it yourself" angle seriously with the lack of PHP-style variable-variables (or Perl non-strict refs): if you need dynamic lookup, you are expected to build a dict for it, not have the language do it for you under the hood.  (Then, being dynamic, it provides ways to dodge this philosophy, such as def __init__ (self, **kwargs): self.__dict__.update(kwargs) mentioned in Norvig's Python IAQ.)

Dynamic languages are easier to weave magic into, but they also require more discipline to understand.  The more the foundation can be moved, the more a programmer needs to keep track of in order to interpret the code in their own head.  (Which is essentially what the process of understanding it is.)

Freezing things into static layers helps with performance and understandability—things that aren’t going to change can be taken for granted—but they have their own price.  Whenever anything in that layer does need to change, the layer needs torn down and rebuilt.  Such as restarting a server for a FastCGI program that has cached some module code that has been updated.  When have you restarted Apache because a new PHP file was uploaded?  Never, because even opcode caches maintain the illusion that the file is reloaded every request.

I half-want to argue against static-ness and freezing, because I can build such amazing dynamic things.  (Unless their complexity exceeds the threshold of my ability to understand it.)  And because it can be difficult or disruptive to rebuild those underlying layers when they change.  On the other hand, the potential performance of such a layer is really attractive.  There’s no guarantee a dynamic system can easily have static parts extracted from it at a later time... but there’s no guarantee a static system can be flexible enough to meet future needs.

Of course, that’s why there is maintenance.

The static problem bites projects the hardest that are making heavy use of libraries they don’t own and don’t want to modify, for whatever reason.  (I treat modification of third-party code as a last resort, because I don’t want to maintain a fork.  But sometimes, that attitude gets back into trying not to modify libraries we do own, ourselves, and it can be a bit of a waste there.)  If the library author doesn’t provide an extension point, such as how you can’t pass your own stream connections into React\ChildProcess\Process, it can result in a need to build an entire separate class, duplicating all the work, just to provide that feature.

Meanwhile, I’ve observed people approach dynamic/flexible/extensible code by ignoring the whole architecture and putting in some new code wherever they can fit it.  When there are a few ways to proceed with a problem, I like figuring out which one is best, and fitting the new code neatly in, like it has been a puzzle piece that was missing the whole time.  Other people just want to get their bug fixed and move on.  If there were less flexibility, it seems like it would make the appropriate spot to fit code into the architecture clearer.  Limiting the options would make the search for where to put new code smaller and faster.