devproxy

Today, I'm pleased to announce the release of devproxy!  It's an HTTP proxy server written in Go (atop github.com/elazarl/goproxy) for QA/testing purposes.  It intercepts requests directed to your production servers and transparently connects them to a staging server of your choice—including HTTPS (CONNECT) redirection.

This lets your staging server perfectly mirror production, including the hostnames and certificates in use, without needing to elevate permissions to edit /etc/hosts every time you want to switch between production and staging.  Instead, you can switch the proxy on/off in your browser; I use FoxyProxy.

I'd like to thank Elazar not only for writing goproxy (it made my life a lot easier) but also for modifying it to support what I needed to do in devproxy.  I'd also like to thank my boss for letting me release this as open source.

How it Works

Normally, a browser connects directly to an origin and requests a resource.  When using a proxy, it includes the real origin along with the resource when it's making a request.  devproxy checks the requested origin, and either passes the request through unchanged, or makes the request to an alternative server (which I shall call the doppelgänger.)

After the connection is made to the origin or the doppelgänger, devproxy does nothing more than copy bytes between the server and browser.  The doppelgänger will see all of the request headers as if the request had been made to the origin, including the origin's Host header.  If a TLS handshake happens, it uses the certificate configured on the doppelgänger to establish the session.

Step by Step

Let us run our application on port 80 of the local host, and configure devproxy to intercept requests to www.example.com and send them to our local application.  We start devproxy listening on port 8111, set the browser to use it as a proxy, and issue a request to www.example.com.

1. The browser connects to devproxy at port 8111, and issues the command "GET http://www.example.com/".
2. devproxy connects to 127.0.0.1:80 and issues "GET /".
3. Accept, Host, and other HTTP headers are passed through.
4. The application at port 80 sees a request made to www.example.com for "/" and responds.
5. devproxy delivers the response to the browser.

Now let's talk HTTPS.  Assume the same proxy setup, except that the application is configured with www.example.com's private key and speaking HTTPS, and of course devproxy is configured to deliver intercepted HTTPS requests to that port.

1. The browser connects to devproxy at port 8111, and issues the command "CONNECT www.example.com:443".
2. devproxy connects to 127.0.0.1:443 and responds with "200 OK" to the browser.
3. The browser sends a TLS client hello.
4. The local application responds with a TLS server hello from its configured key pair.
5. Since that keypair belongs to the server the browser thinks it's connecting to, the connection succeeds.
6. HTTP happens as usual over the encrypted channel.  The browser issues "GET /" with Host: www.example.com.
7. The application sees a request made to www.example.com:443 for "/" and responds.

devproxy is perfectly capable of delivering traffic to other ports, but it might confuse the staging server (and defeat the "staging is as identical as possible to production" ethos) to receive requests to host:port combinations that don't reflect what is actually running on the server.

An Unsolicited Tip: VirtualBox & devproxy

I used to use a VM for development with a NAT card attached and some port forwardings set, e.g. host:9980→guest:80, but that meant having to configure my app with the "visible URL."  There's no way for the app on guest:80 to know what port on the host points back to the app itself, and guest:80 itself wasn't directly accessible from the host.

With a second network card added to the VM, connected to a second network card on a host-only network with a static IP, devproxy can make the guest app visible on the production URL.  devproxy is simply configured to send requests for the production URL to the guest's static IP, the host can route traffic to it without any need for port mapping, and the app can hardcode its production URL for self-referential links.