AWS API Gateway: Returning 404 Errors with OAuth 2.0

As discussed before, we’ve been building out some services using AWS API Gateway.

We have an OAuth 2.0 infrastructure that predates API Gateway, and we’ve had a lot of problems with third parties being able to use the APIs behind API Gateway.  Almost any mistake that can be made with the Authorization header set leads to an unhelpful message from Amazon CloudFront (which technically underpins API Gateway): “not a valid key=value pair” pointing to the access token in the Authorization header.

As it turns out, one of these error cases is a response that should generate a 404 Not Found response, because the URL doesn’t exist in API Gateway.

There’s a workaround to fake 404 messages: build fake endpoints into the API definition.

1. Define “unknown” parameters in order to define URLs like /{unk1}/{unk2}. You’ll have to define one of these for each URL level you’d like to handle.
2. Define a mock integration in API Gateway for these unknown URLs.
3. Define mapping templates.  The mock integration may (I don’t fully understand these) return the status code based on the JSON input. We send a message like {"statusCode":200} and map the 200 OK response that comes back to a {"message":"Path not found. Please check your URL."} response.

Because API Gateway now “knows” about these bad paths, it secretly configures CloudFront to forward them properly, and the Authorization header is no longer intercepted.

In the process of implementing all this, I found out something rather interesting.  When the API Gateway importer claims it “doesn’t support a default response,” what it is saying is that it doesn’t know what to do with a Swagger level default response.

There’s a separate concept of “default response” in the API Gateway extensions that is very much active.  That was where we needed to place the response mapping template, because otherwise, the only thing that showed in the Integration Response for the unknown URLs was “output passthrough” and not our 404 message.

Another way to look at this is, in the API Gateway console, the “method response” corresponds to the responses defined by the standard Swagger schema.  “Integration response” corresponds to the responses defined in the API Gateway extensions.  The response received from the integration (upstream HTTP service or mock integration) may have any status code, and the API Gateway extensions must map all of these codes to one of the valid codes provided by the Swagger response codes.  Errors in this section seem to result in a 500 code being returned instead of the desired status.

Integration response reacts to the real data received from the integration, then turns it into a “method response” which is the external interface that callers of the Gateway may actually see.

I also had an issue with the Import feature available in the API Gateway Console, where it did not seem to preserve the base URL setting from the Swagger file.  The command-line importer handles it fine.  This ended up being a moot point, since I converted our base URL back to / for technical reasons.

(Our endpoints are like /auth/oauth2/token; the first segment is handled by the custom domain mapping, so our base URL was /oauth2, but what if someone misspells that?  I want to be able to return 404 for /oath2/token just as much as /oauth2/taken.)

Finally, I’m embarrassed to admit I have a couple of tips that would have saved me a lot of time and unusable deployments (to our dev stage), had I done them earlier.

First, turn on logging. The CloudFront Logs interface isn’t as bad as it used to be; you can get a pretty good overview of events by heading into a source, ignoring the individual streams, and going to Search All Events. Without a search, recent events appear in chronological order.

And second, build in the Console, then export with API Gateway extensions to see how a given desired configuration should be represented in a Swagger file.  If the authoritative Swagger you’re building doesn’t import the way you expect, change it, export it, and see what API Gateway thinks it should be. Then port it back to your main Swagger file.

I’ve taken the liberty of uploading a minimal example of the “paths” key for one unknown URL and request method here.  This is pretty much taken directly from our OAuth2 Swagger file, hence the application/x-www-form-urlencoded input template type.  That’s what API Gateway is actually receiving in our case.