Today Icelab Learned
 about ops

NGINX proxy_pass DNS caching and Heroku

We occasionally use NGINX to proxy_pass to an app on Heroku so that we have a consistent IP address that can be whitelisted in clients’ firewalls. There’s a quirk in the way that NGINX caches DNS lookups that can cause problems when proxying to Heroku.

By default, NGINX will cache the DNS result (the resolved IP address) for any statically defined URLs for proxy_pass on startup. This is problematic on Heroku because the IP address for the underlying app changes continuously, and so it a cached result falls out of Heroku’s routing mesh we can proxy to the incorrect server.

The solution to this problem is to avoid caching the DNS resolution, at least as much as is possible. You can “trick” NGINX into doing this by setting the proxy_pass value to a variable:

server {
  resolver 8.8.8.8;
  # Set as a variable to force dynamic lookups
  set $proxy_pass_url https://my-app.herokuapp.com;

  location / {
    proxy_pass $proxy_pass_url;
  }
}

Even though the value of the variable never changes, NGINX thinks it might and thus will refresh it regularly.

Creating and storing SSL certificates

As part of the SSL Certificate generation process you’ll need to create a Certificate Signing Request. You can do this with the command:

openssl req -nodes -newkey rsa:2048 -sha256 -keyout server.key -out server.csr

This will generate two files: a public .csr file, and a private .key file

You can view the files with the commands cat server.csr and cat server.key

Here at Icelab we recommend saving this information in a new 1Password secure note named with the following format:

[domain-name.com] SSL CSR

Save the data from both files into the secure note and continue to be happy and confident in the face of adversity :)