Can OpenStreetMap run on Heroku?

It's been a long-term goal of mine to see if we can get the OpenStreetMap Website and API running on Heroku.

So can we? In short - mostly, but it's still got some rough edges.

But why?

OSMF already has the website and API up and running, obviously, and it needs some pretty specialist hardware for the databases. There are no plans whatsoever to move all that to Heroku, so at first glance it seems rather pointless for me to investigate if it actually works. So why am I spending my time on this?

Firstly, it's useful for other people to have their own working copies of the website. They can deploy the same code, but use their copy to map historic features that aren't relevant in OpenStreetMap. Or they can use it to make collaborative fantasy maps, or maps for proposed local developments, or other projects that aren't suitable for putting in the main database.

Secondly, it's also useful for developers. We already have a system on the OSMF development server for testing branches, which allows non-developers to check how proposed features work, debug them and provide feedback on them. But if you have a branch to test you need to ask the sysadmins, and wait for them to deploy it for you, and you don't get full control over the instance if you want to debug it or to dive into the database.

But mainly, I'm doing this because Heroku is a useful yardstick. Even if you aren't interested in using Heroku itself, it's perhaps the best example of a modern Rails hosting environment. If we can make it easy to deploy on Heroku, we'll have made it easy to deploy in all manner of different deployment situations, whether that's in-house in a controlled corporate environment, or using cloud native builder images for Kubernetes, or something else. So I think it's worth using Heroku to find out where the kinks are in deploying our code, and then to iron those kinks out.

Some ironing is required

In no particular order, here are some of the problems I came across today:

So a few minor hiccups, but nothing particularly insurmountable. I was pleasantly surprised! While we work on resolving some of these problems, there's a whole load of things that already work.

How we got here

I would describe getting to this stage as a useful "side effect" of lots of previous work. The most recent was replacing the compiled database functions with SQL equivalents, which are significantly easier to handle when you don't control the database server. Previous work on using ActiveJob for trace processing makes the worker dyno setup possible, and moving the quad_tile code to an external gem made this part of the installation seamless.

Did I mention the price?

So how much does deploying your own copy of the OpenStreetMap website on Heroku cost? Nothing. Zero dollars per month. Heroku gives you 1 web and 1 worker dyno for free on each application that you deploy, and that's all we need. It also offers a free database tier too, which is enough to get started. Of course these have limitations, and you'll need to pay for some S3 storage if you want user images (and traces, in the future). But I think it's worth pointing out that you can spin up your own deployment without having to pay, and dive into testing those new features or creating your own OSM data playground. I'm sure that'll be useful to many people.

This post was posted on 6 November 2019 and tagged 12factor, development, heroku, OpenStreetMap