Styled Museums

The Prisma app became an overnight hit after it launched because of its great photo filters. Rather than just applying some face feature transformation or adjusting the colors, it rerenders a picture in the style of a famous artwork. The results are remarkable and quite recognizable. And it is no secret how this is done. The basic algorithm is described in a paper published a year ago called "A Neural Algorithm of Artistic Style"

Besides the scientific paper and the startup executing on it, there's also an open source implementation of the algorithm. I played around with it a bit and it also works well, although it seems roughly 100x slower than Prisma. It got me thinking, what happens if you use this to re-render pictures of museums in the style of their most famous work? That way you see what the building is like and at the same time what to expect when you go in.

Styled Museums does exactly that. It has the top 100 museums (by their wikipedia page view count) and their most popular works (same measure) and shows them on a world map. It uses the wiki_import frame work to get the data. You can click around and find your favorite museum and see what happens to it.

I think the fact that artistic style transfer is available as a scientific paper, a startup and an open source implementation is indicative of a wider trend. We now live in a world where we have three forms of innovation. The traditional scientific method where publicly financed institutions produce papers describing new ideas; the startup world that funnels large amounts of private money into ideas to see if they come to commercial fruition; and finally the open source world where individuals build something and share it with the world to build up their public profile.

These three engines of innovation aren't silos. Google started out as a scientific experiment, became a startup and a commercial success and now publishes scientific papers and open sources part of their technology. Github is a startup that is not only based on an open source project, but also hosts other open source projects. Twitter open sourced their data processing engine, which now helps academics keep up with what their peers in Silicon Valley are up to.

It doesn't always seem fair. The founders of Google became billionaires with technology developed while being employed by Stanford University, while the inventor of the world wide web works for a non profit. For years Werner Koch maintained the GnuPG email encryption package on the salary of a postman, while the founder of Hotmail is worth more than a 100 million dollars.

The 1980 Bayh-Dole Act in the US explains some of the difference between there and Europe. It allows universities and companies to claim patent rights on research undertaken with federal funding. On one level this doesn't seem right - if the government paid for research, shouldn't the patents end up with the government too? Then again, it turns out that the government isn't particularly good at doing interesting stuff with those patents - startups do much better.

And so we end up with Styled Museums. Inspired by Prisma, a VC funded company, I found the original paper which is based on research paid for mostly by the University of Tübingen, which in turn led me to an Open Source implementation. You can find the code used to get to Styled Museums (of interest is mostly the matching of museums and paintings) on Github, of course.