{"id":1465,"date":"2019-03-01T18:27:12","date_gmt":"2019-03-01T22:27:12","guid":{"rendered":"https:\/\/www.openbible.info\/blog\/?p=1465"},"modified":"2019-03-04T13:18:35","modified_gmt":"2019-03-04T17:18:35","slug":"using-machine-learning-to-enhance-the-resolution-of-bible-maps","status":"publish","type":"post","link":"https:\/\/www.openbible.info\/blog\/2019\/03\/using-machine-learning-to-enhance-the-resolution-of-bible-maps\/","title":{"rendered":"Using Machine Learning to Enhance the Resolution of Bible Maps"},"content":{"rendered":"\n

In a previous post, I discussed how 3D software could improve the resolution of Bible maps<\/a> by fractally enhancing a digital elevation model and then synthetically creating landcover. In this post I’ll look at how machine learning can increase the resolution of freely available satellite images to generate realistic-looking historical maps.<\/p>\n\n\n\n

Acquiring satellite imagery<\/h3>\n\n\n\n

The European Sentinel-2<\/a> satellites take daily photos of much of the earth at a ten-meter optical resolution (i.e., one pixel represents a ten-meter square on the ground). The U.S. operates a similar system, Landsat 8<\/a>, with a fifteen-meter resolution. Commercial vendors offer much higher-resolution imagery, similar to what you find in Google Maps, at a prohibitive cost (thousands of dollars). By contrast, both Sentinel-2 and Landsat are government-operated and have freely available imagery. Here’s a comparison of the<\/a> two<\/a>, zoomed in to level 16 (1.3 meters per pixel), or well above their actual resolution:<\/p>\n\n\n\n

\"Sentinel-2<\/figure>\n\n\n\n

The Sentinel-2 imagery looks sharper thanks to its higher resolution, though the processing to correct the color overexposes the light areas, in my opinion. Because I want to start with the sharpest imagery, for this post I’ll use Sentinel-2.<\/p>\n\n\n\n

I use Sentinel Playground<\/a> to find a scene that doesn’t have a lot of clouds and then download the L2A, or atmosphere- and color-corrected, imagery. If I were producing a large-scale map that involved stitching together multiple photos, I’d use something like Sen2Agri<\/a> to create a mosaic of many images, or a “basemap” (as in Google Maps). (Doing so is complicated and beyond the scope of this post.)<\/p>\n\n\n\n

I choose a fourteen-kilometer-wide scene from January 2018<\/a> showing a mix of developed and undeveloped land near the northwest corner of the Dead Sea at a resolution of ten meters per pixel. I lower the gamma to 0.5 so that the colors approximately match the colors in Google Maps to allow for easier comparisons.<\/p>\n\n\n\n

\"The<\/a><\/figure>\n\n\n\n

Increasing resolution<\/h3>\n\n\n\n

“Enhance!<\/a>” is a staple of crime dramas, where a technician magically increases the resolution of a photo to provide crucial evidence needed by the plot. Super-resolution doesn’t work as well in reality as it does in fiction, but machine learning algorithms have increased in their sophistication in the past two years, and I thought it would be worth seeing how they performed on satellite photos. Here’s a detail of the above image, as enlarged by four different algorithms, plus Google Maps as the “ground truth.”<\/p>\n\n\n\n

\"Comparison<\/figure>\n\n\n\n

Each algorithm increases the original resolution by four times, providing a theoretical resolution of 2.5 meters per pixel.<\/p>\n\n\n\n

The first, “raw pixels,” is the simplest; each pixel in the original image now occupies sixteen pixels (4×4). It was instantaneous to produce.<\/p>\n\n\n\n

The second, “Photoshop Preserve Details 2.0,” uses the machine-learning algorithm built into recent versions of Photoshop. This algorithm took a few seconds to run. Generated image<\/a> (1 MB).<\/p>\n\n\n\n

The third, ESRGAN as implemented in Runway<\/a>, reflects a state-of-the-art super-resolution algorithm for photos, though it’s not optimized for satellite imagery. This algorithm took about a minute to run on a “cloud GPU.” Generated image<\/a> (1 MB).<\/p>\n\n\n\n

The fourth, Gigapixel<\/a>, uses a proprietary algorithm<\/a> to sharpen photos; it also isn’t optimized for satellite imagery. This algorithm took about an hour to run on a CPU. Generated image<\/a> (6 MB).<\/p>\n\n\n\n

The fifth, Google Maps<\/a>, reflects actual high-resolution (my guess is around 3.7 meters per pixel) photography.<\/p>\n\n\n\n

Discussion<\/h3>\n\n\n\n

To my eye, the Gigapixel enlargement looks sharpest; it plausibly adds detail, though I don’t think anyone would mistake it for an actual 2.5-meter resolution satellite photo.<\/p>\n\n\n\n

The stock ESRGAN enlargement doesn’t look quite as good to me; however, in my opinion, ESRGAN offers a lot of potential if tweaked. The algorithm already shows promise in upscaling video-game textures<\/a>–a use the algorithm’s creators didn’t envision–and I think that taking the existing model developed by the researchers and training it further on satellite photos could produce higher-quality images.<\/p>\n\n\n\n

I didn’t test the one purpose-built satellite image super-resolution algorithm<\/a> I found because it’s designed for much-higher-resolution (thirty-centimeter) input imagery.<\/p>\n\n\n\n

Removing modern features<\/h3>\n\n\n\n

One problem with using satellite photos as the base for historical maps involves dealing with modern features: agriculture, cities, roads, etc., that weren’t around in the same form in the time period the historical map is depicting. Machine learning presents a solution for this problem, as well; Photoshop’s content-aware fill allows you to select an area of an image for Photoshop to plausibly fill in with similar content. For example, here’s the Gigapixel-enlarged image with human-created features removed by content-aware fill:<\/p>\n\n\n\n

\"Modern<\/a><\/figure>\n\n\n\n

I made these edits by hand, but at scale you could use OpenStreetMap’s land-use data to mask candidate areas for content-aware replacement:<\/p>\n\n\n\n

\"Data<\/a><\/figure>\n\n\n\n

Conclusion<\/h3>\n\n\n\n

If you want to work with satellite imagery to produce a high-resolution basemap for historical or Bible maps, then using machine learning both to sharpen them and to remove modern features could be a viable, if time-consuming, process. The image in this post covers about 100 square kilometers; modern Israel is over 20,000 square kilometers. And this scene contains a mostly undeveloped area; large-scale cities are harder to erase with content-aware fill because there’s less surrounding wilderness for the algorithm to work with. But if you’re willing to put in the work, the result could be a free, plausibly realistic, reasonably detailed map over which you can overlay your own data.<\/p>\n","protected":false},"excerpt":{"rendered":"

In a previous post, I discussed how 3D software could improve the resolution of Bible maps by fractally enhancing a digital elevation model and then synthetically creating landcover. In this post I’ll look at how machine learning can increase the resolution of freely available satellite images to generate realistic-looking historical maps. Acquiring satellite imagery The […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[34,1],"tags":[],"_links":{"self":[{"href":"https:\/\/www.openbible.info\/blog\/wp-json\/wp\/v2\/posts\/1465"}],"collection":[{"href":"https:\/\/www.openbible.info\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.openbible.info\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.openbible.info\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.openbible.info\/blog\/wp-json\/wp\/v2\/comments?post=1465"}],"version-history":[{"count":3,"href":"https:\/\/www.openbible.info\/blog\/wp-json\/wp\/v2\/posts\/1465\/revisions"}],"predecessor-version":[{"id":1469,"href":"https:\/\/www.openbible.info\/blog\/wp-json\/wp\/v2\/posts\/1465\/revisions\/1469"}],"wp:attachment":[{"href":"https:\/\/www.openbible.info\/blog\/wp-json\/wp\/v2\/media?parent=1465"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.openbible.info\/blog\/wp-json\/wp\/v2\/categories?post=1465"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.openbible.info\/blog\/wp-json\/wp\/v2\/tags?post=1465"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}