Gizmos & Gadgets: New AI Tool Mines Satellite and Aerial Images With Surprising Results

July 2, 2017 – Giving a penny (a coin no longer in existence in Canada) for this artificial intelligence’s (AI) thoughts will buy you a whole lot of interesting data. The AI goes by the name Penny. It is the creation of Stamen Design, Carnegie Mellon University, GBDX, and DigitalGlobe. The latter provides commercial satellite images of cities and other places on the planet.

Census data on median household income by district along with satellite imagery is fed to Penny. The AI, a neural network application, looks at the patterns in the imagery and correlates them with data from the census. Penny learns the patterns and begins to discern the characteristics of neighbourhoods that have high, medium and low incomes. For example, Penny defines a digital image showing baseball diamonds, parking

For example:

• When Penny looks at a digital image that shows baseball diamonds, parking lots and extensive tracts of large uniform buildings, the census data combined with the images make it realize that this pattern equals lower income.

• When Penny sees single family homes on small lots, rowhouses with green space, and variable mid-rise and high-rise building structures, then matches the images to the census data, it defines the area as middle income.

• And when Penny sees large lots with single family homes, tall high rise apartments and lots of surrounding green space, and then incorporates the census data, it defines the area as high income.

Repeatedly running the census data and images from Digital Globe has trained Penny to recognize these patterns and accurately predict income levels by looking at new satellite imagery.

But that’s just the beginning. Change the images to include a proposed new project and then have Penny take a look. The results sometimes show an intuition that is clearly not human. Add a new park to a neighbourhood and depending on the data and existing images, Penny will surprise you. A poorer neighbourhood may not gain the same income boost as a medium or high-income level area when the park gets plunked down in its midsts.

Ask Penny why its conclusions and it logically explains its interpretation and predictions.

For city planners, land developers and architects, Penny’s predictive capability could prove to be an invaluable tool for tinkering. You can imagine all kinds of “what if” scenarios, create an image and place it within an existing one and let Penny tell you the impact of your idea or project.

Penny’s trainer, Aman Tiwari, a computer scientist at Carnegie Mellon, states about his AI’s predictions, “we don’t know whether it knows something that we haven’t noticed, or if it’s just plain wrong.” As a result, Penny’s developers are studying the AI’s non-human intuition to better understand machine learning.

A next generation of Penny will soon be in the works. We should then know from an AI’s perspective whether putting a new high-rise tower in an already highly urban intensified area will have a positive or negative impact on income levels.