No one knows where America’s helipads are, except this neural networkGareth WalkerBlockedUnblockFollowFollowingJun 10Photo by Wayne Chan on UnsplashThe race to the futureI don’t know about you, but I distinctly remember a promise from 1989 that flying cars would be commonplace by 2015.
Better late than never, the reality of on-demand air mobility is rapidly approaching US markets.
In 2016, Uber published its Uber Elevate White Paper, setting its aspirations on providing on-demand air taxis from San Francisco to San Jose for about $130.
Concurrently, Voom (an Airbus company) had already begun providing this service in Sao Paulo at a similar price point and, a year later, expanded to Mexico City.
Both are set to enter US markets and both face a similar challenge.
Dude, where’s my flying car?The main challenge of providing flying taxis doesn’t lie in the technology of the vehicles themselves, but the infrastructure to support them.
In addition to addressing the need for updated airspace regulation and management, beacon placement, and refueling stations, there is an even more fundamental task at hand: mapping where the current helipads are, and figuring out where they should be.
Helipads should be easy to find.
They’re designed to be highly visible from the air, often occupy high value surface areas in downtown locations, and are growing increasingly valuable to the next generation of urban ridesharing solutions.
Yet current data on their locations are incomplete and inaccurate, undermining our ability to strategically plan for their integration into future urban transport systems.
As an Insight Fellow, my project sought to address this problem through training a convolutional neural network (CNN) to recognize helipads in aerial imagery and record their physical coordinates.
Below is a video demo of the project.
If you’re interested in a more in-depth discussion of the project, please read on!Where are the pads?!It might surprise you to hear that the number and locations of helipads in the United States are a huge known unknown for the urban air mobility industry.
The majority of helipads in the US are privately owned and subject to relatively lax requirements from the Federal Aviation Authority (FAA) when it comes to registering and updating their status and location.
In many cases, pads do not currently serve any public or commercial purpose, so they’re also overlooked by popular search engines.
The result is that the FAA database is both incomplete and worryingly inaccurate.
In a 2015 study of the database, researchers noted that coordinates for pads were often wildly off-target and, in 50% of cases, they simply couldn’t be found anywhere near the coordinates provided.
Below are a few examples of the errors which crop up in the FAA database.
Source: FAA’s Heliport Location Database May Be Rife With Errors, Rex J.
Alexander | June 1, 2016Training a convolution neural network (CNN) to spot helipadsThe solution I developed rests on retraining a CNN to recognize helipads in aerial images.
This serves as the engine for scanning an area defined by the user and returning the latitudinal and longitudinal coordinates of a suspected helipad.
The method used is called transfer learning, which takes an existing CNN trained to recognize distinctive features in imagery, and retrains the final classification layer of the network.
In my case, this retraining distinguishes between images of ‘helipads’ and ‘not helipads’.
While the FAA database isn’t perfect, I did find around 3k images of helipads by passing the coordinates listed in the database through the Google Maps API and storing the results.
The ‘not helipads’ category was an interesting challenge: I needed a dataset of aerial images at a similar scale and context (predominantly urban areas) which would have an extremely low probability of containing helipad images.
The solution I came to in the end was to randomly scrape aerial images from downtown Washington DC, as this is a city with a heavily restricted airspace and hence almost no helipads.
Transfer learning applied to ResNet50 CNNTraining resultsThe final accuracy of the model, when tested on a subset of the training data, was 97%.
This means that, in 97% of cases, it successfully distinguished between ‘helipad’ and ‘not helipad’.
As you can see below from the rapid reduction in validation loss over increasing training sizes, the model quickly converged.
Training results of transfer learning (error vs training size)Sampling the real world: sliding window approachIn order to apply this model to a real-world scenario, I needed to find a way to sample aerial imagery and pass it through the CNN.
I decided to implement a sliding window sampling regime, essentially parsing over a user defined area in incremental steps while making requests to Google Maps’ API.
Below is a visualization of this approach for the example area I provide in my video demo.
A 2km² area generates nearly 0.
5 million tile samples from Google MapsData managementData management posed the first challenge because, as you can see, a sliding window approach generates a very large number of samples, each needing to be stored and processed alongside its corresponding latitude and longitude.
For the 2km² area of downtown Los Angeles that I used as my test case, almost half a million unique tiles needed to be collected and analyzed.
The solution here was to develop code that could batch process the downloading and analysis of tiles while periodically updating a data frame to save the location of suspected helipads.
Addressing imbalanced classes through re-samplingMy original training set of images was balanced as a 1:1 ratio of helipads to non-helipads.
In the real world, however, my model will see a vastly larger number of not-helipad images.
This class imbalance led to my original model generating a large number of false positives.
In order to reduce this false positive rate, I re-sampled my training set to include double the number of non-helipads.
This significantly reduced the false positive rate while leaving true positives unaffected.
Below is a visualization of the results.
The image on the left is a heat map of reported positives by a model trained on a 1:1 ratio of pads to not-pads.
The image on the right shows the outputs of a 1:2 ratio.
As you can see, there is significantly less heat over false positives, while the heat over true positives remains (and, in this case, it also corrected a false negative!).
Rebalancing training class data reduced false positives (actual helipads circled in yellow)Dealing with noise in the model using DBSCANAs a window ‘slides’ over a pad, the model will also recognize the pad more than once, generating repeat positives.
This is illustrated in the image below, which shows the raw outputs of the detection model.
The repeat positives clustered around certain buildings are (mostly) cases where a helipad is present, and the isolated red dots are (mostly) false positives.
Red dots represent a positive result from CNN: dense ‘blobs’ are where helipads are located, sparse dots are likely to be false positives.
At this stage, the challenge is two-fold:Consolidating the repeat positives of the model and;Removing the noise of any residual false positives.
My solution was to use a Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering algorithm to further process the results of the detection.
A further advantage of using DBSCAN is that contrary to many other clustering techniques, it does not require you to specify how many clusters should be generated (known in machine learning terms as the problem of ‘choosing k’ ).
Given that the whole point of this enterprise is to identify helipads, any clustering algorithm which required a pre-specified number of clusters would be impractical (we don’t know how many pads are out there!).
Instead, DBSCAN uses the parameters of a minimum number of neighbors and the distance between neighbors as criteria for a cluster, making it a useful tool in spatial analysis.
As the image below illustrates, after some parameter tuning, the model now mostly indicates only the location of helipads (note there is some misalignment due to parallax effects in satellite images of tall buildings).
DBSCAN clustering significantly reduces noise while summarizing repeat positives.
Precision and recall: a business perspectiveWith the CNN and clustering algorithm working well in series, I turned my attention to a real-world case study.
I chose downtown Los Angeles, which has a high density of helipads, but relatively few registered in the FAA database or with Google.
Choosing a small area of the city to apply my model, I ran a script to scrape high resolution images of the area from Google Maps, parsed those images with the CNN, and stored the final clustered results in a data frame of latitudes and longitudes.
As a baseline for comparison, I manually labeled the location of all helipads in the selected area (50) and compared my model’s predictions.
The results were interesting: the model only missed 5 helipads out of 50 (false negatives), but also tended to fixate on other objects with similar geometry, like baseball fields, and falsely identified 22 objects as helipads (false positives).
To better express this performance as a normalized score, it’s helpful to employ the two concepts of precision and recall.
Precision is the fraction of positive predictions which are correct: in other words what proportion of objects that the model identifies as helipads, are in fact helipads?.It is quantitatively defined as:Recall, on the other hand, is the fraction of all potentially correct predictions identified by the model; in other words; what proportion of helipads in the area did the model detect?.It can be quantitatively defined as:We can then combine these useful metrics with a confusion matrix to provide a powerful summary of the performance of the model.
Confusion matrix for final real world test of the modelYou’ve probably noticed that the model performs much better in terms of recall than it does precision, and it’s worth asking what the business implications are of such a disparity.
Which is more important?.In the case of my project, the use case is a ridesharing company evaluating a downtown city area for the availability and location of helipad infrastructure.
The cost of correcting a false positive (that is, manually spotting a false positive and re-labeling it) is relatively small.
However, the cost of correcting a false negative is extremely high (that is, manually checking the whole city for helipads, i.
e.
the whole point of the project).
Therefore, in this particular use case, I would rather be sure my model catches all the pads in the scan, while accepting a few false positives at the same time.
This translates into emphasizing a high recall, while de-prioritizing precision.
ConclusionsI started this project with a clearly defined use case: urban air mobility companies evaluating city infrastructure for market entry.
I also identified a key blind spot in their dataset: the location of existing helipads in US cities.
My project demonstrated that, with a trained CNN, it is possible to scan an area of a city and automate the detection of helipads.
This functionality in itself represents a positive return on investment in terms of automating helipad search processes.
As discussed earlier, it is far cheaper to confirm suggested results from a model than it is to manually go out and generate them yourself.
Scalability and developmentThe work isn’t quite over yet though.
Demoing this project to companies across Seattle also taught me that, while having a convincing MVP is all well and good, it is equally important to have a clear understanding of how the product would scale.
In my case, I believe there are two major avenues for development: optimizing the algorithm to search intelligently over a city, and sorting the results according to usefulness to the user.
I’ve sketched out some thoughts for each below.
Search optimizationAs discussed, this solution is computationally intense.
The current approach to narrowing the task is to ask a user to define ‘areas of interest’ within a city based on industry knowledge.
The example given was the Central Business District of LA, an educated guess as to where an air taxi service may be in high demand.
A better first step could be an economic model which asks, ‘in what areas would we like there to be pads?’, and then applies the CNN model to those areas.
It should also be possible to define areas of interest more objectively, for example, using:Rides/day: The algorithm could prioritize zones where high density commuter routes from the airport terminate.
Distance from airport breakeven distance: Uber has calculated that on a purely distance-based comparison, ridesharing using current helicopter technology could out-perform a taxi in terms of cost after a given distance.
Assuming this cost curve is linear (and that’s a big assumption), a ratio between the distances of a commute and this breakeven distance could provide a further indicator of priority zones.
Probability(average commute speed or less): A p-value of average traffic flow speeds on a given commute to the airport, when compared to all other common commutes to the airport, may give a good normalized measure of its relative congestion.
Property values ($/ft²) and zoning: Property values may give some indication of willingness to pay for rideshares, while zoning would allow for a better targeting of key markets (for example, business travelers).
Result sortingEven with an optimized search, the results are currently returned to the user as a layer on a map, with no differentiation between the ‘quality’ of the pads.
It is reasonable to assume that not all pads are made equal, and for a given region which has broad economic geography, there are still features at the helipad level which would make them better or worse choices.
From interviews with those in ridesharing companies, I know that a huge one is the presence of radio beacons (which would be extremely hard to infer from public data).
However, other features which may also play a role are:Building height upon which the pad rests (do users really want to take an elevator 30 floors up through a private building to use a service?).
Building ownership category: Different building ownership structures may indicate easier or harder bargaining opportunities for lease or purchase of a helipad.
Proximity to restricted airspace: The airspace over cities is heavily regulated, and commercial helicopters tend to be at the bottom of the pecking order.
Buying a helipad next to a high-security government property, for example, may mean frequent denials for permission to fly.
Number of pads/density of pads: I’ve noticed that a single building, or pair of buildings, will often develop more than one pad.
Presumably, there is an economy of scale here which could be leveraged.
So there you have it; by using this product, rideshare companies could now be able to define a city search area, and let the model do the work of pinpointing helipads for them.
This data could then be used as a vital input to developing their market entry strategies.
With some additional investment, this MVP can be scaled to tackle city-wide searches and deliver sorted results to a user.
Are you interested in working on high-impact projects and transitioning to a career in data?.Sign up to learn more about the Insight Fellows programs and start your application today.
.. More details