Client: GNZ, GEOINT New Zealand, NZ Defence Force
Lynker Analytics has successfully trialled machine learning technology to extract building and road features from imagery, which could have major benefits for disaster response.
The New Zealand Defence Force (NZDF) is part of an international consortium of over 31 countries called the Multi-national Geospatial Co-production Program (MGCP) which supports global humanitarian assistance and disaster relief through the production of high-resolution topographic vector data throughout high interest areas of the world.
Over many years NZDF has collaborated with key Pacific Island nations including Fiji, Tonga, Samoa, Tuvalu, Solomon Islands, Cook Islands, Nauru, Niue, Wallis & Futuna, Tokelau and Kiribati to produce and maintain detailed topographic mapping datasets. Features captured include urban areas, landmarks, water courses, land-use and sites that may be used in a humanitarian response effort such as aerodromes or navigational routes.
To enhance the update of mapping datasets through collaboration between NZDF and the Pacific Islands, NZDF approached Lynker Analytics to a trial a Machine Learning (ML) approach to automatically extract building and road features from satellite imagery over four Pacific Island territories: Fiji, Samoa, Tonga and the Marshall Islands which could have key benefits for disaster response support or other humanitarian relief requirements
The goal of the study was to evaluate the effectiveness of ML models to predict and extract buildings, roads as well as the materials used in both structures from imagery.
To target the feature extraction to built up areas, we first used an Inception v3 based classification model trained using transfer learning and fine tuning. This step removed large sections of imagery in irrelevant areas such as the ocean and rural interior. The output of this step is shown below for the main island of Tonga, Tongatapu.
Tongatapu model output with masked area in black
We then used Python, Tensorflow and Keras to build neural network architectures to annotate, classify and segment the imagery. For the land segmentation models (buildings and roads) we used a fully convolutional U-Net inspired architecture with Res-net backbone pre-trained on NZ land cover classified imagery.
This was then fine-tuned on the SW Pacific imagery using active learning. The inference models were run 4-6 times as the training was completed. The models were validated against hold-out data captured by NZDF experts familiar with the locations and F1 scores were calculated for each model.
F1 is the best measure of a model’s overall accuracy as it considers false negative and false positive results along with true positives. In this case the Building model had an F1 score of 0.90 and the Roads model score was 0.93.
The examples below show the inference results for Roads and Buildings in Apia, Samoa with the lighter shade representing higher model confidence at a pixel level. The larger buildings and major transport routes are well described by the models while smaller buildings and minor roads are predicted with slightly lower confidence.
A final step in the process is to threshold the raster output removing lower confidence or grey scale pixels from the vectorisation stage.
Road model output, Apia
Building model output, Apia
Vectorisation and comprehensive GIS post processing was then carried out. This included clipping, sliver detection, geometry size checks, vertex counts, criteria-based dissolve and eliminate tasks and full data re-assembly. Finally, quality checking and validation was run to ensure all features were representative and topologically correct.
The models, vectorisation and GIS post processing have been designed to run on an on-demand AWS large compute instance by NZDF in the future. The ML models were supplied as documented python code with instructions on how to implement within existing workflows. An AWS Machine Image (AMI) was also provided with the software and models.
The Lynker Analytics team hosted a senior NZDF delegation in late 2020 to present and discuss the results and findings. Sarah Hodgson from the NZDF Foundation Data Production team explains
Final building inference, Apia
“the potential time savings from this type of process are significant and once implemented should allow us to detect change faster and focus our human experts on qualitative data revision and other higher value map production tasks”.
Deliverables:
A machine learning pipeline that can be run entirely in the Amazon Web Services (AWS) Cloud using an AWS Machine Image (AMI).
Esri File Geodatabase (FGDB) including Buildings (polygon), Building Centroid (point), Roads, (polyline) Seamlines
Data Quality and Accuracy Report.