Species and habitat mapping in two dimensions and beyond. Structure-from-Motion Multi-View Stereo photogrammetry for the Conservation Community

Abstract

Structure-from-Motion Multi View Stereo (SfM-MVS) photogrammetry is a technique by which volumetric data can be derived from overlapping image sets, using changes of an objects position between images to determine its height and spatial structure. Whilst SfM-MVS has fast become a powerful tool for scientific research, its potential lies beyond the scientific setting, since it can aid in delivering information about habitat structure, biomass, landscape topography, spatial distribution of species in both two and three dimensions, and aid in mapping change over time – both actual and predicted. All of which are of strong relevance for the conservation community, whether from a practical management perspective or understanding and presenting data in new and novel ways from a policy perspective.

For practitioners outside of academia wanting to use SfM-MVS there are technical barriers to its application. For example, there are many SfM-MVS software options, but knowing which to choose, or how to get the best results from the software can be difficult for the uninitiated. There are also free and open source software options (FOSS) for processing data through a SfM-MVS pipeline that could benefit those in conservation management and policy, especially in instances where there is limited funding (i.e. commonly within grassroots or community-based projects). This paper signposts the way for the conservation community to understand the choices and options for SfM-MVS implementation, its limitations, current best practice guidelines and introduces applicable FOSS options such as OpenDroneMap, MicMac, CloudCompare, QGIS and speciesgeocodeR. It will also highlight why and where this technology has the potential to become an asset for spatial, temporal and volumetric studies of landscape and conservation ecology.

Introduction

Relatively new technologies, such as drones (1,2), advances in computational power (3,4), improvements in digital cameras (5), along with classic remote sensing platforms such as kite aerial photography (KAP) and balloons (6,7), have all combined to help create a new opportunity in remote sensing research utilising SfM-MVS photogrammetry. It is these convergent developments that now position SfM-MVS as a cost-effective and democratic tool for conservation research.

SfM-MVS approaches use parallax (i.e. minor displacements in similar images) in conjunction with computer vision techniques, in order to derive 3D structures from 2D data (similarly to how brains use parallax from vision to determine the distance and speed of a moving object). Data such as overlapping digital photographs and GPS/GNSS (Global Positioning System/Global Navigation Satellite System) information are stitched together, adding distance (X and Y) and height (Z) values to pixels (points) in the combined data, producing a “point cloud” (Figure 1). From this, SfM-MVS software is able to output two dimensional orthographic images containing detailed geographical location information, alongside 2.5 dimensional reconstructions (2.5D is used as it relates to the limitations of algorithms to produce true 3D reconstructions from aerial images (8)), such as digital elevation, surface and terrain models (DEM, DSM, DTM) (Figure 2). These data products can be used for mapping habitats (9–11), analysing structure, biomass, topography and change (12–15) but also crucially, we suggest, offer the capability to determine the geographic position of species in both two and three dimensions (16–18), thus being of great value for conservation biologists.

Figure 1:

Screenshot from Agisoft Metashape. Blue squares indicate position of the camera in relationship to the ground when images were captured. The height of drone and it’s velocity allow for the same object to be captured from multiple different angles, which in turn allows the SfM-MVS software to calculate the height of objects captured in the scene.

Figure 2:

Two typical outputs from SfM-MVS software. A) An orthographic map produced using WebODM. The image was taken from the WebODM interface and shows the produced orthomosaic overlaid on top of the corresponding Google satellite image. B) the DEM produced by Agisoft Metashape. Image taken from within Metashape.

Mapping and modelling the distribution of species is an important aspect of planning and management within conservation; especially with regards to land-use change, and where there is potential for biodiversity loss (19–21). Both Bradbury et al. (21) and Davies and Asner (22) have demonstrated the application of Light Detection and Ranging (LiDAR) for delivering improved spatial understanding of species distributions and predictive modelling. Davies and Answer (22) also demonstrate the importance in understanding habitat in 3D with regards to species populations and communities within ecosystems. Whilst very effective, promoted as a conservation technology by the WWF (23), and even suggested as a tool to archive the Earth in 3D (24), some of the downsides to LiDAR or laser scanning are the cost, weight and lack of portability of instruments, which place this technology out of reach for many with small operating budgets or those working in remote areas (25). In contrast, SfM-MVS techniques have been shown to be a viable alternative and much more cost effective, especially when utilised in conjunction with drone derived data (25–27).

Within the Earth observation sector there has been a recent upsurge in the availability of free and open source analysis tools with which to extract information from those data. One such example is the Google Earth Engine, which allows for easily applied satellite remote sensing techniques with regards to conservation issues (28). Furthermore, there is an expanding range of software and literature that can help in using free and open source software (FOSS) for remote sensing (29–32). Combined with the currently available remote sensing and GIS options, opportunities exist for a completely free and open source SfM-MVS processing pipeline for exploiting two-to three-dimensional data. This paper seeks to address the currently available FOSS SfM-MVS software options, alongside some of the scientifically popular commercial offerings and provide guidance in choice that can be of potential benefit to the conservation community.

SfM-MVS software ranges and limitations

In recent years, drones have readily captured the media’s attention, but an equal part of their success within a scientific capacity, is due to the software that allows for the processing of the data they collect. Whilst the number of resources discussing the use of drones in conservation is growing (33,34), along with the concepts and processing of SfM-MVS data (33,35) accessible information about implementing SfM-MVS software options can be hard to find or limited. Alongside this, conservation applicable SfM-MVS research tends to be targeted at the geosciences and remote sensing readership, and as such, might not be as accessible or digestible for those in positions of management or decision making within the conservation sphere.

Within the scientific literature, much of the focus is aimed at quantifying outputs from two major commercial software providers; Agisoft and Pix4D. Both of these popular choices only offer discounts to educational institutions and it is likely that their standard prices of $3499 and $4426 (€3990 converted to USD at time of writing) respectively, could easily put them outside of community-based, grassroots or small-scale conservation budgets. Beyond these two options there are now upwards of 40 alternatives available (36), with most of the commercial software costing as much, and in some cases more than Agisoft and Pix4D (Table 1).

Table 1

FOSS and sample of popular commercial SfM-MVS options showing operating system compatibility, graphics card requirements, GIS ready outputs, active development and price for perpetual licence. *Prices converted to US$ and correct at time of writing.

Many of the commercial options offer trial periods and we recommend that anyone looking to purchase, utilise the trial period for adequate testing of hardware compatibility and ease of processing. There are many that also offer monthly subscriptions, which can help reduce the burden on initial capital expenditure, but may come with limitations on the number of images available to process, or the amount of data available to transmit or store, especially when the monthly subscription is for a cloud processing service (Table 2). As data collection for SfM-MVS and the subsequent data outputs can quickly produce multiple Gigabytes (GB) of data, cloud or online services can also lead to a requirement for an adequate internet connection, and this should be a consideration when in-situ processing with limited resources is required.

Table 2:

Comparison/sample of cloud processing options and costs indicating some of the variability of what is available. *Enterprise option with no data restrictions is price on application.

The available software can be roughly divided into two classes, those that provide survey options (orthographic maps, DEMs and other GIS ready layers), and those that cater more to the recreation of 3D objects via point clouds or meshes. The latter is the result of the growing demand within the gaming and architectural industries and whilst these software have the potential to be utilised, especially where just a point cloud is required, their deployment within the wider conservation community may be more limited, due in part to the extra steps and programmes required to produce data outputs that are likely to be useful in a conservation setting.

Unlike most software used in day to day office work, SfM-MVS software will often leverage all available processing power from a computer, consuming greater levels of energy and generating more heat as a result. If planning to process data in remote locations, it is worth preparing for adequate power supply. Alongside this, there has also been a shift to leveraging graphics cards to process data within certain stages of the SfM-MVS pipeline, and although this can increase processing speeds (Figure 3), it brings with it extra costs associated with necessary hardware. Another potential issue that stems from software that require or benefit from graphics cards, is one whereby not all graphics cards are supported equally, presenting the possibility of owning or purchasing graphics cards that are redundant (Table 1).

Figure 1

Comparison of processing speeds - data processed for this paper using WebODM, NodeMicMac, Agisoft Metashape without GPU, Metashape with GPU.162JPEG images processed on a desktop running Manjaro Linux OS with AMD Ryzen 2700 CPU, 32GB DDR4 3000Mhz RAM and AMD Vega 56 GPU. It is difficult to replicate exactly the same processing parameters for each software, so defaults were used for WebODM and NodeMicMac, medium settings were used for Agisoft Metashape.

Open source SfM-MVS

There has been discussion of the impacts that open source hardware technologies such as Arduino and Raspberry Pi could have in helping the conservation industry innovate (49). It is this same open source community that has not only helped to democratise remote sensing (30,50,51), but is also now helping to liberate the production of maps and 3D modelling by building SfM-MVS software. Whilst Aerial Mapper, Colmap, Meshroom, Regard3D and VisualSFM all have potential, when factoring in the ability to easily create orthomosaic maps, DEMs, DTMs or DSMs, alongside point clouds, of the currently available FOSS options (Table 1), this paper will discuss two that are felt to stand out as having extra value within the conservation community; MicMac and OpenDroneMap (ODM).

MicMac is the more mature software of the two and was initially developed in 2003 by the French National Geographic Institute and French national school for geographic sciences (52) whereas ODM began as a concept presentation at the 2014 FOSS4G (Free and Open Source Software for Geospatial) conference in Seoul. Both have drawn attention from the scientific community, but with ODM being the newer option, the focus has primarily been on its development and initial quality testing (3,4,53,54). MicMac’s maturity has allowed for a deeper scrutiny (55,56), has been utilised as the primary SfM-MVS option in geomorphological studies (57) and has been shown to be comparable to both Pix4D and Agisoft Photoscan (Metashape) within a low sward grassland study (58).

Both are also cross platform, meaning that they can be installed on computers running either Linux, Mac OS X or Windows operating systems. ODM also offers “one click” installation options, although these come with a small fee of $57 each but include additional support and a money back guarantee.

There is even now a MicMac “node” that can be used either stand-alone or alongside the ODM web interface, “WebODM”. The MicMac node has been developed by DroneMapper (59), a software and cloud SfM-MVS provider that have built their commercial services upon MicMac. NodeMicMac is still in its infancy but now provides a user two different SfM-MVS processing options from within the one, WebODM interface. Also, as the original version of MicMac can have a steep learning curve (58), this provides an easier, more user-friendly option for deploying MicMac.

With MicMac and ODM being optimised to run without the need for a GPU, they can be installed on almost any computer hardware using a 64bit computer processor within the last 9 years (32) and as such, in situations where conservation project budgets are limited, older, second-hand computer hardware that is available cheaply will suffice.

Collection of data

Whilst differing in subtle ways, SfM-MVS software all follow a similar processing chain, which has been very well described by Carrivick et al. (35). Creating data products with SfM-MVS depends on the input of data that have been collected with a variety of considerations. Accuracy and precision are not only influenced by the quality and resolution of the sensors used to collect the images, but also by the accuracy and precision of the GNSS receiver providing the spatial information, and where necessary, by the number and position of ground control points (GCPs) utilised (55,60,61). The exception to this is in scenarios whereby actual measurements can be applied to scale a dataset, for example by applying known distances to specific points within a point cloud (62). However, this latter option is only likely useful on smaller scale projects or those that don’t require integration with other geospatial data.

It is important to know what question needs answering beforehand and what scale of error is acceptable. For example, if wanting to produce a fine spatial resolution map, or look at animal species relationship to habitat structure, absolute precision and accuracy might not be as important as it would be with wanting to establish plant below ground biomass derived from above ground biomass (13) or other volume critical measurements (58). Alongside the aforementioned, there are some more useful general principles for data collection which can be applied to any and all SfM-MVS projects;

• Collect data with as high an overlap as is practicable – Nearly every SfM-MVS software will have its own guidelines for the amount of side and forward overlap between images, these range from 60%/60% (63) to 60%/80% (64), though recent studies have shown that higher overlap, upto 90% in both side and forward, is better for accuracy and precision within SfM-MVS derived data (65).

• Collect both nadir and oblique imagery - James and Robson (66) demonstrated that nadir - that is looking straight down - aerial images captured via drone, along with inherent lens distortion of a camera, can induce a systematic error in the form of a doming effect in DEMs. Cunliffe et al. (13) have shown that incorporating oblique imagery can help improve the accuracy of SfM-MVS derived DTMs, whilst Nesbit and Hugenholtz (65) have gone on to quantify that incorporating oblique imagery between 20–35° from nadir, along with higher overlap can improve both precision and accuracy by up to 50%.

• Collect data over a wider area than is required – SfM-MVS data quality deteriorates towards the edges of a reconstruction. Making sure your AOI (area of interest) is central within the data set will ensure there is enough overlap and subsequently be of higher quality (60).

• Cameras with an inbuilt intervalometer simplify data collection – many drones will have either an inbuilt camera mounted on a gimbal, or the ability to trigger a camera via a controller or autopilot, yet if collecting data from a kite or balloon, this will unlikely be possible. So, by choosing a camera that has an intervalometer – the ability to take images at pre-defined time intervals i.e. once every 2 seconds – will simplify data collection. Android apps such as DroneLab Toolkit (67) can also help by providing an intervalometer to cameras in Android phones along with other useful mapping tools.

• Use GCPs – Unless using a camera system or drone equipped with an RTK/PPK (real time or post processing kinematic) GNSS that automatically adds high precision location data into images, the simplest way to adequately warp your model geospatially, is to use GCPs marked using a handheld GNSS (68). For higher accuracy and precision, a DGPS (differential global positioning system) or an RTK/PPK system will be required but can be cost prohibitive. Even with smartphones or cameras that have inbuilt GNSS/GPS and can include this information in images, the use of GCPs can assist in improving the orientation and warping undertaken by the SfM-MVS software and also act as a backup should the GNSS data from a smartphone or camera be of poor quality.

• Time of day – Collecting data around midday, preferably with an overcast/diffuse sky can help improve the overall quality of the data (32). Patchy clouds or bright sunshine, whether early on, or later in the day can all cast shadows which can have an impact when SfM-MVS software stitch the images together.

• Wind – If trying to capture vegetation, especially tall grasses or such that are susceptible to motion from wind, having low or no wind can prove less problematic for SfM-MVS reconstructions. The more stable an object when being captured from multiple positions, the better the possibility for good reconstruction (62,69).

• Setting your camera manually – Mosbrucker et al. (70) give in-depth explanations with regards to relationships between manual settings of cameras, lenses, camera sensors and SfM-MVS, all of which can be important for SfM-MVS outputs. Learn to set a camera manually, starting with obtaining the lowest ISO setting possible, then adjusting aperture, shutter speeds and white balance accordingly (and if possible, manually focused to infinity). This helps to ensure images are taken with consistency in colour, light and subsequently, detail.

• Consideration for camera lens distortion – All consumer cameras have some form of inherent lens distortion (71), which can normally be corrected by SfM-MVS software. However, some cameras, especially sports models have very wide-angle lenses, which create a fish-eye effect. Some SfM-MVS software can automatically correct for this extra distortion, but where not possible FOSS software such as Rawtherapee can be used to correct images before processing (32).

• Collect an extra data set – Forsmoo et al. (58) suggest that, as part of any robust SfM-MVS workflow for ecological purposes, obtaining a replicate data set for comparison can help aid in understanding any error that may arise from the processing through an SfM-MVS pipeline. This could be achieved by collecting twice as many images than is required within the same data capture task.

• Practice and get to understand the software – from our experience, SfM-MVS software will output better or worse data based on slight changes, either in how data is collected, or in how options within the software are utilised. Forums for any given product will usually have answers to problems or can illicit help from those with more experience.

Processing of Data

Whilst SfM-MVS software typically provide easy to use interfaces for uploading and processing of data, understanding what to expect from the differing SfM-MVS software and understanding what to do with the data once it has been processed can be a challenge. A single dataset can appear differently when processed by differing software (Figure 4), and no software that has been tested is without some form of fault or error (72). As such, in scenarios where precise, centimetre level of error measurements are not required, SfM-MVS software choice can play less of a role (58).

Figure 4

Top down comparison of point clouds produced by WebODM, NodeMicMac and Agisoft Metashape. A) The WebODM pointcloud is devoid of much of the finer detail, with nearly all of the canopy missing from the model. There are potentially several reasons that could cause this but include a result of settings being too aggressive and removing points unnecessarily, or from inadequate overlap in the dataset for ODM to cope with. B) NodeMicMac has retained most of the canopy in the production of its pointcloud, but has excluded much of the ground underneath. C) Agisoft Metashope has produced the most complete pointcloud from the dataset, with both the ground and the canopy being shown and with very few patches of missing data underneath the canopy.

Figure 5

Canopy extracted from the NodeMicMac pointcloud. Using a scalar field on the Z axis within CloudCompare, it is possible to extract the canopy by removing points under a certain height. The canopy can then be examined as a separate entity and labelled if necessary, from with CloudCompare.

Surfaces such as rocks and bare earth can provide an easier medium to recreate, due to their static, exposed nature, whereas vegetation can prove more complicated, with wind possibly affecting reconstruction and canopies possibly being the only source of information, with underlying biomass being masked or blocked.

Many of the software packages have inbuilt features such as distance, area and volumetric measurements, along with point classification (to enable feature examination or extraction based on a type such as a tree, building or road as examples). While these functions may suffice, there are also FOSS geospatial options that can further assist in delivering data products. CloudCompare (73) and QGIS (74) are two such FOSS options that, when combined with modern analysis tools such as the FOSS R-package, speciesgeocodeR (75,76) could prove to be of particular interest within conservation management or policy at any level.

CloudCompare

This software is dedicated to the reading, manipulation and conversion of 2.5 or 3D data. It offers a multitude of options that include cleaning, filtering, measuring, labelling and cropping of point cloud data along with the ability to perform statistical and computational analysis, both on individual datasets or between multiple datasets.

CloudCompare has proved a useful tool in a number of scientific studies, including those mapping wild leek (77), habitat suitability modelling for insect conservation (78), spatial ecology studies of coral and megabenthic invertebrates (79), and forest canopy ecology (80).

Functions such as its ability to calculate roughness have been used to quantify erosion and deposition of sediment in peatland forests (81) and climate sensitive alpine vegetation (62), whilst change detection via the M3C2 algorithm (82) has been utilised to monitor coral growth (83) and coastal dune morphology (6). The M3C2 function has also proved useful in developing methodologies for quantifying uncertainty in SfM-MVS data (58,61).

QGIS

QGIS is a software that not only has the ability to process data as a stand-alone GIS desktop application (84), but can also leverage the processing capabilities of other FOSS GIS software such as SAGA (85), GRASS (86), geospatial libraries such as GDAL (87) and Orfeo Toolbox (88), along with programming languages such as R (89) and Python (90).

QGIS has been shown to be useful within conservation research, being used as a main GIS tool in studies such as endemic plant hotspots (91), wildlife management from GPS tracking (92), assessment of conservation area effectiveness with regards to biodiversity and ecosystem services (93) and effects of eutrophication on coral biodiversity (94).

There are a growing number of ‘plugins’ available for QGIS that, when combined with SfM-MVS derived data, could be of benefit to the conservation community too. LecoS (90) adds the ability to automatically analyse the landscape ecology of a raster, the self explanatory ‘Hotspot analysis’ (95), PANDORA for biodiversity ecosystem service assessment (96) and CLUZ (97) for designing conservation and ecological networks. Orfeo Toolbox, although initially a stand-alone programme, can also now be integrated into QGIS via a plugin, enabling a user to perform land cover analysis such as vegetation structure and pattern (98) and assist in conservation planning (99).

A study by Selgrath et al. (100) compared local knowledge with satellite remote sensing for conservation mapping of coral reefs, with QGIS and R serving as the software to process the local knowledge data for map production. It was found that whilst the local environmental knowledge proved useful, where accuracy and precision were considerations, satellite remote sensing still proved the better option, but with a cost approximately 5 times greater.

Utilising SfM-MVS photogrammetry is an opportunity to bridge one such gap by providing both highly detailed maps at a low cost (Figure 6). Freely available satellite images may not be of a high enough resolution, whilst proprietary finer resolution satellite images can be expensive. It can also offer the opportunity for local communities to be involved in the data collection of their local environment.

Figure 6

Using QGIS to retrieve the Bing satellite image of the same location, this shows a comparison of resolution between the orthomosaic produced by A) WebODM and B) Bing satellite image. The WebODM orthomosaic image has a resolution of approximately 5cm per pixel, compared to the approximate 50cm per pixel of the Bing satellite image.

SpeciesgeocodeR

This tool is a relative newcomer to the scientific community and research conducted with it to date has initially focused on the neotropics (101–104). It has been primarily designed for use with big data, dealing with the evolution, biogeography and dispersal of thousands of species across large units of space and time (75,76).

However, with the ability to utilise height/elevation data and by the conclusion presented by Töpel et al. (75) that “the output obtained could be readily used for calculating measures of alpha, beta, and gamma diversity; the identification of neglected areas for conservation; and providing real-time detection of GPS-tagged animals entering and leaving protected areas”, these uses could even be expanded and there exists an opportunity for conservation managers and policy makers to collate and present quantified data in new ways. By combining the high detail outputs of local landscapes from SfM-MVS software, with species data processed by speciesgeocodeR, SfM-MVS can play a crucial role in establishing existing baselines, tracking change and highlighting success and areas for improvement to staff, communities or stakeholders.

Conclusion

Whether commercial or FOSS, SfM-MVS software offer an opportunity for the conservation community to engage in new and novel remote sensing techniques. They can be utilised to create highly detailed maps and 2.5/3D data at local scales, now include FOSS options that are being successfully deployed for scientific research (3,4,53–58,62,105) and can be incorporated into the existing FOSS remote sensing pipeline. With applications for Android smartphones such as the UAV Toolkit (67) or small, lightweight action cameras now available, combined with balloons, kites, drones, or even collecting data on the ground, there is an opportunity within the conservation sphere to start enabling remote sensing even at a grass-roots or community level, or in any situation where funding may be limited but this type of data desired. In support of the conclusion presented by Berger-Tal and Lahoz-Monfort (49), SfM-MVS as part of a FOSS remote sensing pipeline is one such technology that can help the Conservation industry innovate and dismantle the neo-liberal conservation paradigm. Data of this kind not only opens up opportunities to increase understanding within conservation management, but also has the potential to increase impact when dealing in policy. Further, it presents the opportunity to increase engagement and empowerment within local communities, by providing the ability to collect and view data in-situ for a given project (106,107), whilst providing conservation managers and policy makers the ability to collect and present data in ways that were difficult or overly expensive before. Furthermore, with data visualisation being a powerful medium for communication within ecology (108), the extra options for data visualisation from SfM-MVS could be a valuable tool to better inform stakeholders and engage them in pressing conservation issues and cutting-edge science.