Abstract
The lack of high-resolution field data on the abundance, species and distribution of mosquitoes is a serious impediment to effective control of mosquito-borne disease, yet the availability of high-throughput, low-cost surveillance techniques remains a bottleneck in generating such data. Here, we establish that commercially available mobile phones (including low-cost basic models) are a powerful tool to probe mosquito activity, by sensitively acquiring acoustic data on their species-specific wingbeat sounds, together with the time and location of the human-mosquito encounter. We survey a range of medically important mosquito species to quantitatively demonstrate how acoustic recordings supported by spatio-temporal metadata enable rapid, non-invasive species identification. As proof-of-concept, we carry out field demonstrations where minimally-trained users map local mosquito fauna using their personal phones. Thus, by leveraging the global mobile phone infrastructure with the potential for engaging citizen scientists, our approach enables continuous large-scale acquisition of mosquito surveillance data in resource-constrained areas.
The lack of high-resolution field data on the abundance, species and distribution of mosquitoes is a serious impediment to effective control of mosquito-borne disease, yet the availability of high-throughput, low-cost surveillance techniques remains a bottleneck in generating such data. Here, we establish that commercially available mobile phones (including low-cost basic models) are a powerful tool to probe mosquito activity, by sensitively acquiring acoustic data on their species-specific wingbeat sounds, together with the time and location of the human-mosquito encounter. We survey a range of medically important mosquito species to quantitatively demonstrate how acoustic recordings supported by spatio-temporal metadata enable rapid, non-invasive species identification. As proof-of-concept, we carry out field demonstrations where minimally-trained users map local mosquito fauna using their personal phones. Thus, by leveraging the global mobile phone infrastructure with the potential for engaging citizen scientists, our approach enables continuous large-scale acquisition of mosquito surveillance data in resource-constrained areas.
Frequent, widespread, and high resolution surveillance of mosquitoes is essential to understanding their complex ecology and behaviour (1, 2), in order to predict disease risk and formulate effective control strategies against mosquito-borne diseases like malaria, dengue and Zika (3, 4). Mosquito populations vary heterogeneously across urban and rural landscapes, further fluctuating with seasonal or climatic trends and human activities. Hence, the direct monitoring of mosquito species and abundance in field settings is necessary to shape appropriate and timely vector control measures (5,6). Yet, a paucity of such ecological data continues to remain a significant bottleneck in disease control efforts, particularly in resource-poor areas, since current surveillance techniques such as trapping and manual identification are labour, time, and cost intensive. Consequently, although there have been extensive efforts to map mosquito abundance using interpolative mathematical models, their field inputs from entomological surveys are comparatively sparse (7). Therefore, there is a crucial need for novel methods of surveillance that are extremely low-cost yet high-throughput, to adequately sample mosquito populations across large areas while simultaneously maintaining high spatio-temporal resolutions.
A promising candidate to answer this need is acoustic monitoring, where the wingbeat sounds produced by mosquitoes in flapping-wing flight are used to identify different species in the field (8–12). This is based on the hypothesis that sexual selection has led to unique species-specific sound signatures for different mosquito species (13–18). However, the challenges of using expensive microphones to acquire low amplitude mosquito sounds against potentially high background noise levels pose a barrier to the widespread adoption of acoustic surveillance as a field technique (19–21). Low-cost technologies using optical measurement as a proxy for sound are promising in overcoming such limitations (22–25). Yet, the global scalability of such a specific hardware-based solution remains a challenge for large-scale deployment.
Here, we propose a novel solution that uses mobile phones to enable widespread acoustic mosquito surveillance, by using them as sensitive microphones to record species-specific wing-beat sounds from a variety of disease-transmitting mosquitoes for identification and analysis (Fig. 1A). We exploit the insight that these ubiquitous, highly portable devices are optimized for sophisticated audio processing and computing capabilities, and connected by a data transmission infrastructure supporting over 5 billion users globally (26), leading to multiple applications in citizen science and crowdsourced data gathering (27–30). Specifically, the explosive growth in mobile phone use is most pronounced in Africa, Asia and Latin America (26), which also bear the brunt of the impact of mosquito-borne disease (2). This juxtaposition implies that our mobile phone based concept has the advantages of scalability, sustainability and cost effectiveness, in collecting on-the-ground data on mosquito activity in resource-constrained areas with high disease burdens. Our proof-of-concept study highlights the potential of our solution to engage citizen scientists around the world in mosquito surveillance, without the need for specialized equipment or highly trained personnel.
A, Illustration showing the collection of mosquito acoustic data by mobile phone users in different locations. B, Methods to acquire wingbeat sounds from mosquitoes using mobile phones include lab methods like (i) collecting them in cages, and field methods like (ii) following mosquitoes in free-flight, or (iii) capturing them in inflated bags. C, Spectrogram for a flight trace acquired from an individual female Anopheles gambiae mosquito using a 2006 model Samsung SGH T-209 flip phone. The wingbeat base frequency at every instant is computationally identified by a simple automated algorithm and marked with a black line. (Top) The time-averaged spectrum of this flight trace shows the distribution of acoustic power among the base frequency and multiple harmonics. D, The variations in wingbeat base frequency of the mosquito during this flight trace are represented by a probability distribution of the frequency identified in each window of the spectrogram. (Top) Raw base frequency data is represented as a violin plot with an overlaid box plot marking the inter-quartile range, black circle representing mean frequency, gray vertical bar for median frequency, and whiskers indicating 5th and 95th quantiles.
We acquire acoustic signatures from free flying mosquitoes by orienting the primary microphone of a mobile phone in the direction of a mosquito, and using an in-built audio recording application to record and store the sound produced by the mosquito’s wingbeats (Fig. 1A,B, Supplementary Audio SA1). Mosquito sounds have relatively low complexity, comprising a single fundamental frequency with several overtones, which we extract using the short time Fourier transform (STFT) (Fig. 1C). These sounds are sexually dimorphic with males having higher frequencies than females, and show natural variations in the fundamental frequency which are captured by a base frequency distribution characteristic of the given species (Fig. 1D). The female wingbeat frequency is typically between 200 to 700 Hz, which overlaps the voice band (300 to 3000 Hz) in which phones are designed to have maximum sensitivity. Since mosquitoes rarely fly at speeds over half a meter per second, the Doppler shift of frequency during free-flight is small (1 − [330 − 0.5/330 + 0.5] ≈ 0.3%, i.e. < 2 Hz) when compared to the observed natural spreads of up to 100 Hz in base frequency distributions. The use of mobile phones as recording platforms additionally provides automatic registration of relevant metadata, such as the location and time of data acquisition, which adds valuable secondary information for species identification and spatio-temporal mapping. Such acoustic and spatio-temporal information can be crowdsourced from many users, to generate large data sets that map the distribution of mosquito species at high resolutions (Fig. S1).
To establish our fundamental premise that mobile phone microphones are indeed high fidelity acoustic sensors, we first assessed whether mobile phones faithfully record the spectral composition of sound produced by mosquito wings during flight. We measured the wingbeat frequency of female Culex tarsalis mosquitoes in tethered flight using two independent modalities, by synchronizing acoustic recordings with high speed videography (Fig. 2A). For spectrograms derived from mobile phone audio (Sony Xperia Z3 Compact) and high speed video recordings, and aligned in time to within 2 ms, we find an exact match in frequency in each time window to within a computational error margin of 2 Hz (Fig. 2B). The respective distributions of the base frequency have low variances with maximum density occurring in the same bin (Fig. 2C), and are indistinguishable by the 2-sample T-test (significance level α = 1%). This corroborates the spectral accuracy of mobile phone recordings based on an independent optical reference standard.
A, Schematic of experimental setup for recording a tethered mosquito using mobile phones, with synchronized high speed cameras or high performance microphones as visual and auditory reference standards. Synchronization on the order of microseconds is achieved using a piezoelectric buzzer and LED controlled by a microprocessor. B, Overlaid spectrograms for female Culex tarsalis mosquitoes obtained independently using high speed video (magenta) and mobile phone audio (cyan), aligned to within 2 ms and showing a spectral overlap (blue) within 2 Hz across all time instances. The mobile phone data is noisy but faithfully reproduces the base frequency peak of 264 Hz and the first two overtones. C, Base frequency distributions from video and audio are indistinguishable by the 2-sample T-test (n = 165, α = 1%). D, Signal-to-noise ratio (SNR) estimates over distance from a standardized sound source show that mobile phone microphone performance within a 100mm radius is superior or comparable to high performance studio microphones. E, SNR over distance for the wingbeat sound produced by a tethered female Cx. tarsalis mosquito (normalized for a source amplitude of 45 dB), provide working limits where phones can detect the audio signal - 50 mm for the low end T-209 feature phone and 100 mm for the iPhone 4S and Xperia Z3 Compact smartphones. F, Variation of the base frequency distribution sampled by 8 different phones is low compared to the natural variation within a population of about 200 lab-reared Anopheles stephensi females. Raw data are shown with overlaid box plots marking the inter-quartile range, black circles for mean frequency, gray vertical bars for median frequency, and whiskers indicating 5th and 95th quantiles. G,H, The Jensen-Shannon divergence metric for base frequency distributions (G, lower left triangle) shows low disparity, ranging between 0.144 and 0.3, against a minimum of 0 for identical distributions. Likewise, the Bhattacharya distance (H, upper right triangle) shows high overlap, with values between 0.935 to 0.986, against a maximum of 1 for identical distributions.
As we propose using mobile phones as wingbeat acoustic sensors under field conditions, it is crucial to establish working limits within which their built-in microphones are sensitive enough to reliably acquire low amplitude mosquito sounds. Since the technical specifications of many commercially available mobile phone microphones are not openly available, we experimentally compared a range of mobile phone models having diverse feature capabilities to two reference electret condenser microphones under identical conditions. This provides a direct comparison of mobile phone microphones to the gold standard in acoustic sensing. We first used a piezoelectric buzzer of constant amplitude (77 dB at source) and frequency (500 Hz) as a standardized sound source, to show that both smartphones (iPhone 4S, Xperia Z3 Compact) and low-end feature phones (SGH T-209 clamshell model) had signal-to-noise ratios (SNR) that were comparable to the reference microphones over distances of up to 100 mm (Fig. 2D). Next, to gauge suitable working distances for the specific application of acquiring mosquito sound, we simultaneously recorded wingbeat sound from tethered mosquitoes using the reference microphones and mobile phones. Curves of mobile phone SNR over distance indicate that all the phones tested, including a decade-old basic phone (SGH T-209), are capable of acquiring detectable wingbeat sound up to at least 50 mm from a mosquito (Fig. 2E). This is a working distance that we have found to be practically achievable with reasonable ease when making free-flight measurements in the field. Smartphones like the Xperia are capable of signal detection even at up to 100 mm in quiet environments, making it still easier for users to record mosquitoes (Fig. S2).
For our proposed surveillance technique to scale to the broadest possible user base, citizen scientists must be able to engage in acoustic data collection using any commercially available mobile phone that they own. The varying sensitivity observed among phone models highlights the imperative that most mobile phones should still collect quantitatively comparable acoustic data from mosquitoes. We tested this for a collection of eight different commercially available cellphones (Fig. 2F, ranging in price from ~ $20 to ~ $700), where female mosquitoes of similar age from a lab-reared population of the malaria vector Anopheles stephensi were confined in a cage, and recorded by manually following them in free-flight. Quantitatively, both mean and median frequencies obtained by each phone lie well within the interquartile range of frequencies obtained by every other phone, and differ by less than 5% of each other (Fig. 2F). The distributions of wingbeat frequency all have high degrees of mutual overlap, as measured by Bhattacharya overlap distances (BD) between 0.93 to 1 (Fig. 2H). We further computed the Jensen-Shannon divergence metric (JSD) between each pair of phones (Fig. 2G), which had low values below 0.3 corroborating that wingbeat frequency sampling is relatively insensitive to the phone used. Thus, our data demonstrate that a diverse range of both smart and feature phones provide highly similar acoustic spectra from the same population of mosquitoes, as required of a truly universal platform for crowdsourcing mosquito identification via audio signal acquisition. Further, the JSD also provides upper bounds on the variation inherent in sampling the same population in different experiments, allowing us to establish a criterion for the minimum statistical distances required between wingbeat frequency distributions of different species in order to distinguish them.
The difference between wingbeat frequency distributions among mosquito species has a profound impact on the probability of correct species identification in acoustic surveillance (24, 25, 31, 32). To evaluate this, we carried out a broad survey of frequency distributions for lab-reared populations of female mosquitoes from 19 major mosquito vector species under similar experimental conditions (Fig. 3A). Our analysis is exclusively based on free-flight acoustic data acquired by the 2006-model SGH T-209 feature phone (~ $20), to demonstrate the capacity for acoustic identification using a low-end phone with very basic functionality. The vast majority of all possible pairwise combinations of species in our study (184 out of 190) had JSD greater than the maximum value of 0.3 for different samples of the same species computed earlier in Fig. 2G, indicating that acoustic differences between species are typically significantly greater than the variations in sampling a single species using different phones (Fig. 3B).
A, Distribution of base frequencies for lab-reared female mosquitoes of 19 medically relevant species, for recordings obtained with the 2006 model T-209 low-end feature phone (except Cu. incidens, Cx. pipiens and Cx. quinquefasciatus, recorded using iPhone models). B(lower left triangle), Jensen-Shannon divergence metric for base frequency distributions. Distributions are spaced apart with high J-S divergence in most cases, with only four pairwise combinations having J-S divergence around 0.3 - the maximum divergence for the same species across different phones. C (upper right triangle), Classifiation of species pairs according to the possibility of distinguishing them using mobile phones —(i) no frequency overlaps, hence distinguishable by acoustics alone, (ii) overlapping frequency distributions, but not geographically co-occurring hence distinguishable using location, (iii)overlapping frequency distributions but distinguishable using time stamps, (iv) partially overlapping frequency distributions but no location-time distinctions, hence distinguishable but not in all cases, (v) indistingishable due to highly overlapping frequency distributions with co-occurrence in space and time. D,E, Variations in base frequency distribution (D) for field-recorded sounds corresponding to wild mosquitoes having a wide (about two-fold) variation in body size and wing area (E), showing small differences between individuals compared to the variation within each flight trace.
We explored our species survey data in depth to identify different scenarios where acoustic data from mobile phones can be combined with automatically registered metadata such as timestamps and location coordinates, to facilitate quick differentiation between common medically relevant vector species in the field (Fig. 3C,S3). In the simplest cases, species with completely non-overlapping frequency distributions, such as Anopheles gambiae and Culex pipiens (JSD = 1), can easily be distinguished by sound alone (Fig. S3A). Although some species in our large dataset (Fig. 3A) have overlapping frequency distributions, location metadata from the phone allows us to overlook these pairs on the basis of spatial distribution, such as the European An. atroparvus and South Asian An. dirus (JSD = 0.26 < 0.3) (33) (Fig. S3B). Other pairs of species overlapping in both frequency and spatial distribution can be distinguished by metadata such as timestamps (24,25), for instance the night-biting An. gambiae and day-biting Aedes aegypti (JSD = 0.37 ~ 0.3) (Fig. S3C). Co-occurring and highly similar species that are reasonably distinguishable acoustically include the arboviral vectors Ae. aegypti and Ae. albopictus (JSD = 0.55 > 0.3) (31), and the closely related species Cx. pipiens and Cx. quinquefasciatus (JSD = 0.65 > 0.3) (Fig. S3D). Morphologically indistinguishable vector species like the Anopheles gambiae s.l. complex are of particular interest for acoustic identification (32,34,35). Our results for four members of this complex imply partial distinguishability, based on mostly mutually non-overlapping interquartile ranges for An. arabiensis, An. quadriannulatus, An. gambiae and An. merus, with JSD for all but one species pair ranging between 0.61 and 0.91 (Fig. S3E). However, the pair of An. arabiensis and An. merus is potentially indistinguishable (JSD = 0.29 < 0.3) (Fig. S3F) without additional knowledge of their specific local ecology, such as the distribution of saltwater breeding sites for the halophilic An. merus. Nevertheless, despite a very few limiting cases, the broadly species specific nature of acoustic data, combined with the discriminatory power of phone provided time and location metadata, makes mobile phone based acoustic surveillance an extremely useful screening tool to gain broad insights into mosquito populations at a glance (Fig. 3C).
To demonstrate the efficacy of acoustic surveillance using mobile phones in the field, we collected data in a variety of settings from urban to rural, both indoors and outdoors. We recorded acoustic signatures from mosquitoes that were either free-flying, taking off from rest, or captured in inflated Ziploc bags (Fig. S4, Supplementary Audio SA2-7). The high amplitude and distinctive narrow-spectrum characteristics of mosquito sounds allowed us to easily identify them within spectrograms, as the SNR remained high due to manual control of microphone position and orientation relative to the mosquito. These sound signatures were matched against our frequency distribution database (Fig. 3A, data for males not shown) to identify the respective species, which we also confirmed by capturing the respective specimens for morphological identification by optical microscopy. Such field data also allowed us to explore variations in wingbeat frequency among mosquitoes exhibiting considerable variations in body size and wingspan within the same species (Fig. 3D,E). Despite the dependence of wingbeat frequency on factors such as nutrition, age, temperature (36) and size (34), our treatment of this measurement as a distribution over time rather than a single discrete value allows greater comparability between individuals of a given species. Interestingly, in field recordings of Ochlerotatus sierrensis mosquitoes varying almost two-fold in size (Fig. 3E), the difference in mean frequency between each specimen was about the same as the inter-quartile range for individual flight traces obtained from a single mosquito (Fig. 3D). This indicates that frequency variations within flight sequences of several seconds - perhaps due to aerial maneuvers - may contribute as much to the widening of frequency distributions as do variations between individuals. Thus, wingbeat frequency can be a robust identifying characteristic for different species in the field, when treated as a distribution over time for longer flight traces of a few seconds.
Finally, we assessed the feasibility of our approach for spatio-temporal mapping of mosquitoes in the field through citizen science, with small-scale proof-of-concept field trials carried out at Ranomafana village in Madagascar (RNM) and Big Basin Redwoods State Park in California, USA (BBR). First, we acquired curated acoustic signatures associated with morphologically identified specimens of the local mosquito fauna caught in traps (Fig. 4A,B). This subsequently formed the basis for acoustic identification of mobile phone recordings in the field, collected by 8 to 15 volunteers using their personal mobile phones, who were given around 15 minutes of training in acoustic data collection (sample field recordings in Supplementary Audio SA8-10). In the maps constructed using this field data (Fig. 4C,D), the power of crowdsourcing in comparison to many traditional surveillance techniques is reflected both in the volume of data per time (~ 60 recordings over 3 hours in RNM, comparable to the number collected overnight in a CDC light trap at the same location; ~ 125 recordings over 3 hours in BBR), and also the spatio-temporal fine-graining on the level of minutes and tens of meters. In the multi-species ecology of RNM, complementary gradients in the density of each species were evident across the village from riverside to hillside, possibly influenced by extremely local factors such as drainage, density of humans, or presence of livestock (Fig. 4C). Spatial variations in mosquito population density were clearly seen even for the single species that dominated the relatively uniform terrain of BBR (Fig. 4D), and parsing of the data by recording time further revealed the rise and fall in biting activity over an evening (Fig. 4D inset). These pilot field trials highlight the extremely local variations in mosquito distributions together with their circadian activity patterns. This indicates that mobile phone based crowdsourcing for mosquito surveillance is uniquely equipped to simultaneously optimize both scale and resolution of ecological measurements for the spatiotemporal mapping of mosquito vectors.
A, Sample spectrograms from female Culex spp. (top) and Anopheles spp. (bottom) mosquitoes captured in the field at Ranomafana village in Madagascar. B, Frequency distributions for field-caught Culex spp. and Anopheles spp. mosquitoes in Ranomafana, forming a reference for identification of recordings from either species at this field site. C, Map of Ranomafana village showing distribution of female Culex spp., Anopheles spp., and Mansonia spp. mosquitoes, from mobile phone data recorded by 10 volunteers over the approximately 1 km X 2 km area. Each square represents one recording, and black circles indicate locations where volunteers reported encountering no mosquitoes. The map shows a complementary spatial gradient from riverbank to hillside in the relative proportion of Anopheles spp. and Culex spp. mosquitoes. Further, mosquito hotspots are interspersed with points having a reported lack of mosquitoes, highlighting the potential importance of micro-factors such as the distribution of water and livestock. D, Spatio-temporal activity map for female Oc. sierrensis mosquitoes in the Big Basin Park field site, using data collected by 13 hikers recording mosquitoes with their personal mobile phones, over a 3-hour period in an approximately 4.5 km X 5.5 km area. Each brown square represents one Oc. sierrensis female recording, and black dots represent sites where hikers reported encountering no mosquitoes at all. (Inset top left) Temporal distribution of the overall mosquito activity data depicted in (D) based on recording timestamps, showing the rise and fall of activity in each hour of the field study.
In summary, we have demonstrated a method for acoustic surveillance of mosquitoes using mobile phones, by presenting quantitative analyses of mobile phone acoustic signal quality and differentiation between mosquito species, further supported by preliminary field data collected by volunteers and organized into spatio-temporal maps. The involvement of local volunteers in our study underlines that almost anyone with a mobile phone can quickly be trained to contribute data towards mosquito surveillance efforts. With the proof-of-concept presented here, we highlight the potential for building high-density mosquito maps with the participation of citizen scientists, particularly in disease-prone locations where high human population density coincides with complex mosquito ecology. The advent of machine learning and data mining techniques create tremendous scope for the automated processing of crowdsourced acoustic mosquito surveillance data (24, 25). This could boost our capability to dynamically assess mosquito populations, study their connections to human and environmental factors, and develop highly localized strategies for pre-emptive mosquito control (37). Since the critical missing link in enabling such advances at present is the capacity to generate large quantities of mosquito ecological data on fine-grained space and time scales (4), our mobile phone based solution holds great promise as a scalable, non-invasive, high-throughput and low-cost strategy to generate such data, by leveraging widely available hardware and an existing network infrastructure. Thus, we propose a citizen science effort driven by the mobile phone based mapping framework established in this study (Fig. S1), which will enable people to take the initiative in tracking disease vectors within their own communities, expand surveillance efforts in resource-limited areas where they are needed the most, and bring about new big data driven approaches for eliminating vector-borne disease.
We acknowledge Amalia Hadjitheodorou, members of the Prakash Lab, and data collection volunteers for assistance in field experiments, and Mainak Chowdhury for inputs regarding analysis of the data. We particularly thank Menja Brabaovola and Malalatiana Rasoazanany for their invaluable participation in field studies in Madagascar. We are grateful to Ellen Dotson and Mark Benedict at the US Centres for Disease Control, the Luckhart and Coffey labs at UC Davis, the Zohdy and Mathias labs at Auburn University, and the Animal Flight Lab at UC Berkeley, for providing us with access to their mosquito cultures for collecting audio data. We further acknowledge Big Basin Redwoods State Park, California, USA, where we collected acoustic signatures of mosquitoes on hiking trails. We are thankful to the Centre ValBio in Ranomafana, Madagascar for enabling our field studies in the area, and to PIVOT and particularly Andrés Garchitorena for coordinating our field based efforts. We are grateful to Nona Chiariello and the Jasper Ridge Biological Preserve at Stanford University, the Santa Clara County Vector Control District and the San Mateo County Mosquito Control District for their support and assistance with providing access for field work, materials for cultures, and assisting with identification of field specimens. The materials for cultures in our lab were provided by BEI Resources and NIAID. Finally, we thank Desirée LaBeaud, Erin Mordecai, John Dabiri, and all members of the Prakash Lab for illuminating discussions about our work and comments on our manuscript.
HM acknowledges support from a HHMI International Student Research Fellowship, EAC from a Stanford Mechanical Engineering Graduate Fellowship, and FH from a NWO Rubicon Fellowship. MP acknowledges support from the NSF Career Award, HHMI-Gates Faculty Fellows program, the Pew Foundation Fellowship and the MacArthur Fellowship. This work was additionally supported by the Coulter Foundation, the Woods Environmental Institute at Stanford University, the NIH New Innovator Grant and the USAID Grand Challenges: Zika and Future Threats award.
The authors declare that they have no competing financial interests.
References
