Invisible clouds of particulate matter move across EuropeWhere do these clouds of PM10 and PM2.
5 come from?Ignaz WandersBlockedUnblockFollowFollowingMar 31In a previous post I explained how I set up my outdoor air quality sensor, and how it produces open data that is part of a citizen science project.
(https://medium.
com/@ignazw/air-quality-measurements-fcc32d59dca7)It's open data, so anyone can analyze it.
In this post, I'll show you how you can use the Luftdaten.
info website to find large-scale patterns of air pollution.
The caseLuftdaten.
info shows a snapshot of all sensor measurements across the world on a map (http://deutschland.
maps.
luftdaten.
info/#6/51.
165/10.
455).
Most sensors are in Europe so I will limit my view on Europe.
Sample image from Luftdaten.
infoA snapshot is taken about every five minutes and the map is updated.
I wondered what it would look like if we make a time-lapse movie of the snapshots.
Can we find any patterns?Fetching the dataTo create a time-lapse movie, we need to capture each snapshot.
Old maps are not maintained on Luftdaten.
info, so we have to collect them ourselves.
Capturing the snapshots requires performing a HTTP request to the web server every five minutes.
I do this with the help of a small script running on a raspberry pi 3.
And I want to capture two images for every snapshot: an overview of a large part of Europe, and a more detailed one over Belgium, where I live.
On the raspberry pi I use the chromium web browser in headless mode, i.
e.
, without display.
#!/bin/bashchromium-browser –headless –disable-gpu –screenshot –window-size=1920×1080 http://deutschland.
maps.
luftdaten.
info/#7/50.
958/8.
218mv screenshot.
png .
/data/new/de`date +”%Y%m%d%H%M%S”`.
pngYou may have to install or upgrade the chromium browser on your system.
I use version 72 of the browser.
Make sure you run it headless and disable gpu usage (to avoid unnecessary libraries loaded into memory).
The luftdaten.
info web site renders the map using javascript, but the chromium browser executes the javascript by default.
You take a screenshot, and you define the size of the image.
The default image is called "screenshot.
png" so rename it to something unique, like the time it was taken.
Then edit the crontab to run this script every five minutes.
The size of one image of this size is more than 3 MB, so you need to make sure it doesn't fill up your disk.
I run a script on a daily basis that copies all images to external storage space, from where I can pick it up for further usage.
I do the further processing, which consists of annotating the screenshots and turning it into a movie, on MacOS X.
I annotate each screenshot with the data and time it was taken.
I use the package ImageMagick for that, installed via HomeBrew.
#!/bin/bashfor f in $(ls .
/images/raw/*.
png)dofn=${f##*/}anno=${fn%.
png}magick .
/images/raw/$fn -pointsize 72 -gravity southwest -annotate +150+50 $anno .
/images/annotated/$fnmv .
/images/raw/$fn .
/images/tmp/$fndoneThis script reads all file names that I prepared in a directory, and are copies of the images collected on the Pi.
It contains the date and time of the screenshot in the filename.
For each file I grab the date and time from the file name, and set it as the annotation on the image.
The +150+50 defines the number of pixels from the edge.
At this location it is very well readable.
I move the original image after annotation to a temporary directory, which I can delete after I am sure the processing was OK.
Next is to make a movie out of the annotated images.
For this I use the ffmpeg package, which you probably need to install just like you did with ImageMagick.
The following script creates the movie.
#!./bin/bash# create movie for Germanyffmpeg -f image2 -pattern_type glob -framerate 5 -i ‘images/annotated/de*.
png’ -s 1920×1080 -f mp4 -vcodec libx264 -pix_fmt yuv420p germany.
mp4The -f flag defines the image format.
You need to define globbing if you want to use that to collect a list of images.
You need to set the framerate in fps, then the actual input files.
Next is the size, the format, and the codec of the output movie.
The pixel format definition is to ensure quicktime can play the movie as well.
And finally you name the output file.
It's as simple as that.
Once you have a few scripts ready, it's easy to rerun this many times.
The time-lapse movieHere are two time-lapse movies spanning the dates 25–31 March 2019.
One clip shows a large part of Europe, and another clip zooms in on Belgium.
PM clouds over EuropePM clouds over BelgiumDiscussion of the resultsThe Europe movie indicates that high-levels of PM are due to large-scale PM clouds moving across Europe.
No doubt there is a relationship with weather, but where do all these particles come from?The Belgium movie zooms in on Brussels.
You might expect the PM levels correlate with traffic rush hour, but that's not clear from the movie, and needs further study.
Neither does a correlation with open fireplaces in the country side show up.
If such correlations exist, they are well below the levels of the macro movements of the PM clouds across Europe.
I didn't expect this behavior a priori.
We all seem to blame traffic, and no doubt it contributes, but something more important is going on.
We'll dig into this in further posts.
Citizen science is cool!.. More details