OffSetLab

A year has gone by already and looking back it was quite eventful so I wanted to recap the highlights for the fun of it.

We traveled Europe with what days off we had at disposal and went to:

• Croatia because we wanted to see family and friends
• Iceland during the winter, where we saw Aurora Borealis and geothermal fields with geysers, and almost lost our fingers from taking pictures without gloves
• Germany to see the valley of Rhine from Frankfurt
• Spain to see Madrid and the surrounding small cities (Alcalá de Henares, Toledo, El Escorial)
• Scotland to see Edinburgh
• England to see Bristol, Bath and Stonehenge (so Vesna could take a picture in front of the house where Jane Austen once lived)
• Malta, which we covered quite extensively (because it's small), where we saw some nice megalithic structures and the less known, but still standing, Blue Grotto natural sea arch

We also roamed across Ireland with our friends and visited:

• Malahide and some beaches around Dublin
• Tayto amusement park north of Dublin for some thrills
• Sugarloaf mountain, Powerscourt gardens and Blessington greenway south of Dublin
• Arklow on the way to Wexford and Waterford
• Aran islands (Inis Mór), as well as caves with stalactites in the Burren
• Cork and Limerick, which are both beautiful

Concerts:

• Incubus, which was planned. It only took us 15 years to see them live since going together to see a cover band
• Why?, which was not planned, but was great none the less. They had a 10 year anniversary of their Alopecia album

We've been the best man and maid of honor to our best man and maid of honor. Woohoo.

I did an Inktober challenge as preparation for drawing characters in a video game (we may or may not be making); inking for the whole month of October every day. I produced a number of drawings and I'm slowly getting back into it after a long hiatus.

I changed jobs earlier this year and am now contributing to helping people with what I know. My new job is in IT health sector.

Vesna gained confidence in her career switch to front-end coding and got a job as well.

I dared to climb an artificial wall. I think I could do it recreationally, but I'm yet to figure out how to do it, if at all.

I published these articles semi-regularly. It's OK. I do need to push myself to get them out more frequently.

I replaced the cameras in my cell phone. I talked about the Fairphone earlier. It was easy enough to replace them and I needed to do it since I was taking photos a lot because of all the trips.

Because of the photos, I found a way to enrich the GoPro photos with geotags through my cell phone GPS and also made and published a photo tagging script that outputs KML. It creates placemarks for it on specific spatial and temporal coordinates with the photo attached. It figures out what's in the photo with Keras library and puts the results as a name for the placemark. The KML file can then be opened with Google Earth so I can keep track of where my photos were taken.

We visited Octocon which was enjoyable if a bit predictable. We also got Worldcon tickets. It's coming next year to Ireland, which is convenient.

We visited a Marxism conference to see how the community thinks.

We visited Porterhouse brewery and saw the beer making process.

We hosted some guests and went to the the usual sightseeing spots in Dublin with them.

We celebrated 15 years of our relationship and four years of marriage... and four years of being in Ireland.

We used our anniversaries as an excuse to buy new toys: we bought PlayStation 4 to play the exclusives and a projector so we don't need to go to the cinema anymore. To be honest, we've been talking about doing it for five years now.

We did go to the cinema with usual frequency, but this year didn't really see anything that stuck with us. I guess we hit a rut.

We're still playing pen and paper games with two parties over Skype. It's been a couple of years now.

We're still donating blood regularly and we didn't have any serious illnesses throughout the year.

We kept the apartment in order, replaced the window locking mechanism, bought a new table.

I made a GreaseMonkey script for Reddit so I could color code posts from the favorite subreddits.

Regarding other projects I have active, I kept the code in order and did minor upgrades like adding SSL on the OffSetLab because browsers are forcing me to, even though the site is static.

Bought Lazy Nezumi Pro. It's a great piece of software.

Reported a bug on Krita.

Wrote this recap. Fingers crossed that the next year is going to be great, too.

What happens when you have a camera that has no GPS, but you still want to attach spatial coordinates to the produced images? This post builds upon the last series of posts that dealt with displaying the spatial photo coordinates on Google Earth and people should be aware of the dangers of EXIF tags beforehand.

Some years ago I bought GoPro Hero 4 that doesn't have the GPS module. Newer models have it, but Hero 4, and a lot of other cameras don't. This puts me in a position of having the following use case: in possession of a number of photos that are not spatially tagged. For the solution to work, the timestamp function on the camera needs to be properly set up. I frequently forget to sync up the time on my camera when switching timezones, so this is a reminder that it needs to be checked. The prerequisite is that, at the very least, your camera needs to support tagging the images with timestamps. Well, that and a smartphone handy.

In case of Hero 4, I can sync up with Android and the date/time can be adjusted accordingly.

Assuming you have your camera ready and the timestamps work on it, you can take photos whenever and however you like, but, before you go wild with it, you need to somehow keep track of where you are. Nowadays, most of us own some sort of smartphone that has integrated GPS, and I'm guessing that the majority of the smartphones in question have a version of Android OS installed on it. You can always take the approach detailed in using Google Location history to enrich the photos with geotags; however, this cannot work for me since I turned off the location history in the first place some years ago, which is where the GPSLogger app comes in play for me.

Get the GPSLogger from the Play Store or wherever you get your apps from and start it up to periodically log the coordinates in a file. You keep your phone near you, so it will perfectly tag your location within the reach of your camera. Because it tracks GPS coordinates, it strains your battery, so keep that in mind. I was in Cork, Ireland: 51.897222; -8.47, last weekend so I got up in the morning before the trip and set the logging up with the default settings.

My battery doesn't have a big capacity (2420 mAh), but I don't use the phone that much. Still, I had to have a powerbank near me just in case. In the end the phone managed to log for the whole day without me needing to recharge it. If you have battery drain issues, you can refer to the FAQ of the GPSLogger app for some tips and tricks, like lowering the frequency of logging, but also keep in mind that spatial locations will give out approximate data anyway.

After a day out in the open where you take your photos, and at the same time your phone happily tags your location, you end up back home with a number of images in your camera and a GPS GPX file on your phone. Now all you need is a PC that will combine what you have.

I use exiftool on Ubuntu or WSL for the purpose of combining them. The easiest way to get it is installing it via APT. The package name is libimage-exiftool-perl:

sudo apt install libimage-exiftool-perl

I make sure that I place all the images from the camera and the gpx file in the same folder and it's really easy from there:

cd image_folder
exiftool -geotag gps.gpx *.jpg

That's really all there is to it. The tool uses the gpx to tag your images with the locations provided from it. Because the locations are approximate at best, for the purposes of checking out the images, it's good enough.

Now with the set of images that are geotagged, you can reference the script from one of the previous posts and make a KML for you to check the things on Google Earth.

Happy tagging!

This is the third post in the series about developing a script that uses your photos to create a workable KML that shows where the photos were taken on Google Earth. The first one talked about the script basics and the second one introduced reverse geocoding.

The generated KML still has a problem that the name of the photo is just a file path for it.

I wanted to rectify that since it looks ugly on Google Earth. To do that, I needed to know what was in each image. The process where you have half a thousand photos and need to rename them properly can be tedious. A Machine Learning algorithm is better suited for the task. The solution is called image classification and in the case I'm going to describe, it uses convolutional neural network on a pre-trained dataset from ImageNet to make things simple. In the background it does operations on an array of values to figure out what's in an image, which is an array of pixels in itself. Brandon Rohrer explains how convolutional neural networks work if you want to learn more about it.

I attended an information science university. Parts of my curriculum were focused on machine learning so the things that today's industry considers hot are pretty much more of the same, just with more resources.

Now, I cannot have what Google has at its disposal in terms of the dataset and I don't want the script to be overly complex for an ordinary user to use. The usual process is making a tagged dataset to train your model, but I wanted to have a pre-trained model so people didn't need to think about it, there was no need for fine tuning and they could get it out quick and dirty. Fortunately, there's a way today.

Enter Keras. Keras is a machine learning library written in Python. It uses several back-ends, but by default it relies on Google's own TensorFlow library. Installation is simple as it just relies on installing the tensorflow and keras libraries on your system:

pip install tensorflow keras

You don't have to do this because it's covered in the script requirements. Chances are it will work out of the box, and that's what I'm aiming for, but if you want to leverage your hardware resources, introduce GPU support if you have an Nvidia card, or Intel optimization for TensorFlow on Linux systems. You can check out the instructions for installing those yourself. What it usually involves is installing the tensorflow-gpu package or wheels with pip from Intel themselves. The instructions go beyond the scope of this article, which simply aims to provide the means for quick tagging of photos where the speed is not that critical and the user usually has modest resource capabilities.

Keras already has an access to the pre-trained models and the first time you're running an evaluation on the image, it downloads the model and puts it in a hidden keras directory in your home. This script uses ResNet50 application, whose model is around 100 MB and is trained on the ImageNet dataset.

When it classifies each image, the results are displayed as a certainty estimate and a guess from the algorithm, ordered from the best to worst. What happens then is that the first two estimates are taken together and separated by a "/" character. This is what ends up as the name on the placemark.

For the moment I am happy with the resulting script. Reverse geocoding and machine learning have shaped it up nicely.

The image classification results are not going to be perfect, but it should definitely save you quite a lot of time having the image names prepopulated with terms. You can then error correct manually those that you don't find accurate if you want.

Implementing the ResNet50 Keras application was very simple in the end and is good enough without fine-tuning.

Then again... The categories could be automatically translated as well. So I included TextBlob and powered up the automatic translation. You just run:

python geotag-gallery.py --folder=/absolute/path/to/the/image/folder/ --language=hr

The language parameter is optional. It will default to English if you don't put in anything.

Beware, though. TextBlob is not a robust solution since it's using a public facing 3rd party API and you might experience HTTP error 503 depending on Google's whims so you might be better off not using that feature since it's not guaranteed to work and is experimental at best.

So there you have it. Like I said before, you can download it from the repository. Pull requests are always welcome.

A while ago I made a Python script that takes your images and extracts the latitude and longitude coordinates for KML from them. The KML can then be opened with Google Earth where you can enjoy the fancy display of your photos attached to map locations. The script works in the way that it creates placemarks in KML that are correctly placed on the map. When one clicks the placemark, it opens a popup with the image associated with that location.

However, the description is just a timestamp of when an image was taken. That bugged me and there was a better way to get the information about the location, too. The format I opted for was "Photographed at {timestamp} near {location_description}". Quite better than just "{timestamp}".

Since I already had the timestamp to work with, that part was easy, but the other one, where I needed to figure out the location was a bit more tricky. I relied on a public openstreetmap reverse geocoding API to get the information about the specific coordinates. To comply with their terms of service, I needed to avoid bulk querying and query the service only once per second. I also needed to keep a cache of locations so the repeated coordinates didn't need to query the service.

The algorithm in question is simple enough. To get that, first check the cache if it has the location already and return that. If not, pause for two seconds because I don't want to take any chances and then query the service with the requests library, cache the response and get the location description. The requests naively retries for a couple of times just to be sure and drops it if it times out.

The result is a bit more easy to comprehend.

Now, the name that gets into the title also bugs me, but I'll explain the fix I did in the next post.

I decided to write a Python script that would create a Google Earth KML file with the list of geotagged photos that can be showed in the application. My reasoning for it was that a while ago Google decided to shut down Panoramio, the service they had acquired previously.

The service was of a photo-sharing type; much like Google Photos or Flickr today. The added ability that set the service apart from others was that the Google Earth application had a map layer comprised of photos stored on Panoramio. With Panoramio gone, the layer from Google Earth was removed as well.

I was not a user of it, but I liked looking at photos on Google Earth, thinking of places I wanted to visit. I wanted to share where I was traveling with my friends and family. Of course, provided that they used the same set of tools.

To achieve this the media needs to be enriched with geolocation information. The practice itself is called geotagging. JPEG image format, common with a number of digital cameras, can have metadata added to it (in contrast to PNG for example). The specific metadata is called EXIF and can contain information like the timestamp when the image was taken, camera type, location, etc. All the information necessary to create a KML file that Google Earth can read.

Now, before I get into the details about the Python script, I want to point out that EXIF information has a number of privacy and security issues. A number of TLA agencies are targeting EXIF information and using it for their purposes. For example, thanks to Edward Snowden's whistleblowing, we know that the NSA is targeting EXIF tags.

A number of examples associated with geotagging concerns are presented by Friedland and Sommer. A burglar can track your geotagged information and find out where you live, when you're not at home and steal things from your house.

Social networks make it incredibly easy to collect information on you and distribute it among unwanted people. One should be careful about the use of it, especially with kids nowadays gaining access and doing things they don't understand, enabling predators to exploit the information.

In your travels, you might visit wilderness and snap some photos that you would post online. One of more curious examples of abuse is poachers looking for rare animal photo geotags to figure out where to go. You want to be careful and think twice before you share information in this manner. Your camera or smartphone may be using GPS technology to attach the coordinates to an image.

Having said all that, you might think that EXIF information is not for you or that you want to strip that information before posting it online. It can be done, both stripping the EXIF information and preventing it from being saved in the first place.

I store my photos locally on an external disk and look at them offline. I don't frequent social networks so I don't have much trouble with geotagging information. I wanted to see where and when I was for personal use. That's why I created the Python script mentioned in the beginning of the article. You are free to use it too.

The necessary information I got was mostly from the article showing how to create placemarks for use in Google Earth. I wanted to automate the process and save KML instead. I could do that with Python.

The resulting script works only for Python 2.7.x for now because the PyKML dependency is not Python 3 ready so make sure you have the latest Python 2.7.x installed on your system. I install the dependencies in the virtual environment with virtualenvwrapper and can run it without trouble, but you might want to install them differently. Anyway, they are provided in the requirements.txt file in the repo. You can sift through the README.rst file there for more information. The usage of the script is pretty straightforward and it goes something like this::

$ python geotag-gallery.py --folder=/path/to/folder/with/geotagged/images

Keep in mind that it will parse the folder and the nested folders located in it. I use WSL on Windows to point it there or Linux locally so I can find my way around. It is supposed to work on both. The result of the script processing is the KML file in the specified folder with relative path to the images so you're not supposed to move it from there. You can open the KML with Google Earth and see where the images were taken according to the provided coordinates.

It also takes the timestamp so you can replay your steps and see when you were taking the photos.

I hope you find it useful or that it gives you an idea for a more comprehensive system you might want to work on. The script is on the BitBucket repository and under the GPL license so you can do whatever you want with it provided you share the code and give credit where it's due. Without further ado, you can download the Python script for creating KML from geotagged images in the mentioned repo. I put the download information deliberately at the end because I want people to know what they are doing and read through the article first.