offset \ˈȯf-ˌset\ noun

a force or influence that makes an opposing force ineffective or less effective

Convert KML Placemarks to Zones for Home Assistant with Python

I have a habit of tracking the places we've been to in Google Earth Pro. I have a list of folders that are named after countries and, inside them, a list of places that we've been to. Sure, I can export to a KML/KMZ format but Google Maps refuses to load the file because of the size. I would also like to display the places in the Home Assistant, but there's no easy way and I really don't want to over-engineer this.

Home Assistant is self-hosted and I can control it, so I decided to whip up a quick script in Python to help me accomplish seeing the places in Home Assistant.

  1. I opened Google Earth and selected all the places, then exported them to a KML file that I called places.kml. I saved it to a location where I put the Python script.

/media/images/google-earth-places.png
  1. The script is in a file called kml2zones.py and safely nested in a virtual environment on Python 3.10.x. I installed the requirements with pip (pykml and PyYAML) and ran the script python kml2zones.py which in turn output the zones.yaml file in the same folder. Here are the contents:
from pykml import parser
from yaml import dump

# load kml into object
with open("places.kml") as f:
    root = parser.parse(f).getroot()

# iterate through the kml,
# extract what is needed in to a dict,
# then append that dict to to the list
data = []
for country in root.Folder.Folder:
    for placemark in getattr(country, "Placemark", []):
        try:
            coordinates = str(placemark.Point.coordinates).split(",")
        except Exception:
            coordinates = str(placemark.MultiGeometry.Point.coordinates).split(",")
        data.append({
            "name": str(placemark.name),
            "latitude": float(coordinates[1]),
            "longitude": float(coordinates[0]),
        })

# dump the list to a yaml format file
with open("zones.yaml", "w") as f:
    dump(data, f)
  1. I copied over the zones.yaml to the Home Assistant configuration folder and edited the configuration.yaml in it to have: zone: !include zones.yaml at the end, then I restarted the Home Assistant.
  2. I planned to load them in the Map card, but that card accepts only a list of entities which are generated from zones when the Home Assistant loads. I got the list of entities with Developer Tools in Home Assistant. Under Template, I put the following:
{% for item in states | map(attribute='entity_id') | list %}{% if item.startswith("zone") %}
  - entity: {{ item }}{% endif %}{% endfor %}

/media/images/home-assistant-template-zones.png

Good enough of that black magic, except the first result in the list which I had to indent manually. I copied the entities.

  1. I created a new tab on the dashboard with only one card so it expands, the type was Map, I added two zones in visual editor, as well as set the zoom to 3, then switched over to the code editor. This showed me how I wanted it to look like.

/media/images/home-assistant-map-card.png
  1. I removed those two zones under entities and in their place put the clipboard content obtained from step 4. Then I searched for the ones that I had from before in that list that are not from the zones.yaml (like zone.home or the local stores that I previously had saved in the Home Assistant database) and removed them. I finished editing the card.

/media/images/home-assistant-map.png

That's pretty much it. It really shows how it's easy to do things with Python.

As of this moment, I did a number of small stupid scripts for doing various things. Some of them grew into a full fledged service like Mellow. Mellow was more an opportunity to play with what AWS has to offer than anything else. The kml2zones.py could maybe be converted into a service as well but I cannot be bothered and, what's more likely, people would not really use it, but it is there if someone wants to nibble on it and help themselves. Even executing it without a file, through REPL itself.

Some of the scripts I did end up as a snippet, some of them as a repository. Of the top of my head I made:

  1. Mellow - the conversion from Trello board to Coggle mind map
  2. kml2zones.py - the conversion from KML file to zones.yaml for Home Assistant
  3. geotag-gallery - making a KML file with placemarks from photos and tagging them with convolution AI results
  4. led-morse - blinking LED light on Arduino in a Morse code
  5. Shamrock - getting plant data from a 3rd party service.

Probably more. Writing those things is easy and the possibilities are endless.

The script in this article is an example of how things can be done quick and dirty if one only wants to invest a bit of time. I remember seeing some videos of people using Python as a scripting language much like a macro in an operating system. It would open browser, start some pages, click through some of the options and so on. Generally automating a plethora of mundane tasks. I'm sure people could utilize it in their own life, too.

Room-Level Presence Tracking

While ordering PC components, I also ordered Raspberry PI Zero 2 W. Ideally I'd have one more so I can have three and a proper Paxos distribution, but they are notoriously difficult to obtain these days. In the end I used my original PI 4 and Zero 2 W to create a two-node system for room level presence detection. The third one will have to wait so I might go for it this year if it turns available.

I installed Raspbian on the PI Zero 2 W and connected it to the network. Then I made the IP static on the router so it's not changing anymore. The same is done for the Raspberry PI 4, but in reality, all the devices that are our own have a static IP in the network. It's easier to address them and the configuration is not always catering to the dynamic ones.

The project I used was room-assistant and I installed it to both devices via Docker. PI Zero 2 W only has room-assistant, and the PI 4 has Mosquitto, Home Assistant and Room Assistant. The Room Assistant describes that you should have local.yml configuration and mine is as follows on PI 4:

global:
  instanceName: "Living Room"
  integrations:
    - homeAssistant
    - bluetoothLowEnergy
cluster:
  autoDiscovery: false
  weight: 2
  port: 6425
  peerAddresses:
    - <IP_v4 of the Raspberry PI Zero 2 W>:6425
homeAssistant:
  mqttUrl: "mqtt://<IP_v4 of the Raspberry PI 4>:1883"
  mqttOptions:
    username: <mqtt username>
    password: <mqtt password>
bluetoothLowEnergy:
  maxDistance: 20
  allowedlist:
    # bluetooth mac of the device you want to track,
    # for example, phone. There's a chance LE physical address
    # is rotated, if that's the case, you can go into
    # Home Assistant Android app, go under:
    # Settings -> Companion App -> Manage Sensors -> BLE Transmitter
    # and enable it, the format will be
    # UUID_WITHOUT_DASHES-major-minor
    - 112233445566

PI Zero 2 W has the same configuration file, but with a different instanceName (being in bedroom), weight of 1 so it's not picked as the leader if possible (the higher the number, the bigger the chance it will be the leader, and also that's why I'd like to have three devices at least - Paxos, remember), peerAddresses has the IP of the Raspberry PI 4, but the rest are the same as the configuration above. Be sure to change the values where they are required.

As for the Mosquitto, it's an MQTT broker that is coming recommended for Home Assistant use. Room Assistant detects a Bluetooth device, then sends a message to the queue. After that, the Home Assistant picks it up.

I am not very happy with the devices reporting their location. It would depend on the load of the device and the quality as well. I tried using bluetoothClassic integration at first, but switched to bluetoothLowEnergy because it was more accurate. Still, I think I might have to change the devices themselves since the signal drops, or investigate more into it to tweak the configuration. The walls will also affect this.

On the other side, once the Home Assistant picks up the integration, all that is left is to detect the presence. The integration makes sensors out of your devices, for example: sensor.fairphone_3_ble_room_presence. To get it into automations, it's the easiest to create the template automation trigger somewhat like this:

{{ is_state("sensor.fairphone_3_ble_room_presence", "Living Room") }}

When the above triggers the automation, the action can follow. For instance, turn off the lights in the other parts of the apartment, start home theater if it's evening, things like that... That said, I'll keep using it and see if I can improve on it.

2022 Recap

The year was filled with good things and the bad. Mostly good. It's hard to complain looking back. After all, we did more than expected.

I am keeping my publishing frequency of once a month, but like I said before, I lost some of the data when the SSD failed along with a plethora of components so the two consecutive articles were done late December. The restrictions were loosened up so we got to travel, but I miss just staying at home in a way. I also published Cuply in the Summer but need to come back to it to see it in action this year and iron out any kinks.

Moving on to the recap. A friend said that revenge traveling was trending. Being holed up during the pandemic meant that we saved enough money. Coupled with some additional planning meant that we didn't waste our days off. Usually our trips are self-organized and follow a bit of a template, we prefer to take accommodation near the city centre, and then rely on hop-on-hop-off buses and free walking tours to get a better sense of the space and history.

Trips:

  • Poland (Krakow, Auschwitz, Wieliczka salt mine, a lot of it was focused on what happened during the WW2, but we definitely found time for vodka tasting)
  • Croatia (back home, Zagreb and Papuk nature park)
  • Hungary (Budapest, saw some friends from Ireland that were visiting their family)
  • Austria (Vienna, the capital of the old Austro-Hungarian empire that Croatia was a part of)
  • Slovakia (Bratislava, the last destination of our trifecta trip; in retrospect it would've been great if we had the time to visit the cities traveling via Danube, as all three are on it, but it was still great. The architecture is very familiar in the remnants of the empire)
  • Spain (Valencia, a futuristic looking city that reminds us of the Star Trek's San Francisco version, with a lot of green surfaces converted from the old dried up river bed)
  • Spain (Barcelona, visiting a work colleague)
  • Andorra (Andorra la Vella, while visiting Barcelona, it was an opportunity to visit this snowbound microstate)
  • France (Ax-les-Thermes, all part of the same day trip from Barcelona)
  • USA (Boston, Somerville, New York, visiting for work and an opportunity to see another continent. Saw four out of five boroughs in NYC, a neighborhood music festival in Somerville and took some lovely tours in Boston)
  • Switzerland (Zürich, long layover towards Greece, we plan to come back)
  • Greece (Athens, Volos, Rhodes, Santorini, Mykonos, Delos, visiting the islands)
  • Turkey (Istanbul, Kusadasi, and a bit of Turkey which is very European, well, at least the parts we were at, and Istanbul is magnificent)
  • France (Paris, Versailles, a trip with Vesna's parents)
  • Spain (Tarragona, to see this old Roman Empire city with all the accompanying ruins, then Barcelona, visiting for work, this time saw the inside of Sagrada Familia)
  • USA (New York, and work again, and some other vistas, the Wicked musical in Gershwin theater on Broadway and the last borough which was Bronx, where we visited the botanical gardens)
  • Romania (Brasov, Bucharest, Sibiu, Sinaia, visiting castles, all the gothic and vampire things. Carpathian mountains are beautiful, and Peles Castle is an amazing modern castle)
  • Croatia (back home and Zagreb again)

As for domestic travel, a couple of friends came over so we revisited the usual spots in Dublin and around our place, as well as finally went on a tour of the cemetery. My parents visited at one point so that was a blast, too. However, we did some domestic travel outside of Dublin, too, taking a road trip with friends:

  • Derry in Northern Ireland (a quick visit on a rainy day)
  • Malin Head (the northernmost point of the island)
  • Glenveagh (the national park)
  • Slieve League (beyond amazing vistas, we took a boat ride as well)
  • Ballyshannon (a small trip around the city)
  • Marble Arch Caves in Northern Ireland (going underground with all the stalactites and stalagmites)

/media/images/slieve-league.jpg

Concerts and events:

List of things we bought:

  • Grow light for the plants which worked with a pump for the plants to survive in the tub while we were away on one occasion
  • ESP32 feather board for the automation of the standing desk
  • Aqara door and window sensors for turning of the heating if the room is ventilating
  • D&D things, books, papers, inks...
  • 3D printing safety equipment and miniature painting equipment
  • Cocktail making set and then some metal straws and picks
  • Bluetooth speaker for the bathroom. Unfortunately it's not that smart to be always on, but works
  • Soldering iron kit for working on the Cuply project
  • Life is Strange comic set, so I've read a bit this year
  • Logitech Z407 PC speakers because the old ones died and I couldn't repair them. I'm not on that level yet
  • New backpacks for Vesna and I, a picnic blanket and travel pillows
  • Zemismarat curtain motor because I got tired of waiting for the IKEA to release a fitting item... that and the old curtain nearly fell on my head because the old axis got worn down
  • External Samsung SSD for backing up things locally
  • Tefal deep fat fryer because our local fast food joint closed down and we were panicking
  • Kitchen utensils and replacing/repairing things in the apartment
  • This is Vegan Propaganda book I got from Vesna and I highly recommend it
  • New poster covers
  • Bathroom curtain
  • Raspberry PI Zero 2 W
  • Desktop PC components that died (CPU, PSU, Cooler, MoBo, RAM)

Other important events:

  • Got a confirmation that my citizenship application was being processed, but it will probably take another year and a half from the time of this writing
  • PC died, but managed to repair it
  • Contracted COVID-19 most likely while traveling from France
  • Started wall climbing, seems it will stick and we need to get fit
  • Finished hydroponics, i.e. the Cuply project
  • Didn't draw, but must do something this year
  • Played some games, but only finished Ori and the Will of the Wisps
  • Read some comic books and books: Life is Strange, Persepolis, This is Vegan Propaganda
  • Got into cocktail mixing
  • Reminded myself we're getting older
  • Got back to planning trips
  • Didn't ride the scooters, but need to do something about it this year
  • Started 3D printing

Right now it's back to planning out this year and making some resolutions yet again.

Last of the Season

The temperature finally stabilized in the apartment. We got some things cleaned up and met up with some friends for a pint or three. Right now we're waiting to play another N7 session, but no go so far. The fireworks outside make our friends' dog scared so I'm writing a bit.

We made a cocktail for tonight, another vegnog variant.

Vegnog

Vegnog

It comes in two parts. The ingredients are as follows with the first component being infused then added to the rest and blended away.

Ingredients:
  • 3 cups oat milk
  • 6 tbsp brown sugar
  • 4 pods cardamom
  • 1 pinch saffron
  • 2 pieces clove
  • 1 piece star anise
  • 1 1/2 cups spiced rum
  • 1 cup plant based whipped cream
  • 1/4 tsp ground cinnamon
  • 1/2 tsp vanilla extract
  • 1 pinch ground nutmeg
Instructions:

Put 1 cup of oat milk, brown sugar, cardamom pods, saffron, cloves and star anise in a saucepan and heat up for it to infuse. After it's done, let it cool and remove the solids. Then add the infused milk with the rest of the ingredients in a blender and blend for a minute until it becomes frothy. Let it cool in a refrigerator for later. Drink responsibly.

Happy New Year!

The News of Our Death Have Been Greatly Exaggerated

But our PC died and some drafts with it, so I didn't publish anything last month.

It was a painful experience trying to get the new components and checking out the extent of damage. Suffice to say, I managed to cannibalize the old PC and augment what I could with a solid hit on our wallet. The delivery was a nightmare and it took weeks to sort it out. Some issues are still ongoing.

The new rig was assembled just a few hours before our flight to Croatia. I couldn't do anything but wait until we came back.

We spent approximately a month in Croatia, working, visiting friends and spending time with family, but it was enough to start missing Ireland quite a lot. After all, our lives and routine are centered around that small piece of the planet.

Going forward, I need to set up better accessibility to the server, a better backup and a contingency plan... For watering the plants. Luckily there was no damage as the biological processes in plants slowed down so they didn't require water as much. We were also worried about the freezing weather breaking the water containers on the balcony left over from 3D printing, but it was all fine.

We came to an empty apartment. Everything was seemingly in order. We hit the store to get some food. We didn't even manage to unpack. That will happen today. I turned on the heating remotely before arrival but one of the radiators needed to be bled so it took longer than expected for room temperature to climb. It's finally OK at the time of this writing.

I'll publish something tomorrow as well just to keep my publishing frequency at least 12 times a year. For what is worth, we're alive, well and back.