A few weeks ago I got addicted to installing SmartThings in my house, and I wrote of my initial woes getting that set up. This is just to say I managed to get mostly everything working as desired, mostly using the default provided “SmartApps”. I am still having consistent problems with the Android app, but it’s been mostly limited to things like viewing history and now actually controlling the devices. Things have been pretty reliable, to my delight. A few sensors report being opened when they are not once or twice a day, but I’m still tracking that down.

I’ve been looking into the SmartThings documentation to understand how programmable the system is and make some apps to fine tune my setup. So far I haven’t done much of value other than gut the door control app to make it log the door instantly upon being closed, but I am working on something more involved.

I wanted to make a simple graph of all the temperature sensors, and then possibly compare to other data. I thought this would be a simple process. First I wrote a “Web Services SmartApp” that my external service would query in order to get the data. It seems that was the only way, instead of directly querying SmartThings and not forcing the users to install an app themselves. I guess this is a security measure so the access is configurable on the phone alone with all the other SmartApps. Anyway, writing that app was pretty straightforward. I did have to learn Groovy, which is based on Java, but for doing what I needed, it was not hard.

Next came writing the bit that sits on my web server and pulls and processes the data. This has so far proven to be a convoluted process.

I began by researching JavaScript chart libraries. I used D3.js for Weatherbit, and thought I’d use that, but I’ve long since forgotten how to work with it, so I looked at some other options. After a few minutes of not so serious consideration, I decided to go with Google Charts.

Then for the back end, I hoped to use Python, as I’ve been looking for excuses to get more experience with it. Using Python is a bit more involved than using PHP though, as I can’t just make a Python file and access it with the browser. Or, that’s possible, but not recommended. Typically Python apps are run with their own sort of separate server. My problem was I wanted to have a quick way to create silly little projects without so much overhead for each one, and stuff to maintain forever after.

I chose to set up uWSGI such that for each app I’d need a config file for uWSGI and an instruction in the Apache config to send requests for a certain path to the uWSGI socket for that app. This seems reasonably maintainable.

Then I needed to choose a Python framework to handle basic templating. I wasn’t very familiar with the options, but after some reasearch, I chose Web.py for it’s supposed simplicity but also for being well tested. I’ve since come across some others like Flask and Bottle and more, but they seem newer and less tested, and I couldn’t discern any benefit in my case.

So then I was ready to code the app! I thought! I was querying my SmartApp just fine using the access token provided in SmartThings’s IDE, but I needed people to be able to log into my app and have the app get the token itself. This is especially complicated since I didn’t fully understand if the user would need to create the SmartApp themselves by copying and pasting my code into their IDE, or if they would simply be able to go to a URL and have it installed automatically. After some testing, it seems the later is the case. So, I did a fair amount of putzing with querying SmartThings from Python, but struggled to get the process working. This was largely because I didn’t know how to get the detailed logs of my code since it was being run through uWSGI and the messages were not reaching the uWSGI log. I was using requests_oauthlib since I assumed it would simplify token management, but it added confusion because I didn’t have a direct comparison with the SmartThings documentation in which they used a Ruby app and a Ruby OAuth library that used different parameters than anything I found in Python. It shouldn’t be so complicated, but without the logging, it was a bit of a black box for me, as I couldn’t tell what headers and requests were actually being made. I eventually got it working, though!

Next I had to consider how to build the charts with time series data from a bunch of sensors. I figured I’d have to build a table where every row is a timestamp and a value for one of the columns, as the timestamps don’t necessarily match up. So I considered a database structure, and weighed SQL vs NoSQL. Since working with MongoDB and Python is pretty easy, I decided to try that approach. So I had to decide on a document structure, weighing whether I should group temperature or other sensor readings for a time period into a single document, or just have thousands of tiny documents with a temperature and a time, and the device ID. I’m still figuring out what to do there, and it will probably take some trial and error.

But stepping back, I realized I’d need to have a way for my server to query SmartThings on a schedule so the user can see their history even if they don’t log into my app for a while. That led down a bunch of other roads I abandoned. At first I thought I could use the uWSGI task scheduler API, but it seemed it wouldn’t work unless I could import uwsgi from within the Python app, but I can’t do that since I am using a system level compiled uWSGI. I needed to do that since I have Python apps using various versions of Python, and couldn’t have a separately compiled uWSGI in each virtual environment. That was a whole other detour. So, I decided I’ll just write the SmartThings Python class in such a way I can call it on the command line via crontab and have it update everything it can. This means I need to store the tokens for each user and plan a way to allow them to access the app again in the future without logging back in through SmartThings, as that would generate a new access token and maybe disconnect the data in my database. I’m still pondering this aspect, and struggling to structure this program in a sensible way!

I’m sure I left out a bunch of other detours I’ve made today, but that’s at least a sense of the joy this has been so far.