I am recording my thoughts about this process here because I will forget someday. Perhaps this information will help someone else.
The last two years my son’s school was offering books with a disconnected book interface. That is to say the website they were using to host to page turning books had a separate site to host audio. In the end it was very hard to for the kids and parents to use. This year they started using a new service that basically combines the two as a video. So my books aren’t being made any longer.
I took the PDF and .mp3 files they uploaded and (through a cumbersome process) I combined them into an ePub book. Each page had a separate audio player control at the bottom of the page. This was great because it made it much easier for kids to back up the audio so they could hear something again. The other huge benefit of this method is ePub files are portable, so many parents loaded their homework books up on their iPads and had them review on the ride to school!
I discovered that ePub files are an archive compressed in the ZIP format, and by changing the file extension from .epub to .zip I could view into the structure of a book. Download one of the last books from here for an example. I didn’t use any fancy software to create the books, I used a couple of AppleScript droplet scripts to speed my process (PDF2PNG, ePub_Zip, ePub_UnZip), but I hand edited the HTML and split the large audio file page by page in Audacity.
Here is a quick update summarizing the changes I have made over the past few days. I am convinced I have removed 99% of the memory leaks. The memory on my Raspberry Pi stayed pretty stable yesterday.
The summary of what has changed is this:
import gc was added to garagemonitor.py (the Python Garbage Collector).
gc.collect() was added to the end of my main loop (so its runs on each cycle and tries to free any orphaned variables/objects).
keep_garage_monitor_active.sh bash script was created to researt the garagemonitor service when the script is no longer in the process list (for some reason it has terminated a couple of times on me, I am not sure why).
The Status web page no longer reload the entire page every 10 seconds. Instead, it now uses AJAX to query the Web Service every 5 seconds. It now only reloads the page when the status has actually changed. This saves workload and bandwidth for the server, and moves more of the work to the client’s browser.
The Status web page background grows bright then returns to the normal color to indicate when the status has been updated.
I may even rework the status webpage all together again so it never reloads the entire page again. I am looking at the jQuery Mobile framework right now.
I have added a wish list item to my My Raspberry Pi Garage Monitor. I added the ability for users to be able to issue requests from buttons on the Status Page. The first button issues a “Status Refresh” request.
Action Request Web Service
For safety I decided not to make the Pi accessible to the Internet, so I setup a second web service. The button submits a form which stores the request in a file. A web service which reads that file is queried by the Raspberry Pi every so often. Right now I have garagemonitor.py setup to query the service every 60 seconds.
I may add the ability to close the garage door on the Open Status Page. However, there are some issues to consider.
We do not want the web service to be able to open the door. (For safety reasons.)
The door could be closed remotely while someone is backing out of the garage. (For example, my wife checks the status at work and closes the door while I am backing my car out.)
Note: this is part 3 of a series. You can see all of the articles I have written on my Garage Monitor here.
A Possible Solution
OK, so after going down the route of debugging and tracing, I realized I had no idea what I was looking for. So I started thinking about what I should be doing to cleanup. I realized that I wasn’t freeing the memory of the objects that I was using in the functions. So, I started trying to figure out how to do that.
So after making some modifications, I have been randomly taking readings by running the free command. Over the last 2-3 hours the free memory is staying quite stable. In fact for a couple of the readings the free memory actually goes up: 349816, 349824, 349576, 349700, 349584, 349576, 349584! 🙂
There is some kind of memory leak happening. I first noticed a problem yesterday when the python script stopped running. The Raspberry Pi was still responding to my SSH requests, and when I ran “ps aux | grep garagemonitor.py”, the script was no longer in memory.
As a troubleshooting step I disabled some of the superflous functions (such as temperature monitoring and startup LED flashing). I have been watching how much memory is free over the past hour or two. The free memory has gone from 299596 to 288784. So it is obvious to me that this is going to run out of memory at some point as well.
I have been wanting to learn to develop for mobile devices since I got my first Windows Mobile phone six years ago. At the time my attention was occupied with the impending birth of my first son, and the traumatic transition from working life1 to becoming a stay-at-home parent.
I want an iPhone… never mind, I want a Droid!
Two years ago my wife and I were wanting to treat ourselves to iPhones… then Verizon launched the Motorola Droid.2 By this time our second son was 1-year-old and I was too busy raising two young boys.
The path to an iPad (and to an iPhone for the wife).
This summer, Google launched the “Honeycomb” version of Android meant for Tablet devices.3 Although I had trouble justifying why I needed one, I bought an Android tablet. I found it quite handy and my 5-year-old, my wife, and I all got addicted to playing Plants vs Zombies. It was clear we needed a second tablet. To justify buying an iPad we did some research and found a bunch of nice educational apps/games for our boys to play and learn at the same time… oh and PvZ was available on the iPad as well. My wife enjoyed the iPad enough that she traded her Droid for an iPhone 4S.