Python and cybersmart Caps

I recently got DSL at home (after 22 working days), and signed up with a local ISP (cybersmart - you can tell them bwhittington@cybersmart.co.za referred you) because it was the cheapest 3 Gig cap available. What I also like about them is they provide a very simple web page that helps you keep track of your usage. I knew a bit of python, and I wanted to further my skills, so I decided to tap out a python script to bring my cap usage to the desktop. The result:

What this re-iterated for me is that python is awesome, linux is awesome, gnome is awesome, and python is awesome. You can get python wrappers for just about everything, and more and more programs include python as an avenue to build extensions (my first non-django, python project was a rhythmbox plugin that pauses the music when your screensaver turns on).

So I started out on the problem. It was a pretty straightforward issue: download the html from cybersmart, walk the DOM, grab the elements with the relevant data, typecast the data to a floating point number, and, for cream, calculate daily averages (per day usage, per day remaining). Finally, display the data. For this little project I used urllib, pyxml, pynotify (a thin wrapper around libnotify), datetime and time (you can download the full file as an attachment to this blog post).

We start with the data, the cybersmart page looks like this:

The relevant bits of information are the "Used for this month" and "Total left for month". Luckily these are stored in elements with class names that are relevant (although they do reuse the same class name for multiple things, instead of doing a class="used usedTotal" and a class="remaining remainingTotal", and styling the relevant classes, while getting the semantic meaning, and visual identity across).

So, we download the file (sending relevant log in details as a POST), and send it to the HtmlReader class of pyxml:

print("Downloading HTML")
data = urllib.urlencode(
  {   "accountName":"account@cybersmart.co.za",
      "password":"password",
      "login":"login"})
f = urllib.urlopen(
      "http://www.cybersmart.co.za/getAccountDetails.cgi",
      data
    )
doc = f.read()
f.close()
reader= HtmlLib.Reader()

Then we parse the document, and walk the DOM tree, pulling out relevant info (sidebar: I couldn't do this more efficiently than pulling out all the td's and then checking each class entry, the getElementByClass doesn't seem to work out properly?? I also cheat because there are multiple savedTotal elements, but I am only worried about the last one):

print("Parsing Document")
dom = reader.fromString(doc)

print("Walking the tree")
td_elements = dom.getElementsByTagName("td")
remaining=''
used=''
for td in td_elements:
        temp = td.attributes.getNamedItem('class')
        if temp:
                if temp.value == "savedTotal":
                        remaining=collect_text(td)
                if temp.value == "usedTotal":
                        used=collect_text(td)

Now we have strings in our "remaining" and "used" variables which we can easily typecast to floats and process them further. I used some helper functions that I found on the net for dealing with number of days past and remaining in the month, I didn't note where they came from, so apologies to the author.

print("Caluclating averages")
daysleft = 
    mkLastOfMonth(datetime.datetime.now())-datetime.datetime.now()
daysleft = daysleft.days
remaining_float = float(remaining)
used_float = -float(used)
days = int(datetime.datetime.now().strftime("%d"))

Then, the magic...displaying the data as a notification bubble is flipping simple, you just import pynotify, initialise it with your namespace, and tell it what to display. You can get wrappers that allow you more flexibility for your notification (like images etc.), but I didn't need it.

print("Displaying data")
usage=
   "%.2f Gb used in %s days\nUsage average of %.2f Mb" % 
   (used_float,days,(used_float*1000)/days)

projected=
   "\n\n%s days left to use %s Gb\ndaily allowance of %.2f Mb" % 
   (daysleft,remaining,(remaining_float*1000)/daysleft)

content = ''.join([usage,projected])
pynotify.init("cybersmart")
n = pynotify.Notification("Cybersmart Remaining Cap",content)
n.show()

It is not the most wonderful, or extensible code ever, but it works for me, and hopefully this can help someone else do something else. I would like to find a platform agnostic way to do notifications, and I have found some code to build a platform agnostic systray icon (which could keep track of you usage daily as a graph, and pop up a detailed notification when clicked). Splitting the code into classes would allow it to become more extensible in that multiple ISPs could be catered for using different approaches, with the same display code.

I am stoked with how easy it was to roll it together (POSTing, downloading, parsing, processing and display in just over 30 lines), how easy it was to hook into libnotify (the freedesktop notification lib) for display. I have upgraded from edgy to feisty (which bumped python from 2.4 to 2.5) and it wasn't bothered either. You will need to install python-xml, and python-notify. The nice thing is that this is also an appropriate area for python, since it is just message passing.

Update 24/06/2007 - Jonathan pointed out BeautifulSoup as a more appropriate alternative to htmllib

As a side note, I fell in love with python when I learnt it so I could program in Django. I have slowly been trying to spread my python wings beyond simply writing web apps, because it is a flipping rad language, with awesome readability, and crazy power. I suppose you could say I script in python, because python is interpreted at run time, then all the java programmers are scriptors too, so ne ner ne ner ne ner.

Bookmark/Search this post with: