Extracting data from the "ambient weather observerIP 3.0" module

I am a big fan of weather and love to see all types of weather. I’m a CERT member, a HAM radio operator, and a trained weather spotter. I even have a Personal Weather Station (PWS) set up and posting to Weather Underground (check it out here). I am using the WS-1200-IP from Ambient Weather with the Observer IP Module which automatically and painlessly updates Weather Underground.

I love my set up, but there is one feature missing from the Observer IP module. It does not have a clean way to extract data and is not capable (as of this post) to be configured to send the data to other locations such as a FTP site or a CGI script. I could upgrade to a fancier PWS such as the Davis Instruments 6163 Wireless Vantage Pro2 Plus and get a console that can be connected to a computer… but that requires more money for the weather station plus dedicating a computer to run the collection and upload programs.

Now, the Observer IP comes with a nifty web interface to help you configure it and has a “Live Data” page that gives you some of the measurements. The page even refreshes, so you can watch the data. It does have several problems though… it’s very ugly… it is not mobile friendly… and it is only available on your local network. You can judge for yourself…

I happened to be doing some coding and web development and on a whim, decided to look at the source code for this page. Here’s what I saw:

<tr>
 <td bgcolor="#EDEFEF"><div class="item_1">Indoor Temperature</div></td>
 <td bgcolor="#EDEFEF"><input name="inTemp" disabled="disabled" type="text" class="item_2" style="WIDTH: 80px" value="38.1" maxlength="5" /></td>
 </tr>
 <tr>
 <td bgcolor="#EDEFEF"><div class="item_1">Indoor Humidity</div></td>
 <td bgcolor="#EDEFEF"><input name="inHumi" disabled="disabled" type="text" class="item_2" style="WIDTH: 80px" value="35" maxlength="3" /></td>
 </tr>

Now… if you haven’t seen it yet, there’s something important to notice! The individual input fields have a name, and a value assigned. This is awesome… thank you to whomever developed the firmware for the Observer IP. Why are these important you ask? Because we can use them to extract the data and use it elsewhere! In order to grab the data, we need something that can parse a web page and pull out what we need. I chose Perl as it is something I know and it has all the right tools… and it is one of my preferred languages. I’m sure other languages are capable of achieving the same results, but lets take a look at an example (and yes.. it may be ugly… it’s a first pass and it works!):

#!/usr/bin/perl
use HTML::TagParser;
use Data::Dumper;
use Compass::Bearing;

my $obj = Compass::Bearing->new();


my $html = HTML::TagParser->new( 'http://192.168.1.201/livedata.htm' );
 #print Dumper(\$html);
 my $elem = $html->getElementsByName( "inTemp" );
 $inTemp=$elem->getAttribute("value") if ref $elem;
 my $elem = $html->getElementsByName( "inHumi" );
 $inHumi=$elem->getAttribute("value") if ref $elem;

 my $elem = $html->getElementsByName( "outTemp" );
 $outTemp=$elem->getAttribute("value") if ref $elem;
 my $elem = $html->getElementsByName( "outHumi" );
 $outHumi=$elem->getAttribute("value") if ref $elem;

 my $elem = $html->getElementsByName( "AbsPress" );
 $absPress=$elem->getAttribute("value") if ref $elem;
 my $elem = $html->getElementsByName( "RelPress" );
 $relPress=$elem->getAttribute("value") if ref $elem;

 my $elem = $html->getElementsByName( "windir" );
 $windir=$elem->getAttribute("value") if ref $elem;
 my $elem = $html->getElementsByName( "avgwind" );
 $avgwind=$elem->getAttribute("value") if ref $elem;
 my $elem = $html->getElementsByName( "gustspeed" );
 $gustspeed=$elem->getAttribute("value") if ref $elem;

 my $elem = $html->getElementsByName( "rainofhourly" );
 $rainofhourly=$elem->getAttribute("value") if ref $elem;
 my $elem = $html->getElementsByName( "rainofdaily" );
 $rainofdaily=$elem->getAttribute("value") if ref $elem;
 my $bearing=$obj->bearing($windir);

 print "Inside: $inTemp / $inHumi% | Outside: $outTemp / $outHumi% | Difference: ",abs($inTemp-$outTemp),"\n";
 print "Pressure (Absolute/Relative) $absPress inches / $relPress inches\n";
 printf "Wind: $avgwind mph from $bearing ($windir degrees) | (Gusts: $gustspeed mph)\n";
 print "Hourly Rain: $rainofhourly inches | Daily Rain: $rainofdaily inches\n";
 

Let’s break down the requirements… First, I am using HTML::TagParser so I can grab a page and parse it. I am also using Compass::Bearing to convert my windspeed direction to a compass direction. Data::Dumper is there just for debugging. You’ll need to install all 3 packages if you don’t have them installed already. Note that Compass::Bearing requires GD, so if you are on a Mac like me, you’ll need to install it (Homebrew is awesome for this!). If you are on Linux, you should be good. Windows guys… I don’t know what to tell you. Also, you’ll probably need URI::Fetch (I think) if it isn’t installed. You’ll get an error if you’re missing modules. So let’s look at the first couple of lines after the “use” lines…

my $obj = Compass::Bearing->new();
my $html = HTML::TagParser->new( 'http://192.168.1.201/livedata.htm' );

The first line sets up my Compass::Bearing object. We’ll use this later. The second line pulls the HTML page from my Observer IP.

my $elem = $html->getElementsByName( "inTemp" );
$inTemp=$elem->getAttribute("value") if ref $elem;

Next, I use HTML::TagParser to pull the tags by “name” and then get the “value” attribute and assign it to a variable. You can find all the tag names by viewing the source code for your live data page.

my $bearing=$obj->bearing($windir)

This line converts my wind direction (which is presented as a number and represents the degrees on a compass) into a direction such as “N” or “NNW” or “SW”.

Finally, I am printing out the results…

print "Inside: $inTemp / $inHumi% | Outside: $outTemp / $outHumi% | Difference: ",abs($inTemp-$outTemp),"\n";
print "Pressure (Absolute/Relative) $absPress inches / $relPress inches\n";
printf "Wind: $avgwind mph from $bearing ($windir degrees) | (Gusts: $gustspeed mph)\n";
print "Hourly Rain: $rainofhourly inches | Daily Rain: $rainofdaily inches\n";

…which looks like this…

Inside: 36.9 / 32% | Outside: 26.2 / 36% | Difference: 10.7
Pressure (Absolute/Relative) 30.13 inches / 30.47 inches
Wind: 1.1 mph from W (281 degrees) | (Gusts: 2.5 mph)
Hourly Rain: 0.00 inches | Daily Rain: 0.00 inches

Since you have the values, you could add code to save the values to a file, post it to a form or other API, or even create a page and upload it to your website. I am using GeekTool on my Mac to display some other information, so I added a geeklet that updates every 15 seconds to show my weather station information on my desktop. Very cool!

I hope this post helps other weather and computer geeks.