Temperature Data Logging
I have an interest in weather data collection and analysis. Because of my interest in weather and strong background in electronics hardware and computer software both for the PC and embedded control systems, weather data collection and analysis seems like a natural thing to do.
This is a project that was hard to find a home for on this site. It is partly related to a few other topics covered here. It uses a micro-controller, so
it would fit under the projects heading. It uses software based off of the
CAT program used to control a transceiver via serial communications,
so it could fit under the software projects heading as well. It is related to the weather page
in the fact that it was used to log outdoor temperatures on a minute by minute basis. It is also related to the house page
in the fact that it was used to monitor indoor temperature as part of an energy study.
In the end it wound up here on its own page with links from all of the above.
My intent here is not to focus on the technical details of the hardware and software but to cover the results obtained by the device,
therefore I will briefly outline some technical details in the next section, which can be skipped over without loss of continuity on the topic.
This project is a work in progress but I have gone far enough with it to publish some of the details to date.- 12/22/2007
Originally I started a few temperature data logging experiments with a digital voltmeter(DVM) that had a serial output. This allowed extended logging of
temperature data to a PC via the serial port by measuring the resistance of a thermistor directly. The DVM was a Radio Shack type with a program called
Meter View that came with it. The DVM hooks up to the computer while Meter View runs, logging data to a file. Once the data is in the file, it can be
imported to a spreadsheet. From this point the spreadsheet contains an algorithm that converts the thermistor resistance to temperature.
Another way to log temperature that I used was to put the thermistor in series with a voltage source connected to the sound card of the PC.
By using the line level input it is possible on some sound cards to read DC voltage values directly. It all depends on how the A to D converter is
hooked up to the sound card. The beauty of this method is that you just record the values using sound recording software. The drawback is that the
sample rate is so high that the sound files grow large in size quickly. This method would be best suited for temperature events that span minutes and
not days.
Using the sound card method it is possible to write software that will extract the data from the sound file and convert it to another format that can be
read by a spreadsheet or some other post processing software. The conversion is basically from a binary file with hexadecimal data that represents the
voltage levels to the format that the user requires, such as CSV(comma separated variable) text. The samples will occur at a rate that the sound recording
program is set to. Common values are 4000,8000,10050,16000...up to 44100 samples per second. Once again this is a tremendously high sampling rate for
temperature recording, but it is possible to do.
Micro-controller based weather logging
The basic concept of it is to use a micro-controller board with an analog to digital(A to D) input and a serial output. Hooked up to the A to D input line
is a sensitive thermistor. Inside the micro-controller there is software that reads the input and produces an output message on the serial port for a PC
to read once per second. The message protocol would be based off of the protocol used to control a transceiver, a modified CAT command protocol.
The message would consist of a time stamp and the current temperature.
The time stamp comes from the internal real time clock of the micro-controller and the temperature comes from the thermistor converted to a digital
output and then scaled using an algorithm to map the thermistors temperature to resistance map into a digital value to be transmitted across the bus.
It is also possible to log the data internally within the micro-controller RAM. The time stamp and the temperature take up only 4 bytes so it is possible
to log a few thousand records internally and then perform a 'dump' of the values to the PC. With 60 minutes in an hour and 24 hours per day 1440 records
are created per day. To dump the memory to the PC, a command is sent across the serial port to the micro-controller that instructs it to send all of the
recorded time stamp and temperature data to the PC via the serial bus. During this process the PC will echo the data back to the micro controller which
will acknowledge that a match has occurred or it will resend the data again if it is corrupted. This is a rather robust method of ensuring that no
data is lost in the communications process. Once again the protocol is much the same as the way the CAT control program sends its commands and returns
a status. This logging ability makes the device work a lot like the commercially available Hobo data loggers.
A way to extend the devices data logging ability over long periods of time,would be to set the sample rate lower, to something like once every 10 minutes
or an hour. Another possibility would be to use a small internal buffer lets say of 10 samples representing 10 minutes of temperature data.
A statistical operation such as averaging could be performed on it and then the result could be stored. In this way every ten minutes an average of
the previous ten minutes of temperature is stored. Because the data is time stamped, there is no dependance on sequencing. It would be possible to set
some threshold,say +/- one degree and only store records that change by this amount. This would save a great deal of memory when the temperature remains
static for long periods of time. When the data is downloaded it is always possible to reconstruct it because of the time stamp. An algorithm could
post process the data filling in the missing slots in time with the previous data knowing that it has not changed.
In this way the data can be converted back to a form usable by a sequential spreadsheet.
Finally of course is the fact that the time stamp only goes up to 23:59:59, which can be compressed into 2 bytes using
Radix50 conversion. The same conversion could apply to the data as well compression the MM/DD/YY
format into just 2 bytes.
Micro-controllers typically have multiple inputs, so it is possible to log multiple temperatures simultaneously with resorting to a multiplexing scheme.
A lot of data loggers claim multiple inputs, but these are not always simultaneous. Sometimes a round robin type of multiplexor is used on some loggers.
This device could also be used to measure wind speed and direction. Using a shaft encoder to generated pulses related to position would work for both
the speed and direction. The speed sensor pulses could be fed directly to a special micro-controller interrupt routine that would measure the time
difference in microseconds between pulses and from that output use an algorithm to derive speed. Wind direction could once again use a shaft encoder,
however this time the micro-controller would have to use so called quadrature decoding of the pulses. Quadrature decoding means that 2 sets of pulses
are generated from the encoder with a 90 degree phase angle between them. This makes it possible using a interrupt driven timer and a state machine to
derive not only position but direction as well. The best setup for wind direction would also have a switch (or some other noncontact, magnetic sensor)
to provide an absolute reference at true north. By having one fixed reference, direction is always calibrated against true North. The wind speed and
direction inputs would connect to digital inputs on the micro-controller, these are ones that sense 0 or +5, switch type inputs and pulses.
To measure relative humidity a capacitance can be used where the dielectric constant of the material changes due to the presence of moisture in the
air. A digital pulse train would be sent to the capacitor. When the pulse turns on then off, the capacitor will drain through a resistance that is in
the circuit. A time after the pulse is removed the voltage will be measured, the voltage will have a relationship to the capacitance and therefore
the humidity via the RC time constant. The constant is such that after T = RC, time in seconds, R in Ohms and C in Farads the voltage appearing at the
terminals will be 1/e of the original charging voltage.
For barometric pressure there are devices that can be obtained to measure it electronically as well. The price of these devices must be falling because
there are device capable of reading barometric pressure electronically that are < $50 at this point(2007). I am sure that a device in an IC package could
be obtained readily.
Data via Radio Modem- I have seen in the past where data can be sent via radio links. Specifically, amateur packet radio. It would be possible to use a similar concept for the weather data logging. The object of this would be to format the data into a fixed protocol that would be transmitted via a radio modem from various points. It would be possible to form a mesh network as well. In this method the radio/micro-controller units would work in concert to forward the data from one unit to the next until it arrives at the final collection point. From this point it is only a matter of data post processing to create something like superimposing the data on a real time map for instance.
Final Thoughts- Obviously weather prediction is based on models. It is a type of finite element analysis. Increasing the density of data that
the model uses will increase accuracy to a great degree. This is mathematically equivalent to adding more decimal places in a recursive function. The more
decimal place the more recursions before the model diverges ( sometimes chaotically) from reality. This is all related to the folding the dough problem,
butterfly effects, etc... But, back to the point which is that we want to get as close to measuring the butterflies as possible. Ideally with the
current technology it would be possible to mount a small robust and reliable weather station on every cell tower. Also it would be possible to provide
portable units of the same nature to agencies to deploy near weather events of interest in order to increase the amount of data to feed into weather
prediction models. The units would use the present infrastructure of the cellular phone system as the communications hub.
If the micro-controller in each of these units were lets say supercharged a bit, lets say they used a decent chip. I am thinking either DSP or a dual
core with one being a math co-processor chip. It may be possible to do distributed computing of the weather model. What I mean is, in effect each of the
weather stations are doing a small piece of the algorithms for the entire weather model.
This leverages thousands of small computers each with the computing power of a graphing calculator, gaming system or network router. Actually it would
be closer to a melding of parts of all three.
The main computer then just does the final gather and assimilation of the model data. That would make for a very powerful system.
The fact that there is enough redundancy in the system (if the data is stored/forwarded in the proper manner) makes it a very robust system that would
be able to tolerate dropouts and faults in small sections of the system without losing the ability to model well.
A few examples
I intend to add a few data logging examples here.
Original Build Date:12-01-2007
Last updated 01-20-2008