I use a couple Tilt Hydrometers to track and log the fermentation data for my beers. While there are a few different ways that one can upload the data to the internet, I noticed that there wasn’t an automated way to upload to Azure Log Analytics. TiltPi is built on a customized variant of Raspbian. The current version utlizes Raspbian Stretch, which in turn is based on Debian Stretch. Microsoft supports Linux and has an OMS agent available for both x86 and x64 variants of Linux. They don’t, however, currently support the ARM platform which the Raspberry Pi boards are built on. Attempting to install the OMS Agent via the script from GitHub fails. While the agent is built on Ruby and Fluentd which are both compatable with ARM64, other components such as omi are not. If all you need is syslogs, there is a fluentd plugin for OMS which Microsoft supports. Unfortunately, the Tilt app does not write to the syslog. It writes to it’s own CSV formatted log. That leaves us with two options: write a daemon to forward new entries to syslog or utilize the cloudurl and write the data directly to Log Analytics using the REST API. The first sounded like more effort than I wanted to put into this project and would likely be purpose built for this and this alone. The second method though, sounded a bit easier, and more flexible. Thankfully, Microsoft has a nice bit of documentation regarding the HTTP Data Collector API on their docs website.
I was able to use this information to build a framework for writing data directly to Azure via a PHP script. Microsoft’s documentation thankfully has all the information we need. The examples helped with some of the basic pieces that I needed to get going, however, I did have to do a bit of playing around to get the signature to work since translating from one language to another is not always straight forward. .NET for example requires byte arrays when calling encode/decode/hash functions whereas PHP just wants the raw string. When calling the API, part of the HTTP header is a Signature that authorizes you and tells Log Analytics where to put the data. The basic formula is base64_encode(sha256_hmac(base64_decode(<SharedKey>), <HashData>)). Let’s break that down. First, we need to create our HashData. This is a string formatted as follows: <SendMethod>\n<PayloadLength>\n<ContentType>\nx-ms-date:<RFC1123Date>\n<SendResource>. All of these are detailed in the documentation, but essentially we have “POST\n<PayloadLength>\napplication/json\nx-ms-date: <RFC1123Date>\n/api/logs” where <PayloadLength> is the length of our data we are going to send and <RFC1123Date> is the current date/time in RFC 1123 format. Once we have that string, we need to ensure that it is encoded as utf8. Once we have the utf8 encoded version of the string, we pass that to hash_hmac and have it create a sha256 hash using that and our base64 decoded key. We then do a base64_encode on the resulting hash and create our signature; “SharedKey <WorkspaceID>:<EncodedHash>”. After we have that, we can use the curl libraries to send the data to OMS. The code for our function looks like this:
public function Send_Log_Analytics_Data($Payload) {
#Get current timestamp (UTC) and Payload length
$Rfc1123Date = gmdate('D, d M Y H:i:s T');
$PayloadLength = strlen($Payload);
#Generate the Signature
$StringToSign = "{$this->SendMethod}\n{$PayloadLength}\n{$this->SendContentType}\nx-ms-date:{$Rfc1123Date}\n{$this->SendResource}";
$Utf8String = utf8_encode($StringToSign);
$Hash = hash_hmac('sha256', $Utf8String, base64_decode($this->PrimaryKey), true);
$EncodedHash = base64_encode($Hash);
$Signature = "SharedKey {$this->WorkspaceId}:{$EncodedHash}";
#Build the URI and headers for cURL
$Uri = "https://{$this->WorkspaceId}.ods.opinsights.azure.com{$this->SendResource}?api-version={$this->SendApiVersion}";
$Handle = curl_init($Uri);
$Headers = array("Authorization: {$Signature}", "Log-Type: {$this->LogType}", "x-ms-date: {$Rfc1123Date}", "time-generated-field: {$this->TimeStampField}", "Content-Type: {$this->SendContentType}");
#Set cURL Options
curl_setopt($Handle, CURLOPT_POST, true);
curl_setopt($Handle, CURLOPT_HEADER, true) or die("Failed to set cURL Header");
curl_setopt($Handle, CURLOPT_HTTPHEADER, $Headers) or die("Failed to set cURL Header");
curl_setopt($Handle, CURLOPT_POSTFIELDS, $Payload);
curl_setopt($Handle, CURLOPT_RETURNTRANSFER, true) or die("Failed to set cURL Return Transfer");
#Execute cURL and return the result
$Result = curl_exec($Handle);
return curl_getinfo($Handle, CURLINFO_HTTP_CODE);
}
Now that we can send data, what data are we sending? The Tilt app sends the following pieces of information to whatever cloud url you specify in the logging tab:
- Timepoint (Excel datetime value)
- Temp (deg F)
- SG (Specific Gravity)
- Beer (Name of the Beer)
- Color (Color of the Tilt)
- Comment (blank unless you manually send data and add a comment)
We can take this post data and create our payload by converting it into a JSON formatted string.
$TimeStamp = gmdate('Y-m-d\TH:i:s.v\Z');
$Entry = "[{\"Timepoint\": \"{$TimeStamp}\", \"Temp\": {$Temp}, \"SG\": {$SG}, \"Beer\": \"{$Beer}\", \"Color\": \"{$Color}\", \"Comment\": \"{$Comment}\",\"Source\": \"TiltPHP\"}]";
Since we’re doing this in real time as data is sent, we are ignoring the timestamp that the Tilt app has created and instead using our own. This lets us skip having to convert the Excel formatted date into one that Log Analytics can use. If you need to use the Excel date, it’s just the number of days that have passed since 12:00AM 1900-01-01. The entry we created above is what we pass to the Send_Log_Analytics_Data as the $Payload parameter. And now, we can sit down, have a beer, and watch data flow into Log Analytics. If you’re interested in looking at the full code, I’ve posted it up on my GitHub space. Soon I’ll write something up on how to do something with that data 😉