I recently found out that Telstra has a SMS API, and for now, it’s free to use for up to 1000 SMSes a month for Australians (sorry rest of the world).
As I’ve been building up Zabbix for our work environment where SMSes are used regularly to alert techs to critical alerts, this piqued my interest as this means that for now, I could have SMS alerts for my homelab, and potentially implement the same solution at work.
Continue reading »
In the previous post, I got the Raspberry Pi outputting the values from the temperature sensors via SNMP.
In this post, I’ll be configuring Zabbix to monitor and record these values so that I can see how cold it was last night.
You’ll need to note down the OIDs that were configured on the SNMP agent in the last post so you can use them in this post.
The Raspberry Pi will need to be monitored in some way by Zabbix in order for us to record the temperatures from the SNMP agent that we’ve set up.
You’ll need to add the Raspberry Pi as a host in Zabbix, and give it an IP address under the SNMP interfaces section. I’ve added the SNMP OS Linux template to monitor the rest of the Pi stats, but that’s not required for monitoring temperatures.
So I got a bit sidetracked from my tear down of the Ritmo CE-3590 NAS, and ended up building a temperature sensor with some DS18B20s and a Raspberry Pi.
Today’s post will detail how I configured the Pi to output the temperature values to the SNMP agent that I’m running on the Pi.
Since I used the DS18B20s, they are polled over a single wire with a 4K7 resistor across the VCC and Data wires.
I ended up using a whole bunch of CD Audio cables like this one –
to create the wiring for the sensors. If I had the pins to use between the cables, I could have just joined cables together, but as I didn’t, I cut and soldered multiple cables together. At the end of the wires that will plug into the Pi, I spliced in the 4K7 resistor and heatshrinked it so hopefully it won’t short anything out.
It ended up very messy, but it seems to have worked.
I used the Red wire for 3.3V, the white wire as data, and the black as ground.
Red then gets plugged into the Pi header at Pin 1 which is 3.3V power
White gets plugged into Pin 7 which is GPIO 4
Black gets plugged into Pin 6, which is a ground. This could be any of the other grounds, but this is the closest one.
I’m using Raspbian on my Pi, so any instructions are done with a Raspbian flavour. Most distros should have the same setup though.
The Pi needs to be configured to use the w1-gpio device tree overlay (dtoverlay).
This is done by adding the line dtoverlay=w1-gpio to the end of the config.txt file on the /boot partition.
Once this dtoverlay has been activated, you should be able to see your sensors under /sys/devices/w1_bus_master1/. They should appear as 28-0215**. On my Pi, the sensors appear like so –
/sys/devices/w1_bus_master1/28-0215012018ff
/sys/devices/w1_bus_master1/28-021500ce50ff
/sys/devices/w1_bus_master1/28-021500cfceff
Catting one of the w1_slave nodes under the sensor should output something like this –
57 01 55 00 7f ff 0c 10 05 : crc=05 YES
57 01 55 00 7f ff 0c 10 05 t=21437
The YES confirms the sensor is working, and the temperature is displayed after the t=
In this example, the temperature is 21.437 Degrees Celsius. The output needs to be divided by 1000 to get the temperature to the correct amount of decimal places.
With the sensors plugged in, it’s time to extend the SNMP agent to allow us to poll the temperature sensors over SNMP.
I’ve written a script that will output just the temperature to the SNMP agent, I’ve placed this script in /opt/scripts but you’re free to put it wherever you want. You’ll need to keep the location handy for when we extend the SNMP agent.
I’ve named this script gettemp.sh
#!/bin/bash
#
# Usage: gettemp.sh
# e.g. gettemp.sh 28-0215012018ff
SENSOR=$1
SLAVE="/sys/devices/w1_bus_master1/"$SENSOR"/w1_slave"
OUTPUT=$(/bin/cat $SLAVE | /usr/bin/awk -F 't=' ' { printf $2 } ')
echo $OUTPUT
With this script, you should be able to call the script from a command line, and it should output the temperature for that sensor.
# /opt/scripts/gettemp.sh 28-021500ce50ff
21437
With this script operational, we’ll need to setup the SNMP agent to run this script to provide us the output of this script over SNMP.
To do this, we need to modify /etc/snmp/snmpd.conf. At the end of the file, put in this line –
Make sure you replace the sensor id with your own sensor id. If you want to monitor more than one sensor, add more extend lines, and give them differing names, like temp2, temp3, etc.
extend .1.3.6.1.3.1.1 temp1 /bin/bash /opt/scripts/gettemp.sh 28-021500ce50ff
After you modify the snmpd.conf file, make sure you restart SNMP agent with /etc/init.d/snmpd restart or service snmpd restart.
Once the SNMP agent is back up and running, you should be able to poll the OID we specified above (.1.3.6.1.3.1.1) and get the output of the command. I’ve run the snmpwalk on the Pi itself, polling itself.
# snmpwalk -v 2c -c public 127.0.0.1 .1.3.6.1.3.1.1
iso.3.6.1.3.1.1.1.0 = INTEGER: 1
iso.3.6.1.3.1.1.2.1.2.5.116.101.109.112.49 = STRING: "/bin/bash"
iso.3.6.1.3.1.1.2.1.3.5.116.101.109.112.49 = STRING: "/opt/scripts/gettemp.sh 28-021500ce50ff"
iso.3.6.1.3.1.1.2.1.4.5.116.101.109.112.49 = ""
iso.3.6.1.3.1.1.2.1.5.5.116.101.109.112.49 = INTEGER: 5
iso.3.6.1.3.1.1.2.1.6.5.116.101.109.112.49 = INTEGER: 1
iso.3.6.1.3.1.1.2.1.7.5.116.101.109.112.49 = INTEGER: 1
iso.3.6.1.3.1.1.2.1.20.5.116.101.109.112.49 = INTEGER: 4
iso.3.6.1.3.1.1.2.1.21.5.116.101.109.112.49 = INTEGER: 1
iso.3.6.1.3.1.1.3.1.1.5.116.101.109.112.49 = STRING: "21437"
iso.3.6.1.3.1.1.3.1.2.5.116.101.109.112.49 = STRING: "21437"
iso.3.6.1.3.1.1.3.1.3.5.116.101.109.112.49 = INTEGER: 1
iso.3.6.1.3.1.1.3.1.4.5.116.101.109.112.49 = INTEGER: 0
iso.3.6.1.3.1.1.4.1.2.5.116.101.109.112.49.1 = STRING: "21437"
As you can see from the example, the temperature is output in a couple of different places.
Polling the SNMP agent with one of those locations directly should yield only the temperature.
# snmpwalk -v 2c -c public 127.0.0.1 .1.3.6.1.3.1.1.4
iso.3.6.1.3.1.1.4.1.2.5.116.101.109.112.49.1 = STRING: "21437"
With that, we now have SNMP set up on the Pi, ready to be polled for the temperature.
I needed to get the average availability for a group of servers for some work that I was doing at my day job
These servers were monitored via both SNMP and ICMP, but I only needed to know if they were up or not.
To do that, I created a dummy host so I could set up items on it just for reporting purposes.
On this host, I created a new item of type “Zabbix aggregate”.
For the key, I used the grpavg[] function to average out the ICMP reachability metric for the group of servers. I set the time range for the last day to calculate
grpavg["Server Group","icmpping",last,1d]
I also set a % for the units, and the custom multiplier to 100. The result is a calculation of the average number of servers that are up for the period, in this case I’ve used the last 1 day.
Graphing this would then produce a graph of the average availability of the group “Server Group” for a day.
For my current task at work – moving away from CA Spectrum to Zabbix – I’ve had to integrate NMIS with Zabbix for SMS alerting as we wanted all alerts to filter through Zabbix for easy tracking of SMSes, and also for the ServiceNow integration functions that I have built into my Zabbix instance.
To build the integration between NMIS and Zabbix, I had to create a custom script that NMIS would call which would then send an SNMP trap to Zabbix for processing.
On the Zabbix side, I needed to set up snmptt to process the traps so that Zabbix will recognise them as traps and action them as alerts.
The following sections will detail the bits I’ve setup to get this integration working
This script is what NMIS calls when an alert is generated.
I’ve named this script “snmptrap.pm”, it’s based off the example script in /usr/local/nmis8/lib/Notify on the appliance.
If you name it something different, make sure you update the relevant lines in the script file.
package Notify::snmptrap; ## Update this if you change the name
require 5;
use strict;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
use Exporter;
use JSON::XS;
use File::Path;
use Net::SNMP;
$VERSION = 1.00;
@ISA = qw(Exporter);
@EXPORT = qw(
snmptrap ## Update this if you change the name
);
@EXPORT_OK = qw( );
my $dir = "/tmp/customsnmptrap"; ## This is the log directory, and can also be changed
sub sendNotification {
my %arg = @_;
my $contact = $arg{contact};
my $event = $arg{event};
my $message = $arg{message};
my $trapdestination = "10.0.0.5"; ## This should be your Zabbix server IP Address
my $trapcommunity = "public"; # Use your community string here
my $oid = "1.3.6.1.4.1.4818.1"; # I've used the OPMANTEK MIB OID here, but you can use your own if you want
if ( not -d $dir ) {
my $permission = "0770";
my $umask = umask(0);
mkpath($dir,{verbose => 0, mode => oct($permission)});
umask($umask);
}
# add the time now to the event data.
$event->{time} = time;
$event->{email} = $contact->{Email};
$event->{mobile} = $contact->{Mobile};
my ($sess, $err) = Net::SNMP->session(
-hostname => $trapdestination,
-version => 1, #trap() requires v1
-port => 162
);
if (!defined $sess) {
print "Error connecting to target ". $trapdestination . ": ". $err;
next;
}
my @vars = qw();
my $varcounter = 1;
push (@vars, $oid . '.' . $varcounter);
push (@vars, OCTET_STRING);
# This is where you set up the variables for the SNMP Trap message
push (@vars,$event->{level}.' : '.$event->{node}.' : '.$event->{element}.' : '.$event->{event});
my $result = $sess->trap(
-varbindlist => \@vars,
-enterprise => $oid,
-specifictrap => 1,
);
if (! $result)
{
print "An error occurred sending the trap: " . $sess->error();
}
my $fcount = 1;
my $file ="$dir/$event->{startdate}-$fcount.json";
while ( -f $file ) {
++$fcount;
$file ="$dir/$event->{startdate}-$fcount.json";
}
my $mylog;
$mylog->{contact} = $contact;
$mylog->{event} = $event;
$mylog->{message} = $message;
open(LOG,">$file") or logMsg("ERROR, can not write to $file");
print LOG JSON::XS->new->pretty(1)->encode($mylog);
close LOG;
# good to set permissions on file.....
}
1;
Now NMIS needs to be setup to call the new snmptrap.pm script when an alert is generated.
To setup NMIS, you’ll need to locate the escalation that requires the SNMP trap.
Open the escalations table by going to Setup --> Emails, Notifications and Escalation.
In the escalation table, find the alert that you need SNMP traps for, and add this to the escalation level that requires traps –
snmptrap:Contact – Replace Contact with a contact in NMIS.
Once NMIS has been setup to send traps to the Zabbix server, some configuration needs to be done on the Zabbix side.
You’ll need to set up SNMPTT to receive SNMP traps from the OID you’re using in the snmptrap script.
In this example, I’m just going to set up a catchall to catch the SNMP traps, however you can use SNMPTT to parse different traps and generate different alerts.
Create a file in /etc/snmp called snmptt.conf.catch
In that file, put the following lines in
EVENT general .* "SNMP Catchall" Normal
FORMAT ZBXTRAP $aA $ar
You’ll also need to modify /etc/snmp/snmptt.ini to add the newly created file to the snmptt_conf_files configuration variable.
This can be done by appending the path to the list like this –
snmptt_conf_files = <<END
/etc/snmp/snmptt.conf.catch
END
If there are already lines there, then add the line to the block of text right before the last END
I’ve used the basic snmptrap.fallback method to catch all traps, but you can set up specific alerts in SNMPTT to generate different messages.
On the NMIS server, add an item of type SNMP Trap, and with a key of snmptrap.fallback.
This item will now get any SNMP traps from the Server, you can create a trigger if required to alarm on the SNMP traps, or just keep them for history.
