May 182015

One of my current projects at work is to build up Zabbix as an alerting solution.
This includes using Zabbix to raise incidents in ServiceNow for any alerts that come through.

Initially, I thought that I would need to do lots of scripting, but it turns out I only had to write a simple script to allow Zabbix to raise incidents in ServiceNow.
This was largely thanks to pre-built Python modules that the community have built to allow easy integrations, namely ZabbixAPI and Python-Servicenow.

These 2 libraries made integration easy because they allowed me to concentrate more on the code to join the 2 systems rather than having to figure out how to make them talk to Python, and then talk to each other.

The integration itself will take an alert that is generated by Zabbix and insert the data into ServiceNow as an incident.

Configuring Zabbix

  1. Copy the snippet at the bottom of this post into a file in /usr/lib/zabbix/alertscripts. I’ve named mine
  2. Create a new Media Type.
    Create a new media type by going to Administration => Media types, and click on Create media type.
    Give the Media Type a name, select Script for the Type, and put in the name of the file. Using my example, it would be
  3. Assign the new Media Type to a user.
    The user’s “Send to” for the media type will define the assignment group in ServiceNow.
    Click on Administration => Users, select Users in the drop down on the top right, and click on a user. Once you’re in the user’s configuration page, click on the Media tab, and then click on ‘Add’.
    Select the new Media Type from the type drop down, and then enter in the ServiceNow Assignment Group in the ‘Send To’ box, and click ‘Add’.
  4. Create or modify an existing action to start sending incidents to ServiceNow.
    Click on Configuration => Actions, and open up an Action. Click on the Operations Tab, and click on “New”.
    Click on “Add” in the Send To Users section, and choose the user that has the ServiceNow Media type set up.
    Select the ServiceNow action in the “Send only to” box, and then click on the “Add” button to add the action to the list of actions.
    Once the action has been setup, click on Save.

Once the alert is all set up, whenever the alert is triggered, the script should log an incident directly into ServiceNow.

The below code should be copied and pasted into a file to be used as the script for the Media Type.

import zapi
import datetime
import sys
import urllib2
import os
import servicenow.Connection
import servicenow.ServiceNow

## I've used logging for my own setup, but I've commented it out so that it won't spam a log file unless you uncomment it.
## Just make sure the location that you're storing the logfile is writable by the zabbix user
## In this example, I've used /usr/lib/zabbix/logfiles but this could be anywhere writable by the zabbix user
#f = open('/usr/lib/zabbix/logfiles/snow.log','a')
#f.write('\n\nScript Start :: ''\n\n')

## Zabbix Passes the details via command line arguments.
assignmentgroup = sys.argv[1]
description = sys.argv[2]
detail = sys.argv[3]

## Set Up your Zabbix details
zabbixsrv = ""
zabbixun = "Admin"
zabbixpw = "zabbix"

## Set up your ServiceNow instance details
## For Dublin+ instances, connect using JSONv2, otherwise use JSON
username = "username"
password = "password"
instance = "instance"
api = "JSONv2"

## I've configured Zabbix to only pass the Event ID in the message body.
## If you want more detail in the body of the incident in ServiceNow, you'll need to make sure that eventid is parsed out of detail correctly.
eventid = detail

#f.write('trying to connect to servicenow\n')
conn = servicenow.Connection.Auth(username=username,password=password,instance=instance, api=api)
print "Error Connecting to ServiceNow\n"
#f.write("Error Connecting to ServiceNow\n")

#f.write('trying to create incident instance\n')
inc = servicenow.ServiceNow.Incident(conn)
print "Error creating incident instance\n"
#f.write("Error creating incident instance\n")

#f.write('trying to create new incident\n')

## This is where the fun starts.
## You'll need to set up the following section with the correct form fields, as well as the default values
newinc = servicenow.ServiceNow.Incident.create(inc, { \
"short_description":description, \
"description":detail, \
"priority":"3", \
"u_requestor":"autoalert", \
"u_contact_type":"Auto Monitoring", \
"assignment_group": assignmentgroup})
except Exception as e:
print "Error creating new incident in ServiceNow\n"
print str(e)
#f.write("Error creating new incident in ServiceNow\n")

## This script will retrieve the new incident number from servicenow and put it back into zabbix as an acknowledgement
newincno = newinc["records"][0]["number"]
print "unable to retrieve new incident number\n"
#f.write("unable to retrieve new incident number\n")

zabbix = zapi.ZabbixAPI(url='http://'+zabbixsrv+'/zabbix',user=zabbixun,password=zabbixpw)
#f.write('Acknowledging event '+eventid+'\n')

#f.write('\n\nScript End :: ''\n\n')

Apr 302014

I have Spectrum integrated with Service Now in order to raise Incidents automatically when certain critical alerts come through on CA Spectrum

As a pre-requisite, Spectrum needs a user configured in Service Now with the ability to create records in the Incident table.

  1. Spectrum Configuration
  2. I have alarm notifiers setup to catch events that require an automated incident. The alarm notifier SetScript then calls a custom perl script in order to send the web services request to Service Now. The alarm notifiers are configured

    The SetScript has also been modified to parse out some extra parameters from the policy’s Notification Data. The notification data is accessed from the script through the variable “$NOTIFDATA”.
    I use a comma as a delimiter in the notification data to create fields, so the notification data within the policy looks like this

    Network Team,Network,Infrastructure

    The NOTIFDATA variable is expanded, and then assigned a variable inside the SetScript with the following bash script snippet

    declare -a ENOTIFDATA
    for x in $NOTIFDATA; do
    ENOTIFDATA=( "${ENOTIFDATA[@]}" "$x" )


    The SetScript then calls the Perl script with some arguments. The perl script will make the calls to Service Now.

    $SPECROOT/custom/scripts/SNow/ $AID "$AssignmentGroup" "Autogenerated - A $SEV alarm has occurred on $MNAME" "$EVENTMSG" "$Category" "$SubCat" "$SEV"

  3. Perl Script
  4. The Perl script is built from examples available from ServiceNow.

    The following example will need to be modified to suit your environment – specifically the parameters as different organisations will have different configurations for ServiceNow.


    #use lib '/usr/lib/perl5/custom';

    # declare usage of SOAP::Lite

    use SOAP::Lite;
    use feature 'switch';

    # specifying this subroutine, causes basic auth to use
    # its credentials when challenged
    sub SOAP::Transport::HTTP::Client::get_basic_credentials {
    # login as the itil user

    return 'spectrum_user' => 'spectrum_password';

    # declare the SOAP endpoint here
    my $soap = SOAP::Lite
    -> proxy('');

    # calling the insert function
    my $method = SOAP::Data->name('insert')
    ->attr({xmlns => ''});

    # create a new incident with the following short_description and category
    my @params = ( SOAP::Data->name(short_description => $ARGV[2]) );
    push(@params, SOAP::Data->name(u_requestor => 'Spectrum 9.3') );
    push(@params, SOAP::Data->name(contact_type => 'Auto Monitoring') );
    push(@params, SOAP::Data->name(description => $ARGV[3]) );
    push(@params, SOAP::Data->name(u_business_service => $ARGV[4]) );
    push(@params, SOAP::Data->name(assignment_group => $ARGV[1] ) );

    given ($ARGV[6]) {
    when ("MINOR") {
    push(@params, SOAP::Data->name(urgency => '3') );
    when ("MAJOR") {
    push(@params, SOAP::Data->name(urgency => '3') );
    when ("CRITICAL") {
    push(@params, SOAP::Data->name(urgency => '3') );
    default {
    push(@params, SOAP::Data->name(urgency => '3') );

    # invoke the SOAP call
    my $result = $soap->call($method => @params);

    # print any SOAP faults that get returned
    if ($result->fault) {
    exec 'echo Incident Raising Error. Please check spectrum logs | mail -h smtp.dmz.localnet -s "Issue Raising Incident"';
    # print the SOAP response that get return

    # convenient subroutine for printing all results
    sub print_result {
    my ($result) = @_;

    # convenient subroutine for printing all SOAP faults
    sub print_fault {
    my ($result) = @_;

    if ($result->fault) {
    print "faultcode=" . $result->fault->{'faultcode'} . "\n";
    print "faultstring=" . $result->fault->{'faultstring'} . "\n";
    print "detail=" . $result->fault->{'detail'} . "\n";