Incident Creation Rules
When alerts are received by ServiceNow they can be raised as Incidents. Rules set out the process for what gets raised as an Incident and the properties of the created Incident.
There are two types of rule that govern how incoming SCOM alerts are handled:
- Incident Creation Rules - These are the most common type of rule. They contain criteria that an incoming SCOM alert must match for an Incident to be raised and set what properties the Incident that is raised should have (assignment group for example). They are prioritized against each other.
- Wait Rules - New in our ServiceNow store app version 1.1.8, Wait rules allow an incoming SCOM alert to be held for a specified period of time before being evaluated against Incident creation rules. They are useful for cases where a SCOM alert might open and close itself in quick succession (EG a CPU usage threshold monitor) where you don't want an Incident for SCOM alerts that have closed themselves before someone can take action. More on wait rules, including how to use them with Incident Creation Rules here.
Creating your first Incident Creation Rule
- Log in to ServiceNow, search for "Cookdown" and select Incident Creation Rules
- Select New
- The following screen will look like the screenshot below. Input name, CI to map to, Processing Order, Assignment Group, some Incident Creation Criteria and make the rule Active
Incident Creation Rule simple properties
Name: The name of the Incident creation rule you are creating.
Automatically map CI: If you have our Discovery product, rather than mapping Incidents raised from an Incident Creation Rule to a single static CI you can choose for alerts to be mapped to the correct CI in your CMDB for the Incident. More on hos to use this below.
Configuration Item: If you don’t check the Automatically map CI box, this is the CI that Incidents created will be raised against for all Incidents created by hitting this Incident Creation Rule.
Contact Type: From store app version 1.2.2 you can use Contact Type to set the corresponding incident field.
Alert Condition: The condition that an incoming SCOM alert must meet to match this rule. More on how to build conditions below.
Description: A friendly description of the Incident Creation Rule.
Active: Specifies whether this Incident Creation Rule is active or not (default is inactive).
Processing order: The order that this Incident Creation Rule is processed. Lower numbered rules are processed first (IE 1 is the highest priority) and ServiceNow will not process lower priority Incident Creation Rules once it has found a match on a higher priority one.
Assignment Group: The group that created Incidents from this rule are assigned to.
Incident Creation Rule Alert Conditions
Incident Creation Rules contain Criteria that are used to match Alerts based. They use ServiceNow's standard picker and so can be created using and (IE create Incident if the Alert matches A and B )/or (IE create Incident if the Alert matches A or B) clauses.
The fields available for selection in the criteria builder are those imported from SCOM.
The criteria available are
- Starts with
- Ends with
- Does not contain
- Is not
- Is empty
- Is not empty
- Matches pattern
- Matches regex
- Is anything
- Is one of
- Is empty string
- Less than or is
- Greater than or is
- Is same
- Is different
Automatically Mapping CIs
If you have both our Alert Sync and Discovery products we can automatically map Incidents we raise from SCOM alerts to the correct CI. This process works by populating a field on each CI with its SCOM object ID using our discovery tool, which exposes the SCOM Object IDs themselves, and then checking each incoming SCOM alert against the SCOM Object Ids of discovered CIs.
Setting up automatic CI mapping
To set up this capability, firstly decide which field in your CMDB to store the SCOM object ID in, populate the field with Cookdown Discovery and configure Alert Sync to check this field for SCOM Object ID. For the below worked example we have used the "correlation_id" field for this purpose.
Using the correlation_id field is acceptable in ServiceNow instances apart from where ServiceNow's SCCM connector is in use (as it uses these fields for its own IDs) or any other discovery tool that populates this field
- Agree the field to be used for automatic CI mapping - in this example we use correlation_id
Populate the agreed field using Cookdown Discovery. The see the below sample Discovery payload for an example for populating Active Directory Controllers. Read more on Cookdown Discovery payloads:
Configure Alert Sync to look at the populated field
If the field you have chosen to populate contains data other than a single SCOM Object Id check the "Allow contains for SCOM Object Id Search" box
Use of the contains search is more load intensive on your ServiceNow instance so while we provide this functionality, we recommend you store the SCOM Id in a field by itself
In each Incident Creation Rule that you wish to use automatic CI mapping with, check the Automatically map CI checkbox and provide a fallback CI. The Fallback CI is used where automatic CI mapping can find no CI in your CMDB with a SCOM object corresponding to the SCOM alert that the Incident Creation Rule is evaluating.
- Note that you can use automatic CI mapping for a subset of your Incident Creation Rules
Building Incident Creation Rules
Typically, you will want to raise Incidents against the correct CI and Assignment Group so they are handled as quickly as possible by the most appropriate team, which means that you will want an Incident Creation Rule for each set of "like Alerts".
We recommend a rule for each set of unique CI/Assignment Group pairings your company might need and a backstop rule that creates Incidents for triage. Some of our customers have these Incidents triaged by a first-line support team and re-routed short term while the list of Incident Creation Rules are built out. Others use their SCOM team to triage incidents raised against this backstop rule as an untuned SCOM environment can generate a large number of unwanted Alerts (and therefore Incidents) making your SCOM Admins the best placed to tune SCOM and prevent the generation of these Alerts (have a look at our free Easy Tune product which can help with this).
The result of this exercise will be a list of Incident Creation Rules that will create incidents and handle routing them to the correct team. To illustrate this point here is a simple example:
The SQL Team are separate from the rest of IT and need to handle SQL related Incidents, they will want created Incidents raised against a SQL CI. All other Incidents are routed to a first-line Infrastructure team who try to fix the most simple issues and triage/re-assigned Incidents they cannot address themselves.
In the Incident Creation UI the resultant rules will look like this:
The criteria of each Rule will be as follows:
Advanced Incident Creation Rules and scripting
The use of Advanced creation scripts overrides the default behavior of the product and the alert criteria method of specifying Incident Creation Rules conditions. The script-based options are set on a per Incident Creation Rule basis, so it is possible to use the standard criteria based method for some rules while scripting other rules. Also within a single Incident Creation Rule, it is possible to script the create and update behaviors separately from each other, so you can configure a rule with an advanced update use case while using the basic criteria-based system for creation.
There are 3 possible scripts that can be configured which are on the Advanced Creation and Advanced Updates tabs:
- Advanced Creation
- This script runs when a new SCOM alert comes into the ServiceNow instance. You can use this option to script Incident Creation behavior, to re-open resolved incidents and populate incidents with custom information (such as a specific property of the monitored object such as IP address or performance readings from the Alert)
- Advanced Updates - Incident update behavior
This script executes when an alert comes in that already has an incident tied to it in ServiceNow, any of the properties on the ServiceNow incident can be updated from the new alert. Note that the Incident doesn't have to have been raised via an Alert to use this functionality but does need to be associated with one prior to the update event.
- Advanced Updates - Alert Update behavior (only effective when SCOM is configured for bidirectional sync)
- This script runs when a SCOM requests updates on an Incident if you want to fill custom fields or make adjustments to anything headed to SCOM this is where it can be performed.
Advanced Creation Example Script
Also baked into the product and displayed when Advanced Creation is enabled:
Advanced Update: Alert Update Example script
Also baked into the product and displayed when Advanced Update is enabled:
Advanced Update: Incident Update Example script
Also baked into the product and displayed when Advanced Update is enabled