Areas of Concern

ServiceNow has a number of different components that can each warrant its own troubleshooting steps depending on what is happening. The following ‘areas of concern’ are in the order that you would most commonly experience them (you can’t raise Incidents or tasks before you have your alerts imported for example).

Alerts Not Being Created

It is also worth ensuring that you have an active Outbound Notification connection in SCOM that can successfully connect to your instance.

In our experience issues with Alerts not being created in this manner are related to hardened ServiceNow instances not allowing the service account access to part of the import set functionality. For example, we have seen service accounts that were allowed to create records but not insert values. In this example, the error was around coalescing on Alert Id (which was always null because the service account could not write to it).

A good quick check would be to use the service account and then an Admin account to try and push sample data into the import set. In the following example, we use PowerShell, however, you could easily use tools such as Postman or curl if preferred.

$Instance = 'yourinstancenamehere'
$Credential = Get-Credential -Message 'Enter some valid ServiceNow credentials for the scenario you are testing'

If ([Net.ServicePointManager]::SecurityProtocol -notlike '*Tls12*'){[Net.ServicePointManager]::SecurityProtocol += [Net.SecurityProtocolType]::Tls12}

$Headers = @{
    Accept = 'application/json'
    'Content-Type' = 'application/json'
} 

$Bareminimumalert = @{
    u_alertid = 'testdata'
}

Invoke-RestMethod -Method Post -Credential $Credential -Uri "https://$Instance.service-now.com/api/now/table/x_oklt_cookdown_sc_scom_alert_import" -Body ($Bareminimumalert | ConvertTo-Json) -Headers $Headers

If the admin account succeeds and the service account does not you should check over your ACLs paying specific attention to the following tables:

• x_oklt_cookdown_sc_scom_alert_import

• sys_import_set_row

• sys_import_set

The x_oklt_cookdown_sc_scom_alert_import table extends the sys_import_set_row table, so ACL entries on this table should take preference.

If these rules are in place and active it would be worth engaging in ACL debugging and contacting support.

Back to top

Incidents and/or Tasks Not Being Raised

In order to raise Incidents or other types of tasks, you need to have an Alert and a matching Creation Rule.

1. Firstly you should make sure that you have Alerts available in the 'SCOM Alerts' table by pushing them from SCOM or by creating them manually using the minimum requirements as described here

2. Next, you should check that you have a ‘Creation Rule' in place to raise incidents by going to the 'Creation Rules’ table

   1. Make sure you have at least one Creation Rule

   2. Make sure that the Creation Rule is active

   3. Make sure that the Creation Rule has a filter (Alert Condition)

      

3. If you have Post-Processing or other advanced settings enabled consider disabling them temporarily

4. Copy the text version of the alert criteria from the main table view

5. Navigate to the SCOM Alerts table

6. Add your search query to the end of your URL to check that you have matching alerts

   1. Clear out any existing query after x_oklt_cookdown_sc_scom_alerts_list.do

   2. Add %3Fsysparm_query%3D

   3. Add the query that you copied in step 4

   4. Press Enter

      

7. If you do not see any alerts in the table at this stage you may need to adjust your filter

8. If you have alerts matching the criteria check to see if they have matched the creation rule at the bottom of the alert:

   

   1. If it has not matched a Creation Rule at all

      1. Try to ‘re-run’ the alert as described here

   2. If it has matched a Creation Rule but has not raised an alert

      1. Check the history tab to see if there are any messages and investigate any errors

      2. Impersonate the service account and then try to re-run the alert

      3. Investigate any errors seen such as the one described here

      4. If disabling Post-Processing or other scripts has allowed you to generate an Incident/Task proceed to Custom Script Issues below

9. If you are not able to determine what is going wrong consider reaching out to our support team

Back to top

Alerts Updates Not Being Returned to SCOM

Much like alerts not being created, it’s more likely to be an issue caused by the SCOM side or in transport (proxies or firewalls getting in the way) than the ServiceNow side. Make sure that you do have an active Inbound Notification connection that can connect to ServiceNow.

In order to sync back to SCOM the incident needs to be linked to the SCOM alert, the SCOM alert needs to be linked to a Creation Rule, and the alert needs to have the 'Updates for SCOM?' option checked (should be automatically set when a linked incident is updated and removed when SCOM pulls the updates, though you can set this manually for testing/troubleshooting purposes):

1. The first thing to do is hurry up and wait

   1. When Connection Center successfully pulls the alert it will mark the alert as polled (by default every 60s)

   2. The alert will remain checked for a further 4 minutes in case a re-run needs to occur for some reason

   3. The Checkmark will be removed and the alert will no longer be available to Connection Center to poll unless another change happens to reset the checkmark

      

2. If this does not occur the next thing to check is that the API is successfully presenting the alert to the Service Account

   1. Impersonate the service account (not required in all scenarios, but is the most accurate representation)

   2. Use the API Explorer (or another service such as PowerShell, Postman, or curl) to call our API

      1. Select API Explorer

      2. Set Namespace to x_oklt_cookdown_sc

      3. Send the request

   3. Verify that the status is 200 and that the response body contains your alert/incident in the body:

      

   4. If it does not, repeat the steps using an admin account to check for permissions issues

   5. If the admin account returns the details as expected, verify that the service account has the correct roles assigned

   6. If the roles are already correct, or the admin account does not return the expected data it is probably worth looking at ACL debugging or getting in contact with our support team

If you do not have access to the Rest API explorer and/or impersonation in ServiceNow you can also try troubleshooting with PowerShell. You can adjust the script described here to suit your needs.

Back to top

Alerts not being closed in SCOM on Resolved in ServiceNow

Usually, this is caused by using Advanced Alert Updates on your creation rules. By default, this does not include any closure logic and also does not set custom field 5 in SCOM.

If you aren’t customizing your alert updates we would recommend that you disable the Advanced Alert Updates feature. Otherwise, you can put in logic to replicate the default closure behavior:

  if (incidentRecord.state == 6 || incidentRecord.state == 7) {
      scomAlertUpdatePayload.ResolutionState = parseInt(255);
  }

Back to top

Custom Script Issues

Back to top

Logging

Back to top

Specific Issues

Alerts in ServiceNow Show a Different Time/Timezone to the Alert Itself in the SCOM Console

Problem

Cause

In ServiceNow, the time of each alert is displayed in the timezone that you (the user logging into ServiceNow to view the alert) as.

Solution

Change the timezone of your ServiceNow user account in your profile to the correct one:

Back to top

Creation Rule Users See Strange UI Behavior

Problem

A non-admin user of ServiceNow is reporting strange UI behavior when using creation rules. This typically includes the Configuration Item picker and script-able elements not displaying as expected.

Solution

This is generally caused by the user not having access to certain tables within ServiceNow. Typically Configuration Items. Granting access to these elements resolves the issue (typically through the ITIL role). If you do not require this functionality Alert Sync will continue to operate as normal without them and the role can be left off if desired.

Further information on Connection Center and any related roles can be found here.

Back to top

When Using a Caller That Is Not the Service Account, Alert Closed Incident State Will Not Resolve or Close an Incident

Problem

You have set your creation rule to use a caller other than your service account and the ‘Alert Closed Incident State' from the 'Advanced' tab is set to Resolved, Closed, or Cancelled. When a SCOM alert is closed the Incident remains on its previous status and does not get moved to the desired state.

Cause

Cookdown and ServiceNow both adhere to ITIL best practices by default. They allow a caller to resolve or close their own Incidents but require resolution/closure notes and codes when resolving Incidents for another caller. If at the time of SCOM alert resolution the Incident is linked to a different caller (set either by the creation rule or through other means) by default the Incident will not close. You may also see this behavior using the service account as the caller in a customized instance.

You can verify this is the case by impersonating the service account and updating an alert manually. You will receive an error similar to the following:

Data Policy Exception: The following fields are mandatory: Resolution code, Close notes

Solution

Use the service account as the caller where possible

Workaround

1. Rather than using ‘Alert Closed Incident State’ in your creation rules use 'Advanced Incident/Task Updates' to script the resolution of the ticket, setting the required fields as appropriate.

2. In ServiceNow create or adjust your ‘Data Policy' and 'Data Policy Rules’ to exempt Incidents raised by creation rules from needing to fill in these fields.
   Specifically how you approach this would be down to your organizational requirements, however, a suggestion might be to exempt any Incident created/updated by your service account, with a certain contact type, or a combination of both.

Back to top

Creation Rule Fails to Create Task, Catalog Task, and Service Task

Problem

When a new SCOM alert comes in, a creation rule set to create a Task, Catalog Task, or Service Task fails to raise a task of this type

Cause

By default, ServiceNow does not provide users or the ITIL role with access to these tables. You may find that there are no ACL entries for these roles or you may find that there are not any ACL entries for those tables at all…

Solution

Provide the Service account with appropriate access to these tables to use these task types. For example, you could provide the ‘catalog_admin' role to the service account to provide access to create Catalog Tasks. However, rather than doing this, we strongly recommend that you create your own ACL to provide access to the required table via the 'x_oklt_cookdown_sc.scom_sync_service’ role or another role of your choice.

Back to top

Alert updates continue without a creation rule

Problem

You remove a creation rule that was associated with a number of alerts with linked tasks. Alert updates continue to occur after the creation rule is removed.

Cause

This is by design, alerts will continue to sync using the default settings. If you need advanced alert updates, you should disable the creation rule until the alerts/tasks have all been groomed at which point you should be able to remove it without falling back on default behaviors.

Testing Get/Post from a script

Uncomment sections of the script as you require them:

$Instance = 'instanceName'
#Youer instance name

#$Proxy = 'http://proxy.address'
#Uncomment and adjust if requiring a proxy

if (-not $SNOWCredential) {
    $SNOWCredential = Get-Credential -Message 'Please enter your servbice account or other valid servicenow API credentials'
    #Valid ServiceNow credentials, best to start with service account and then try others
    }

$Bareminimumalert = @{
    u_name = 'Alertname'
    u_alertid = [guid]::NewGuid().Guid
    u_description = 'Fake alert'
    u_monitoringobjectparentids = [guid]::NewGuid().Guid
}
#Absolute bare minimum that we can expect to raise an incident.

$PostSplat = @{
    Method = 'Post'
    Uri = "https://$Instance.service-now.com/api/now/table/x_oklt_cookdown_sc_scom_alert_import"
    Credential = $SNOWCredential
    Body = ($Bareminimumalert | ConvertTo-Json)
    Headers = @{Accept = 'application/json';'Content-Type' = 'application/json'}
}

$GetSplat = @{
    Method = 'Get'
    Uri = "https://$Instance.service-now.com/api/x_oklt_cookdown_sc/scom_incident_status"
    Credential = $SNOWCredential
    Headers = @{Accept = 'application/json'}
}

if ($Proxy) {
    $PostSplat.Proxy = $Proxy
    $GetSplat.Proxy = $Proxy
}

Get-Date

# Uncomment/comment out the scenario(s) you wish to test:

#$Post = Invoke-webrequest @PostSplat
#Send our fake alert up

#$Get = Invoke-RestMethod @GetSplat
#Get all available alerts

$Post
$Get

Get-Date