ServiceNow JSON Payloads

ServiceNow JSON Payloads (commonly referred to as simply “the payload” in our documentation) - are blocks of JSON containing parameters in place of actual values. The cookdown Discovery engine replaces these parameters with actual values for each object.

This article will explain the components of these payloads. For more information on how these payloads are set up in the product see this page.

Once you understand payloads, check out the Discovery Payload Library for payloads covering some common technologies that we have made for you.

The Basics

Payloads

The below payload (again for Windows Server) is an example of the sort of payload you would create for a new discovery, it is highlighted to explain the components that make it up.

{
"items": [
{
"className": "cmdb_ci_win_server",
"lookup": [ ],

"related": [ ],
"values": {
"ip_address": "Microsoft.Windows.Computer:::IPAddress",
"dns_domain": "Microsoft.Windows.Computer:::DomainDnsName",
"os_domain": "Microsoft.Windows.Computer:::DomainDnsName",
"os_version": "Microsoft.Windows.OperatingSystem:::OSVersion",
"cpu_core_count": "Microsoft.Windows.OperatingSystem:::LogicalProcessors",
"ram": "Microsoft.Windows.OperatingSystem:::PhysicalMemory",
"os": "Microsoft.Windows.OperatingSystem:::OSVersionDisplayName",
"virtual": "Microsoft.Windows.Computer:::IsVirtualMachine",
"short_description": "SID: Microsoft.Windows.Computer:::ActiveDirectoryObjectSid AD OU:Microsoft.Windows.Computer:::OrganizationalUnit",
"host_name": "Microsoft.Windows.Computer:::HostServerName",
"name": "Microsoft.Windows.Computer:::NetbiosComputerName"
}
}
],
"relations": [ ]
}

Grey = ServiceNow Identification and reconciliation syntax/format for JSON payloads. Relations, Lookups and Related entries are explained in another article

Orange = The table in your ServiceNow CMDB that you wish to push CIs from the selected SCM class into (in this case cmdb_ci_win_server). Note that the Identification and Reconciliation Engine which will be consuming the payloads expects the table label rather than its name (the name of the cmdb_ci_win_server table is “Windows Server” for example). The list of tables present in your CMDB can be found by opening CI Class Manager in your ServiceNow instance and for a generic list see here.

Purple = Attributes of the CI Class into which you plan to push values. Each CI Class will contain a set of attributes that must contain values. These will either be attributes set as mandatory in your CMDB or attributes set in Identification Rules' Identification Entries for each CI Class (used to identify discovered CIs as unique or not based on rules and attribute matching.) If data is not provided for a mandatory attribute, the Identification and Reconciliation engine will reject the incoming CI. Read more on Identification Rules/Entries here.

Green = The values to be pushed into the defined attribute. These are typically Cookdown Discovery keys (explained on the Create Discovery page), strings, integers, and booleans. You can provide multiple keys for a single attribute and mix them in with regular strings (assuming the attribute supports the type of data you plan to push to it). It is worth noting that some attribute has a max character length so be careful to not push more to an attribute than it can hold (or excess characters are stripped from the provided data).

Example Payload for Windows Servers

Payload added to Cookdown Discovery for Windows Server

{
    "items": [
      {
        "className": "cmdb_ci_win_server",
        "lookup": [],
        "related": [],
        "values": {
          "ip_address": "Microsoft.Windows.Computer:::IPAddress",
          "dns_domain": "Microsoft.Windows.Computer:::DomainDnsName",
          "os_domain": "Microsoft.Windows.Computer:::DomainDnsName",
          "os_version": "Microsoft.Windows.OperatingSystem:::OSVersion",
          "cpu_core_count": "Microsoft.Windows.OperatingSystem:::LogicalProcessors",
          "ram": "Microsoft.Windows.OperatingSystem:::PhysicalMemory",
          "os": "Microsoft.Windows.OperatingSystem:::OSVersionDisplayName",
         "virtual": "Microsoft.Windows.Computer:::IsVirtualMachine",
          "short_description": "SID: Microsoft.Windows.Computer:::ActiveDirectoryObjectSid AD OU:Microsoft.Windows.Computer:::OrganizationalUnit",
          "host_name": "Microsoft.Windows.Computer:::HostServerName",
          "name": "Microsoft.Windows.Computer:::NetbiosComputerName"
        }
      }
    ],
    "relations": [    ]
  }

Payload passed from Cookdown Discover to ServiceNow for a single Windows Server

{
  "items": [
    {
      "className": "cmdb_ci_win_server",
      "lookup": [],
      "related": [],
      "values": {
        "ip_address": "192.168.1.201, fa80::b900:ef93:d1bc:53fc",
        "dns_domain": "cookdown.com",
        "os_domain": "cookdown.com",
        "os_version": "10.0.14393",
        "cpu_core_count": "2",
        "ram": "4193844",
        "os": "Microsoft Windows Server 2016 Datacenter",
        "virtual": "True",
        "short_description": "SID: S-1-5-21-1291999999-842597620-722999999-8601 AD OU:CN=Computers,DC=cookdown,DC=com",
        "host_name": "AzureHost1",
        "name": "Admin02"
      }
    }
  ],
  "relations": []
}

Relationships

A critical aspect of creating useful payloads is the relationships they contain. At their simplest, relationships link together CIs and some CI Classes cannot exist without a relationship to something else (EG SQL Databases cannot exist without a relationship to a SQL DB Engine). All CIs have recommended relationship types and possible relationship types. Read more about relationships in ServiceNow. ServiceNow instances ship with these but as they are customizable in each CMDB, the best place to see the suggested/available relationship types is to look in CI Class manager in your ServiceNow instance:

Example Relationships: SQL Database, SQL DB Instance, and Windows Server

The below payload will push SQL Databases, SQL DB Instances, and Windows Servers:

{
  "items": [
    {
      "className": "cmdb_ci_db_mssql_database",
      "lookup": [],
      "values": {
        "short_description": "Collation: Microsoft.SQLServer.Windows.Database::Microsoft.SQLServer.Core.Database::Collation Recovery Model: Microsoft.SQLServer.Windows.Database::Microsoft.SQLServer.Core.Database::RecoveryModel Udatability: Microsoft.SQLServer.Windows.Database::Microsoft.SQLServer.Core.Database::Updateability",
        "name": "Microsoft.SQLServer.Windows.Database::Microsoft.SQLServer.Core.Database::DatabaseName",
        "database": "Microsoft.SQLServer.Windows.Database::Microsoft.SQLServer.Core.Database::DatabaseName",
        "sys_class_name": "cmdb_ci_db_mssql_database"
      }
    },
    {
      "className": "cmdb_ci_db_mssql_instance",
      "lookup": [],
      "values": {
        "instance_name": "Microsoft.SQLServer.Windows.Database::Microsoft.SQLServer.Core.DBEngine::InstanceName@Microsoft.Windows.Computer:::NetbiosComputerName",
        "sys_class_name": "cmdb_ci_db_mssql_instance"
      }
    },
    {
      "className": "cmdb_ci_win_server",
      "lookup": [],
      "values": {
        "name": "Microsoft.Windows.Computer:::NetbiosComputerName"
      }
    }
  ],
  "relations": [
    {
      "type": "Contains::Contained by",
      "parent": 1,
      "child": 0
    },
    {
      "type": "Runs on::Runs",
      "parent": 1,
      "child": 2
    }
  ]
}

Looking at the relationship portion of this payload, we see the type of relationship defined (EG “Contains::Contained By”) and the parent/child of the relationship with parent referring to the first half of the relationship type (in this example Contains). The numbers relate to each CIs position in the payload, with the first CI occupying position 0. In the example above, cmdb_ci_db_mssql_database (SQL Database) is in position 0 and cmdb_ci_db_mssql_instance (SQL Instance) is in position 1. The relationship between then is a containment relationship which read in English essentially says the SQL Database is contained by the SQL Instance

{
   "type": "Contains::Contained by",
   "parent": 1,
   "child": 0
}

Regular Expressions (RegEx)

From version 1.5, rather than the Cookdown Discovery engine simply pushing the values returned from a key into ServiceNow we can transform their values using regular expressions. The below video will explain how they work and where to use them

Let us explore this with the example of Linux Servers.

Linux Servers identified by ServiceNow’s own Discovery product use the Netbios name of a Linux Server as the name of each CI and ServiceNow’s default Identification and Reconciliation Engine Identity Entries call out the name field. This means that for Cookdown Discovery to co-exist with ServiceNow Discovery created CIs for Linux Servers the Netbios name must be used in the name attribute.

When looking at the data in SCOM for Linux Servers, you will only find the FQDN of the server (“server01.mydomain.local” rather than simply “server01”). While this data is close, if you were to push CIs from SCOM using their FQDN you would create duplicate CIs in ServiceNow.

With the RegEx feature, we can get the Cookdown Discovery engine to drop everything after the first full stop in the Linux Server’s FQDN (in other words, to get “server01” only from “server01.mydomain.local”)

RegEx Syntax

Cookdown Discovery uses .net so all standard .net RegEx syntax applies to Cookdown Discoveries use of it, Read more on RegEx with .net here and see this handy RegEx cheat sheet with the most commonly used regular expressions.

Regular expressions are noted alongside the key they apply to as per the following syntax (an example line from a payload):

"<attribute name>": "[REGEX(<Cookdown Discovery Key>,'<regular expression>')]",

See below for an example of what this will look like in the product with a specific example for Windows Server

As with all Cookdown Discovery Payloads, the format is JSON and keys are highlighted green, but unlike standard payloads, regular expressions are highlighted yellow

RegEx Example

In this example, we will split the NetBios name from an FQDN for “Server-01.cookdown.local”.

Payload:

{
  "items": [
    {
      "className": "cmdb_ci_win_server",
      "lookup": [],
      "values": {
        "name": "[REGEX(Microsoft.Windows.Computer:::PrincipalName,'\\D*\\d*')]"
      }
    }
  ],
  "relations": []
}

Note that \D matches non-numeric characters, when used with * it will match all non-numeric characters in the key from its start until it hits a number. \d matches all numeric characters and when used with * will match all numeric characters (so will stop matching when a non-numeric character is hit).

Typically in .net regular expressions, the syntax would be “\D*\d*” but in the JSON language \ is a protected symbol so must be escaped with a \. The result in a payload is “\\D*\\d*”

Result of payload for Server01:

{
  "items": [
    {
      "className": "cmdb_ci_win_server",
      "lookup": [],
      "values": {
        "name": "Server-01"
       }
    }
  ],
  "relations": []
}

Lookups and Related entries

The Lookup and Related sections of the payload allow you to fill in reference attributes with values from other CMDB tables. Creating payloads to fill in these fields is simple in principle but it is hard to get right. If, for example, you create a lookup for a CI type that already has an identifier entry somewhere else in your CMDB, duplicate CIs can be created. The following examples for Windows Servers with Network adapters + Clusters with cluster nodes show you how to construct payloads with lookups and related entries.

Lookup Example: Windows Servers with Network Adapters

{
    "items": [
      {
        "className": "cmdb_ci_win_server",
        "lookup": [
  
          {
            "className": "cmdb_ci_network_adapter",
            "values": {
                "ip_address": "Microsoft.Windows.Computer:::IPAddress",
                "mac_address": "Microsoft.Windows.Server.10.0.NetworkAdapter::Microsoft.Windows.Server.NetworkAdapter::MACAddress"
            }
        } 
        ],
        "values": {
          "name": "Microsoft.Windows.Computer:::NetbiosComputerName"
        }
      }
    ],
    "relations": [
          ]
  }

Related Entry Example: Windows Clusters with Cluster Nodes

{
  "items": [
    {
      "className": "cmdb_ci_win_cluster",
      "lookup": [],
      "related": [
                 {
                  "className": "cmdb_ci_win_cluster_node",
                  "values": {
                  "name": "Microsoft.Windows.Computer::Microsoft.Windows.Cluster.Node::ClusterName",
                  "cluster": "Microsoft.Windows.Computer::Microsoft.Windows.Cluster.Node::ClusterName"
                  }
               }               
        ],
      "values": {
        "fqdn": "Microsoft.Windows.Cluster.NodeRole::Microsoft.Windows.Computer::PrincipalName",
        "cluster_type": "Microsoft.Windows.Cluster:::VendorId",
        "cluster_version": "Microsoft.Windows.Cluster:::HighestVersion",
        "cluster_status": "Microsoft.Windows.Cluster::System.ConfigItem::ObjectStatus",
        "cluster_id": "Microsoft.Windows.Cluster.NodeRole::System.Entity::DisplayName",
        "name": "Microsoft.Windows.Cluster.NodeRole::System.Entity::DisplayName"
      }
    }
 ],
   "relations": [
  ]
}

You will notice in both of these examples that the CI you want to lookup/relate to is mentioned but the reference field on the main CI class is not. That is because both Lookups and Related Entries depend on Identification and Reconciliation rule changes to work. See the below rules that facilitate this (which can be found in your ServiceNow instance by searching for “CI Identifiers” in the navbar on the left-hand side of the screen):

Identifier Entry for Network Adapter for example Lookup payload

Note that this identifier Entry is shipped by default with ServiceNow. It is used by ServiceNow’s own SCCM Discovery product. Also, note that this Identifier is for cmdb_ci_hardware - the Windows Server CI Class is parented by cmdb_ci_hardware so inherits its Identifier Entries by default.

Related Entry for Windows Cluster, for example, Related Entry payload

Note that this related entry is not shipped by default with ServiceNow.

Read more about lookup and related entries.

Creating payloads

When getting started with creating payloads they can seem daunting, but once you get going you will see they are simple enough.

Your ServiceNow instance contains a useful tool that can help build payloads called Identification Simulator. It is designed to see what the result of pushing payloads into ServiceNow would be on the CIs that already exist in your CMDB.

It creates all the JSON syntax that surrounds the data you select, builds the JSON for relationships between CI types where CI Classes are dependant on each other, it lets you know which attributes can be used for identifying a CI as unique (as defined in your CMDB for each CI class) and warns you if you try and build a payload without an attribute that has been made mandatory for the CI Class in your CMDB.

It is a valuable tool when building Cookdown Discovery payloads!

The ServiceNow documentation on the Identification Simulator is good and will amply explain how the tool works, but as we are using the tool to build payloads containing Cookdown Discovery keys we have created the below as supplemental documentation

To read more about using the Identification Simulator for building Cookdown Discovery payloads read this page.

Identification and Reconciliation engine limitations

The only limitation with the Identification and Reconciliation engine itself (and so with Cookdown Discovery) is that there is no way to search for records from a table outside of the CMDB itself. Some examples:

  • Company table (core_company)

  • Users table

We have seen customers implement two workarounds for this limitation:

  1. Store the sys_id for the entries in the table Identification and Reconciliation cannot search in SCOM as data in extended classes. The sys_id can simply be pushed into a reference attribute that references data from a table outside of the CMDB

  2. Add a custom attribute into ServiceNow, push data from SCOM into the field (like the owner of a SCOM object) and write scripts in ServiceNow to find the attribute’s value in the desired table (eg search for an email address from the custom attribute in the users table and return the sys_id of that user to store in a user reference attribute)

Search