Time!!! It plays very vital role in our life and with IT systems; specially when we are talking about complex systems like Cisco Unified Contact Center Enterprise (UCCE) or Cisco Unified Communications Manager (CUCM).  There can be diverse effects if these servers/systems are not in sync and cause various issues with overall operations and user performance.

Let’s look at some of important points of keeping these systems in sync.

NTP and Time Synchronization

Packaged CCE requires that all components of the solution have the same time. While time drift occurs naturally, it is critical to configure NTP to keep solution components synchronized.

To prevent time drifts on Live Data reports, the NTP settings on the Rogger VMs, the PG VMs, the AW VMs, and on the Cisco Unified Intelligence Center Publisher and Subscriber VMs must be in sync.

Microsoft periodically releases cumulative time zone updates. These updates include worldwide changes to time zone names, bias (the amount of time in minutes that a time zone is offset from Coordinated Universal Time (UTC)), and observance of daylight saving time. These patches update the information in the Windows registry. When these updates are available, apply them to all virtual machines in the deployment that are running a Microsoft Windows operating system.

Windows Active Directory Domain

The Windows Active Directory Primary Domain Controller (PDC) emulator master for the forest in which the Packaged CCE domain resides (whether same, parent, or peer) must be properly configured to use an external time source. This external time source should be a trusted and reliable NTP provider, and if already configured for the customer’s forest, must be used (and useable) as same source for all other applications as detailed in this section for the Packaged CCE solution.

See the following references for properly configuring Windows Active Directory Domain for NTP external time source:

• How to configure an authoritative time server in Windows Server.

• AD DS: The PDC emulator master in this forest should be configured to correctly synchronize time from a valid time source.

Microsoft Windows Server Domains do not automatically recover or fail over the authoritative internal time source for the domain when the PDC emulator master server is not reachable, due to hardware failure or otherwise. This article, Time Service Configuration on the DC with PDC Emulator FSMO Role, helps describe how you must additionally configure the new target server to be the authoritative internal time source for the domain.

It also covers manual intervention to recover and seize or reassign the PDC Flexible Single-Master Operations (FSMO) role to another domain controller.

Windows Components in the Domain

Windows hosts in the domain are automatically configured to sync their time with a PDC emulator, whether by the PDC emulator master with authoritative internal time source or chained from same in the domain forest hierarchy.

Windows Components Not in the Domain

Use the following steps to set NTP time source for a Windows Server that is not part of a domain:

1. Log in as a user with administrative privileges.

2. In the Command Prompt window, type the following line and press ENTER:

w32tm /config /manualpeerlist:PEERS /syncfromflags:MANUAL

Note Replace peers with a comma-separated list of NTP servers.

3. Restart the w32time service: net stop w32time && net start w32time.

4. Sync w32time service with peers: w32tm /resync.

5. Use the following Service Control command to ensure proper start of the w32time service on any reboot of the server: sc triggerinfo w32time start/networkon stop/networkoff.

Cisco Integrated Service Routers

Cisco IOS Voice Gateways must be configured to use the same NTP source for the solution in order to provide accurate time for logging and debugging.

See: Basic System Management Configuration Guide, Cisco IOS Release 15M&T: Setting Time and Calendar Services.

VOS Components

Components such as Unified Intelligence Center, Finesse, Customer Collaboration Platform, and Unified Communications Manager must point to the same NTP servers as the domain authoritative internal time source.

ESXi Hosts

All Packaged CCE ESXi hosts (including those for optional components), must point to the same NTP server(s) used by the Windows domain PDC emulator master as their external time source. For details on configuring NTP on ESXi hosts, see the VMware documentation at https://www.vmware.com/ support/pubs/ .

NTP with CUCM and IM&P Cluster

NTP in CUCM cluster is very important as VOIP packets are very sensitive to time variants.  Hence CUCM cluster must maintain time synchronization within cluster including IM&P.

NTP Source

CUCM cluster needs a source which can provide NTPv4 output.  If the source is not providing NTPv4 then you will see your NTP is out of sync on CUCM.  Cisco recommends to use either a Linux-based or Cisco IOS-based NTP solution.  You also need to use an IP address of a source, instead of a URL which points to a group of NTP servers.

NTP Master/Client

CUCM Publisher in a cluster syncs its time with external source, so it is an NTP client in this situation. Whereas other components on CUCM cluster, including IM&P servers, rely on CUCM Publisher to get time and remain in sync.  So in this case, CUCM Publisher is the NTP master and other servers in cluster behaves as an NTP client.

Identifying NTP issues

There are several clues that can tell us that the CUCM cluster is having issues with NTP:

• Time on Phones is out of sync (most visible symptom of an NTP issue)
• CDR logs are out of sync
• DB Replication doesn’t work properly

Troubleshoot NTP issues

• Run below command from OS Admin to verify NTP status
  • Utils ntp status
  • If the output shows “unsynchronized” then NTP isn’t working properly
• Try to ping NTP server listed in above output
• Use CLI commands listed below to view, add, delete NTP servers

CLI commands for NTP Servers

While NTP servers are typically specified at install time, here are few commands you can use from the platform cli of the VOS components, to list, add and remove NTP servers.

From the platform CLI:

• To list existing ntp servers: utils ntp servers list

• Use this to add an additional ntp server: utils ntp server add

• Command to delete an existing ntp server: utils ntp server delete (row number of the item to delete). Press Enter.

Time changes in ICM Rogger

You have to be aware of both Windows time and ICM time (established by the Call Routers).

From the ICM perspective, as long as you are not going back in time and potentially writing duplicate records in the ICM Db then you are all set.

So, if you are moving forward in time (e.g. ICM time is 12:00:00.000 and NTP time is 12:01:00.000) then:

– stop ICM on all components,

– change your time,

– follow all Windows rules with setting NTP sources and configuring clients and domain members, including rebooting the servers,

– rerun ICM setup from the \ICM\BIN\ directory on the Routers and PGs and confirm the Disable MDS time sync is disabled on those components,

– restart ICM

If you are going back in time, then you’ll have to document your delta and keep the system down for greater than that delta. For example, if ICM time is 12:00:00.000 and NTP is 11:45:00.000, you’ll have to keep ICM down for greater than 15 minutes in order not to duplicate records in the Db. Otherwise the process is the same as above.

I would love to hear your feedback or if you have any question, please drop them in comments!!