Riverbed Product Alert - Leap second bug on 30 June 2015 may cause System Hang or High CPU

Posted 27/05/2015

Riverbed have informed us of an issue with a potential issue with a kernel bug which can affect some SteelHeads when a Leap Second is added into the time scale.

Issue

On June 30 2015 at 23:59, a (leap) second will be inserted into the time scale.  Devices running NTP for clock synchronization will have one second added to the system time. 

Certain Riverbed products are exposed to a kernel bug that may cause the device to hang or create high CPU leading to system unresponsiveness.
 

Knowledge Base Solution Number: S26022

Solution

Currently this issue is tracked in bugs 227878, 232692 and 235360.
 

• Bug 227878 deals with the update of the tzdata distribution which contains the update of the leap second itself.
• Bugs 232692 and 235360 deal with the merging of a set of kernel patches as provided by the Linux distribution.

The following software releases have these bugs fixed:

• Steelhead RiOS 8.6.2c (expected release end of May)
• Steelhead RiOS 9.0.1a (release date May 15, 2015)
• Interceptor 4.5.2
• CMC 8.6.x
• CMC/SCC 9.0.x

The following software releases are not affected by this issue:

• NetShark 10.8.1 is not affected
• NetProfiler 10.5+ is not affected
  However if NTP is not being used it will not handle the leap second correctly. We highly recommend NetProfiler to use NTP.
• SteelCentral Portal 1.0.0 and RPM Dashboards 2.3.1 should not be affected
  However, if any unexpected behavior is observed, please restart.
• AppResponse 8.x+ is not affected

Disabling NTP in advance of midnight June 30 2015 UTC and July 1 2015 UTC will avoid the leap-second update. The system clock will be incorrect by 1 second until the clock is corrected, either by manually setting the time or after NTP is re-enabled.

In the rare event where unpatched systems are affected by this bug and become unreachable or get in high CPU state that does not subside, a system reload or power cycle will recover the unit.

Environment:   Leap Second bug

Palo Alto KB Article - Leap Second and Leap Year Compliance

Posted: 24/02/2015

Palo Alto have a KB article detailing their Leap Second / Leap Year Compliance.

A leap second is a one-second adjustment that is applied to Coordinated Universal Time (UTC) in order to synchronize atomic clocks with astronomical clocks. The Earth's rotation around it's own axis is slowing down gradually so a second is often added to compensate for this. Without this adjustment, there will be an increasing gap in the time between atomic time and astronomical time.

Note: The last leap second was added June 30th, 2012. The next leap second will be added June 30th, 2015.

The Palo Alto Networks firewall handles the leap second, as well as the leap year, in the following manner:

• Leap second insertions should be picked up by time keeping NTP servers and devices acting as ntp clients will be able to synchronize their clocks with the servers.
  
  
• Palo Alto Networks devices are NTP clients and will pick up the change from any in-house or global server.
  
  
• If your IT department maintains an in-house NTP server, then make sure the leap second insertion has been picked up by this server in order for the clients to be properly synchronized.

Keep up to date with the latest news

The Teneo Technical Support Team will be keeping up to date with all of the latest information and updates, so please get in touch with any questions or concerns at support@teneo.net

To keep up to date with what we know, follow us on Twitter