Traveler server with Red status and long running push threads
We had a customer contact us recently about their Traveler server. The server was reporting a status of Red, CPU on the server was extremely high, and there were a number of these errors in the log file:
Traveler: User CN=User Name/O=Org on thread Push-1ca3 has been running for 8820 minutes.
One correlation for the users reporting this error, they were all Android users. No other device types were reporting this error.
In digging a bit deeper, we saw this error for each affected user in the log:
SEVERE Push-14ad9 User Name[Android_f5a7931af273cbdc] ConnectionNotificationSenderGCM$SendMessageRunnable.run#460 There was an issue sending the notification message to the Google Cloud Messaging (GCM) server. 3 attempts to re-establish the connection have failed, so the notification message cannot be sent. You can test network connectivity including firewall settings by trying https://android.googleapis.com/gcm/send in a web browser on this server. Exception Thrown: java.net.SocketException: Connection reset
Running a trace from the physical server the Traveler server is hosted on revealed an inability to connect to https://android.googleapis.com/gcm/send
Google Cloud Messaging (GCM) became available in Traveler 220.127.116.11. It is the default synchronization option for Android devices when they download the 18.104.22.168 or later IBM Verse client.
The fix to this issue is two-fold:
- Enable GCM push via notes.ini NTS_PUSH_ENABLE_GCM=True and restart the Traveler server.
- Ensure the Traveler server can connect to https://android.googleapis.com/gcm/send