Comprehensive Guide to Resolving Batch Job Failures, GUI Access Issues, and Batch Sending Errors in SMSC

Overview

Issues may arise in the SMSC platform where batch jobs fail to execute, the Management GUI becomes inaccessible, and batch sending operations encounter errors. Common symptoms include "Another operation is in progress" errors, 500 HTTP errors when accessing the GUI, stuck batch jobs, performance degradation, and synchronization delays with the database. These problems can stem from Apache server port conflicts, stuck MGR transactions, NTP synchronization issues, SSH trust problems, MGR node performance degradation, and database synchronization delays.

Symptoms

You may see messages or errors in the MGR such as:

• Configuration update and synching currently in progress, hence updated information is not displayed. Please revisit this screen after some time to view the updated information.
• Item #: 7 - Another operation is in progress. Please try after some time.
• Cannot set distributionListTable

You may see that batch jobs appear completed but have not actually completed (e.g. messages not received.)

Solution

1. Restart Synchronization Process

If an SMPP error is encountered while creating a Job, it could be because of the configuration synchronization.

• Go to Settings -> Network Layout -> Devices -> BAT
• Right click on BAT and click "Resync"

2. Resolve Apache Port 80 Conflicts

• Identify processes using port 80:
  netstat -tulpn | grep :80


• Terminate conflicting processes:

  kill -9 <PID1> <PID2> ...
  

• Restart Apache:

  systemctl restart httpd
  

• Confirm Apache status:

  systemctl status httpd
  

3. Address Stuck MGR Transactions

• Stop the MGR services:

  /usr/TextPass/bin/tp_mgr_stop
  

• Log into MySQL and switch to the MGR database:

  mysql --login-path=mysql_root
   USE mgr_domain_mgr;
  

• List the syncing and saving tables:

  Select * from syncingTable;
  Select * from savingTable;
  

• Backup existing entries:

  CREATE TABLE syncingTable_bk AS SELECT * FROM syncingTable;
  CREATE TABLE savingTable_bk AS SELECT * FROM savingTable;
  

• List the syncing and saving tables:

  DELETE FROM syncingTable WHERE condition; (replace with a proper condition from the select statement result) 
  DELETE From savingTable where condition; (replace with a proper condition from the select statement result);

• or alternatively truncate tables:

  TRUNCATE TABLE syncingTable;
  TRUNCATE TABLE savingTable;
  

• Start the MGR services:

  /usr/TextPass/bin/tp_mgr_start
  

4. Free Up Disk Space on /var/TextPass

• Identify large files:

  du -sh /var/TextPass/* | sort -hr | head -20
  

• Delete unnecessary log files carefully.

• Ensure at least 15%-20% free space on /var/TextPass.

5. Fix SSH Trust Issues for Localhost Connections

• As the "textpass" user:

  ssh-keygen -R localhost
  ssh-keygen -R $(hostname)
  ssh-keyscan -H localhost >> ~/.ssh/known_hosts
  ssh-keyscan -H $(hostname) >> ~/.ssh/known_hosts
  

• Confirm localhost SSH connectivity without trust warnings.

6. Re-Synchronize NTP and Hardware Clock

• Disable NTP:

  timedatectl set-ntp false
  

• Synchronize time immediately:

  ntpdate -u <Your_NTP_Server>
  

• Update the hardware clock:

  hwclock --systohc
  

• Re-enable NTP:

  timedatectl set-ntp true
  

7. Delete Stuck MGR PIDs

• Stop the MGR Service:

  systemctl stop mgr
  

• Navigate to the PID Directory:

  cd /var/TextPass/MGR/pid
  

• Remove All PID Files:

  rm -rf *.*
  

• Start the MGR Service Again:

  systemctl start mgr

8. Verify STV and System Health

• As the "textpass" user, run:

  stv_self_test
  stv_db_importer --status
  stv_poll_collector --status
  stv_poll_config_sync --status
  stv_poller --status
  

• Confirm all components report OK status.

9. Confirm Batch Job Functionality

• Test by creating and sending a small batch job.

• Monitor for successful execution without pending transactions.

If issues persist, review system logs:

• /var/TextPass/MGR/tp_mgr_error_log.txt

  • or /var/TextPass/tp_mgr_error_log.txt

• /var/TextPass/MGR/tp_mgr_change_log.txt

  • or /var/TextPass/tp_mgr_change_log.txt

• /var/log/messages

10. Resolve MGR Node Performance Issues Affecting Batch Sending

• Restart the MGR Service or Server:

  • Restarting restores system responsiveness.

  • Impacts are limited to the MGR GUI and Bulk SMS temporarily, without affecting active SMS traffic.

• Perform System Health Checks:

  • Confirm that system services operate normally after restarting.

• Implement Regular Maintenance:

  • Reboot the MGR server every 6–8 months during a maintenance window.

  • Expected downtime is minimal.

• Additional Preventive Measures:

  • Avoid simultaneous modifications to the same batch job or distribution list.

  • Monitor system logs regularly for early signs of performance degradation.

11. Handle Synchronization Delays When Adding Templates, Distribution Lists, or Batch Jobs

• Wait for Configuration Synchronization:

  • Allow the system time to synchronize configurations with the database.

• Manual Synchronization (if needed):

  • Navigate to Settings -> Network Layout -> Devices -> BAT.

  • Right-click on BAT and select Resync.

  • Retry operations after resynchronization completes.

• General Recommendation for SMPP Errors:

  • Use the same waiting or manual resynchronization approach.

• Important Notes:

  • Avoid modifying or deleting templates linked to active jobs.

  • Prefer creating new templates instead of editing those associated with live traffic.

By following these procedures, most batch job failures, GUI access issues, and batch sending errors in SMSC environments can be effectively resolved.

<supportagent>

Related Tickets

• #60004576, #2958825, #2916715, #2751233

</supportagent>