Overview

The customer is experiencing an issue with high memory usage on their LGP2 node. The problem was identified as the MySQL database consuming a significant portion of the memory, with a utilization rate of 99%. The `innodb_buffer_pool_size` was found to be set to 55GB for LGP2, compared to 12GB for LGP1. The customer was advised to check with their OS admin if this setting was intended and if not, to reduce the `innodb_buffer_pool_size` to a reasonable value between 50%-80% of their RAM. The customer was also concerned about the impact of restarting the MySQL process on their service.

Solution

To resolve the high memory usage issue on the LGP2 node due to MySQL database, follow these steps:

1. Check with your OS admin if the `innodb_buffer_pool_size` was intentionally set to 55GB for LGP. If not, reduce the `innodb_buffer_pool_size` to a reasonable value between 50%-80% of your RAM. Avoid using a fraction of a GB (33.5 , 43.5 etc..).
2. To make the changes, as the root user, navigate to the configuration file by typing `cd /etc/`.
3. Open the file by typing `vim my.cnf`.
4. Using vim, find the line `innodb_buffer_pool_size`, press `i` to start making changes.
5. When finished, press esc, and type `:wq` to save and exit.
6. Restart MySQLd and monitor the RAM afterward.

If you are concerned about the impact of restarting the MySQL process on your service, follow these steps to minimize impact:

1. Make the change to the config file (/etc/my.cnf) as suggested above.
2. Stop LGP - As Textpass run `tp_stop --tp_lgp`.
3. Restart MySQL service on the LGP - As root user run `systemctl restart mysql`.
4. Start LGP - As Textpass run `tp_start --tp_lgp`.
5. Check the status of MySQL service using `systemctl status mysql`.
6. Check LGP status - As Textpass run `tp_status --tp_lgp`.

Summary

The high memory usage on the LGP node can be resolved by adjusting the `innodb_buffer_pool_size` in the MySQL configuration file. Restarting the MySQL service and LGP will be necessary, but the impact on service can be minimized by following the steps provided.

FAQ

1. What is the `innodb_buffer_pool_size`?
   It is a system variable in MySQL that specifies the size of the buffer pool, the memory area where InnoDB caches table and index data.
2. What happens if I don't adjust the `innodb_buffer_pool_size`?
   If not adjusted, the MySQL database may continue to consume a significant portion of the memory, leading to high memory usage on your LGP node.
3. Will there be any impact on my service when I restart the MySQL process?
   The processing of files by the LGP will be affected during the restart. However, these files will be again processed during the next retry as per the Polling intervals configured. It is recommended to do this at a time when there is minimum traffic.