Content Manager Performance Issue, Maintenance and Monitoring

• How to troubleshoot when Content Manager is running slow.
• Check performance before it becomes serious.
  • Free Memory (Step 2)
  • Server Monitoring (Step 9)

Icon

Low on memory or High CPU usage might cause

• data inconsistency, file corruption, installation failure.

You need consider to use a proxy server if you are implementing a large number of players.

Upgrading your Content Manager improves performance.

• Plan Generation (11.02.03 MSSQL 11.03), search (11.02.05), calculate playlist duration (11.03.04), PAM parsing (11.03), loading channels or players page (11.02), updating maintenance jobs with many players (11.02.05), many users logged in(11.03) etc..

Check hardware specifications.

1. A physical network or a virtual network (VMware/Hyper-V)
2. Installed memory (RAM), type of processor and number of CPU core
   1. Run>msinfo32.exe>OK
   2. Type of processor XXX XX GHz xxx Core
   3. Installed memory XXGB

1. Max server Memory XX GB Free Memory XXX MB Number of Processors X
   • System>Serve Settings>Status

• If you are using Hyper-V or VMware, make sure a CPU core is not one for a large network and memory size is enough.
• Max server memory is java maximum memory pool (Step 12)
• If free Memory is not enought
  • Execute the garbage collector (Step9)
  • Increase memory and Set the value (Steop 12)

1. Number of Players. XXPlayers
   • Network>Players

• Confirm requirements from below links
  • Small <50
  • Medium <500
  • Large <2000

1. Provide logs
   1. Check ContentManger.log
   2. Check Config.log Check maxConnectionsPerParition and partitionCount
      1. Version 11.03

1. Before 10.05 maxConnections
1. Send a ContentManager-logs.zip file to a support team.
1. Check Performance. Open Task Manager>Performance>Open Resource Monitor
   1. Check any high usages.
   2. Check which .exe process is high

1. Check available Storage space
   • Resource Monitor>Disk>Storage

1. Open Event Viewer>Check errors in Administrative Events and System. Find disk, CPU and memory related issues.

1. Possible reasons
   1. After update? Has anything changed?
2. Open Server Monitoring and force indexing.
   1. CPU/Memory/Disk usage should not reach to 100%
   2. Execute the garbage collector
   3. Delete All Caches
   4. Determine which process is causing the issue.
   5. (Greater than 11.00) Force indexing.
   6. Restart Apache Tomcat service. Confim the perfomrance.
   7. Send PDF report to a support team.
3. Restart the server.
4. Delete unused media files, frames, playlists, maintenance jobs and players in Content Manager.
5. Check Tomcat Java Memory Settings You must check after increasing the memory.
   1. Example 11.03
   2. To change these settings, please open Windows Explorer and navigate to "C:\Program Files\Apache Software Foundation\Tomcat 7.0\bin". Once there, double click on Tomcat7w.exe, then click on the Java tab

1. Restart Apache Tomcat service.
1. Tuning Content Manager
   1. Max pool size. If you are experiencing symptoms of CPU spikes, poor performance, or Content Manager unresponsiveness after upgrading, try increasing the Max Pool size

1. Under no circumstances should the Max pool size in the UI be set to a value greater than 90% of the database's pool of connections (Max Pool Size <= .9 * database connections)
2. Changing the number of database connections in Postgres may require re-evaluation of Shared memory configuration for Postgres. This needs to be assessed on a case by case basis.
1. Check Player Heartbeat rate
   1. Network>Distribution>Increase Player Heartbeat rate.

1. Check Plan Generation History
   • Example. Version 11.02.03 850 players. It takes less than 2 min.
   • Network>Distribution>Select a server>Plan Generation History

1. Check Miscellaneous Settings. System>Network Settings>Miscellaneous Settings

1. Older than 10.05 and over 100 Players
   1. DB pool settings.
      1. Maximum Threads – Enter the maximum number of concurrent threads this connector can use. The connector will not be able to serve more simultaneous requests than it has threads. The default value is 150. You should set this based on how many players are directly connected to Content Manager. To ensure stable content delivery this value should be set to about 4/3 the number of connected players. Specifying a value of more than 1000 has little effect on Windows Operating systems, because there is a limit to the number of threads available to each process.
      2. Max keep alive - Enter the maximum number of requests that can be kept alive waiting for subsequent requests on this connector. The default is 100, and a value of 1 disables this setting. Generally, Scala recommends disabling this setting on networks with more than 50 players.
      3. Min spare threads - Enter the minimum number of spare threads for the thread pool on this connector. This setting is the number of threads that are available to serve requests when they arrive. The default value is 4. If your network contains more than 50 players, this setting should be set to 25.
      4. Check row counts of database errors (Step 2.b)
2. Send the database to a support team.
   1. Restore an old database which was not slow.