APM in SCOM 2016 | Doomed or Saved?

There is a scary bug with SCOM 2016. If you are using Application Performance Management (APM) feature with SCOM 2016 you may possibly run into an issue where the SCOM 2016 Agent may cause a crash for the IIS Application Pool running under .NET 2.0 runtime. The underlying cause dor this issue is that the APM code of SCOM 2016 Agent utilize memory allocation within the APM code of the Microsoft Monitoring Agent, that is incompatible with .NET 2.0 runtime. This results in a crash  if this memory is later accessed in a certain way. The SCOM 2012 R2 agent doesn’t have this issue since the code that cause this behavior is not present in that version. 

Microsoft has provided a fix for this issue with SCOM 2016 Update Rollup 3. Unfortunately this hotfix seems useless in rectifying this issue. Microsoft is working on another hotfix to correct this behavior. 

There are several workarounds that you can perform in order to remediate this issue.

  • Migrating the Application pool  to .NET 4.0 Runtime;
  • Installing the SCOM 2012 R2 Agent as it’s forward-compatible with SCOM 2016 Server and APM feature will continue to work with the older binaries;
  • Reinstalling the SCOM 2016 Microsoft Monitoring Agent with NOAPM=1 switch in msiexec.exe setup command line to exclude the APM feature  from setup;

There are some additional issues casued by this bug.

SharePoint Central Administration site stops working when SCOM 2016 Agent is installed onto the server

Even though the APM feature is in disabled mode by default when you install the SCOM 2016 agents, it adds a registry setting to load inactive APM into IIS Application pools. If you don’t configure APM in the SharePoint Servers, the application pools will have APM loaded in inactive state without monitoring. It has been reported that the inactive APM may crash SharePoint Central Administration v4 application pool and prevent the application from starting.

Known Workarounds

  • Install SCOM 2012 R2 agent if APM is needed.
  • If APM is not need, re-install the SCOM 2016 agent with “NOAPM=1” from the command line.
Web Site crashes during startup when SCOM 2016 Agent is installed 

As described above, APM adds a registry setting to load inactive APM into IIS Application pools regardless of APM is disabled (but installed) or not. The application pool account needs access top the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\System Center Operations Manager\12\APMAgent registry key,. If it cannot access this registry hive, the inactive APM configuration cannot read that key and the application pool process may crash. As a workaround you can add “Read” access permission for the application pool account to this registry key.