At a customers SCOM infrastructure I faced a very strange problem. The SCOM infrastructure consists of 2 management servers and 1 SQL server. Only one of the management servers stopped in certain time intervals responding to the agents and encountered the following symptoms:
- All agent computer which had this management server as primary management server sent all of a sudden critical alerts.
- In the System event log a huge amount of errors event id 7011 from Service Control Manager.
- The management server did not respond to TCP port 5723
- The management server appeared running also all services seemed to be up and running.
- After restarting the HealthService on the management server everything went o.k. until a couple hours/days later.
I had been browsing through every event log to get a picture about the situation and in the System event log I found many errors, the detailed error description in the event log was…
“A timeout (30000 milliseconds) was reached while waiting for a transaction response from the HealthService service.”
After some investigation there was an event id before the error occurred which finally pointed me into the right direction…
Event 19
“Installation Successful: Windows successfully installed the following update: Definitions update für Microsoft Endpoint Protection – KB2461484 (Definition 1.133.361.0)”
Event 28
“Automatic Updates is now resumed.”
It seemed that the WindowsUpdateClient is making some problems. For me it looked like that the update service would prevent the HealthService from running. There was also SCCM 2012 deployed and after some more investigations I disabled the following service…
…and disabled the Task “Configuration Manager Health Evaluation” on the management server
..then everything ran smoothly and the management server never stopped again.
I know this is not a final solution but it helps at least for the moment. I will try to find a final solution but until then I leave it this way.
I hope you find this helpful…
Stefan
We have startet getting this problem on our SCOM RMS Emulator and SCOM Server – we are running SCOM 2012 SP1, and the “System Center Management Service” gets into “Paused” state after a couple of the Event 7011. It looks like the issue comes just after FEP Definition Updates on the SCOM servers, så maybe these has something to do with the problem ?
Please adwise – this is our production environment.
Yours Erling B. Kjeldsen
University of Southern Denmark
Hi
My solution was to remove the SMS Agent Host (SCCM Agent) and reinstalled it. I would remove the SCCM Agent and see if it helps, if the System Center Management Server starts without a Problem, re-install the SCCM service.
OK?
Cheers,
Stefan
Okay – so after reinstalling the SCCM Agent the error did not come back – will try and get back to you thanks..
I have deinstalled sccm client ccmsetup /UNINSTALL and booted, installed the sccm-klient again, and until now (the last 30 minutes) my SCOM Servers are working as supposed to. The indicator for others is “gray MGMT Server” in “Management Servers view and/or the event 7011 in the System log on the management servers.
I will monitor a couple of days and get back with the results.
Have monitored the two SCOM Mgmt. servers now for a couple of hours. As soon as i start the SMSAgent Host Service on one of the SCOM Servers, the Server gets grayed out on the mgmt. console under Admin, Management servers.
Now i have tried shutting down the SMSAgent Host service, disabling it, and restarted the “System Center Management” Service. As soon as i do this, the Management server changes from Gray to green in SCOM Console.
Is this issue adressed in SCOM/SCCM 2012 SP1 CU1 or CU2 ? We are running SCOM/SCCM 2012 SP1 (no CU) at the moment.
Hi
I don’t know if this problem is solved or in any of the CU’s/UR’s. So it is definetly an issue with the SCCM service same story as mine. Check if there is a corrupt update from FEP and install the latest updates for all SC products.
Cheers,
Stefan
Okay will give it a try in the morning.. thanks so far 😊
Hi again – todays update 🙂
The SCOM Servers are now working, i uninstalled FEP and Uninstalled CCM-Agent left the two servers running 10 hours, and installed the latest SCCM Agent (Push-install) wich also installed FEP (latest version). both SCOM servers were rebooted and everything worked fine – some time.
This morning i found that the problem seems to be on all our SCOM clients (or at least many of them as well.
I get error in SCOM Log on RMS stating as follows:
The health service {5C52FE82-84C9-9F71-282B-6F18757F0F90} running on host (SCOMCLIENTNAME) and serving management group XXXXXX with id {DE67708F-FC33-40E2-ABC8-F70A4D06BBA0} is not available because service has been paused.
At the same time (within 1-2 minutes) i get this alert on the SCOM Client:
Installation Successful: Windows successfully installed the following update: Definition Update for Microsoft Endpoint Protection – KB2461484 (Definition 1.149.544.0).
So now i am thinking – do i have to manually uninstall all my SCCM Clients from our 300 servers, Uninstall FEP mnually from the 300 servers, and do a fresh push on all of them – ARRGH this day just went from great to worse.
Just read your answer once more – i will try to apply CU1 and CU2 to my SCCM/SCOM 2012 SP1 installation and then see if it all goes away 🙂
I have contactet Kent Agerlund from Coretech – Kent is an expert on SCCM 2012, and Kent had an idea that i should check. The FEP automatic deployment rule in my SCCM, had a setting under the “Alerts” settings, i had “disable Operations Manager alerts while software updates run” marked.
This option seems to set the server receiving the update into “Maintenance Mode” in SCOM – and that’s not so great for the Management server. – I have corrected this, and now i wait for 5 hours for the next FEP Definition update to kick in.