Orchestrator Script

Reset Monitor using SCOM 2012 and Orchestrator – A must have Runbook

There is at least one Runbook each SCOM responsible must have. Are you wondering which? Well, I am sure you are always aware of what kind of alert you are closing. Is this alert beeing generated by a rule or by a monitor. If it is generated by a rule you just can close the alert and you don’t have to care about anything. But if the alert is being generated by a monitor you always facing the problem that if you just close the alert itself your monitor will often stay in a critical or warning state. A good practice to close alerts from monitors is to fix the problem itself in your IT infrastructure and then go to the health explorer and reset the monitor. After that the alert will also close.

A similar problem occurs in a more advanced situation. Let’s assume System Center Operations Manager is generating incident tickets in System Center Service Manager through the connector. After the ticket has been closed in Service Manager the alert in Operations Manager will also close. But what if the alert has been generated by a monitor? Well, Service Manager does not reset the monitors in Operations Manager.

For both situation you could build a Runbook in System Center Orchestrator Smiley.

It looks like that….simple huh?

image

O.k. let me explain. The first activity “Monitor Alert” is constantly monitoring the alerts for updates. If an alert is updating its status from “New” to “Closed” e.g. you close an alert using the SCOM console or Service Manager closes the alert. Each time the Runbook will start. The “Monitor Alert” activity will pass the ID from the alert to the“Reset Monitor” activity and then run/trigger the script activity. This script will reset the monitor which has generated the alert. Easy? Yes.

Imagine it does not matter if you are closing the alert or Service Manager will close the alert in both situations if the alert was generated by a monitor the Runbook will do its work.

The “Monitor Alert” activity looks like this…

image

…and the “Reset Monitor” activity and script which I slightly modified from here looks like this…

image

Script Code:

# Import Operations Manager Module and create Connection
Import-Module OperationsManager;
New-SCOMManagementGroupConnection scom01;
# Get alert
$alertid=”Put here published data ID from “Monitor Alert” acitivity]“;
$alerts = Get-SCOMAlert | where { $_.id -eq $alertid};
foreach ($alert in $alerts)
{
# Get IDs
$mrid = $alert.monitoringruleid;
$mcid = $alert.monitoringclassid;
$moid = $alert.monitoringobjectid;
# Get Objects
$monitor = Get-SCOMMonitor | where {$_.id -eq $mrid};
$monitoringclass = Get-SCOMClass | where {$_.id -eq $mcid};
$monitoringobject = Get-SCOMMonitoringobject -class $monitoringclass | where {$_.id -eq $moid};
# Reset Monitor
$monitoringobject | foreach{$_.ResetMonitoringState($monitor)};
};

Make sure you modify line 5 with the published data!

Now every time the monitor gets reset properly and you don’t have to care about it anymore.

Cool, SCOM and SCORCH better together!

52 Replies to “Reset Monitor using SCOM 2012 and Orchestrator – A must have Runbook

      1. Stefan,

        I’ve been playing around with Orchestrator over the last few days and went to implement your runbook above.

        When I either test the runbook or simply just run it, I get a failure on the ‘Run .Net Script’ action with an error alert like this:

        File C:Program FilesSystem Center Operations Manager 2012PowershellOperationsManagerOperationsManager.psm1 cannot be loaded because the execution of scripts is disabled on this system. Please see “get-help about_signing” for more details.

        Now, I know you might be thinking that I haven’t set my execution policy to either ‘RemoteSigned’ or ‘Unrestricted’, but this was the first thing that I did on both the SCOM and SCOrch servers and when I run ‘get-executionpolicy’ on both servers, it comes back with ‘Unrestricted’. I’ve tried rebooting both servers and also recreating the runbook again.

        Any ideas for someone whose PoSh Kung Fu is pretty weak?

        Thanks!

        1. Hi Kevin,

          Cool, great hearing from you.

          Try to start the Powershell in “C:WindowsSysWOW64WindowsPowerShellv1.0powershell.exe” on your Orchestrator server and set there also Set-Executionpolicy unrestricted ;).

          Let me know if it worked.

          Regards,

          Stefan

  1. Stefan, that sorted the problem out straight away – I should’ve known it was something as simple as that – was trying to debug the issue a little too deep when I hadn’t thought about the WOW64 PowerShell.exe instance 🙂

    I’ll know the next time I have this issue for sure now!

    Thank you so much for the rapid response and excellent Orchestrator runbooks – it’s making my ramp-up on it much easier!

    Kevin.

  2. Stefan,
    Thanks for the great post as this will truly be helpful. HOwever, I’m having the following issue using SCOM 2012 and Orchestrator 2012. Using the following script;
    # Import Operations Manager Module and create Connection
    Import-Module OperationsManager;
    New-SCOMManagementGroupConnection ;
    # Get alert
    $alertid=”{Id from “Monitor Alert”}”;
    $alerts = Get-SCOMAlert | where { $_.id -eq $alertid};
    foreach ($alert in $alerts)
    {
    # Get IDs
    $mrid = $alert.monitoringruleid;
    $mcid = $alert.monitoringclassid;
    $moid = $alert.monitoringobjectid;
    # Get Objects
    $monitor = Get-SCOMMonitor | where {$_.id -eq $mrid};
    $monitoringclass = Get-SCOMClass | where {$_.id -eq $mcid};
    $monitoringobject = Get-SCOMMonitoringobject -class $monitoringclass | where {$_.id -eq $moid};
    # Reset Monitor
    $monitoringobject | foreach{$_.ResetMonitoringState($monitor)};
    };

    I get a SUCCESSFUL run when using the RUNBOOK tester, but when actually running the script I get the following error in my Orchestrator Log History;

    Run.NET script – FAILED
    Error Summary Text
    “The term ‘New-SCOMManagementGroupConnection’ is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.”

    Any idea why this is not working for me? Typing the same commands from my shell both x86 and x64 versions work fine???

    1. Update: also see this in event log.

      C:Windowssystem32WindowsPowerShellv1.0powershell.exe
      2544

      Microsoft.EnterpriseManagement.Core.Cmdlets, Version=7.0.5000.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35

      System.DllNotFoundException: Unable to load DLL ‘MOMBIDldr.dll’: The specified module could not be found. (Exception from HRESULT: 0x8007007E)
      at Bid.internalInitialize()

      1. Hi,

        I have seen this error before in other situations.

        Sometimes it helps to copy the ‘MOMBIDldr.dll’ to copy into the mentioned directory here C:Windowssystem32WindowsPowerShellv1.0powershell.exe. But please test it before going into production.

        Regards,

        Stefan
        .

    2. Hi Scorch Lover :),

      You need to go to “C:WindowsSysWOW64WindowsPowerShellv1.0powershell.exe” and open Powershell.

      Set the executionpolicy to unrestricted by typing Set-ExecutionPolicy Unrestricted

      I think after that it should work.

      Regards,

      Stefan

      1. Thanks Stefan, I had already done that. What I just found out is that Orchestrator ‘requires’ a reboot after installing the OpsMgr shell/console even though it does not ‘require’ a reboot 🙂 Guess it can’t load the .dll without the reboot. Just rebooted and it worked like a champ. Thanks again for the great article, look forward to seeing more in the future. Hope this helps someone else!

  3. I get this warning “The requested name is valid, but no data of the requested type was found”. Anyone have any ideas?

    1. You have to replace scom01 in the line “New-SCOMManagementGroupConnection scom01;” with the name of your own Mgmt-Server.

  4. Hi Stefan

    Hello i recive also the Warning/error above in the log history. I also recive the error Bad numeric constant: 80.
    any ideas?

    Andy

    1. Hi Andy

      Hmm, did you copy/paste my code? If so make sure all special characters e.g. -, etc. are what they suppose to be. Sometimes if you copy code from the web, it changes characters, and the code is unusable.

      Try to execute the code just in a Powershell command window, maybe you will see more what it is all about.

      Cheers,

      Stefan

      1. Hi Stefan
        Now it works. It seems that i have not replaced all the characters. Now I have it make with find/replace.

        Thanks for the Blog!
        Andreas

  5. Hi Stefan,
    Thanks for the great post as this will truly be helpful.

    My question is:
    After the monitor alert capture an alert the Runbook is finish.
    I want to continue monitoring new alert.
    What would be the best way to handle it?
    What happen if we got two alert at the same time (we have a big environment – around 600 servers) ?

    Thanks a lot for your great job.
    Gilad

    1. Hi Gilad

      Your welcome, glad I could help.

      The first activity of the runbook is already constantly runnig and “monitoring” for new alerts comming in. Unless you are stopping the runbook there is no need to do anything. The “Monitor Alert” activity is always “on”…

      Orchestrator will handle this for you, all new alerts will be processed separately you don’t have to be concernt about it. I am using this runbook also in a environment of ~600 servers.

      Conclusion: Just build it and test it, there is nothing that can go wrong….

      I hope this helps

      Stefan

  6. Hi Stefan,

    I’m using the Runbook and we see something strange.
    you may be able to help me with that.
    I see in the log that the trigger for the monitoring alert is activate every few second.
    i don’t now if there is a problem in SCOM or in ORCHESTRATOR but its cause us a very heavy traffic.
    i also see that we got the same alert every few second.
    i wrote the output to the log.

    This is example of the log:

    <1The monitor has been initialized for the first time or it has exited maintenance mode>
    <1The monitor has been initialized for the first time or it has exited maintenance mode>

    us you can see we got the same alert every ten second. (its only example of 2 alert but we got a lot of alerts)

    Do you know what is the problem?
    Thanks for your help,
    Gilad

  7. Hi,
    Thanks for this useful runbook example! I implemented it yesterday but encountered a problem:
    Each time an alert is processed by the runbook (Powershell) and is resetted, the “Monitor Alert” discovers it again and again (as for SCORCH, it seems to be an updated alert). I’ve tried to add the filter “MonitoringObjectHealthState” does not equal “Healthy”, but it did not work as expected. Do you have any recommendations?
    By the way, I haven’t used the “Run .NET Activity” but the “Execute PS Script” from Codeplex IP, as I wanted to run it as a different user than the SCORCH service account.
    Thanks in advance!

      1. Hi!

        I managed to fix Klaus’ problem. I added an extra activity to update a customfield in the alert. This results in the parameter LastModifiedBy of the alert beeing edited to the account that SCO uses to connect to SCOM.

        If you then add “LastModifiedBy does not equal … (the same account)” to the Monitor Alert filter, SCO will discover the alert only once.

        This offcourse only counts if the alert doesn’t get closed by SCO in the first place (with that account).

        Regards,
        Matthias

        1. Hi Matthias

          Thanks for your comment.

          Well this could work or just write another kind of “mark” into the customfield and check this “mark” as an additional paramater on the Monitor Alert Parameter.

          Cheers,

          Stefan

  8. Hi Matthias, Stefan,

    can you post the lines you added to the script? About LastModifiedBy….. I run in the same problem.

    Regards,

    Martin

  9. Hi Matthias, Stefan,

    I could configure the script like you said, i added to the alert a CustomField: Closed by Orchestrator
    The only thing I Need to figure out is that i have a lot of warnings with following value:

    Cannot validate argument on parameter ‘Class’. The argument is null or empty. Supply an argument that is not null or empty and then try the command again.

    $monitoringobject = Get-SCOMMonitoringobject -class $monitoringclass | where {$_.id -eq $moid};
    I guess the error is an empty value at $monitoringclass.

    Can you help me! Maybe we can Change the script to say if $Monitoringclass empty go to next….

    Regards,
    Martin

    1. Hi Martin

      Try to re-type the pipe character it might be a copy / paste error which causes the command to fail. If it doesn’t work, re-type the other characters.

      Cheers,

      Stefan

    2. Has this issue been resolved? The script works fine for me and then I will see 3 or 4 of these warnings.
      Thanks,
      Gregg

  10. Hi,
    I ran into the following problem. I get an error: Invalid language type specified. Could someone help?

    Thanks,
    Arne

  11. If I ran “C:WindowsSysWOW64WindowsPowerShellv1.0powershell.exe” I got this error. Now I changed the .Net Script back to PowerShell and the error is gone.

  12. Arne,
    I had the same issue. The problem in my case is/was that the executed powershell ran in an environment where the OperationsManager module was not registered and I was therefore not able to do import-module OperationsManager. However, I created a powershell script which I put on all Management Servers (c:userspublicdocumentsscriptsscomps.ps1) and which is “included” in all powershell command channels
    scomps.ps1:
    Import-Module “C:Program FilesSystem Center 2012Operations ManagerPowershellOperationsManagerOperationsManager.psm1”
    . “C:Program FilesSystem Center 2012Operations ManagerPowershellOperationsManagerStartup.ps1”
    . “C:Program FilesSystem Center 2012Operations ManagerPowershellOperationsManagerFunctions.ps1”

    and my command channel is as followed:

    Full path to cmd:
    C:WindowsSystem32WindowsPowerShellv1.0powershell.exe

    CMD parameters (example where I set a special resolution state to grab it via Orchestrator):
    . c:userspublicdocumentsscriptsSCOMPS.ps1; Get-SCOMAlert -Id (‘$Data/Context/DataItem/AlertId$’) | Set-SCOMAlert -ResolutionState 13;

    Startup folder:
    C:WindowsSystem32WindowsPowerShellv1.0

  13. Has anyone seen the issue for the line
    $monitoringobject = Get-SCOMMonitoringobject -class $monitoringclass | where {$_.id -eq $moid}
    that results in high memory consumption and never completes when the object is an interface on a network device? I see the following event id 26373 on the SCOM management server:
    The query resulted in 230773 rows.
    This may cause high memory consumption, and affect server performance.
    Consider using buffered mode for this query.

    1. We see this aswell. We recieve alot of Event 26373, OpsMgr SDK Service
      ___
      The query resulted in 182678 rows.
      This may cause high memory consumption, and affect server performance.
      Consider using buffered mode for this query.
      TypeId: Sytem.Entity
      Criteria:
      ___
      Any thoughts?

      1. I resolved it by using this modification for the monitoringobject variable
        $monitoringobject = Get-SCOMMonitoringobject -id $moid

  14. Stefan,

    This is an excellent runbook!
    I have added a little thing in our situation. This runbook reset also monitors for alerts that were closed by the system. We don’t need to reset this ones as well so I added to the scomalerts def:
    | where {$_.resolvedby -ne “System”}

  15. The Script can be faster and easier: The Runbook starts the Powershell-Script once for each Alert.
    So, you don’t need the for-each-loop. All used Commandlets accept Id as Parameter so you don’t need to querry all Objects and Pipe it to where {$_Id -eq …). With this enhancements no MonitoringClass-Instance is needed. And last but not least Get-SCOMMonitringobject returns only one Instance. Again there is no for-each needed.

    # Import Operations Manager Module and create Connection
    Import-Module OperationsManager;
    New-SCOMManagementGroupConnection hgf-int-vm-273
    # Get alert

    $alert = Get-SCOMAlert -Id “`d.T.~Ed/{A28FF5FE-996F-4211-84A8-16836A554471}.Id`d.T.~Ed/”

    # Get IDs
    $mrid = $alert.monitoringruleid
    $moid = $alert.monitoringobjectid

    # Get Objects
    $monitor = Get-SCOMMonitor -Id $mrid
    $monitoringobject = Get-SCOMMonitoringobject -Id $moid

    # Reset Monitor
    $monitoringobject.ResetMonitoringState($monitor)

  16. Hey Stephan ! Thanks for the awesome work ! … is that id on $alert = …. just the published data from the monitor alert activity ?

  17. Hello Stefan,

    thanks for this helpful Runbook. I have one Question:
    I am getting the error that my user has not enough rights to do the resolve. It’s my SCO User. I don’t finde the possibility to change the user. So should I give the SCO user rights at SCOM?

    Regards, Doreen

  18. Great Runbook!
    But one thing I have noticed when running this against Dependency Monitors is that the script has problem reseting the Health.

    Tried to run this script and manually put in the AlertID and got this answer

    3A9369B9-BA14-A5E0-DF81-7079B0255079
    The monitor belonging to instance of the selected entity
    was forced to reset its state. If monitor is unit monitor, it will contribute with
    ‘healthy’ state to its aggregate rollup monitor, eventually causing whole instance
    health state recalculation. When monitor is not unit monitor, it forces reset of
    every leaf unit monitor that rolls up its state thru this monitor. This task has no
    effect on selected dependency rollup monitor.

  19. Hi Stefan Roth,

    I implemented your runbook and it runs, but it runs ervery 10 seconds. I don’t have so much monitores to be closed. What can I check?

Leave a Reply to klaus Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.