Understanding an Heart Beat Exclusion properly

edited October 2013 in General
Hello,
We are testing this product because we are considering moving away from our current monitoring software. I have the software installed. During the install I selected the option to start the Heartbeat automatically. Since that time I am getting a ton of emails. I tried to exclude some events from coming through.
This is what my exclude looks like.
Actions
Default Email
Heartbeat Alert

Log
Application

Event Severity
Error

Filter Settings
Exclude

Details
ES EventSentry
Category Service Monitoring
EventID 10100

The rest is default

These are the events I am getting

EVENT # 6095
EVENT LOG Application
EVENT TYPE Error
OPCODE Info
SOURCE EventSentry
CATEGORY Service Monitoring
EVENT ID 10100
COMPUTERNAME ABC
DATE / TIME 10/24/2013 1:06:06 AM
MESSAGE The status for service trustedinstaller (Windows Modules Installer) changed from Running to Stopped.

Additional Service Information:

Startup type: Manual
Executable: C:\Windows\servicing\TrustedInstaller.exe
Service account: localSystem

I saw in the readme it says excludes preclude includes what am I missing?

Thank you in advance.

Comments

  • Hi Jason,

    It would appear that you defined the exclude fields correctly. Is it possible that package where this exclude filter is located has not been assigned to all your hosts? Also the include filter that is forwarding the events may be in a package that has the 'Ignore exclude filters' options checked.

    A helpful tool in this case would be the Filter Test Utility. If you use the EventSentry Event Log Viewer, you may right click any event and select Test against Filter Rules. This will show you all filters that match the event when it occurs. This utility can be accessed in Tools > Utilities as well.

    http://www.netikus.net/software/eventsentry/index.html?console_testingfilterrules.htm

    I hope this helps.
  • When I run that test it just opens the exclude.
    I only have on test machine so I don't follow on all hosts above. Some of this is just new so it seems a bit foreign.
    Is it required to put information on the event details?
    I may just have to turn this off is there a way to disable this globally?
  • I also checked the ignore exclude filter for package heartbeat and it was not checked.
  • Jason,

    The alerts you are getting are coming from service monitoring (from one of the service monitoring packages under "System Health"), not from the Heartbeat Monitor. The HB monitor is only responsible for checking the uptime of remote hosts through PING and TCP.

    I'm surprised that you are getting those emails since the service you listed above is excluded by default - you should not be getting alerts about it.

    How many hosts are you currently monitoring - is it just one machine where you ran the setup?

    Do you know which email action sends you the alert? You can add the $FILTER variable to the subject to see which filter triggered the email.

    In which package is the exclude filter you created? Is it an existing package or a new package? Is the package assigned globally?
  • I have installed the light version one one machine. All off the notifications are coming from this machine.
    To make things easier I did a fresh install it is totally defaulted I have not changed or added anything at this point.
    I am still getting these email messages.
    The filter variable is already there and this is what it returns.
    ES [1] EventSentry:Service Monitoring:10100 by Email Critical Events

    I do not know what action it is. Is it critical Events? How would I know what package it is?
  • Thanks for the clarifications, that helps. We'll look into the default install to make sure that it does not send excessive notifications going forward. Thank you for letting us know.

    I would like to start investigating why the exclude filter you created is not working. To which event log package did you add the exclusion filter? Did you create a new package, or did you add the filter to an existing package?
  • At this time I have not added a package or an exclusion filter. I would like to create one to filter these events. When I tried before I created a new filter under the heartbeat package. What package should I create the exclusion filter under?
    Also if we can get this working can I import this configuration in to the paid version?
  • Yes, you can easily upgrade from the light to the full version. Simply uninstall the light version and choose to keep the configuration. Then, install the full version.

    It doesn't matter which package you create it under, for as long as the package is assigned to the correct computer and/or group.

    I would suggest that you create a new package (right-click Event Log Packages) with a descriptive name (e.g. "My Exclusions"), and make the package global (as described before).

    The fastest way to create the exclusion filter is to open the "Application" event log and find the event in question. Right-click the event and select "Add Exclude Filter". In the resulting dialog, select the package you just created before (or you can select an existing one if you didn't create a package) and create a descriptive name (e.g. "Ignore Service Status Changes").

    Then, select the actions you want this excluded from (I would select the Default Email) and verify that the filter properties are correct.

    Then save the configuration and you should be all set.

    Please let us know if that works - thanks!
  • Ok I added this as requested.
    What actually is supposed to happen when you run the test? When I run it, it shows the rule is that correct?
  • Yes, by default the filter rule test shows all the matching filters only. If you select "Verbose", then you can see all filters and why they match or do not match.

    Did the notifications stop now?

    I also wanted to point out that you can exclude those alerts by activating two of the other service monitoring packages.

    If you expand the "System Health" packages, then you will notice a total of 3 service monitoring packages (Services Workstations, etc.). Right-click any of the packages which are not global, and select "Global". Then save the configuration.

    This should exclude the services right inside the service monitoring feature as well - so the alerts not even generated in the first place.

    Please let me know if that helped. Thanks!
  • Ok understood. This is a good exercise just to see how this all works any way.
    Another thing I noticed is that if an exclude is created it may cause an issue with an event that is needed. For instance our example is 10100 and I am getting other notifications that use the same eventid. I assume this is where the content filters come in. I added "The status for service trustedinstaller" with text match type wilcard match (default).
    Is that the correct way to only get events that match this string? Is that enough or do I have to put the entire and exact message in the content filter? Lastly can you tell the software not to even write those messages to the event log?

    I am also trying to follow this article as this is a Windows 2012 server with Storage Spaces volumes.
    I am not sure how to tell this is actually running. So far I don't see any events or notifications up to this point.

    http://www.eventsentry.com/blog/tag/windows-2008-software-raid-mirror-event-log-failed-notification-email-alert
  • Generally speaking you are correct, the exclusion you setup will apply to all service status change events, and in most cases you will want to know when a critical service changes its status from running to stopped for example.

    You will need to use wild cards such as the asterisk character when specifying a filter text, e.g. *trustedinstaller* would do the trick.

    You can also use insertion strings, please see http://www.netikus.net/software/eventsentry/configpackagesfiltersfilterpropertiescontentfilter.htm for more information.

    We also have a number of tutorials on how to create filters, please see http://www.eventsentry.com/support/tutorials. They explain the setup of filters with a variety of scenarios.

    Yes, you can tell EventSentry not to write the service status changes to the event log by excluding the service from being monitored to begin with. This is what I had suggested earlier by making all the service monitoring packages global. This tutorial has some information on this as well: http://www.eventsentry.com/support/tutorial/topic/service-monitoring/step/1.

    I'm not exactly sure regarding the issue with the 2012 machine - do you have a software RAID configured? You will have to setup the VBS script from the blog article and schedule it from EventSentry, this will continuously poll the status and log an error when a fault is detected.
  • The filter with the * worked thanks for that.
    I don't know if I want to exclude the services from being monitored I have to read that a couple more times.

    I have followed the directions from that article and told the software to write to the event log but I don't know how to make sure it is working. I don't see anything in the event log.
    If I go ahead and pay for a license can I call in for support?
  • The best thing to do, in regards to the blog post, would be to create that VBScript and run it from the command line. I'd be happy to help you through the forums, but it would probably be a good idea to open a new discussion / thread for this.

    And yes, if you purchase 1 or more licenses then you can definitely email or call for support.
Sign In or Register to comment.