Tim Miller Dyck from the mailing list has reported this problem.
He was testing out ICMP latency thresholds. They were working fine (he can see alerts triggering at the expected latency values).
However, He was not able to get threshold alerts to stop during scheduled outages where the ICMP latency threshold is marked to be ignored. The alerts continue throughout the outage.
Other types of thresholds are stopping during scheduled outages as expected – just the ICMP latency outages aren't working for him.
Checking the source code I've found that LatencyStoringServiceMonitorAdaptor is not checking if the node is on outage before applying the thresholds, and that's the reason of the problem.