Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IDEA] Advanced Clustering Breaks Advanced Alerting Functionality #5415

Open
RunnenLate opened this issue Sep 9, 2022 · 4 comments
Open
Labels
enhancement New feature or request Internal-Issue-Created An issue has been created in NextGen's internal issue tracker RS-9335 triaged

Comments

@RunnenLate
Copy link

RunnenLate commented Sep 9, 2022

Summary
When using Advance Alerting in conjunction with Advanced Clustering the alerting does not work in a manner that is useful for determining if a channel is not receiving messages. The clustering configurations are not independent of each other in that if you start alerting on server A it will also start on server B, yet the monitoring of message flow is independent in that Server A will only count messages received on server A and server B will only count messages received on server B. Thus, the alert will trigger from server B even if messages have been received on server A. Either the server configuration needs to be independently managed in that turning an alert on A should not turn on the alter for server B or more preferable message monitoring should be calculated as a sum of the messages received between both servers.

Describe the bug
When Using Advanced Clustering plugin and the Advanced Alerting plugin alerts are triggered independently of the other server's behavior. For example, if a channel is processing inbound messages on port 5000 and an alert is configured to trigger if has received <= 0 messages within a minute. If a message is received on server A at 30 seconds from start then Server B does not appear to see the inbound message and will trigger the alert one minute since it didn't receive the message directly. Alerting should be checking that the channel has received a message, not each server has received a message.

To Reproduce
Setup up two identical servers with the clustering plugin installed
Create an HTTP channel
Configure alerting with the following configurations:
189231677-6df95471-4201-46e0-a5a7-bdac989e3203
Continuously HTTP messages to the port for the channel you just created using the IP:Port for server A.
Altering will trigger after 1 minute to Level 1, 2 minute to Level 2.
Sending a message to each individual server within a minute period will cause the alert to return to normal

Expected behavior
Alerts do not trigger based on individual server message processing by by channel processing as a whole between multiple servers. (E.G. if server A processes all messages then server B should understand that and not trigger)

Actual behavior
Alter is triggering based on individual server processing, not cluster processing.

Environment (please complete the following information):
Tested on
CentOS 7 mirth Connect version 3.10.1 with SQL database
Windows server 2019 with Mirth Connect version 4.1 with Mysql database

Workaround(s)
None with Clustering Plugin enabled

@RunnenLate
Copy link
Author

After speaking with support this is an issue because the load balancing functionality of the clustering plugin when using the appliance does not work as explained. I've been told that if a message is sent to the VIP it should balance messages between both servers. But in practice, all messages will go to a single server, the one that holds the VIP. The configuration state it should be a round robin
image

@RunnenLate
Copy link
Author

Changing this over from a bug to an idea as I see how this functionality is supposed to work. Altering should have the ability to monitor a channel based on the aggregated data between both servers instead of each individually. This would be needed if you the servers are set up in a failover instance.

@RunnenLate RunnenLate changed the title [BUG] Advanced Clustering Breaks Advanced Alerting Functionality [IDEA] Advanced Clustering Breaks Advanced Alerting Functionality Sep 16, 2022
@pladesma pladesma added enhancement New feature or request triaged Internal-Issue-Created An issue has been created in NextGen's internal issue tracker RS-9335 and removed bug Something isn't working labels Sep 23, 2022
@pladesma
Copy link
Collaborator

@RunnenLate I've created an internal ticket to address your feature request. Regarding the load balancing issue you're experiencing with appliances, that is a known issue that's on our radar.

@RunnenLate
Copy link
Author

RunnenLate commented Jan 12, 2024

@pladesma Is there any update on this or what needs to happen to get this worked on? Can you reference me the internal ticket for this feature?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Internal-Issue-Created An issue has been created in NextGen's internal issue tracker RS-9335 triaged
Projects
None yet
Development

No branches or pull requests

2 participants