More configuration options requested #92
Comments
|
Hi Arjen,
Without seeing your exact code I'm going to make some assumptions here so
bear with me.
1. For something like an email handler, this sounds like a command handler
and so you should only define this once.
If you have defined the handler once but host it in two applications that
each own their own queue, then this will multiplex the message and hence
handle it in two places. In this case you'd need to decide which app
handles this command and only define the handler there.
2. You're correct in that if you send a message and no process is listening
then the message should be queued up. This will only happen if an
application queue has been subscribed to the message topic. This just means
you register a handler for the messages you're interested in and start the
app so @node-ts/bus can set all the topic/queue subscriptions. At this
point if you stop the app and send a message then it will be queued.
If you don't do this, then the message gets sent to a topic that has no
queue subscribers and so is discarded.
Hope this helps. If this isn't your situation then feel free to send a code
example and I can provide a more specific response.
Cheers
Andrew
…On Wed, 7 Oct 2020, 18:11 Arjen Haayman, ***@***.***> wrote:
Right now I'm missing options to more finegrained configuration of the
message queues. Some behaviour I'd like to change are:
-
When I'm running multiple instances of -say- an email-handler and I
notice that both handlers get the message resulting in multiple mails. How
to prevent this
-
When I send a message while there's no process listening to that
message it is discarded. Isn't the whole idea of message queues that
messages can sit in the queue patiently until they're handled? I'm quite
surprised about this behaviour and would like to have some options to
override this
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#92>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOAR7K3W3NF5XHB2GLB7H3SJQPCLANCNFSM4SHBU4LA>
.
|
|
Hi Andrew, The email handler was just an example. But what if I have some heavy load service that I want to have horizontally scaled? I'd like to be able to start up a bunch of handlers that each get assign a message round-robin. Isn't that a valid use case that should be possible? |
|
Absolutely, and this should work out of the box. If you have a single
handler and you start that same node app several times then each should be
receiving from the same queue and provide the horizontal scalability.
Are you seeing something different?
…On Wed, 7 Oct 2020, 19:46 Arjen Haayman, ***@***.***> wrote:
Hi Andrew,
The email handler was just an example. But what if I have some heavy load
service that I want to have horizontally scaled? I'd like to be able to
start up a bunch of handlers that each get assign a message round-robin.
Isn't that a valid use case that should be possible?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#92 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOAR7PDDGJH6OYX5KWTXB3SJQ2G5ANCNFSM4SHBU4LA>
.
|
|
That's just it: each instance gets the message but I'd like only one instance to get the message. I'm afraid I'm only a newby concerning message queueing but I expect that it's possible to configure that a message is sent to a single queue and the first handler pops the message from the queue so the other handlers don't get it. More info maybe: I was running the apps on different machines. The 'sending' app was using messageQueue name 'app1' and both receiving apps were using messageQueue name 'app2' (the real names are hidden for privacy reasons). Could this be the reason why both app2 apps were interfering with each other? |
|
btw out of curiosity and highly offtopic: are you Dutch? |
|
The setup you described sounds correct in order to get multiple instances
processing the one queue without duplication. Do you have a basic code
example I could have a look at? It might be easier to spot what's going on.
And yes I'm half Dutch. The surname is a dead giveaway :)
…On Wed, 7 Oct 2020, 20:02 Arjen Haayman, ***@***.***> wrote:
btw out of curiosity and highly offtopic: are you Dutch?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#92 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOAR7NFCNY3T3GEDXG3YN3SJQ4FFANCNFSM4SHBU4LA>
.
|
|
it's not a problem that two apps share the same queue name? This morning I fixed a bug where app1 and app2 both used the same queue name and although both were connected I got the message there weren't any handlers so the messages got dropped (case 2 in the original post, so that's solved now). So I don't know what happens exactly when 2 apps use the same queueName but it appears that is causing troubles |
|
I'm afraid my code is too complicated atm to serve as an example |
|
Yes, each application needs its own dedicated queue. Each instance of an
application should share the same queue as it's other instances.
I hope I'm understanding your question correctly
…On Wed, 7 Oct 2020, 21:07 Arjen Haayman, ***@***.***> wrote:
it's not a problem that two apps share the same queue name? This morning I
fixed a bug where app1 and app2 both used the same queue name and although
both were connected I got the message there weren't any handlers so the
messages got dropped (case 2 in the original post, so that's solved now).
So I don't know what happens exactly when 2 apps use the same queueName but
it appears that is causing troubles
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#92 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOAR7MCK5CGMLZEYNFV46DSJRDWXANCNFSM4SHBU4LA>
.
|
What about postgres ? I have the same setup and postgres module is failing to save/retrieve data from the database, always erroring, because of race condition that happens when multiple containers handles messages and tries to save or upsert workflow data. issue #134 |
Right now I'm missing options to more finegrained configuration of the message queues. Some behaviour I'd like to change are:
When I'm running multiple instances of -say- an email-handler and I notice that both handlers get the message resulting in multiple mails. How to prevent this
When I send a message while there's no process listening to that message it is discarded. Isn't the whole idea of message queues that messages can sit in the queue patiently until they're handled? I'm quite surprised about this behaviour and would like to have some options to override this
The text was updated successfully, but these errors were encountered: