Slow message receiving/processing

[rjkroll]

Hi,

We have a new lab results channel where we receive normal HL7 results messages through TCP/IP. Last night, our Lab sent roughly 7,000 messages and it took almost 4 hours to "receive": them all and I am not sure why.

Our channel is setup as such:
TCP/IP Listener
Source Queue : OFF (Response after processing)
Response : Auto-generate (After Source Transformer)
Process Batch : No
Max Processing Threads: 1
Max Connections: 10
Receive Timeout (ms) : 30,000
Keep Connection Open: Yes
The channel is just parsing the message, getting the PID and the necessary OBX segments, then calling an insert into a SQL database.

Is there anything "off" with the settings above that can possible be adjusted to speed things up? I am wary of just changing things willy-nilly, but I know there has to be something we can do so that 7,000 messages do not take 4 hours to receive and import.

Any help or tips would be appreciated.

[jonbartels]

A common strategy is to turn on destination queues. However this means that if your INSERT fails then you cannot respond back with a NACK to the caller. Without queuing you have one thread processing messages off of those 10 connections and a single threaded source connector with no destination queues is single threaded on the source and the destination.

A related strategy is to leave destination queues off but increase your source threads. This will have 10 source threads with one destination thread each and let you process 10 messages in parallel.

Source threads are best if you need to generate ACK/NACK based on the INSERT being successful.
Destination queues are best if you can receive a message fast, queue it, respond to the caller, then INSERT it when you're ready.

I would encourage you to review queueing and threading in the MC user guides. You're wise to not go willy nilly and to think it through first!

[rjkroll]

Thanks. Your info about queuing made me think about what we were trying to accomplish with the channel. I made some tweaks and to allow for us to receive the messages without caring about waiting for the Insert, because in these cases, the Insert will likely never fail. This helped cut the receiving time of actually getting the messages from 4 hours to 10 minutes. It still takes around 15 minutes for the Inserts to actually happen, but a total of 30 minutes (which I am sure I can cut down some more time) as opposed to 4 hours is great.