Add a "before poll" script

[pacmano1]

Add a "before poll" script to allow the rotation of values as needed.

For example I may want to update a server name in an SFTP poller each time it polls.

Discussed in Mirth Slack some time ago.

[elemle]

Yes, that would be excellent. A before poll script. Please add this feature ASAP to all polling channels.

[elemle]

I would call this script "Prepoll" or "Prepolling".

[cturczynskyj]

This is an interesting proposal. I would like some more examples of use cases for this to help gauge value.

[elemle]

Instead of writing 20 channels to loop through twenty different folder/file locations, a Prepoll script would allow one to use a list or any data structure to populate the directory and filename global or global channel map values that one channel (with 20 threads) would process.
${maps.get('3271aaaa-b56e-4c40-b55d-90da24f4d5f0_directory')}
${maps.get('3271aaaa-b56e-4c40-b55d-90da24f4d5f0_filename')}
...each thread would grab the next one to process via setting these values in the g or gc Prepoll script

[pacmano1]

Common for file readers.

I have 160 sftp servers I need to poll. (This is real). I need 160 channels for this. There is no trigger to rotate a value if the last poll was empty since there is no message.

I suppose there is an esoteric method using functions w/velocity as the file reader source and I attempted to get this working with @agermano but never was able to get it working.

Hence a pre-polling script.

[elemle]

I have another use case. Imagine that you have a folder and 100 places drop an SFTP file into it once a week.
Now there is a policy change that we no longer accept files via SFTP but we will still poll a remote (approved) location to pick up the file. Now I have one channel that needs to poll 100 different locations every Monday morning.

[pacmano1]

I think that is sorta said already.

[elemle]

Maybe, but the more traffic this page gets the sooner this will be implemented.

[elemle]

I wrote a channel that implements a Prepoll script by using a poll file that is created on deploy, this file is read first and then in the channel's preprocessor script I check if the message is the
text I have in this file which just says "Prepoll Script". That block runs and sets the polling field values by grabbing them from (however you want), then the next poll will look for that file and sets the polling back to the polling file afterward processing work is done. I have added channel pauses for the channel while I debug and perfect the code, but I think they may be removed later. This channel is not perfect enough to post here yet. But if you are good at mirth connect
​​​​​​​and re-read this post you can figure out how to do the same thing. After a few hours of work I think it is working correctly now.

[pacmano1]

Lost me there. A preprocessor script runs once for each message. So you read some marker file and then via java read the actual file manually building each row to pass to destinations? Maybe post your channel over at the forum when you are done.

I'm sure there are a few ways to do this but I would not consider them as easy as a pre-poll script that executes on poll and thereby is guaranteed to run.

[tonygermano]

The best a pre-processor script can do is set values for the next poll, but can't change anything at the actual time of polling. Like @pacmano1 said, it also won't fire if there are no messages created from the current poll values, so it will get stuck at the same settings until a message qualifies.

I have a working solution creating a custom java object that can be called from the source connectors which can return different values at poll time, but it's not a simple solution. The object necessarily must be stored in the globalChannelMap. This causes an issue at reprocess time if message processing relies on the current state of the object, because the state may be different than the one which originally generated the message.

@cturczynskyj Ideally, a "pre-poll" script would also have the ability to inject sourceMap values to track state within the messages that are to be generated.

[elemle]

Ah maybe. I did not care to change polling time. It just polls every 1ms. It won't get stuck because I add a .dot to the poll filename, until the next poll completes, Then remove the dot so it will poll again, same thing if it errors or the other file does not actually exist. It works, but I am still trying to test with multiple threads. Ideally I think I want each thread to have its own poll file. Still working on that.

[elemle]

Also I do not manually build anything. There are two polls for each file I actually want. The first poll is my poll file, which then let's me run code in the pre-processor and change poll values. The second poll will poll the actual file that needs processed. Then its back to poll one again. etc. etc.

[agermano]

I think you have the wrong agermano in here

[pacmano1]

LOL - sorry, I was on my phone and being lazy about searching. @tonygermano was the correct tag.

[pacmano1]

But this is a great feature everyone needs!! Upvote it. Selfish I am.

[tonygermano]

@elemle It will get stuck if you update the poll values, and there are no files that qualify with those values. You won't be able to rename your poll file until it finds a file as there is nothing else that would trigger any of your code to run.

[elemle]

I took care of this already by using java.io.File.exists() to check if the file is there before changing the polling values.

[elemle]

Pacmano1, it is not selfish. Mirth connect is the future. It mixes Monolithic with Microservices in a controlled manner. (Only terrible design with Mirth Connect is that it allocates memory for how many threads you set the channel to, instead of allocating them as needed.)
https://www.google.com/logos/2010/pacman10-hp.png

[elemle]

I made several changes to the channel today and ended up not needing the dot file rename that didn't work anyways.
Go here to get the channel------(I am StickyBandit)--------> https://forums.mirthproject.io/forum/mirth-connect/support/178332-parameterized-polling

[ChristopherSchultz]

Another option for pre-poll might be to say "yeah, go ahead and skip this poll process" to stop the process. Kind of like return false from a filter, except, ya know... from the pre-poll script.

[twest-mirthconnect]

@pacmano1 @elemle @tonygermano @ChristopherSchultz

Does anyone want to work on this as a PR? It has value just not sure it has enough ahead of many other priorities we already have for at least the next 6 months.

[pacmano1]

@pacmano1 @elemle @tonygermano @ChristopherSchultz

Does anyone want to work on this as a PR? It has value just not sure it has enough ahead of many other priorities we already have for at least the next 6 months.

Not a java developer. I have to defer.

[elemle]

At one point years ago I did download the Mirth Connect source code and built it. Nowadays, I don't have the time to keep the codebase up to date, let alone contribute because my regular job involves too many different projects etc.. Who knows maybe someday I can work for Mirth instead and I could implement the prepoll script.