[DISCUSSION] Broadcast functionality

[roziscoding]

Hi everybody.
I'm opening this so we can discuss how the broadcast functionality should work.
It is established that this will use the data stored by the chat-members plugin but, besides that, there are a few decisions we need to make:

1. Should it be able to filter chat types to send messages to?
2. Should it be available as an installable plugin that can be called from within a grammY bot (like bot.api.broadcast()), or should it be intended to be used alongside grammY (something like runner)?
3. What's a nice API shape?
4. Should it support an external queue service, or should it keep the jobs in-memory?
5. How should it handle timeouts and other errors?
6. Anything else you'd like to add

Also, please share your use-cases and experience with broadcasting in Telegram. How do you do it? How do you want to do it? What works for you? What doesn't?

Feel free to participate, no matter how much experience you have with grammY, telegram bots, or chatbots in general :)

[rojvv]

@KnorpelSenf enable the Discussions feature and convert this to a discussion?

[mrmaster009]

Here is what i feel

1. Yes , It Should have a filter For Chat Types to send messages to
2. It sould be a different plugin so we could specify the broadcast method easily like ( broadcaster.sendMessage() or some other calls )
3. I can't think of any ;)
4. It should keep jobs-in memory
5. It should have some time lag between two calls & If timeout errors are hit , it should retry doing it unless the call is successful. And for other errors , like 403: Forbidden , Bot is blocked by user , it should skip executing for that particular I'd to the next one
6. More things that could be added:
   something like ::
   • broadcaster.progress() : which tells how many chatIDs have be done ( either skipped or executed )
   • broadcaster.failed() : How many chatID were skipped during the broadcast upto where it was called
   • broadcaster.on('start'): to resolve when broadcast has started
   • broadcast.on('over') : to resolve when the broadcast has completed & return how much time it took to complete

I want to use broadcasting to send some message ( includes any type of message - text , photos , video , audio , etc .... )
As most of people have did , I tried broadcasting with a loop , but everytime it fails at some places , either doesn't broadcast to all users or stops in middle
I requested for the plugin , so obviously , my experience is not good with Broadcasting

Good Luck . Hope Speedy Development. Can't wait for it 😜

[KnorpelSenf]

The two main things to consider are rate limits and reliability.

Telegram will respond with 429 if rate limits are hit. This needs to be respected. The interesting part is that organic bot load does not count towards these rate limits.

What should happen if the bot crashes after broadcasting the message to only some of the targeted chats? Do we need to persist the progress as we go?

[mrmaster009]

Yes , The Broadcast progress needs to be continued as soon as bot restarts again
The whole point for the plugin is to stop it from crashing bot ( which even a loop will do ) ;)

[roziscoding]

Having the progress stored implies some kind of persistence layer for the queue, which kinda rules out a memory-only queue.
Perhaps we could have differente queue adapters, and the persistent adapters would be able to transparently pick up from where they left off?
PS: I use persistence here to talk about storing things outside the memory of the process that is running broadcast. It doesn't necessarily have to be literal persistence

[KnorpelSenf]

If the storages are able to enumerate all chat members in a stable fashion and start at a given offset, then we are able to implement everything via the existing storage adapters. All we really need to store is an index at which we left off. This means that we can store a mapping broadcasting task identifier → last chat identifier. After every sent message, we update this value to the next chat.

As a performance improvement, we could avoid writing this value after every single sent message. Instead, we only do this every n messages. This will increase the risk of sending duplicate messages on very rare occasions, but reduce the storage writes by a factor of 1/n.

Also, we could consider to never wait for the storage write. That way, rate limits will automatically lead to higher guarantees.

[AbbassAlmusawi]

I'm very interested in this thing because I need it in my bot work, Unfortunately, I cannot develop the plugin with you, because I haven't experience in building libraries, but I have some suggestions:

1. Not be built-in plugin, but you can install it like Runner
2. You can (start, pause, continue, stop) broadcast
3. Broadcast continue if exist after restart the bot
4. Can take any array of IDs and broadcast to them
5. Should be filter to chats (group, private, channel)
6. Can broadcast with any available method (sendMessage, copyMessage, sendPhoto, forwardMessage)
7. Built-in auto-retry plugin to retry send request after 429 error seconds end
8. Can set time between every request
9. I have txt file Contains errors encountered when send broadcast to large users could be help?

Maybe it's stupid suggestions but I do what I can to help :)

[KnorpelSenf]

Closing due to inactivity.