look into changing over to brpoplpush

[tj]

atomic goodness for queues, could get rid of my hack of a zpop

http:https://redis.io/commands/brpoplpush

[maboa]

Any idea if/when this is coming? It looks like we could really do with this feature too!

Keep up the great work BTW and thanks! :)

[behrad]

I'm thinking of doing this on a major version release, specifically 1.0.0 since it requires a very big code change, and kue redesign on redis sets/lists usage.

But you currently should have no problems in many deployment situations with the existing implementation.

[maboa]

Thanks Behrad! afaict this will only be an issue if something fails. We need to detect that situation so we can resubmit.

[behrad]

when something bad fails, everything can be an issue, event with the atomic brpoplpush

[pgte]

As I see it, with atomic brpoplpush the work stays in redis. Without it, there's a brief windows between a pop and a push where, if the node.js process goes down, the work is lost. Am I right?

[behrad]

right @pgte , in the current Kue code we have seen a couple of recovery codes from this situation

[behrad]

A good example of how to provide priorities:
https://github.com/josiahcarlson/rpqueue/blob/master/rpqueue/__init__.py#L226

[landau]

Any movement on this?

[behrad]

I'd planned to start the refactoring to atomic Kue 1.0 early 2015, however it was my very busy days. I'm thinking of starting it in a few months, and remember this is a really huge refactoring effecting all Kue

[behrad]

if you have a local redis instance (or one without poor network connection) you won't face queued stuck jobs. however active stuck jobs is something caused 95% by your app code.

[pawawat]

Hi,
Any progress on this?

[behrad]

I've been very busy for a couple of months...

[vieks]

up ?

[rturk]

Hi @tj any plans to work on this?

[behrad]

brpoplpush is one of methods of reaching atomicity/consistency but in favor of loosing priorities...
I don't want to drop priority queues, and implementing them with brpoplpush adds some complexities.
Another method is using lua scripts which is being preferred and implemented in v1 branch currently.

[rturk]

@behrad. That's the point: atomicity/consistency is very important for me, priority not. Most my jobs will complete in less than 1s. Appreciate your thoughts about bee-queue?

[behrad]

have you tested v1 branch @rturk ?
you should align with your requirements if it fits you the bee-queue @rturk

[mir-nawaz-10p]

@behrad any updates with the 1.0.0 release?

[Palisand]

Hello from 2018, how's 1.0.0 looking?

[MetinSeylan]

hello from future, our cars can flying but kue is still under 1.0.0