Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can I setup multiple queues in 1 application? #62

Open
hanachan1026 opened this issue Jan 27, 2022 · 3 comments
Open

How can I setup multiple queues in 1 application? #62

hanachan1026 opened this issue Jan 27, 2022 · 3 comments
Labels
support Someone needs help

Comments

@hanachan1026
Copy link

When I setup single queue, it works like a charm. But if I want to have multiple queues, since we can only specify one queue url in SQS_QUEUE I don't know how to setup.

The reason why I want multiple queues in 1 application is that I want to know if a specific job is still processing. To do that I separate queues for each jobs and long polling the status of queue.

@hanachan1026 hanachan1026 changed the title How can I setup multiple queues? How can I setup multiple queues in 1 application? Jan 27, 2022
@mnapoli
Copy link
Member

mnapoli commented Jan 27, 2022

I think you would have to create a new queue connection in config/queue.php (per Laravel's documentation).

And then:

  • duplicate the worker function
  • duplicate the worker.php file and change the queue connection and queue name used in there (I mean these lines:
    'connection' => 'sqs', // this is the Laravel Queue connection
    'queue' => getenv('SQS_QUEUE'),
    )

@mnapoli mnapoli added the support Someone needs help label Jan 27, 2022
@aristidesneto-bnw
Copy link

aristidesneto-bnw commented May 30, 2022

@mnapoli Could you tell why this implementation does not work?

I'm dynamically getting the queue name through the event variable. It doesn't show any error, but also the message is not sent to the queue.

# File: worker.php

return function (array $event, Context $context) {

    /** @var Application $app */
    $app = require __DIR__ . '/bootstrap/app.php';

    $kernel = $app->make(Illuminate\Contracts\Console\Kernel::class);
    $kernel->bootstrap();

    $message = $event['Records'][0];

    $queueName = explode(':', $message['eventSourceARN'])[5];

    $queueUrl = getenv('SQS_PREFIX') . $queueName;

    Log::info("Message '{$message['messageId']}' sent to '{$queueUrl}' ");
    
    return $app->makeWith(LaravelSqsHandler::class, [
        'connection' => 'sqs',
        'queue' => $queueUrl,
    ]);
};

References:

Would you help me?
Thanks

@alexw23
Copy link

alexw23 commented May 30, 2024

Did you figure this out?

I haven't tested this but you should be able to put an environment variable per handler.

constructs:
    primary-queue:
        type: queue
        worker:
            handler: Bref\LaravelBridge\Queue\QueueHandler
            runtime: php-81
            timeout: 60 # seconds
            environment:
                 QUEUE_HANDLE_CONNECTION: primary
     secondary-queue:
        type: queue
        worker:
            handler: Bref\LaravelBridge\Queue\QueueHandler
            runtime: php-81
            timeout: 60 # seconds
            environment:
                 QUEUE_HANDLE_CONNECTION: secondary

Then override your the existing service provider definition of $connection, in your own AppServiceProvider.

$this->app->when(QueueHandler::class)
            ->needs('$connection')
            ->give(env('QUEUE_HANDLE_CONNECTION'));

Ideally this would all be magically resolved by QueueHandler instead based on the Job Queue ARN itself. I'll see if I can put together a PR later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
support Someone needs help
Projects
None yet
Development

No branches or pull requests

4 participants