concurrent/async jobs

[wessel-techtribe-nl]

If multiple jobs are waiting with batch mode = false and each jobs takes 2 seconds, they are executed sequentially instead of concurrently (asynchronously).

If I run multiple runners (with bin/consoledtc:queue:run) at the same time concurrent handling works ok, but then batch mode (which should enforce sequentially if I understand it correctly) doesn't work.

This is in symfony 4 with ORM.

Is this something in my configuration?

[mmucklo]

@wessel-techtribe-nl Sorry for the delay in response.

If multiple jobs are waiting with batch mode = false and each jobs takes 2 seconds, they are executed sequentially instead of concurrently (asynchronously).

This is correct

If I run multiple runners (with bin/consoledtc:queue:run) at the same time concurrent handling works ok, but then batch mode (which should enforce sequentially if I understand it correctly) doesn't work.

No batch mode doesn't enforce the order. What batch mode does is if you submit multiple jobs with the same arguments, and they are all waiting to get executed (say the queue is backlogged by 1000 jobs), the jobs with the same arguments will all "clump" or "batch" together as one job (with only a single execution). Perhaps "batch" is the wrong moniker here. Maybe it should be called something different, like "merge" mode or something.

The idea is if you have a burst of activity and the queue gets backed up, and say you have an event that sends you a page once per minute by enqueueing a job every minute. You won't suddenly get a flurry of pages once the queue starts to catch up if you set batch mode on. Another example could be a password-reset email that gets enqueued, but back logged, and the user retries 100 times to send it, it can batch or merge those jobs as a single email-reset job, so they'll only get a single email once the queue catches up.