[IMP] queue_job_cron_jobrunner: channel #750
Conversation
|
Hi @ivantodorovich, |
Koudja
force-pushed
the
14.0-foodles-queue_job_cron_jobrunner-channel
branch
3 times, most recently
from
656b232 to
70b10c4
Compare
| channels = {} | ||
| for queue_job in self.search([("state", "=", "started")]): | ||
| if not queue_job.channel: | ||
| continue | ||
| channels[queue_job.channel] = channels.get(queue_job.channel, 0) + 1 |
There was a problem hiding this comment.
suggestion: what do you think?
| channels = {} | |
| for queue_job in self.search([("state", "=", "started")]): | |
| if not queue_job.channel: | |
| continue | |
| channels[queue_job.channel] = channels.get(queue_job.channel, 0) + 1 | |
| channels = defaultdict(int) | |
| for queue_job in self.search([("state", "=", "started")]): | |
| if not queue_job.channel: | |
| continue | |
| channels[queue_job.channel] += 1 |
| result = queue_job | ||
| break | ||
| self.flush() | ||
| self.env.cr.commit() # pylint: disable=E8102 |
There was a problem hiding this comment.
suggestion: Maybe add a comment to explain why you can't use a different cursor (lock never released by Odoo)?
Koudja
force-pushed
the
14.0-foodles-queue_job_cron_jobrunner-channel
branch
3 times, most recently
from
53454b8 to
d72fbca
Compare
this make a lot of open file descriptors that we got the limit while instentiate jobrunner from queue_job_cron_jobrunner in the current channel implementation OCA#750
this make a lot of open file descriptors that we got the limit while instentiate jobrunner from queue_job_cron_jobrunner in the current channel implementation OCA#750
|
This PR require #754 to avoid opened file descriptor leak ! |
|
This PR has the |
this make a lot of open file descriptors that we got the limit while instentiate jobrunner from queue_job_cron_jobrunner in the current channel implementation OCA#750
petrus-v
force-pushed
the
14.0-foodles-queue_job_cron_jobrunner-channel
branch
from
c6ff271 to
918702c
Compare
|
@Koudja this is still in draft state, is this intentional? |
In fact we are still supervising the system and keep improving it as long we met difficulties ! yesterday we've implement the stop_processing feature to avoid to be kill by the cpu time limit in case there is a lot of job in the queue, this is not yet deployed to our production (and I'm going to somme logging). So I suppose keeping it in draft state as long we have not get some stabilities is probably better ! But review and feedback are welcome otherwise so not sure what suits most OCA conventions in such case ? |
petrus-v
force-pushed
the
14.0-foodles-queue_job_cron_jobrunner-channel
branch
from
918702c to
1c94494
Compare
|
draft is good I think, because of the label I nearly merged it |
| "queue_job_cron_jobrunner.stop_processing_threshold_seconds", "5" | ||
| ) | ||
| ) | ||
| if next_cron_job_runner_trigger.nextcall <= ( |
There was a problem hiding this comment.
after few investigation, I've the feeling the nextcall is not yet uppdated so we are probably dealing with the current started date so we would have to compute it...
There was a problem hiding this comment.
because odoo changed it on a different cursor !...
|
I'm wondering... now that the job scheduler (misnamed jobrunner) can run on odoo.sh (#668), queue_job_cron_jobrunner could maybe be simplified to only run jobs that are in state |
…job list wait the next cron trigger instead to avoid to reach the limit_time_real_cron limit.
petrus-v
force-pushed
the
14.0-foodles-queue_job_cron_jobrunner-channel
branch
from
1c94494 to
867608d
Compare
As of today, we are not using the mentioned patch and have configured our setup to avoid spawning the job scheduler/runner, as suggested in the module’s documentation. To be honest, we haven’t spent much time evaluating this patch, so we might have missed some key information. However, we do have some concerns:
If I understand correctly, your idea is to have the queue jobrunner focus on managing job states—ensuring that tasks are enqueued while respecting channel capacities—while letting the ir.cron execute them based on priority. This approach would remove the need for HTTP workers to process jobs. That sounds like an interesting direction and avoid to duplicate the channel implementation logic. Would you be open to further designing this together or discussing the long-term vision for the module? |
Implement channel on queue_job_cron_jobrunner