Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Thread/Process pools #61

Open
PromyLOPh opened this issue Feb 21, 2016 · 1 comment
Open

Use Thread/Process pools #61

PromyLOPh opened this issue Feb 21, 2016 · 1 comment

Comments

@PromyLOPh
Copy link

Using thread or process pools and queues instead of starting a new thread one for each grab (or subtask) would limit the number of jobs running at the same time and thus keep the server load at manageable levels.

@HarryC145
Copy link
Member

load average: 150.87, 129.18, 115.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants