-
Notifications
You must be signed in to change notification settings - Fork 50
Open
Description
Hello,
I would like to contribute and add a bulk crawl function.
It would use multiprocessing to optimize the crawling duration.
The user can add the desired domains to a list in the file and execute it with the CL.
If needed I will also enable a text file, where the user adds all the domains.
@amitupreti: Please provide feedback
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels