Skip to content

Commit

Permalink
Update spider.py
Browse files Browse the repository at this point in the history
  • Loading branch information
mevljas committed Jan 19, 2025
1 parent 576ecc9 commit 4e53082
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion spider/spider.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,9 @@ async def run_spider(database_manager: DatabaseManager):

# create a new page inside context.
browser_page = await browser.new_page(
# pylint: disable=line-too-long
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36"
) # pylint: disable=line-too-long
)

# Prevent loading some resources for better performance.
# await browser_page.route("**/*", block_aggressively)
Expand Down

0 comments on commit 4e53082

Please sign in to comment.