Scrapy

The Udemy course is outdated, here is a youtube video I followed https://youtu.be/s4jtkzHhLzY . Scrapy for beginners

Info

Beautiful Soup’ can help extract specific elements from a target web page, while ‘Scrapy’ can manage asynchronous data retrieval, increasing efficiency

Scrapy is a more robust, feature-complete, more extensible, and more maintained web scraping tool. Scrapy allows you to crawl, extract, and store a full website. BeautilfulSoup on the other end only allows you to parse HTML and extract the information you’re looking for.

Scrapy video with vscode, https://youtu.be/ALizgnSFTwQ

Scrapy

The udemy course I have is outdated- this video looks more practical- https://youtu.be/y3nUE-RNETY- create conda python virtual environment- robot. Txt file contains certain information about rules and regulations of scrapping and can be seen by adding /robot.txt to website name- selector gadget chrome tool- scrapy shell example.com, gives interactive shell to find css query selector.-…

Scrapy medium

https://link.medium.com/MwfZhtFZ7yb- scrap my all bookmarked medium post and save as md file