Python Libraries Used in Web Scraping http-client requests 48,858 2.28.2 (11 days ago) Feb 14 2011 aiohttp 13,174 3.8.3 (4 months ago) Jul 26 2019 httpx 9,877 0.23.3 (19 days ago) Jul 26 2019 pycurl 935 7.45.2 (a month ago) Feb 25 2003 treq 553 22.2.0 (11 months ago) Dec 28 2012 html-utils bleach 2,463 5.0.1 (6 months ago) Feb 21 2010 cli-tool youtube-dl 116,598 2021.12.17 (1 year, 1 month ago) Feb 22 2012 you-get 46,483 0.4.1650 (a month ago) Sep 01 2012 scraper youtube-dl 116,598 2021.12.17 (1 year, 1 month ago) Feb 22 2012 trafilatura 722 1.4.0 (3 months ago) Jul 17 2019 you-get 46,483 0.4.1650 (a month ago) Sep 01 2012 blocking cloudscraper 2,712 1.2.68 (13 days ago) Dec 28 2012 html-extractor html2text 1,349 2020.1.16 (3 years ago) Dec 14 2008 newspaper 12,365 0.2.8 (4 years ago) Dec 28 2012 trafilatura 722 1.4.0 (3 months ago) Jul 17 2019 readability 2,206 0.8.1 (2 years ago) Jun 30 2011 extruct 728 0.14.0 (2 months ago) Oct 27 2015 sumy 3,007 0.11.0 (3 months ago) Oct 20 2013 photon 9,316 1.1.9 (4 years ago) Aug 24 2018 extractnet 70 2.0.7 (2 months ago) Dec 11 2020 text-parsing sumy 3,007 0.11.0 (3 months ago) Oct 20 2013 html-parser beautifulsoup - 4.11.1 (9 months ago) Jul 26 2019 html5lib 1,004 1.1 (2 years ago) Jul 30 2007 cssselect 266 1.2.0 (2 months ago) Apr 14 2012 feedparser 1,532 6.0.10 (8 months ago) Jun 15 2007 xmltodict 1,532 6.0.10 (8 months ago) Jul 30 2007 pyquery 2,170 2.0.0 (26 days ago) Dec 05 2008 parsel 864 1.7.0 (2 months ago) Jul 26 2019 requests-html 12,893 0.10.0 (3 years ago) Feb 25 2018 untangle 569 1.2.1 (6 months ago) Jun 09 2011 html5-parser 643 0.4.10 (1 year, 4 months ago) Jun 03 2007 gazpacho 694 1.1 (2 years ago) Dec 28 2012 selectolax 722 0.3.12 (22 days ago) Mar 01 2018 chopper 22 0.5.0 (6 months ago) Jul 24 2014 framework scrapy 45,593 2.7.1 (2 months ago) Jul 26 2019 scrapyd 2,576 1.3.0 (1 year, 11 days ago) Sep 04 2013 autoscraper 4,845 1.1.14 (6 months ago) Jul 26 2019 scrapydweb 2,649 1.4.0 (3 years ago) Sep 30 2018 gerapy 2,895 0.9.12 (27 days ago) Jul 04 2017 ruia 1,641 0.8.5 (4 months ago) Oct 17 2018 photon 9,316 1.1.9 (4 years ago) Aug 24 2018 dude 327 0.1.2 (2 years ago) Feb 20 2022 gui scrapydweb 2,649 1.4.0 (3 years ago) Sep 30 2018 gerapy 2,895 0.9.12 (27 days ago) Jul 04 2017 scrapy scrapy 45,593 2.7.1 (2 months ago) Jul 26 2019 scrapyd 2,576 1.3.0 (1 year, 11 days ago) Sep 04 2013 scrapydweb 2,649 1.4.0 (3 years ago) Sep 30 2018 gerapy 2,895 0.9.12 (27 days ago) Jul 04 2017 splash 3,759 3.5 (2 years ago) Apr 25 2014 captcha unicaps 553 22.2.0 (11 months ago) Jul 27 2020 captcha_solver 201 0.1.5 (3 years ago) Feb 17 2015 microformats extruct 728 0.14.0 (2 months ago) Oct 27 2015 http-utils httptools 1,084 0.5.0 (4 months ago) Dec 28 2012 css cssutils 47 2.6.0 (5 months ago) Aug 04 2007 selenium selenium 25,562 4.7.2 (a month ago) Apr 25 2008 requestium 553 22.2.0 (11 months ago) Dec 28 2012 browser-automation selenium 25,562 4.7.2 (a month ago) Apr 25 2008 playwright 7,853 1.29.1 (19 days ago) Feb 24 2021 requestium 553 22.2.0 (11 months ago) Dec 28 2012 splash 3,759 3.5 (2 years ago) Apr 25 2014 playwright playwright 7,853 1.29.1 (19 days ago) Feb 24 2021 seo advertools 746 0.13.2 (3 months ago) Dec 28 2012 data-output xhtml2pdf 1,963 0.2.8 (7 months ago) May 19 2011 data xhtml2pdf 1,963 0.2.8 (7 months ago) May 19 2011