Scrapy A free and high level scraping and web crawling framework for writing spiders to crawl and parse web pages
Download

Scrapy Ranking & Summary

Scrapy Tags

framework web framework crawl scrape website scrape crawl website

Scrapy Description

A free and high level scraping and web crawling framework for writing spiders to crawl and parse web pages Scrapy is a fast and open source high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages.Scrapy can be used for a wide range of purposes, from data mining to monitoring and automated testing. Here are some key features of "Scrapy": Simple: · Scrapy was designed with simplicity in mind, by providing the features you need without getting in your way. Productive: · Just write the rules to extract the data from web pages and let Scrapy crawl the entire web site for you. Fast: · Scrapy is used in production crawlers to completely scrape more than 500 retailer sites daily, all in one server. Extensible: · Scrapy was designed with extensibility in mind and so it provides several mechanisms to plug new code without having to touch the framework core. Portable: · Scrapy runs on Mac, Linux, Windows, and BSD. Open Source and 100% Python: · Scrapy is completely written in Python, which makes it very easy to hack. Well-tested: · Scrapy has an extensive test suite with very good code coverage. Requirements: · Python · Libxml2

Scrapy Related Software