[leanpub] Website Scraping with Python (eng)

[leanpub] Website Scraping with Python (eng)

Автор: Gabor Laszlo Hajba
Формат: PDF, ePub, Mobi
Страниц: 122

Книга о парсинге веб-сайтов и обработке данных в Python.
Какой инструмент использовать?
Насколько быстро работают инструменты?
Я попытаюсь ответить на эти вопросы в этой книге и помочь лучше понять работу сайта с помощью BeautifulSoup4 и Scrapy.
Я выделяю некоторые проблемы, с которыми сталкиваюсь, и, возможно, вы также можете столкнуться с решениями.

Спойлер: Описание

A book about website scraping and data processing in Python. Which tool to use? How fast the tools are? I try to answer these questions in this book and help to gain a better understanding of website scraping with BeautifulSoup4 and Scrapy. I highlight some problems I encountered and perhaps you can encounter too -- with solutions.

Спойлер: Содержание

• • Preface
    • What will I do exactly?
    • About the programming language
    • Some extra feature
    • Prerequisites
    • Length of the book
    • LeanPub
  • Some brief words about the project
  • 1. BeautifulSoup – The ancestor of JSoup
    • 1.1 Some words about BeautifulSoup
    • 1.2 Configuring the download timeout
    • 1.3 Configuring the proxy
    • 1.4 Changing the XML-parser behind BeautifulSoup
    • 1.5 Some of the errors that happened
    • 1.6 Bits and pieces of the solution
    • 1.7 Printing a CSV line
    • 1.8 Conclusion
  • 2. Scrapy – another way to gather data
    • 2.1 Some words about Scrapy
    • 2.2 Setting up the project
    • 2.3 Configuring the download timeout
    • 2.4 Configuring the proxy
    • 2.5 Working with offline data
    • 2.6 Bits and pieces of the solution
    • 2.7 Errors which happened
    • 2.8 Exporting the data as a CSV
    • 2.9 Accessing settings
    • 2.10 Scrapy as a library
    • 2.11 Conclusion
  • 3. Performance of the solutions
    • 3.1 The dataset
    • 3.2 Introducing the test environments
    • 3.3 The test scenarios
    • 3.4 BeautifulSoup compared with itself
    • 3.5 Comparing Scrapy with itself
    • 3.6 Comparing both tools
  • 4. Creating plots with Python
    • 4.1 Simple examples
    • 4.2 Display multiple data ranges
    • 4.3 Displaying the averages
    • 4.4 Displaying the legend
    • 4.5 Formatting the plot
    • 4.6 Conclusion
  • 5. Some thoughts on functional programming
    • 5.1 The idea behind functional programming
    • 5.2 First class functions
    • 5.3 Currying
    • 5.4 Writing declaratively
    • 5.5 Using map and reduce
    • 5.6 Using recursion
    • 5.7 Pipelining
    • 5.8 Applying functional programming
    • 5.9 Conclusion
  • 6. Parallel working
    • 6.1 Why Should I care, I use Scrapy?
    • 6.2 Parallelism in Python
    • 6.3 An example
    • 6.4 CPU-bound tasks
    • 6.5 Possible errors you can encounter
    • 6.6 Conclusion
    • 6.7 Sources
  • 7. Two real-life projects
    • 7.1 Sport clubs spider
    • 7.2 Bloso scraper
    • 7.3 Conclusion
  • Extra! Extra! Read all about it!
    • Introducing the requirements
    • Starting the project
    • The items
    • Defining the spider
    • Exporting the data into a database
    • Offline data
    • Performance
    • The sources
    • Next step