以下操作基於Windows平臺。

開啟CMD命令提示框：

輸入 如下命令：

開啟專案裡的items.py檔案，定義如下變數，用於儲存。

class MaoyanItem(scrapy.Item):
    # define the fields for your item here like:
    # name = scrapy.Field()
    movie = scrapy.Field()
    actor = scrapy.Field()
    release = scrapy.Field()
    score = scrapy.Field()

然後開啟專案裡的spiders資料夾內的maoyanTop100.py檔案。

# -*- coding: utf-8 -*-
import time
import scrapy
from scrapy.http import Request
from maoyan.items import MaoyanItem


class Maoyantop100Spider(scrapy.Spider):
    name = 'maoyanTop100'
    #allowed_domains = ['maoyan.com/board/4']
    #start_urls = ['http://maoyan.com/board/4']
    
    def start_requests(self):
         for offset in range(0,100,10):
            self.url = 'http://maoyan.com/board/4?offset={}'.format(offset)
            yield Request(self.url,callback = self.parse)
            time.sleep(1)

    def parse(self, response):
        context = response.css('dd')   #分析得知所有的電影item均在該標籤內
        for info in context:
            item = MaoyanItem()
            item['movie'] = info.css('p.name a::text').extract_first().strip()
            item['actor'] = info.css('.star::text').extract_first().strip()
            item['release'] = info.css('.releasetime::text').extract_first().strip()
            score = info.css('i.integer::text').extract_first().strip()
            score += info.css('i.fraction::text').extract_first().strip()
            item['score'] = score
            yield item
 

開啟settings.py檔案，加入如下兩行程式碼：

USER_AGENT = 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'
FEED_EXPORT_ENCODING = 'gbk'

然後在命令提示符輸入:

scrapy crawl maoyanTop100 -o maoyan.csv

爬取結束後會生成一個csv檔案，效果圖如下：