1. 程式人生 > >Python的scrapy框架爬蟲專案中加入郵箱通知(爬蟲啟動關閉等資訊以郵件的方式傳送到郵箱)

Python的scrapy框架爬蟲專案中加入郵箱通知(爬蟲啟動關閉等資訊以郵件的方式傳送到郵箱)

前面關於傳送郵件的部落格參考:普通郵件部落格——點選開啟連結

                                             帶附件的郵件——點選開啟連結

準備:
1、建立scrapy爬蟲專案

2、程式碼主要是兩部分:

呈上程式碼
第一部分是傳送郵件的程式碼:

import smtplib
from email.mime.text import MIMEText
 
import logging
class EmailSend(object):
    def __init__(self):
        self.logging = logging.getLogger('Waring')
        self.email_host = 'smtp.qq.com'
        self.email_port = '465'
        self.email_pass = '授權碼(或密碼,不推薦密碼)'
 
    def send_text_email(self, from_addr, to_addrs, subject, content):
        self.logging.warning('send_text_email is willed 丟棄')
        self.logging.error('send_text_email is None')
        message_text = MIMEText(content, 'plain', 'utf8')
        message_text['From'] = from_addr
        message_text['To'] = to_addrs
        message_text['Subject'] = subject
 
        try:
            # 在建立客戶端物件的同時,連線到郵箱伺服器。
            client = smtplib.SMTP_SSL(host=self.email_host, port=self.email_port)
            login_result = client.login(from_addr, self.email_pass)
            if login_result and login_result[0] == 235:
                print('登入成功')
                client.sendmail(from_addr, to_addrs, message_text.as_string())
                print('郵件傳送成功')
            else:
                print('郵件傳送異常:',login_result[0], login_result[1])
        except Exception as e:
            # print('連線郵箱伺服器異常:',e)
            self.logging.error('連線郵箱伺服器異常:{}'.format(e))
    def send_image_email(self):
        pass
 
    def send_word_email(self):
        pass
 
    def send_video_email(self):
        pass
 
"""
1. start_spider 
"""
 
 
注意:上述程式碼中可自定義傳送內容(文字:send_word_email,視訊:send_video_email....)主要是在爬蟲專案中呼叫該模組傳送郵件(包含引數:

from_addr, to_addrs, subject, content
)

爬蟲專案程式碼如下:

# -*- coding: utf-8 -*-
import scrapy, time
from datetime import datetime
from ..emailsend import EmailSend
 
class EmailtestSpider(scrapy.Spider):
    name = 'emailtest'
    allowed_domains = ['baidu.com']
    start_urls = ['http://www.baidu.com/']
 
    # 在爬蟲啟動和關閉的時候,分別傳送郵箱,通知爬蟲管理者。
    def start_requests(self):
        email = EmailSend()
        content = '爬蟲啟動時間:{}'.format(datetime.now())
        email.send_text_email('傳送郵箱@qq.com', '接收郵箱@qq.com', '爬蟲結束', content)
 
        for url in self.start_urls:
            yield scrapy.Request(url=url, callback=self.parse)
 
    def parse(self, response):
        time.sleep(10)
        print('123')
 
    def closed(self, reason):
        # 爬蟲關閉的時候,會呼叫這個方法
        email = EmailSend()
        content = '爬蟲關閉時間:{}'.format(datetime.now())
        email.send_text_email('傳送郵箱@qq.com', '接收郵箱@qq.com', '爬蟲結束', content)

注意:1、示例程式碼中郵件內容是當前時間(爬蟲開始結束時間:作用可以計算爬蟲速度!),當傳送訊息是需要呼叫,要注意傳送的引數是:
‘傳送郵箱@qq.com’, ‘接收郵箱@qq.com’, ‘爬蟲結束’, content

    2、最後爬蟲結束是的closed函式

closed函式對應的是爬蟲專案中(原始碼)的close函式,如下:


原文:https://blog.csdn.net/qq_33472765/article/details/81022979
版權宣告:本文為博主原創文章,轉載請附上博文連結!