Python的scrapy框架爬蟲專案中加入郵箱通知(爬蟲啟動關閉等資訊以郵件的方式傳送到郵箱)
阿新 • • 發佈:2018-11-27
前面關於傳送郵件的部落格參考:普通郵件部落格——點選開啟連結
帶附件的郵件——點選開啟連結
準備:
1、建立scrapy爬蟲專案
2、程式碼主要是兩部分:
呈上程式碼
第一部分是傳送郵件的程式碼:
import smtplib from email.mime.text import MIMEText import logging class EmailSend(object): def __init__(self): self.logging = logging.getLogger('Waring') self.email_host = 'smtp.qq.com' self.email_port = '465' self.email_pass = '授權碼(或密碼,不推薦密碼)' def send_text_email(self, from_addr, to_addrs, subject, content): self.logging.warning('send_text_email is willed 丟棄') self.logging.error('send_text_email is None') message_text = MIMEText(content, 'plain', 'utf8') message_text['From'] = from_addr message_text['To'] = to_addrs message_text['Subject'] = subject try: # 在建立客戶端物件的同時,連線到郵箱伺服器。 client = smtplib.SMTP_SSL(host=self.email_host, port=self.email_port) login_result = client.login(from_addr, self.email_pass) if login_result and login_result[0] == 235: print('登入成功') client.sendmail(from_addr, to_addrs, message_text.as_string()) print('郵件傳送成功') else: print('郵件傳送異常:',login_result[0], login_result[1]) except Exception as e: # print('連線郵箱伺服器異常:',e) self.logging.error('連線郵箱伺服器異常:{}'.format(e)) def send_image_email(self): pass def send_word_email(self): pass def send_video_email(self): pass """ 1. start_spider """ 注意:上述程式碼中可自定義傳送內容(文字:send_word_email,視訊:send_video_email....)主要是在爬蟲專案中呼叫該模組傳送郵件(包含引數: from_addr, to_addrs, subject, content )
爬蟲專案程式碼如下:
# -*- coding: utf-8 -*- import scrapy, time from datetime import datetime from ..emailsend import EmailSend class EmailtestSpider(scrapy.Spider): name = 'emailtest' allowed_domains = ['baidu.com'] start_urls = ['http://www.baidu.com/'] # 在爬蟲啟動和關閉的時候,分別傳送郵箱,通知爬蟲管理者。 def start_requests(self): email = EmailSend() content = '爬蟲啟動時間:{}'.format(datetime.now()) email.send_text_email('傳送郵箱@qq.com', '接收郵箱@qq.com', '爬蟲結束', content) for url in self.start_urls: yield scrapy.Request(url=url, callback=self.parse) def parse(self, response): time.sleep(10) print('123') def closed(self, reason): # 爬蟲關閉的時候,會呼叫這個方法 email = EmailSend() content = '爬蟲關閉時間:{}'.format(datetime.now()) email.send_text_email('傳送郵箱@qq.com', '接收郵箱@qq.com', '爬蟲結束', content)
注意:1、示例程式碼中郵件內容是當前時間(爬蟲開始結束時間:作用可以計算爬蟲速度!),當傳送訊息是需要呼叫,要注意傳送的引數是:
‘傳送郵箱@qq.com’, ‘接收郵箱@qq.com’, ‘爬蟲結束’, content
2、最後爬蟲結束是的closed函式
closed函式對應的是爬蟲專案中(原始碼)的close函式,如下:
原文:https://blog.csdn.net/qq_33472765/article/details/81022979
版權宣告:本文為博主原創文章,轉載請附上博文連結!