1. 程式人生 > >使用ip代理池爬蟲時,requests模組get請求出現問題_AttributeError: 'str' object has no attribute 'get'

使用ip代理池爬蟲時,requests模組get請求出現問題_AttributeError: 'str' object has no attribute 'get'

問題描述:專案使用ip代理池對網頁進行資料爬取,但是requests模組get方法出現問題,出錯如下:

  File "E:\project\venv\lib\site-packages\requests\api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "E:\project\venv\lib\site-packages\requests\api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "E:\project\venv\lib\site-packages\requests\sessions.py", line 524, in request
    prep.url, proxies, stream, verify, cert
  File "E:\project\venv\lib\site-packages\requests\sessions.py", line 699, in merge_environment_settings
    no_proxy = proxies.get('no_proxy') if proxies is not None else None
AttributeError: 'str' object has no attribute 'get'

程式碼如下:

proxies = "http://" + proxy
res = requests.get(url=url, headers=headers, proxies=proxies)

原因分析:

api.py中get方法定義如下:

def get(url, params=None, **kwargs):
    r"""Sends a GET request.

    :param url: URL for the new :class:`Request` object.
    :param params: (optional) Dictionary, list of tuples or bytes to send
        in the body of the :class:`Request`.
    :param \*\*kwargs: Optional arguments that ``request`` takes.
    :return: :class:`Response <Response>` object
    :rtype: requests.Response
    """
:param params: (optional) Dictionary, list of tuples or bytes to send
    in the body of the :class:`Request`.

可以知道,傳入的引數必須是字典型別

修改後程式碼:

            proxies = {
                "http": "http://" + proxy
            }
            res = requests.get(url=url, headers=headers, proxies=proxies)


執行成功