Блог

Я хочу очистить некоторые страницы с помощью selenium с помощью прокси-серверов. Если прокси-сервер выходит из строя, я хочу перезапустить драйвер. После первой ошибки драйвер перезапускается, но затем я буду получать эту ошибку каждый раз:

 ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x00000202DCD57D30>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=64403): Max retries exceeded with url: /session/42a03ba502142ea987a58acc4c940329/execute/sync (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x00000202DCD57D30>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))
 

Мой код:

 def my_get_selenium(driver,proxy,url):
    driver.get(url)
    time.sleep(2)
    print(driver.title)
    if 'Attention Required! | Cloudflare' in driver.title:
        with open("blacklisted_proxies.txt",'a') as f:
            f.write(proxy   "n")
        raise ValueError('Custom error 2...')
    while True:
        if 'attention required! ' not in driver.title.lower():
            if 'just a moment' not in driver.title.lower():
                break
            else:
                print("Waiting 0.5 seconds")
                time.sleep(0.5)
        else:
            raise ValueError('Custom error 1..')
    if 'www.g2.com' not in driver.title:
        soup = BeautifulSoup(driver.page_source, 'html.parser')
        return soup
    else:
        raise ValueError('Custom error 2...')
 

 def new_get_data(url,categories_list,driver,proxy):
    try:
        soup = my_get_selenium(driver,proxy,url)
    except:
        print("Retrying...")
        all_good_proxies.remove(proxy)
        time.sleep(2)
        driver.quit()
        time.sleep(2)
        driver,proxy = get_chrome_page()
        return new_get_data(url,categories_list,driver,proxy)
    name = soup.find('div', {'itemprop': 'name'}).text
    ...
 

Я перепробовал много разных вещей из других подобных вопросов, но безуспешно