2022-11-15

How to resolve "coroutine 'Launcher.killChrome' was never awaited" error?

I am trying to automate searching on a website. I am using requests_html in headless mode for it.

I have written the following code in ipython. This works fine if k<=20 otherwise it just hangs up. I neither understand the cause of this problem nor how to debug it.

k=20
import asyncio
from requests_html import AsyncHTMLSession
from bs4 import BeautifulSoup
async def asyncurl(url):
    r = await asession.get('https://pricehistoryapp.com/')
    script = """
    () => {
    setTimeout(function(){
        var element = document.getElementsByTagName("input");
        
        document.getElementsByClassName("px-3 py-2 outline-none rounded-none disabled:bg-gray-200")[0].setAttribute("value", "%s");
        document.getElementsByClassName("px-3 py-2 outline-none rounded-none disabled:bg-gray-200")[0].click();
        document.getElementsByClassName("px-3 py-2 outline-none rounded-none disabled:bg-gray-200")[0].dispatchEvent(new Event('change', { 'bubbles': true }));
        document.getElementsByClassName('bg-secondary-600 text-white py-1 font-bold shadow-lg')[0].click();

    }, 1190);
    }
    """%(url)
    await r.html.arender(sleep=7, keep_page=True, scrolldown=1,timeout=105,script=script)
    return r.html.html
results=[]
start=time.time()
ii=0
#urls=list of urls
for i in range(1,5,1):
    if __name__ == '__main__':
        try:
            asession = AsyncHTMLSession()
            list_of_lambdas = [lambda url=url: asyncurl(url) for url in urls[k*(i-1):k*(i)]]
            results.append(asession.run(*list_of_lambdas))
            asession.close()
            ii+=1
            print("Completed records till "+str(i*k)+" itteration: "+str(ii)+" records in "+str(time.time()-start)+" seconds")
        except:
            continue
name=[BeautifulSoup(p,'lxml').find('h1',{"class":True}).text for pp in results for p in pp]

Please guide me toward a workaround to this problem. Thanks in advance.

When I try to get out of hang by ^+C it gives below error repeatedly but remains hung.

<ipython-input-3-76d514892a35>:11: RuntimeWarning: coroutine 'Launcher.killChrome' was never awaited
  continue
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
^C^CTraceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/pyppeteer/launcher.py", line 153, in _close_process
    self._loop.run_until_complete(self.killChrome())
  File "/usr/lib/python3.8/asyncio/base_events.py", line 592, in run_until_complete
    self._check_running()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 552, in _check_running
    raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running

Also, any other methods for speeding up automated searching (like running the code inside for loop in multi-threads) would be appreciated.



No comments:

Post a Comment