Is there a way to guarantee that coroutines have been started before doing something that will allow them to finish?
I have a method that fetches some data from an API using IDs that are passed to it. Before this happens, the IDs are placed in a queue so that multiple (up to 100) can be used per API call. API calls are carried out by a method called flush_queue, which must be called at the end of my program to guarantee that all of the IDs that were added to the queue will have been used. I wrote an async function that takes in an ID, creates a future that flush_queue will eventually set the result of, and then returns the data obtained for that ID when it's available:
async def get_data(self, id):
self.queued_ids.append(id)
self.results[id] = asyncio.get_running_loop().create_future()
if len(self.queued_ids) == 100:
await self.flush_queue()
return await self.results[id]
This works well for the case where the queue is flushed because there are enough IDs to make an API request; the problem is, I need to ensure that each user id is added to the queue before I call flush_queue at the completion of my program. Therefore, I need each call to get_data to have started before I do that; however, I can't wait for each coroutine object returned by each call to finish, because they won't finish until after flush_queue completes. Is there any way to ensure that a series of coroutines has started (but not necessarily finished) before doing something like calling flush_queue?
from Recent Questions - Stack Overflow https://ift.tt/3mciQTu
https://ift.tt/eA8V8J
Comments
Post a Comment