Progress bar stuck at 0 when using geedim with Celery for asynchronous downloads

[mht2953658596]

Description:
Hi,

I am using the geedim library in combination with Celery to perform asynchronous downloads of Google Earth Engine images. However, I frequently encounter an issue where the download starts and the progress bar appears, but it gets stuck at 0 and doesn’t update. There are no errors or warnings in the logs, making it difficult to debug the problem.

From my understanding, geedim uses multithreading to download multiple tiles simultaneously. Given this, I would expect the progress to update rather than remain at 0.

I would like to understand the following:

Is there a known issue with using geedim in conjunction with Celery for asynchronous downloads?
Could this be related to how geedim handles multithreading, or is there an internal mechanism that might conflict with Celery's task execution model?
Are there any debugging tips or configurations I should check to diagnose this problem?

[mht2953658596]

[mht2953658596]

In my understanding, downloading multiple tiles should be non-blocking. If one tile takes too long to download, the task for that tile should be terminated and the tile should be downloaded again in a separate thread. This way, the progress won't be stuck at 0%.

[dugalh]

Hello. I am not familiar with Celery, but it is possible you are exceeding one of the Earth Engine quotas by using it with geedim. The download should not get stuck though but either timeout eventually, or raise an error if it runs out of retries.

Are you able to provide a minimal code example that reproduces the problem, preferably without Celery?

[mht2953658596]

thanks for your reply!However， I am confused by your statement. GEE does indeed have quota limits. If I reach the maximum quota limits while downloading imagery with geedim, the download will get stuck at 0%. According to your statement, running another script to download imagery at this point should also be stuck at 0%, since it was previously blocked. However, this is not the case in reality. When one script gets stuck and reaches the maximum quota limits, the other script can still download imagery. Also, I checked the documentation, which mentions limits on maximum concurrency and the maximum number of requests per minute. According to this, geedim should not stay stuck at 0%, because while it might hit the maximum concurrency for this minute, it might not be at maximum concurrency in the next minute.

[dugalh]

I am not sure what the cause of the problem is. Running multiple geedim downloads concurrently could quickly exceed the quotas, so I mentioned the quota limits as a possibility. Generally, exceeding the limits should be avoided. If limits are repeatedly exceeded, geedim should eventually cancel the download and raise an error.

If this is not happening, or you otherwise believe this is a geedim bug, please provide a minimal code example that reproduces the problem so that I can look into it.