Skip to content

Asynchronous Tasks

The asynchronous tasks endpoints allow for processing large geospatial data requests that may take a significant amount of time to complete. Instead of a single, long-running synchronous request, this endpoint provides a mechanism to initiate a task, monitor its progress, and download the results once ready.

Asynchronous endpoints have different limits than synchronous endpoints - see Limits for details.

Basic Usage Flow

The general flow for using the asynchronous tasks endpoint involves three main steps:

  1. Initiating a Task: You begin by submitting a large polygon request, specifying the polygon geometry and the desired layer IDs. The API will respond with a unique task_id for your request. This is a quick, synchronous call that simply registers your task.

  2. Monitoring Task Status: Once a task is initiated, it enters a processing queue. You can periodically query the API using the task_id to check its current status. The status will indicate whether the task is pending, processing, complete, or has encountered an error. It is crucial to wait until the task status is COMPLETE before attempting to download results. If the task status is ERROR, details about the failure will be provided.

  3. Downloading Results: After the task has successfully completed, the API provides a download_url. This URL can be used to retrieve a zip file containing the processed results.

    Download URLs are short-lived

    The download_url is generated fresh each time you request the task status, and is only valid for 15 minutes. It is not a permanent link, so it should be used to download the results shortly after it is returned rather than being stored for later use.

    If it expires, request the task status again and the response will contain a new download_url valid for a further 15 minutes.

    This is a deliberate security measure - long-lived URLs which grant access to confidential data (such as the locations you requested) could otherwise be stored or leaked and remain usable indefinitely.

    Results are retained for 1 week, after which they are deleted and the task status is returned as EXPIRED with no download_url. You must therefore download your results within 1 week of the task completing.

Zip File Structure

The downloaded zip file contains the following:

  • TIFF files: For each requested layer that contained data within the specified polygon, a GeoTIFF file is included, named after its layer_id (e.g., your_layer_id.tif). These files contain the geospatial data for the respective layers.

If the request contained more than one polygon, the TIFF files are nested in one directory per polygon instead (e.g., your_polygon_id/your_layer_id.tif). The directory name is the id of the corresponding GeoJSON feature, or the polygon's zero-based position in the request if the features have no id.

  • manifest.csv: This CSV file provides a summary of all requested layers and their processing outcomes. It contains the following columns:
  • layer_id: The identifier of the requested layer.
  • result_code: The status of the processing for that layer. Common values include POLYGON_RESULT_CODE_OK ( indicating data was found and a TIFF file is included) or POLYGON_RESULT_CODE_OUT_OF_BOUNDS (indicating no data was found for that layer within the specified polygon).
  • area_km2: The area of the polygon in square kilometres.
  • polygon_id: The polygon the row refers to, named as described above. This column is only present when the request contained more than one polygon.

For a multi-polygon request there is one row per polygon per requested layer, so a polygon may be in bounds for some layers and out of bounds for others.

Example

If a task was created using the Python SDK with these layer IDs:

layer_ids = [
    "US-1_3ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.10-PERCENTILE50-v2.0",
    "US-1_3ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.30-PERCENTILE50-v2.0",
    "US-1_3ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.20-PERCENTILE50-v2.0",
    "US-1_3ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.00-PERCENTILE50-v2.0",
    "US-1_3ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.40-PERCENTILE50-v2.0",
    "US-1_3ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.50-PERCENTILE50-v2.0",
    "FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.20-PERCENTILE50-v3.1",
    "FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.30-PERCENTILE50-v3.1",
    "FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.10-PERCENTILE50-v3.1",
    "FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.40-PERCENTILE50-v3.1",
    "FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.00-PERCENTILE50-v3.1",
    "FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.50-PERCENTILE50-v3.1",
    "FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.10-PERCENTILE50-v3.1",
    "FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.30-PERCENTILE50-v3.1",
    "FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.40-PERCENTILE50-v3.1",
]

poly = polygon(...)

client = Client(...)
create_resp = client.async_tasks.create_large_polygon_task(poly, layer_ids)
# ...poll for result

After downloading and extracting the zip file, it might look like:

manifest.csv
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.20-PERCENTILE50-v3.1.tif
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.30-PERCENTILE50-v3.1.tif
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.10-PERCENTILE50-v3.1.tif
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.40-PERCENTILE50-v3.1.tif
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.00-PERCENTILE50-v3.1.tif
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.50-PERCENTILE50-v3.1.tif
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.10-PERCENTILE50-v3.1.tif
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.30-PERCENTILE50-v3.1.tif
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.40-PERCENTILE50-v3.1.tif

And the manifest:

layer_id,result_code,area_km2
US-1_3ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.10-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OUT_OF_BOUNDS,250.75
US-1_3ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.30-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OUT_OF_BOUNDS,250.75
US-1_3ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.20-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OUT_OF_BOUNDS,250.75
US-1_3ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.00-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OUT_OF_BOUNDS,250.75
US-1_3ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.40-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OUT_OF_BOUNDS,250.75
US-1_3ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.50-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OUT_OF_BOUNDS,250.75
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.20-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OK,250.75
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.30-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OK,250.75
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.10-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OK,250.75
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.40-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OK,250.75
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.00-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OK,250.75
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.50-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OK,250.75
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.10-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OK,250.75
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.30-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OK,250.75
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.40-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OK,250.75

Note that the 'out of bounds' results do not have an associated GeoTIFF in the output file.

Multiple polygons

If the request contained more than one polygon, the results are nested per polygon and the manifest gains a polygon_id column. For a request with two features, bristol and cardiff, the zip might look like:

manifest.csv
bristol/FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.20-PERCENTILE50-v3.1.tif
bristol/FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.30-PERCENTILE50-v3.1.tif
cardiff/FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.20-PERCENTILE50-v3.1.tif

With this manifest, where cardiff was out of bounds for one of the two requested layers:

layer_id,result_code,area_km2,polygon_id
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.20-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OK,120.5,bristol
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.30-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OK,120.5,bristol
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in50-PLUVIAL-DEFENDED-DEPTH-dT1.20-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OK,80.25,cardiff
FLOOD_MAP-1ARCSEC-NW_OFFSET-1in75-PLUVIAL-DEFENDED-DEPTH-dT1.30-PERCENTILE50-v3.1,POLYGON_RESULT_CODE_OUT_OF_BOUNDS,80.25,cardiff

Every feature must either have an id or none of them may; a mix is rejected. Feature ids must be unique and cannot contain path separators, since they are used as directory names.