Skip to content

Concepts

Asynchronous API Requests: Technical Overview

This section provides a high level technical explanation of how our asynchronous API endpoints work. We'll examine the internal processes, advantages of the approach, and best practices for implementation. For authentication details, please refer to the Authentication & Security section.

Asynchronous Processing

The asynchronous communication flow is more complex, but your client isn't blocked until the data preparation is finished and the approach can deal with much bigger data. The flow is visualized in the diagram below: 

sequenceDiagram
    participant Client
    participant POST Endpoint
    participant GET Endpoint
    participant Backend
    Client->>POST Endpoint: POST /v3/devices/status (with timerange and fileformat)
    POST Endpoint->>Backend: Start data preparation
    Backend-->>POST Endpoint: Return request_uuid
    POST Endpoint-->>Client: Response with status "Submitted" and request_uuid
    Note over Client: Wait and start polling...
    Client->>GET Endpoint: GET /v3/devices/status/{request_uuid}
    GET Endpoint->>Backend: Check status
    Backend-->>GET Endpoint: Status information
    GET Endpoint-->>Client: Status "In Progress"
    Note over Client: Wait and request again...
    Client->>GET Endpoint: GET /v3/devices/status/{request_uuid}
    GET Endpoint->>Backend: Check status
    Backend-->>GET Endpoint: Status information
    GET Endpoint-->>Client: Status "In Progress"
    Note over Backend: Data preparation completes
    Note over Client: Further request after waiting
    Client->>GET Endpoint: GET /v3/devices/status/{request_uuid}
    GET Endpoint->>Backend: Check status
    Backend-->>GET Endpoint: Status with URLs
    GET Endpoint-->>Client: Status "Completed" with download URLs
    Client->>Backend: Request file via download URL
    Backend-->>Client: Delivers result file

What happens:

  1. Request Submission: When you submit a request to an asynchronous endpoint, the API immediately returns a request identifier (request_uuid) and places your actual data request in a processing queue.
  2. Background Processing: In the background the system starts to prepare your requested data. This happens independently of the HTTP connection that submitted the request. Your client can do something else during the processing.
  3. Status Tracking: The API provides your client with the state of your request over the GET endpoint. The state always starts with Submitted when you send your request to the POST endpoint for the first time.
  4. Result Storage: Once processing is complete, the results are stored in a temporary cloud storage location and download URLs are generated. The URL is provided by the GET endpoint.
  5. Result Retrieval: You use the provided URLs to download the result files directly from storage.

Request Status Values

Status Description
Submitted The request has been received and queued for processing.
In Progress The backend is actively preparing your data.
Completed Processing is finished. Download URLs are available in the response.

Polling Best Practices

When polling the GET endpoint for status updates, keep the following in mind:

  • Polling interval: Wait at least a few seconds between requests. Polling too aggressively wastes quota and adds unnecessary load on the API.
  • Timeout handling: Set a maximum number of retries or a total timeout for your polling loop to avoid infinite waits if something goes wrong on the server side.
  • Download URL lifetime: Pre-signed download URLs are temporary. Download your files promptly after receiving a Completed status.

Technical Advantages

The asynchronous approach offers several technical benefits:

  1. No Time Constraints: Processing can take as long as needed without concern for HTTP timeouts.
  2. Larger Data Sets: There's no practical limit to the size of data that can be returned, as results are provided as downloadable files rather than in the HTTP response.

Info

If you require the same data multiple times you can cache the URL and download the file multiple times. This avoids unnecessary API calls on our side.

File Formats

When submitting an asynchronous request, you must specify the desired output file format. The following formats are supported:

Format Description
CSV (default) Comma-separated values. Human-readable text format, compatible with most tools.
PARQUET Columnar binary format. Efficient for large datasets and analytics workflows.
JSON JSON array format. Well-suited for downstream processing in web applications.
ORC Optimized Row Columnar format. High performance for Hive/Spark environments.
AVRO Row-based binary format with embedded schema. Good for data pipelines.
TEXTFILE Plain text output. Use when a simple, unformatted output is required.

For most use cases, CSV is the simplest choice. Use PARQUET or ORC when working with large volumes of data in a data warehouse or analytics environment.