Managing Consolidation Queries


SummaryA basic overview of the types of scheduled queries and specifics on working with Consolidations using various API endpoints.

With Alooma there are two types of queries, both of which are scheduled: Consolidation and Custom scheduled queries. For more information about queries see Queries. Here, we'll focus on Consolidations.

Query functions

Just like with all other API work, it all begins with authenticating and getting an API instance. Once you have an API Instance, you can get a list of query objects as well as create/schedule/delete them by defining and then using the endpoints/functions below.

import alooma
import requests

class endpoints:
    # Consolidation
    CONSOLIDATION = 'consolidation/'
    CONSOLIDATION_BY_ID = 'consolidation/{id}'

    # Consolidation V2
    CONSOLIDATION_V2 = 'v2/consolidation'
    CONSOLIDATION_STATE_V2 = 'v2/consolidation/{query_id}'
    CONSOLIDATION_RUN_V2 = 'v2/consolidation/{query_id}/run'

def create_consolidation(configuration, custom_variables=None):
    """ Create Consolidation Given Configuration

        :param configuration: requires event_type and frequency
        :param custom_variables: custom variables to add to consolidation 
    if "deployment_name" not in configuration:
        deployment_name = api.get_deployment_info()['deploymentName']
        configuration["deployment_name"] = deployment_name
    if custom_variables:
        if "custom_variables" not in configuration:
            configuration["custom_variables"] = {}

    url = api.rest_url + endpoints.CONSOLIDATION_V2
    res = api._Client__send_request(, url, json=configuration)

    return res.json()

def get_consolidations(api):
    """ Return List of Consolidations """

    url = api.rest_url + endpoints.CONSOLIDATION_V2
    res = api._Client__send_request(requests.get, url)

    return res.json()

def get_consolidation_by_event_type(api, event_type):
    """ Return List of Consolidations """
    params = {"event_type": event_type}
    url = api.rest_url + endpoints.CONSOLIDATION_V2
    res = api._Client__send_request(requests.get, url, params=params)

    cons = res.json()
    if len(cons) == 0:
        raise Exception("Consolidation DNE: %s" % event_type)

    return cons[0]

def delete_consolidation(api, query_id):
    """ Delete Consolidation """
    url = api.rest_url + endpoints.CONSOLIDATION_STATE_V2.format(
    res = api._Client__send_request(requests.delete, url)

    return res.ok

See the API Reference for more information.

Creating Consolidations

For most inputs, you create a consolidation by passing a configuration to create_consolidation. For some webhook-based inputs, you'll also need to define custom_variables for the create_consolidation call. These inputs include:

  • Custom Webhooks

  • iOS/Android App

  • Java App

  • JavaScript App

  • Python App

An example of creating a consolidation for a webhook-based input in Redshift:

rs_configuration = {"event_type": event, "run_at": "*/15 * * * *", 'query_type': 'incremental'}
rs_custom_variables = {"role": "PUBLIC", 'ri_column': '_metadata__timestamp', 'all_ri_cols': '_metadata__timestamp DESC'}

create_consolidation(configuration=rs_configuration, custom_variables=rs_custom_variables)

An example of creating a consolidation for a webhook-based input in Snowflake:

sf_configuration = {"event_type": event, "run_at": "*/15 * * * *"}
sf_custom_variables = {"role": "PUBLIC","ri_field":"_metadata.@timestamp", "all_ri_fields": ["_metadata.@timestamp"]}

create_consolidation(configuration=sf_configuration, custom_variables=sf_custom_variables)
Get all scheduled queries and consolidations

Return a list of query objects (custom scheduled as well as consolidation queries).

Get consolidation by event type

Return a query object by event type.

get_consolidation_by_event_type(api, event_type)
Delete consolidation

Remove a consolidation by event type.

con = get_consolidation_by_event_type(api, event_type)

delete_consolidation(api, con["id"])
Create consolidation

Create a new consolidation.

configuration = {
    "event_type": event_type,
    "run_at": "0 * * * *"
custom_variables = {"role": "PUBLIC"}
create_consolidation(api, configuration, custom_variables=custom_variables)

Search results

    No results found