Documentation

Managing Consolidation Queries

Abstract

SummaryA basic overview of the types of scheduled queries and specifics on working with Consolidations using various API endpoints.

With Alooma there are two types of queries, both of which are scheduled: Consolidation and Custom scheduled queries. For more information about queries see Queries. Here, we'll focus on Consolidations.

Query functions

Just like with all other API work, it all begins with authenticating and getting an API instance. Once you have an API Instance, you can get a list of query objects as well as create/schedule/delete them by defining and then using the endpoints/functions below.

Copy
import alooma
import requests

class endpoints:
    # Consolidation
    CONSOLIDATION = 'consolidation/'
    CONSOLIDATION_BY_ID = 'consolidation/{id}'

    # Consolidation V2
    CONSOLIDATION_V2 = 'v2/consolidation'
    CONSOLIDATION_STATE_V2 = 'v2/consolidation/{query_id}'
    CONSOLIDATION_RUN_V2 = 'v2/consolidation/{query_id}/run'

def create_consolidation(configuration, custom_variables=None):
    """ Create Consolidation Given Configuration

        :param configuration: requires event_type and frequency
        :param custom_variables: custom variables to add to consolidation 
    """
    if "deployment_name" not in configuration:
        deployment_name = api.get_deployment_info()['deploymentName']
        configuration["deployment_name"] = deployment_name
    if custom_variables:
        if "custom_variables" not in configuration:
            configuration["custom_variables"] = {}
        configuration["custom_variables"].update(custom_variables)

    url = api.rest_url + endpoints.CONSOLIDATION_V2
    res = api._Client__send_request(requests.post, url, json=configuration)

    return res.json()

def get_consolidations(api):
    """ Return List of Consolidations """

    url = api.rest_url + endpoints.CONSOLIDATION_V2
    res = api._Client__send_request(requests.get, url)

    return res.json()

def get_consolidation_by_event_type(api, event_type):
    """ Return List of Consolidations """
    params = {"event_type": event_type}
    url = api.rest_url + endpoints.CONSOLIDATION_V2
    res = api._Client__send_request(requests.get, url, params=params)

    cons = res.json()
    if len(cons) == 0:
        raise Exception("Consolidation DNE: %s" % event_type)

    return cons[0]

def delete_consolidation(api, query_id):
    """ Delete Consolidation """
    url = api.rest_url + endpoints.CONSOLIDATION_STATE_V2.format(
                                    query_id=query_id)
    res = api._Client__send_request(requests.delete, url)

    return res.ok
        

See the API Reference for more information.

Creating Consolidations

For most inputs, you create a consolidation by passing a configuration to create_consolidation. For some webhook-based inputs, you'll also need to define custom_variables for the create_consolidation call. These inputs include:

  • Custom Webhooks

  • iOS/Android App

  • Java App

  • JavaScript App

  • Python App

An example of creating a consolidation for a webhook-based input in Redshift:

Copy
rs_configuration = {"event_type": event, "run_at": "*/15 * * * *", 'query_type': 'incremental'}
rs_custom_variables = {"role": "PUBLIC", 'ri_column': '_metadata__timestamp', 'all_ri_cols': '_metadata__timestamp DESC'}

create_consolidation(configuration=rs_configuration, custom_variables=rs_custom_variables)

An example of creating a consolidation for a webhook-based input in Snowflake:

Copy
sf_configuration = {"event_type": event, "run_at": "*/15 * * * *"}
sf_custom_variables = {"role": "PUBLIC","ri_field":"_metadata.@timestamp", "all_ri_fields": ["_metadata.@timestamp"]}

create_consolidation(configuration=sf_configuration, custom_variables=sf_custom_variables)
Get all scheduled queries and consolidations

Return a list of query objects (custom scheduled as well as consolidation queries).

Copy
get_consolidations(api)
Get consolidation by event type

Return a query object by event type.

Copy
get_consolidation_by_event_type(api, event_type)
Delete consolidation

Remove a consolidation by event type.

Copy
con = get_consolidation_by_event_type(api, event_type)

delete_consolidation(api, con["id"])
Create consolidation

Create a new consolidation.

Copy
configuration = {
    "event_type": event_type,
    "run_at": "0 * * * *"
}
custom_variables = {"role": "PUBLIC"}
create_consolidation(api, configuration, custom_variables=custom_variables)

Search results

    No results found