Documentation

Storage Integration Supported File Types

Abstract

SummaryDetails about "Storage" integration types. This is general information about how Alooma works with these types of inputs. See the Storage input specific articles for how to configure the actual inputs.

Alooma has a class of integrations we call "storage" integrations—ones that specifically store your files of various types and formats, and that Alooma can connect to and pull to process them. This includes Avro, Azure, Box, Google Drive, Google Cloud Storage, S/FTP, and S3.

Note that the files may be packaged and compressed with Tar as well as GZip. The following file types and formats are supported:

  • Delimited with header (CSV/TSV) - Comma separated values or tab separated values. You can also select the delimiter character, the file quote character (optional), and the escape character (optional). Each row in the file will appear in Alooma as an event with a field for each column.

    For example, a CSV file containing:

    id, timestamp, value
    1, 1480500113.994287, 0.4269831606281763
    2, 1480500116.441592, 0.4875648468121073

    Will generate two events:

    {"timestamp": 1480500113.994287, "id": 1, "value": 0.4269831606281763}
    {"timestamp": 1480500116.441592, "id": 2, "value": 0.4875648468121073}
    

    If your CSVs don't have headers, check out how to add them.

  • Excel (.xls/x) - Each file is an Excel file with extension .xls or .xlsx. Each row in the Excel file will become an event in Alooma.

  • JSON file - Each file contains a single JSON object, which will become a single Alooma event. For example:

    {"timestamp": 1480500113.994287, "id": 0, "value": 0.4269831606281763, "triggers": []}
  • JSON lines - Each line must be a JSON object, which will each become an event in Alooma. For example:

    {"timestamp": 1480500113.994287, "id": 0, "value": 0.4269831606281763, "triggers": []}
    {"timestamp": 1480500116.441592, "id": 3, "value": 0.4875648468121073, "triggers": [8294]}
    {"timestamp": 1480500115.746223, "id": 1, "value": 0.34203259903426253, "triggers": [9960, 498, 6714, 1743, 7070]}
  • JSON array - The file includes an array of JSONs, which will each become an event in Alooma. For example:

    [{"timestamp": 1480500113.994287, "id": 0, "value": 0.4269831606281763, "triggers": []}, 
    {"timestamp": 1480500116.441592, "id": 3, "value": 0.4875648468121073, "triggers": [8294]}, 
    {"timestamp": 1480500115.746223, "id": 1, "value": 0.34203259903426253, "triggers": [9960, 498, 6714, 1743, 7070]}]
  • Other type of data - Loads the data as is (raw), meaning that each line is imported as an event without parsing or processing. You can use a transform function in the Code Engine to parse the events. Each line will show up as a message field in your event).

  • XML - If the file is formatted in XML, we will JSONify it (turn the XML tags into JSON fields) and turn it into a single event in Alooma.

Have a file type we don't support you'd like to use? Contact us!

Search results

    No results found