Documentation

FTP or SFTP Integration

Alooma can read and replicate all of the data in files from an FTP or SFTP server. This allows you to, for example, merge arbitrary data from an S/FTP server with client usage data in your data destination.

Connecting to S/FTP

Files on your S/FTP server can be packaged and compressed using Tar or GZip, and can contain either one event per line, a single JSON object, or an array of JSON objects.

  1. Add your FTP or SFTP input from Alooma's plumbing screen.

  2. Select whether your server is FTP or SFTP.

  3. Specify the connection info for your server:

    • Hostname and Port of the server.

    • Username and Password for the server.

  4. If you chose FTP, you have an opportunity to define if the connection is via SSH.

  5. If you chose SFTP, you can choose to add our public key (recommended) or provide a password.

  6. Optionally, give a File Prefix (the prefix of the files you'd like to upload, for instance mydir/sys_log). If a prefix is entered, other files are ignored.

  7. Define which files to import: New (from the time you create the input, existing files will be ignored) or All files currently in that folder path.

  8. Define the file format of your files:

    • JSON lines - Each line must be JSON object. For example:

      Copy
      {"timestamp": 1480500113.994287, "id": 0, "value": 0.4269831606281763, "triggers": []}
      {"timestamp": 1480500116.441592, "id": 3, "value": 0.4875648468121073, "triggers": [8294]}
      {"timestamp": 1480500115.746223, "id": 1, "value": 0.34203259903426253, "triggers": [9960, 498, 6714, 1743, 7070]}
    • Delimited with header (CSV/TSV) - Comma separated values or tab separated values. You can also select the delimiter character, the file quote character (optional) and the escape character (optional). Each row in the file will appear in Alooma as an event with a field for each column.

      For example, a CSV file containing:

      Copy
      id, timestamp, value
      1, 1480500113.994287, 0.4269831606281763
      2, 1480500116.441592, 0.4875648468121073

      Will generate two events:

      Copy
      {"timestamp": 1480500113.994287, "id": 0, "value": 0.4269831606281763}
      {"timestamp": 1480500116.441592, "id": 3, "value": 0.4875648468121073}
      

      If your CSVs don't have headers, check out how to add them.

    • Other type of data - Loads the data as is (raw), meaning that each line is imported as an event without parsing or processing. You can use a transform function in the Code Engine to parse the events. Each line will show up as a message field in your event).

    • Contact us if you'd like to load files that are complete JSONs or JSON arrays. 

Keep the mapping mode to the default of OneClick if you'd like Alooma to automatically map the file events directly to your data destination. Otherwise, they'll have to be mapped manually from the Mapper screen.

Note

If there are any null bytes in rows in your data, those rows will be discarded and will not appear in the Restream Queue.

That's it! You're ready to import your data from your FTP or SFTP server into Alooma!

Search results

    No results found