Documentation

Shopify Integration via REST API

Follow these instructions to create a basic REST API input for your existing Shopify account. What we describe below is the most basic of implementations — you can add much more complexity. In our example, we start by creating the REST API input and then we show how you can use the Code Engine to convert the line_item sections in the Shopify Orders records into individual events for uploading into your data destination.

  1. Create a private application for authentication in Shopify using the instructions here: https://help.shopify.com/api/getting-started/authentication/private-authentication#generate-credentials-from-the-shopify-admin

  2. Next, add a REST API input via the Plumbing page in Alooma. Specify a name and provide the URL and authorization username/password. The URL is something like: https://<your-name.myshopify.com/admin/orders.json> and the authorization is the API key/password from step 1 above.

  3. Now specify the pagination. For Shopify, the pagination type is Incremental. Set the Page Parameter to "page" and the Initial Value to "1".

  4. Enter the endpoint parameters. In our example we're configuring the input to retrieve Orders. You can add more parameters and more complexity to extend the functionality to help with handling the data. See Shopify's REST API reference for options.

    URL Parameter

    Value

    Type

    limit

    250

    text

    status

    any

    text

  5. Enter a Data Field value of "orders".

Here’s what an example Shopify input looks like:

shopify_REST.png

There's more you can configure, but this is a solid start. See the Alooma REST API Integration documentation for more information.

So, now you have the data, but you may want to clean it up or process it prior to importing it into your data destination. One example of how to do this is to break up the records (which may contain multiple events) into individual events. See the following section for an example of how to use the Code Engine to create individual events from the Orders records for loading into your data destination.

Example: Creating individual events from records

The instructions above cover creating the input, but the data you retrieve — the Orders records — likely still require some additional processing prior to import into your data destination (because they contain multiple events). In this example, we'll take the line_item section from a record and convert it into a series of events for input.

Here's our example record. You can see the two line_items:

Copy
{
    "id": 32674523,
    "name": "Dylan Hercher",
    "email": "dylan@alooma.com",
    "closed_at": "2018-01-17T09:12:04-08:00",
    "created_at": "2018-01-16T20:29:34-08:00",
    "updated_at": "2018-02-01T18:08:58-08:00",
    "order_number": 73857,
    "line_items": [
        {
            "id": 12345,
            "variant_id": 45550183171,
            "title": "The Alooma Code Engine",
            "quantity": 1,
            "price": "amt",
            "sku": "DSFSDHEW-13",
            "fulfillment_status": "fulfilled"
        },
        {
            "id": 12346,
            "variant_id": 45550183192,
            "title": "The Alooma Mapper",
            "quantity": 1,
            "price": "amt",
            "sku": "DSFSDHEW-14",
            "fulfillment_status": "fulfilled"
        }
    ],
    "billing_address": {
        ...
    },
    "shipping_address": {
        ...
    },
    "client_details": {
        ...
    },
    "customer": {
        ...
    },
    "_metadata": {
        ...
    }
}

We'll ignore the rest of the information for now. In the Code Engine, we created a module named Utils and imported it into the main module. The Utils module does the actual work of breaking out the line_items into records. First, the Utils module:

Copy
def _extract_data(event, field, remove_extract=True):
  next_field = field.split('.')[0]
  remaining_field = ".".join(field.split('.')[1:])
  
  if next_field not in event:
      raise Exception("Event is Missing Field %s" % field)
  
  if not remaining_field:
      if remove_extract:
          return event.pop(next_field)
      return event[next_field]
  
  return _extract_data(event[next_field], remaining_field, remove_extract=remove_extract)

def extract_embedded_data(event, field, event_type=None, primary_key_fields={}, remove_extract=True, allow_empty=False):
  
  try:
    new_events = _extract_data(event, field, remove_extract=remove_extract)
  except:
    if allow_empty:
      return []
    raise
  metadata = deepcopy(event['_metadata'])
  
  if isinstance(new_events, list):
    for new_event in new_events:
      # Assign Primary Key When Applicable
      for primary_key_field, primary_key_name in primary_key_fields.iteritems():
        primary_key = _extract_data(event, primary_key_field, remove_extract=False)
        new_event[primary_key_name] = primary_key
      # Handle Metadata Copy
      new_event['_metadata'] = metadata
      new_event['_metadata']['event_type'] = event_type or field
  else:
    # Assign Primary Key When Applicable
    for primary_key_field, primary_key_name in primary_key_fields.iteritems():
      primary_key = _extract_data(event, primary_key_field, remove_extract=False)
      new_events[primary_key_name] = primary_key
        
    # Handle Metadata Copy
    new_events['_metadata'] = metadata
    new_events['_metadata']['event_type'] = event_type or field
    new_events = [new_events]
      
  return new_events

Next, our main module looks like this:

Copy
   import utils
# ------------------------------
# ---------- SHOPIFY -----------
# ------------------------------

def transform_shopify_orders(event):

  event_type = event['_metadata'].get('event_type') or event['_metadata']['input_label']
  if event_type.startswith("shopify."): # Already Processed
    return event

  event['_metadata']['event_type'] = 'shopify.orders'
  all_events = [event]

  primary_field_map = {"id": "order_id", "name": "order_name"}
  all_events += utils.extract_embedded_data(event, "line_items",
                                            event_type='alm_shopify.order_line_items',
                                            primary_key_fields=primary_field_map)

  return all_events

# ------------------------------
# ----- DO NOT EDIT BELOW -----
# ------------------------------
def transform(event):

  func_name = 'transform_' + event['_metadata']['input_label'].lower().replace(' ', '_').replace('-', '_')
  if func_name in globals():
    f = globals()[func_name]
    return f(event)

  else:
    return None

After running the record through the Code Engine, here's the output with the line_items broken out into events:

Copy
[
    {
        "customer": {
            ...
        },
        "billing_address": {
            ...
        },
        "name": "Dylan Hercher",
        "client_details": {
            ...
        },
        "created_at": "2018-01-16T20:29:34-08:00",
        "updated_at": "2018-02-01T18:08:58-08:00",
        "id": 32674523,
        "_metadata": {
            ...
        },
        "shipping_address": {
            ...
        },
        "closed_at": "2018-01-17T09:12:04-08:00",
        "order_number": 73857,
        "email": "dylan@alooma.com"
    },
    {
        "sku": "DSFSDHEW-13",
        "title": "The Alooma Code Engine",
        "order_id": 32674523,
        "price": "amt",
        "order_name": "Dylan Hercher",
        "fulfillment_status": "fulfilled",
        "_metadata": {
            "@parent_uuid": "",
            "@uuid": "53fe36-8f30-84C-b09c-42144v22",
            "event_type": "alm_shopify.order_line_items",
            "input_type": "rest_endpoint",
            "@timestamp": "2018-03-12T23:00:45.595Z",
            "pulled_at": "2018-03-12T22:58:35.280915",
            "base_url": "https://alooma.myshopify.com/admin/orders.json",
            "input_id": "7382423-846-43a9-ic7-03b23c270",
            "restream_count": 0,
            "input_label": "shopify_orders",
            "snapshot_number": 0,
            "schema_url": "schema?id=7382423-846-43a9-ic7-03b23c270&schema_object=shopify_orders"
        },
        "variant_id": 45550183171,
        "id": 12345,
        "quantity": 1
    },
    {
        "sku": "DSFSDHEW-14",
        "title": "The Alooma Mapper",
        "order_id": 32674523,
        "price": "amt",
        "order_name": "Dylan Hercher",
        "fulfillment_status": "fulfilled",
        "_metadata": {
            "@parent_uuid": "",
            "@uuid": "53fe36-8f30-84C-b09c-42144v22",
            "event_type": "alm_shopify.order_line_items",
            "input_type": "rest_endpoint",
            "@timestamp": "2018-03-12T23:00:45.595Z",
            "pulled_at": "2018-03-12T22:58:35.280915",
            "base_url": "https://alooma.myshopify.com/admin/orders.json",
            "input_id": "7382423-846-43a9-ic7-03b23c270",
            "restream_count": 0,
            "input_label": "shopify_orders",
            "snapshot_number": 0,
            "schema_url": "schema?id=7382423-846-43a9-ic7-03b23c270&schema_object=shopify_orders"
        },
        "variant_id": 45550183192,
        "id": 12346,
        "quantity": 1
    }
]

If you have questions or want to know more about how we did the transformation, please feel free to reach out.

Search results

    No results found