What is SnapLogic?
Founded in 2006, SnapLogic provides platform-agnostic connection for data, applications, and APIs. Their enterprise integration cloud provides a unified integration platform as a service (iPaaS), which they call the Elastic Integration Platform.
How it works
SnapLogic uses pre-built, intelligent connectors, called Snaps, for 400+ applications and data stores. These connectors simplify the effort of moving data from one database to another. Hybrid batch and streaming support provides data movement flexibility.
Snaps use Snaplex, a self-upgrading, elastic execution grid, for data processing behind the firewall, in Hadoop or in the cloud. Snaplex streams data between applications, data sources, and platforms.
There are several types of Snaplex:
- Cloudplex, which is hosted in the cloud.
- Groundplex, which is behind the customer’s firewall.
- Hadooplex, which uses Yet Another Resource Negotiator (YARN) to execute pipelines. SnapLogic can run natively on a Hadoop cluster. Users can create Hadoop-based pipelines without coding.
- Sparkplex, a data processing platform with a collection of processing nodes (containers) that take data pipelines, convert them to the Spark framework, and then execute them on a cluster. Users can create Spark-based pipelines without coding.
Big data integration uses three base tools:
- Designer: A user interface based in HTML5 to specify and build “pipelines”, or integration workflows out of Snaps with drag and drop. These pipelines can be streaming or accumulating. The accumulating type collects all data from the input source before emitting it from the pipeline. Accumulating pipelines are used for more complex data manipulations.
- Manager: An application that controls and monitors data integration, and administers data and process flow lifecycles. This interface also administers users, projects, security, single sign on (SSO) and password encryption.
- Dashboard: Used for viewing data integrations, including performance, utilization, and health. The interface includes drill-down capabilities and provides for triggered event notifications.
- Data Snaps for all major SQL databases and data sources: MySQL, SQL Server, Oracle, Teradata, Cassandra, Trilium, Amazon Dynamo DB, Confluent, Amazon Redshift.
- Analytics Snaps for a variety of systems: HDFS Read/Write, Anaplan, Google Analytics, Amazon Dynamo DB, Cassandra, Amazon Redshift, Birst.
- Core Snaps used for data analysis on common systems and file types: CSV Read/Write, REST, Filter, Spreadsheet Reader, Unique, ERP, Fixed Width Reader/Writer, Field Cryptography, Email Snap, Sequence, Sort, XML Read/Write, Transform, JSON Read/Write, Transform.
- An SDK and APIs are available to build or embed custom Snaps and integration flows into other applications and platforms.