Load MongoDB data to Amazon Redshift in minutes.
Copy your MongoDB data to Amazon Redshift to improve the performance of your queries at scale and to generate custom real-time reports and dashboards. Combine your MongoDB data with other data sources such as mobile and web user analytics to make it even more valuable.
How it works
By leveraging the Alooma enterprise data pipeline, you can easily integrate, connect, and watch your MongoDB data flow into Amazon Redshift.
Add MongoDB as a data source.
Map MongoDB data to Amazon Redshift.
See MongoDB data flow into Amazon Redshift.
Loading MongoDB Data to Redshift
Tailing the OpLogSuitable for Real-time replication of semi-structured MongoDB collections. What is it The OpLog is a built-in MongoDB mechanism for chronologically logging changes made to all documents in all databases. These changes include INSERTS, UPDATES and DELETES. How it works
- Dump an initial snapshot of the desired collections
- Continuously read the changes recorded by the OpLog
- Automatically translate the collections aggregated schema to the output schema
- Load INSERT, UPDATE and DELETE operations to a staging table
- Consolidate the staging tables to exact copies of the original collections
- Use a read-replica to refrain from sharing production resources.
- When using a sharded replica set, make sure to configure all shards for replication.
MongoDB to Redshift - Should you do it yourself or use a third party solution?
When performing an initial collection dump, make sure the format matches the OpLog format, since it may vary with some data types e.g. timestamps.
When reading update operations from the OpLog, make sure to fetch the whole document, as the OpLog contains only the data that has changed.
By nature, the documents in MongoDB don’t conform to strict schemas. This effectively requires frequent schema changes on your data warehouse. To avoid this, you can use the Alooma Mapper to update schemas automatically.
Calculate how much it would cost to build your own data pipeline in-house here.
Export your MongoDB data along with all of
your other data sources to Amazon Redshift.
Extract, transform and load your MongoDB data to your data warehouse using Alooma's ETL stream-based data pipeline as a service. Replicate your MongoDB data to improve the performance of your queries at scale and to generate custom real-time reports and dashboards. Combine your MongoDB data with other data sources such as mobile and web user analytics to make it even more valuable.