How to Load Salesforce Data Into Redshift

by Garrett Alley  
6 min read  • 18 Sep 2018

Getting your Salesforce data into your Redshift data warehouse is the first step in setting up a powerful analytical workflow and getting valuable insights from your data. Typically, having that data together with data from your various other sources in Redshift delivers a compounding effect. The more data you have, the better your analysis, the better your insights.

Challenges with loading Salesforce data into Redshift

At first, the idea of moving data from Salesforce into Redshift may sound straight forward. You simply use the Data Loader export wizard and select the objects you want to export (and whether you want to export soft-deleted records). But what if you have huge Salesforce objects with gigabytes of data? You'll need a secure and robust way to pipe that data into Redshift. That's likely to require a lot of time and specialized resources.

And usually around this point you realize that you might want to set up a way to repeat this task. Whether you want to perform the same migration into Redshift periodically, or you want to add different objects (or even different data sources besides Salesforce), you'll need someone to build a method for scheduling, tracking, and logging the process. And it will need to be scalable. Oh, and you'll need to make sure you have a way of catching and handling any errors that occur along the way.

You may also notice an opportunity to scrub PII (personally identifiable information) or enrich the data with things like geolocation data or currency conversion before the data is uploaded.

It's not an easy task, and unless you already have a seasoned team in place, you will need to train or hire someone who has the expertise to do the work. In reality, by the time you factor in security concerns, headcount, training, and technical complexity, you realize that you are, in essence, building your own ETL platform, just to extract your Salesforce data.

An easier Salesforce to Redshift solution: Alooma

We recommend that you don't build a custom ETL tool and take on all of the technical challenge and resource costs. The better solution is to use a modern ETL platform designed to move data from Salesforce (and other sources) into Redshift and make strategic transformations along the way.

Alooma is the enterprise data platform built for the cloud. With built-in support for Salesforce and Redshift, and bolstered by enterprise security and scalability, it's the ideal solution.

Steps to load data from Salesforce to Redshift

Getting your Salesforce data into Redshift is incredibly simple with Alooma. Let's break down the process.

  1. On the Plumbing page, click "Add new input" and select Salesforce from the list of integrations.
  2. salesforce to Redshift step 1

  3. Enter your account information as required, name the input, and then select all the objects you want to import.
  4. salesforce to Redshift step 2

  5. That's all there is to it. Once you save your input, assuming your credentials are correct, your Salesforce data will automatically begin importing into Redshift. See our Salesforce documentation for more information.
  6. salesforce to redshift step 3

Of course, there's a lot more you could do along the way:

  • You could use the Code Engine to transform/enrich/cleanse data as it flows from Salesforce to Redshift.
  • You could change how the schema is mapped, via the Mapper; however, most of the time Alooma's powerful auto-mapping works just fine.
  • You could click on the Live tab for your Salesforce input and monitor the data flow. Or click the Samples tab to see examples of the actual data being loaded.

What's next?

Put Your Data to Work: Now that you have your Salesforce data in Redshift you can, for example, merge client usage data with payment data to get insights on your business.

Bust Data Silos: Don't just work with data from Salesforce. Perform an information census and look for data silos within your company. Integrating multiple data sources into Redshift is straightforward and simple, and each new source — whether it's a stream, a database, a file, etc. — potentially increases the usefulness and impact of your analysis.

Automate the Process: Using an enterprise data platform means you can automate data extraction and transformation from multiple sources without having to build out your infrastructure.

Benefits

Enterprise scalability and performance: The Alooma platform provides horizontal scalability, handling as many events from as many data sources as you need.

Security at the core: The Alooma platform is built around a robust and flexible security architecture, providing full visibility and control over data. SOC 2 Type II, HIPAA and EU-US Privacy Shield, GDPR compliant, Alooma does not store any data permanently and encrypts all data in motion.

Guaranteed data integrity and reliability: The Restream Queue, Alooma's intelligent data integrity engine, is your safety net for ensuring zero data loss. The Restream Queue collects all the events that were not loaded to Redshift, for whichever reason, making them easy to fix and enabling you to "restream" them into Redshift later.

Flexible data enrichment: The Code Engine, a stateful, python-based processing engine, enables on the fly data enrichment for sophisticated use cases, such as real-time alerts, sessionization, anomaly detection, and more. Customize your data exactly how you want by writing real code to transform data on the stream.

Simple yet powerful data management: The Mapper automatically infers schemas, maps schema changes, or enables customization of mappings to your liking, ensuring you meet all your data governance requirements.

Cost effective: You won't need to hire or train staff to build the process, saving time and money. You won't need to buy more machines or processing power as your data grows, and adding new data sources to import into Redshift is a breeze.

Getting started

Ultimately, you want the process of getting insights from your data to be as simple as possible. The fewer steps, the lower the cost, the better. And if you can get data from other sources thrown in without requiring custom coding or processes, you're even further ahead of the game. Using the cloud to store and process that data is the natural next step.

Alooma was designed and built for the cloud. Alooma enables businesses to use all of their data to make better data-driven decisions, providing Data Scientists and Data Engineers the ability to integrate, cleanse, enrich, and bring together batch or streaming data from various data silos at any time to any destination.

Alooma makes the whole process of getting your Salesforce data into Redshift simple and affordable.

Ready to get started? Alooma can help. Contact Alooma today to learn more about how a Salesforce and Redshift integration solution can benefit your business.

Like what you read? Share on

Get your data flowing

Contact us to start using Alooma for free

Get Started

This might interest you as well

Schedule a free demo!

We'll show you how Alooma can integrate all of your data sources in minutes.