Making big data analysis accessible throughout your company

by Andy Granowitz  
5 min read  • 23 May 2016

Thanks to the likes of Google Analytics, Mixpanel, and others, it's easy to measure app engagement. The problem is these services only report on a predefined set of metrics and user segments. Also, it's hard to analyze other customer interactions, such as sales calls or support interactions, alongside app usage data. When you need more custom analysis or to combine product usage with other data sources, you'll need to query the raw data. I'm talking user IDs and timestamps.

There are a number of great modern ETL tools that centralize app engagement and other customer interaction data into a data warehouse like Redshift or BigQuery. Having all of your analytics data in Redshift and queryable by SQL is very powerful: it allows you to answer nearly any question you have of your data.

But there's a Big Problem™: only some people know how to write SQL. Engineers and savvy analysts will be comfortable writing SQL, but less technical members of your team also need access to the data. What's the best way to make everyone at your company data driven, regardless of their SQL knowledge?

We've found three ways to to help everyone at your company benefit from SQL.

Make it easy and fun to share SQL

Sharing queries can be an important, rewarding part of your team's culture.

At Wagon, we encourage this in two ways. We have a dedicated Slack channel for posting ad-hoc queries, data, and charts. Slack is a great place to show off your hard work! We also keep a repository of "source of truth" queries that are code reviewed, commented, and frequently updated. Having a place for ad-hoc sharing and more standard reports solved a lot of confusion.

Wagon SQL shared on Slack

Reuse and modify your teammates' work

Many questions can be answered with the same underlying SQL. Make it easier to browse other people's queries by having them in one place and with reasonable comments. We see a lot of people tweaking a query by changing a where clause to use a different customer ID or log line. If you're using data from another analytics service, many provide starter queries. Here are a few: Amplitude, Google Analytics, Segment, and Snowplow.

Learn some SQL

SQL isn't that hard to learn. We joke that you can double your salary by learning how JOIN works. We recommend Udemy's online SQL course or if you prefer in person, check out General Assembly's SQL bootcamp.

This might interest you as well