What are common Python based open source ETL tools?
The are quite a bit of open source ETL tools, and most of them have a strong Python client libraries, while providing strong guarantees of reliability, exactly-once processing, security and flexibility. The following blog has an extensive overview of all the ETL open source tools and building blocks, such as Apache Kafka, Apache Airflow, CloverETL and many more.
If you are looking to build an in-house solution, there are some points that you need to take in account. Rami Amar, one of the co-founders of Alooma (I work there! shameless plug! :D ), wrote a great piece about the challenges and the pitfalls in the process. I recommend you to carefully read it before you embark on the journey of building it in-house.
Published at Quora. See Original Question here