Hacker News new | past | comments | ask | show | jobs | submit login

I thought they used Piper (based on Airflow) for workflows at Uber. https://eng.uber.com/managing-data-workflows-at-scale/



That is for data workflows (your etl jobs basically). This is for services.


If you already have a service that requires ETL, Cadence might be a good choice as well. You will need more boilerplate and setup compared to Airflow/Piper, but you can share data structures between ETLs and services that use it and have more control over execution and deployment as you own your workers.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: