Hacker News new | past | comments | ask | show | jobs | submit login

We used them for a single data source at GitLab (Zendesk). Worked pretty well! But we quickly hit a road block where what you could get via the UI wasn't all the tables available. We wound up forking it and adding to the tap. Basically all of our extractors and loaders are going to follow the Singer spec for taps and targets - it's a pretty nice model.

Internally, we're using a tool called Meltano which is aimed at solving just your problem. Most of our data warehouse is coming from external business ops tools (Salesforce, Zuora, Zendesk, Marketo, etc.) and we're using dbt for transformations w/ Looker as the BI layer. Definitely check our primary analytics repo [0] as all of our code is out in the open. Feel free to ping me if you have more questions - tmurphy at gitlab.

[0] https://gitlab.com/meltano/analytics/




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: