Hacker News new | past | comments | ask | show | jobs | submit login

Don't forget the validation!

Webapps are dumb middleware that pipes data from the database into an http stream - but it needs to determine which database calls to invoke and sanitize all the incoming junk.




We use arrow to do the validation 'for free': it has a typed columnar schema, so by passing the schema, we get the recordset validation without inspecting values.

we still need to do semantic validation, but w/ arrow, now we do that in bulk and on gpus :)




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: