> developers don’t want GPUs. They don’t even want AI/ML models. They want LLMs.
Is there not a market for the kind data science stuff where GPUs help but you are not using an LLM. Like statistical models on large amounts of data and so on.
Maybe fly.io customer base isn't that sort of user. But I was pushing a previous company to get AWS GPUs because it would save us money vs CPU for the workload.
There is a market but it most likely requires a thick software layer to enter the competitive space. Modal Labs, Anyscale, and Outerbounds are examples of companies competing for "data science stuff" and have thick software layers over the VMs.
not from fly.io, but my experience is that most data scientists will just prefer to lump it with the tools they know (pandas / R) on CPUs, rather than delving into things like rapids https://rapids.ai -- even if it makes things faster/cheaper.
I might have had a bad sample set so far. But the "doing statistics" bit seems to be the interesting thing for them. the tooling doesn't really factor into solutions/plans that often. and learning something new because "engineer say it shinier" doesn't really seem to motivate them much :/
Do many DS use Google Colab and click the GPU option? That made me think GPUs would be more popular (due to speed).
Also GPUs may be used when productionizing work done by DS but maybe I am in a tiny niche here of (Data Science) intersection (Scale up) minus (Deep learning LLM etc.)
Is there not a market for the kind data science stuff where GPUs help but you are not using an LLM. Like statistical models on large amounts of data and so on.
Maybe fly.io customer base isn't that sort of user. But I was pushing a previous company to get AWS GPUs because it would save us money vs CPU for the workload.