Where is that improvement coming from? Hardware is already here to compute gemm ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		pdhborges 53 days ago \| parent \| context \| favorite \| on: AI capex is so big that it's affecting economic st... Where is that improvement coming from? Hardware is already here to compute gemm as fast as possible.

leakyfilter 53 days ago [–]

Raw gemm computation was never the real bottleneck, especially on the newer GPUs. Feeding the matmuls i.e memory bandwidth is where it’s at, especially in the newer GPUs.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact