Google already incorporates site speed into their rankings, separate from AMP.

However, if you want to get into Google's mobile carousel above the normal search results, you have to serve AMP pages and let Google cache them.