Yes it’s a different model + backend and obviously the extrapolation will never be as good as experimental values. but, 1. We have only used the multiplier value 3.4, and not the exact throughput from Lambda’s experiment. 2. We have also used the same input/output sequence length as Lambda's experiment. 3. Also our extrapolated value is inline with the specs of H200 when compared to Mi300x

Hot Aisle: https://hotaisle.xyz/pricing/

Lambda: https://lambdalabs.com/service/gpu-cloud#pricing

In this one we were only using 3.1 405B FP8. We took one model to simplify the setup and were mostly looking at the memory saturation effect. So basically we compared inference metrics of the same model. I suppose comparing 3.1 and 3.2 will be difficult as they are different models entirely. But open to ideas

Thanks Jack. This one too... https://news.ycombinator.com/item?id=42331355