11/12/2023 0 Comments Amd![]() The main gaps were due to a lack of software support and optimizations for the relevant models. Hardware is not necessarily the reason why AMD lagged in the past. We put it here as a reference point to provide more information.Īt a high-level, we can find that AMD 7900 XTX is comparable to RTX 3090 Ti from the hardware spec perspective. It is harder to compare the price of 3090Ti as that was a previous generation. RX 7900 XTX is 40% cheaper than RTX 4090.Lantency sensitive LLM inference is mostly memory bound, so the FP16 performance is not a bottleneck here. 4090 has 2x more FP16 performance than 7900 XTX, while 3090 Ti has 1.3x more FP16 performance than 7900 XTX.All have 24GB memory, which means they can fit models of the same size. ![]() AMD is one potential candidate.įrom the spec comparison, we can see that AMD’s RX 7900 XTX is a good match for NVIDIA’s RTX 4090 and RTX 3090 Ti. Support to a broader class of hardware accelerators. In the meantime, with the high demand for compute availability, it is useful to bring Most of the performant inference solutions are based on CUDA and optimized for NVIDIA GPUs. ![]() There have been many LLM inference solutions since the bloom of open-source LLMs. Besides ROCm, our Vulkan support allows us to generalize LLM deployment to other AMD devices, for example, a SteamDeck with an AMD APU. More specifically, AMD Radeon™ RX 7900 XTX gives 80% of the speed of NVIDIA® GeForce RTX™ 4090 and 94% of the speed of NVIDIA® GeForce RTX™ 3090Ti for Llama2-7B/13B. MLC-LLM makes it possible to compile LLMs and deploy them on AMD GPUs using ROCm with competitive performance. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |