LM Studio with Phi 3.5 on Snapdragon X Elite No NPU support yet
I upgraded from my Surface Laptop 4 to a Surface Pro 11 with the Snapdragon X Elite processor and have been very satisfied so far with the performance, battery life, and the compatibility. I wanted to see if LM studio would run on the Surface Pro with the Snapdragon. To my surprise they have recently released LM Studio in a tech preview for Windows on ARM. So let’s look!
Installing LM Studio Tech Preview for Windows on ARM
First of all, you will need to go to the official LM Studio site and download the installer for Windows on ARM. You can do that here: LM Studio – Discover, download, and run local LLMs.
Below is a screen capture from the LM Studio site, where you can download the Tech Preview for Windows on ARM.
After installing LM Studio it will launch after the installation. The program seems snappy with native ARM support on the Snapdragon X Elite processor.
Downloading and installing Phi-3.5 LLM from Microsoft
Once you have LM Studio downloaded and installed, you can download and install the new Phi-3.5. Just search for Phi-3.5. Click the green Download button on the right side in the description of the LLM model.
You will see the Downloads open in LM Studio and you will see the download of the model progress as it comes down.
After it downloads, from the LLM downloads, you can click the button Use in New Chat to start chatting with the model immediately.
No support for the Surface Pro 11 NPU as of yet
One thing I was a little bit disappointed in was there is no support for the Surface Pro 11 NPU as of yet. If you pull up your performance in task manager, you will see the NPU is baselined. However, it is pretty amazing to see how well the Snapdragon X Elite handles the tasks even without the help of the NPU. I didn’t see any extremely slow performance on chats, it was actually rather quite snappy.
Wrapping up
It is great to see that support for Windows on ARM is gaining traction. I think the Snapdragon X Elite has ushered in a new era for Windows devices on ARM and the performance is akin to MacBooks with the M-series processors. Performance has been really good for me. It will be exponentially even better when LM Studio can integrate support for the NPU in the Surface Pro 11. Let me know if you guys are starting to delve into Windows on ARM.
My Surface Pro 11 runs native Arm64 versions of most of my apps, even VPN (Private Internet Access), and the ones that aren’t native arm64 emulate flawlessly. As of Aug 29, 2024 an NPU version of LM Studio (Llama.cpp) is the thing I remain excited to see but, like the article says, local chat performance with just the Snapdragon X Elite cpu is currently very good.
This is a good list of Windows ARM software : https://armrepo.ver.lt/
Joe,
Thank you for the comment and insights you have on the NPU with LM Studio. I am really impressed with how well the Snapdragon X Elite handles the LLM compared to an x86 CPU. Also, great resource for ARM software as well. Thanks for sharing!
Brandon