Performance: Real-world use
Living with the Qualcomm-based Surface Pro X is somewhat like living with a Windows 10 S machine: Within the ecosystem of Microsoft apps—Edge, Office, the basic apps like Mail and Calendar—the Surface Pro X feels decently quick. It performed the mainstream tasks that I need to accomplish (web browsing, word processing, video playback, email, and so on) quickly and responsively. Slack (though just the 32-bit version) worked just fine. It’s almost immediately responsive as startup, which makes it feel fresh and perky. The beta version of Edge (or Edgeium) is very fast. Adobe’s Creative Cloud is heading to the Surface Pro X, but isn’t here quite yet.
Once you exit that safe room, though, it’s a mixed bag. Pages loaded snappily with both Chrome and Firefox Nightly (supposedly optimized for ARM)—but only after remaining totally blank for a second or two on a fast home broadband connection. Forget about playing games: The Microsoft Store app pushed games like Mah-Jongg at me, and wouldn’t run the Xbox beta app. I resigned myself to testing Asphalt 8, about the only game I could run when I’ve tested previous Qualcomm platforms, too.
Remember, even though the version of Windows 10 on the Surface Pro X is a 64-bit operating system, it won’t run a 64-bit app that’s been written for the X64 architecture. In fact, there’s a whole list of caveats that Microsoft itself has published: Some games won’t work (anti-cheating software won’t run on ARM), some antivirus apps won’t work, and so on. It’s a minefield that threatens to blow up what you want to do on an ARM-based machine, and as such requires careful navigation.
In reviewing the Surface Pro X’s prices online, I also skimmed some of the Amazon reviews, and I have to agree with this statement: “Only a subset of Windows software will run on this computer, and only a subset of that subset will run at full speed.”
Because of the limitations of the ARM architecture, we were able to run only a few of our normal suite of benchmarks. We also had to create a new, smaller group of comparable products. We were nevertheless able to bracket the Surface Pro X’s performance: It’s somewhat less than some of Microsoft’s older products that are based on Intel’s 7th-gen Core chips.
Our comparison products include the Lenovo Yoga C630, which uses the SQ1’s predecessor, the Qualcomm Snapdragon 850; our Editor’s Choice-award winning tablet, the Lenovo ThinkPad X1 Tablet (3rd Gen); the Microsoft Surface Go tablet; the first-generation Microsoft Surface Laptop, and the Microsoft Surface Laptop 3 (Core i7). We didn’t have a Surface Pro 7 tablet to test, but the Surface Laptop 3 (Core i7) uses the same Ice Lake processor.
For our test suite, we first chose the PCMark 10 Apps benchmark, which restricts testing to the three main Office apps (Word, Excel, and PowerPoint, plus the EdgeHTML version of Microsoft Edge.) It’s about as real-world as you can get. Remember, Qualcomm claimed that it could perform as well as a Core i5 with its 8cx in this test. Our results put it a notch lower on the totem pole.
To emphasize multimedia applications, including video editing, photo manipulation, and light gaming, the PCMark 8 Creative benchmark fills the bill. Here, again, the Surface Pro X is outpaced by older, supposedly slower machines.
Underwriter Labs made a special version of its 3DMark graphics benchmark for ARM-powered machines, that can be run on both x86 as well as ARM-based systems. This allows us to do some cross-platform benchmarking of the Surface Pro X’s integrated Adreno GPU, and compare it to the integrated chips in the Core and Ryzen processors. Here, the Surface Pro X does surprisingly well.
Performance: The battery life boondoggle
Distilling a product down into a single attribute can be a dangerous stereotype. On the other hand, Qualcomm’s whole argument for the Snapdragon 8cx or the Microsoft SQ1 has been all-day battery life, with near ubiquitous connectivity. Unfortunately, the Surface Pro X’s battery life was all over the map.
Manufacturers have been pushing hard for us to test products in the real world and avoid synthesized use cases. But they aren’t repeatable. That’s the kicker—we don’t know if, or when, an update to Photoshop will impact performance, or by how much. With that said, we do test each product under real-world scenarios, as well as with recorded benchmarks.
In this case, we took four separate approaches. Typically, we loop a 4K video until the battery expires, with earbuds plugged in. This simulates one experience: watching downloaded movies on a transoceanic flight. In this case, because the Surface Pro X lacks a headphone jack, we tried two things: We streamed audio over Bluetooth to a pair of headphones, and separately, we used a USB-C to 3.5-mm dongle to perform the same test.
We tested using PCMark 10’s battery test, performing a scripted mix of Office apps and Web browsing. I also used the Surface Pro X in a day’s work, working on the Bay Area’s BART subway on the way to work, and then on a desktop performing my day’s work.
The Surface Pro X’s battery life is decent for a tablet or a laptop, but well short of the lofty promises of “25 hours." Yes, the size of the battery matters. So does the fact that the SQ1 is a semi-custom chip that’s been tweaked for performance.
But let’s take a look at the numbers. We’ve used the dongle-streaming number as the most comparable to our other tests.
- Streaming over Bluetooth: 445 minutes, or 6 hours, 25 minutes.
- Streaming over a USB-C dongle: 521 minutes, or 8 hours, 41 minutes.
- PCMark 10’s battery test: 503 minutes, or 8 hours, 23 minutes.
- Real-world: 346 minutes, or 6 hours, 46 minutes.
That last number was a bit of a shocker, as you would think that'd be the most accurate test of the Surface Pro X’s battery life. But my tests simply reflected what I do during an average day: I drove to BART, found a seat on the train, connected via the Surface Pro X’s integrated cellular connection, and used Excel, Word, the Mail app, and the built-in Edge client. At work, I remained on cellular.
It’s worth noting that I had performed a battery rundown test the night before, so I had only managed to charge it to 72 percent by the time I left. But that meant that I was at 4 percent battery by 12:49 PM, which extrapolated out to under 7 hours of use. Unacceptable.
Surface Pro X: Some of the glitches
Living with the Surface Pro X also forces you to acknowledging that not everything works—a deal-breaker right away for some, especially where apps are concerned. But I was really annoyed to pull out the Surface Pro X, connect it to the tried-and-true Surface Dock—and then discover that the mouse cursor showed up extremely intermittently. (It later fixed itself after an update.) We also use a third-party VPN to connect to our domain, which refused to connect at all either over cellular or ethernet. Then the Microsoft Movies & TV app (which we use for our video rundown battery test) started glitching, starting and stopping over and over. What more gotchas lie in wait?
Conclusion: This is not a Surface “Pro” anything
After years of cookie-cutter upgrades, the Surface Pro X delivers a well-thought-out physical redesign. Again, however—and this is getting tiring by now—app compatibility, performance issues and bugs complicate Microsoft’s effort. Calling it an “Edgebook” isn’t quite right, because the Surface Pro X runs Office and other Microsoft apps well. But it still feels somewhat short of an actual PC.
One redeeming factor that we haven’t touched much upon is the price: $999 for our base model isn’t bad...but it’s still almost $200 more than the Core i5/8GB/128GB model Microsoft Surface Pro 7.
We reviewed Lenovo’s ThinkPad X1 Tablet (3rd Gen) eighteen months ago, with a 13-inch 2K display, an 8th-gen Core processor, two USB-C ports with Thunderbolt 3, and a nanoSIM slot—and a bundled pen and keyboard. These all match or exceed what the Surface Pro X delivers. The ThinkPad’s big letdown was battery life, which the Surface Pro X easily surpasses by about 90 minutes—but it’s not even close to the promises of truly all-day battery life, which Qualcomm previously delivered upon in earlier machines like the Asus NovaGo and Samsung Galaxy Book 2.
The question that I hope Microsoft is asking itself is: What compelling reason has it given for customers to buy the Surface Pro X? Always-on connectivity? It’s not a unique argument, and one that Intel itself is gunning for. We haven’t tested Microsoft’s Surface Pro 7, but we’d assume that that’s its best tablet, not the Surface Pro X. With this tablet, we can’t see why Microsoft would use the term “Pro” at all.