AROWANA - Benchmarking Eight Windows Versions on Pentium 3/4

So many options, which to choose?

In the span of six years, Windows has been incarnated into eight distinguishing versions, and even more if you count the separate editions coming in different levels of premium, including Windows NT Workstation and Server. You might know that each successive version of Windows grows more bloated, arbitrarily shortening the lifespan of hardware.

By how much does a newer operating system drag down an old system, exactly? The difference could be drastic on an underpowered machine, but in the case of something that's pretty beefed up, what you go with may not be too important.

Test Subjects

Two computers were used in the Arowana benchmarks.

The Late 440BX Build

• Intel Pentium III 800MHz
• 512MB PC100 SDRAM
• 3dfx Voodoo3 3000 AGP
• Sound Blaster AWE64 Value

The Dell Dimension 8100

• Intel Pentium 4 1.5GHz
• 1GB RDRAM ^[1]
• nVidia GeForce 2 Ultra
• Turtle Beach Santa Cruz PCI Sound Card ^[2]

Frame Rates

All tests were conducted at a resolution of 1024x768.

WinQuake

Running the first timedemo in WinQuake provides consistent results across all operating systems, around 24 frames per second for the Pentium III and 41 to 43 FPS for the Pentium 4. There is an oddity in the Pentium 4 tests, though: when running this under Windows NT 4.0, 2000, or XP, it caps at 30 FPS. This may be the result of vertical sync being forced by GDI32 or DirectDraw.

Of course, these frame rates are too low for anyone who wants to play Quake with software rendering. It's more practical to run WinQuake at a lower resolution (even 640x480 should do); this is more so for putting it in perspective with the upcoming tests using 3D acceleration.

GLQuake

Our first real insight into how choosing the right version of Windows matters lies in the results above. Windows 95 OSR2 has the constraint of no support for SSE optimizations, which many nVidia drivers make use of. Due to this, either version of Windows 95 can take a serious punch in the frame rate with SSE-optimized drivers, but the Voodoo3 is unaffected because its official driver does not use SSE.

Given SSE-capable CPUs didn't start showing up on the market until 1999, you'd think Windows NT 4.0 might not be able to use them, either, but an update to Service Pack 5 or later adds it in. AGP support is also not available in Windows 95 by default, but can be added via the USB supplement, and the same applies to Windows NT 4.0 when Service Pack 3 or later is installed. As for Windows 95 RTM, it is impossible to install AGP support there, so AGP cards are forced to function in PCI mode.

For the Pentium III system, most versions land around 90 frames per second, but Windows 95 RTM also forces vertical sync with the OpenGL driver installed, so it's stuck at whatever the monitor's refresh rate is set to, being 60 FPS in this case. As for Windows XP, it's down to 86 FPS, evidencing that this operating system simply is not appropriate for old systems. Expect to see this pattern further down.

For Windows versions with all the necessary hardware optimizations available, the Dell Dimension 8100 reaches around 224-226 FPS, while Windows 95 OSR2 is stuck at 142. If you really want to run Windows 95 on a Pentium III or faster, go with a 3dfx card.

Quake II

On the Pentium 4, Quake II is hardly any different in its frame rate for all SSE-capable versions of Windows, just a tad few frames under GLQuake. On the Pentium III build with the Voodoo3, most versions hit around 80 FPS, but of course Windows 95 RTM is stuck with the same as usual given its forced vertical sync.

You could argue that Windows XP's slower 73 FPS timedemo has to do with there never being an official Voodoo driver made to run reliably under Windows XP; it's exhibited various issues, but seriously... further down at the bottom, more tests will show Windows XP truly is bloated.

Quake III Arena

Most people benchmarking Quake III Arena will probably play back DEMO0001, but I haven't found any such thing in my Steam copy, so I use FOUR instead.

The results between the two versions of Windows 95 might be of interest. Even with Windows 95 RTM's lack of AGP support, it's still pretty much the same as that of Windows 95 OSR2 with the USB supplement. I'm not exactly certain of the cause of this, but you might guess that vertical sync is not so much of a concern anymore, as all of their results are somewhere under 55 FPS.

Another thing I really want to bring up is the much faster benchmark in Windows NT 4.0 compared to everything else. I'm almost certain this has a lot more to do with the late version of the driver I used, as finding specific versions of nVidia drivers for Windows NT is a pain in the ass. It doesn't help that the official website points me to nonfunctional download links for their old drivers. Typical nVidia!

Unreal

This one is a real bugger to get working silky smooth. If you're using a 3dfx card, you can have Unreal use its native Glide API, and you'll easily be all set to go... but if you can't use that, you may have a lot to wrestle through. I couldn't get OpenGL to consistently work on my GeForce2 Ultra for these tests, even though I could do it all fine previously in a smaller benchmark. It's a shame, really, because Unreal can be very fast with the OpenGL renderer if it does work.

For the Pentium 4, I had no choice but to use the Direct3D render, which was absolutely miserable. It actually ran slower than the Pentium III in all versions, sinking to a measly 13 FPS in Windows 95. Due to Windows NT's limitations in DirectX, I could not test that version at all. By contrast, the Pentium III's Voodoo3 makes Unreal a joy to run, consistently providing a reliable frame rate under under 60 FPS.

Unreal Tournament

Unreal Tournament doesn't have the same issue as the original Unreal when it comes to using OpenGL, so I was able to use that on the Pentium 4. Still, the differences between the Pentium III and 4 systems were hardly remarkable when running UTBench. In truth, I don't think I know how to correctly benchmark this game; I know there's a little field called "Minimum Desired Framerate" that I could've used.

These results just make it seem as if the jump from roughly 1999 standards to those of 2000 just aren't charismatic, as the general improvement of the Pentium 4 is only about 5 frames above the Pentium III's ~35 FPS. I suppose there's a lot of truth to it, as Pentium 4 CPUs are horribly inefficent compared to equally clocked Pentium III ones...

Serious Sam: The First Encounter Demo

The results from Serious Sam are much more like something I would expect. Being a game from 2001, it won't run as fast as the earlier titles on a Coppermine Pentium III (averaging 40 FPS on my build), but a Pentium 4 will handle it just fine with a good graphics card, something like 85 FPS.

Yes, I played the demo version here; never really made any time to buy and play the full version, but this was pretty fun for all I've done to test it out.

3DMark Results

In the case of 3DMark and other benchmarking software, these programs tend to be very picky about what they're installed to, which is why I was only able to test four Windows versions for each version of 3DMark I ran. As with the game benchmarks, these are run at 1024x768 and use Direct3D.

3DMark 99 MAX

3DMark 99 MAX seems to force vertical sync no matter what options I set, which is why the 3DMarks are all roughly around 4700 in both releases of Windows 95 and 98. Even so, 3DMark 99 MAX counts another score it calls "CPUMarks", which, as the name implies, is more focused on measuring raw CPU performance. Windows 98 decimates Windows 95 in this field, as even the first edition has SSE support.

The 3DMarks on the Pentium 4 machine are hardly that much higher than those of the Pentium III, averaging around 5500 to 5800. Still, the CPUMarks are much higher, peaking at 19,005 CPUMarks in Windows 98 First Edition.

3DMark 2000

3DMark 2000 insists on Windows 98SE at minimum. This version of Windows completes the benchmark faster than any other on both systems, topping out at 3,174 and 6,875 3DMarks, respectively. Windows 2000 is strangely much slower than Windows ME and XP on the Pentium 4 machine, regardless of the service pack installed.

Workstation Software

But Windows NT is not a gaming operating system, is it? It can be, but it's generally intended for more professional usage. With that in mind, there has to be a reason to justify it over Windows 9x...

We're not using symmetric multiprocessing, and most any hardware we have installed will work just fine under Windows 9x, save for the 1GB of RAM in the Pentium 4 build. Windows NT gives you a more reliable kernel, a much improved filesystem, and a considerably more secure environemnt, but is it faster than Windows 9x?

3ds MAX R2.5

DOLPHINS.MAX is a sample animation that comes with 3ds MAX R2.5. To put my computers under a different kind of stress, I will render the first frame of this animation at 640x480 on each version of Windows.

Windows NT and 2000 pull far ahead in the benchmarks compared to anything Windows 9x. Their pure 32-bit kernels seem to be much better designed to running 32-bit software efficiently. Even Windows XP happens to be faster than Windows 9x thanks to its roots in the Windows NT architecture, but it is still slower than NT4 and 2000, reinforcing my point from earlier.

Photoshop 5.5

Here, a glass filter is applied to a large image, with the distortion level set to 14 and the smoothness level set to 11. All you need to know is that once again, the Windows NT family is much faster at applying this filter than Windows 9x. A difference of 5 seconds may not seem like much, but it can really add up over extended usage. Always value shaving a second off a routine if said routine is to be done 1,000 times!

Premiere 5.0

I created a spontaneous video that doesn't really have much meaning other than to have some original raw material to work with so I can see how fast Premiere encodes a video to Intel's Indeo codec. Same pattern from before - Windows NT is faster, Windows 9x is slower, Windows XP is bogged down... need I say more?

So what do I go with?

Originally, I suggested Windows 98 Second Edition due to its very broad compatibility, but over the course of 2019 I've started to heavily favor Windows 95 because the Internet Explorer-based shell in newer versions of Windows was growing to be highly insufferable. I can't stand to waste five seconds hopping from one directory to another over and over again, which is why I've lately recommended Windows 95 even for Pentium II systems.

Of course, if you have a Pentium III or similar CPU which has SSE support on hand, running Windows 95 is a surefire way to waste potential in some of your programs. Fortunately, a solution has been devised for this - a little something called Redtoast serves as an alternative to both the stock Windows 98SE installation and the much outdated 98lite by replacing the native shell with that of Windows 95's. It's very underdeveloped so far, but is already a delight to work with.

Of course, if workstation software is what you're after, I would prefer to use Windows NT 4.0 if at all possible. It takes significantly more effort to get everything working right under there, but once you have everything in place, it's a lot of fun, and will do a much better job of handling intensive workloads coming from professional creative programs.

Wasn't this uploaded in 2019???

The Arowana benchmark video was finished in early 2019 and posted on the latter half of that year, but the bulk of this project's work was done in July 2018 and edited on an on/off basis.

Footnotes

¹ Modifications were applied to SYSTEM.INI in Windows 9x to limit the memory available to the system and VCACHE, as Rudolph Loew's PATCHMEM utility was not freely available at the time of benchmarking.

² Sometimes, a Sound Blaster 128 PCI was used because the Santa Cruz card is not compatible with Windows 95. I doubt these make a sigificant impact on overall system performance.