3DMark Vantage

3DMark Vantage is the latest version of the popular benchmark from Futuremark. With a Windows Vista requierment as well as several brand news tests it has all the chance to become as popular as its predecessors.

INTRODUCTION TO 3DMARK VANTAGE

Few benchmarks have had such an impact amongst gamers as the 3DMark benchmarks from Futuremark. Whether it is to verify that you are getting the expected performance out of your system or to brag about how fast your system is, 3DMark has always been the benchmark to use.

As the last version came out 2 years ago many gamers have been anxiously waiting for a new version that more accurately can test today’s cards and today it is finally here: 3DMark Vantage.

3Dmark Vantage is the latest in a long line of video card benchmarks from Futuremark. Just as its predecessors its main goal is to test your video card in a variety of tests created to push the video card as much as possible. In addition to the video card tests it also has some pretty nifty CPU tests in it.

WHAT DO YOU NEED TO RUN 3DMARK VANTAGE?

Earlier versions of 3DMark had one default setting which yielded an “official” score. With 3Dmark Vantage Futuremark has chosen to instead give us 4 different presets which all will yield an official score. Let’s first look at the general requirements of all presets:

Processor
Requirement: SSE2 support
Recommendation: A dual-core processor with performance equivalent to Intel Core 2 Duo E6600, AMD Athlon X2 6000+, or higher

Graphics Card
Requirement: Fully DirectX10-compliant graphics hardware

System Memory
Requirement: Windows Vista minimum requirement
Recommendation: 2 GB or more

Hard Disk
Requirement: 1 GB of free hard disk space

Operating System
Requirement: Windows Vista with Service Pack 1

As you can see this benchmark, just as PCMark Vantage requires Windows Vista. In the review guide, from where these requirements are taken from, it does say Service Pack 1 as a requirement for Vista but in the actual benchmark it just says Windows Vista.

In additions to the above requirements each preset also has some special needs:

• Entry – 128MB DX10 Video Card and a monitor capable of displaying 1280×1024 resolution.
• Performance – 256MB DX10 Video Card and a monitor capable of displaying 1280×1024 resolution.
• High – 512MB DX10 Video Card and a monitor capable of displaying 1680×1050 resolution.
• Extreme – 512MB DX10 Video Card and a monitor capable of displaying 1920×1200 resolution.

I’m sure not everyone will be able to run the benchmark with the High or Extreme presets but it is nice to have some “official” presets that push the cards even further.
 

DIFFERENT VERSIONS

As with PCMark Vantage, Futuremark has decided to no longer offer a free version that can be run over and over again. Futuremark themselves say this is because of the increases amount of work needed to create the benchmark, especially for Windows Vista. While earlier versions allowed you to run the default benchmark for free, the free trial edition of 3DMark Vantage only allows you to run the benchmark once before you have to pay for it.

There are 4 editions of the benchmark:

• Trial Edition – Allows you to run it once for evaluation purposes
• Basic Edition ($6.95) – This edition allows you to benchmark your system as many times you want with the Performance-preset. You will need a network connection to view the results.
• Advanced Edition ($19.95) – This version includes all 4 presets as well as 6 extra feature tests. You also can change the settings. A network connection still is required to view the results.
• Professional Edition ($495) – This edition isn’t really aimed at the consumer. It’s for commercial users and in addition to the features of the advanced edition also has support for benchmark automation through command line scripting, includes technical support as well as allows you to view results without having to be online.

You can easily upgrade to either version by entering a registration code.

THE TESTS – GPU

3DMark Vantage introduces two brand new GPU-tests. Forget the tests in earlier versions, these two new GPU-tests are not connected to the older ones at all.

Graphics Test 1: Jane Nash

The first test is called Jane Nash. It follows a girl, presumably Jane Nash, as she breaks into the bad guys lair and steals a boat (which oddly enough is sponsored by Sapphire).
The Jane Nash test scene represents a large indoor game scene with complex character rigs, physical GPU simulations, multiple dynamic lights, and complex surface lighting models. It uses several hierarchical rendering steps, including for water reflection and refraction, and physics simulation collision map rendering.

The following features are specific to this scene:

• Lots of static objects
• Lots of complex dynamic skinned objects
• Cascaded shadow maps using PCF filtering
• Very few instanced objects
• No ray-marching (volumetric) effects
• Cloth simulation
• Anisotropic materials (math-heavy)
• Caustics
• Hierarchical rendering passes to render water reflection and refraction

While this test looks pretty crappy with the Entry-preset it quickly looks pretty impressive as the features are turned on with the more advanced presets. It also is a test that quickly brings even the fastest cards to its knee.

Graphics Test 2: New Calico

Of course we cannot have a 3DMark without a space scene. The “New Calico” test features lots of flying objects as well as some nice huge explosions on the surface of a planet.

The following features are specific to this scene:

• Almost entirely consists of moving objects
• No skinned objects
• Variance shadow mapping shadows
• Lots of instanced objects
• Local and global ray-tracing effects (Parallax Occlusion Mapping, True Impostors and volumetric fog)

 

THE CPU TESTS

The CPU-tests in 3DMark06 were pretty boring. As they only were running at 1-3 fps it wasn’t even fun to watch. In 3DMark Vantage Futuremark has spiced the CPU-tests up a bit.

CPU Test 1: AI

The first CPU-test tests the CPU by making it helping a bunch of planes find their way around several airplane race courses.

The AI test features a high-intensity workload of co-operative maneuvering and path-finding artificial intelligence calculations. The test setting is an airplane race course crowded with planes, all attempting to navigate through a series of gates while avoiding collisions with each other and the ground. The test load consists of the movement planning for each airplane. The workload is entirely parallelized, and can utilize multi-core CPUs to the fullest. Faster CPUs will be able to compute more frequent and timely movement plans for the airplanes, resulting in smarter flight routes.

CPU Test 2: Physics

This test is a bit more interesting. It uses the CPU or an installed AGEIA PhysX physics accelerator to calculate the physics as a bunch of planes fly through a obstacle course which is designed to generate lots of crashes.

The Physics Test features a heavy workload of future generation game physics computations. The scene is set at an air race, but with an unfortunately dangerous configuration of gates. Planes trailing smoke collide with various cloth and soft-body obstacles, each other, and the ground. The smoke spreads, and reacts to the planes passing through it.

If you have a GEIA PhysX physics accelerator you can use that in this test but you can also turn it off and chose to use the CPU for the calculations. Both choices will still generate an approved score. This test does not test physics run on a GPU as that is done in the two GPU tests.

FEATURE TESTS

In addition to the 4 tests that together create a 3Dmark-score, 3Dmark Vantage also has a few more feature tests that each are designed to isolate a specific set of graphics hardware functionality, and exercises it to the limit.

Feature Test 1: Texture Fill
Yup, we’ve seen this in every 3DMark program since the beginning. Does anyone actually look at this anymore?

Feature Test 2: Color Fill
Another test that has been around for a while. The render target is in 16-bit floating-point format, currently the most relevant format for HDR rendering output.

Feature Test 3: Parallax Occlusion Mapping (Complex Pixel Shader)

This test draws frames by rendering a single rectangle (two triangles) on screen, seen from an animated camera position. The pixel shader uses the Parallax Occlusion Mapping technique to simulate complex geometry under the surface of the rectangle. Heavy ray-tracing operations against a huge depth-map determine the actual intersection of the view ray with the geometry. Further ray-tracing determines visibility of that point from multiple animated light sources. Finally, the surface is shaded using the relatively complex Strauss shading model. This test represents a very complex, heavy pixel shader, containing massive amounts of texture reads (ray-tracing) and dynamic flow-control (ray-tracing, looping over multiple lights), as well as traditional lighting calculations (Strauss). All the geometry on screen is rendered on just two triangles, and simulated entirely in the pixel shader.

Feature Test 4: GPU Cloth

This test features physical simulation of cloth on the GPU. The simulation is performed as a vertex simulation using a combination of vertex shader and geometry shader stages, with several simulation passes needed for each simulation step. Stream out is used to cycle the cloth vertices from one simulation pass to the next. This test stresses the vertex shading, geometry shading and stream out features of the hardware.

Feature Test 5: GPU Particles

This test features physically simulated particle effects on the GPU. The simulation is performed as a vertex simulation, with each vertex representing a single particle. Stream out is used to cycle the particle vertices from one simulation pass to the next. There are hundreds of thousands of particles in the test, all individually simulated, and colliding with a height map. The particles are rendered by expanding each vertex to a full rectangle in the geometry shader. The test stresses the vertex shading, stream out.

Feature Test 6: Perlin Noise (Math-heavy Pixel Shader)

This test features multiple octaves of Perlin noise evaluated in the pixel shader. Each color channel has its own noise function for added computational load. The Perlin noise function is a standard building block of procedural texturing approaches, and is very math-intensive to compute in a pixel-shader. This feature test emphasizes the arithmetic computing power of the graphics hardware.
 

CALCULATING THE SCORES

The final 3DMark score is calculated according to specific formulas taking in account each subscore. The GPU Score is formed by taking a weighted arithmetic mean of the raw scores of the two graphics tests. The CPU score is formed by taking a weighted arithmetic mean of the raw scores of the two CPU tests.

While the subscores are calculated the same for all the presets, the 3DMark scores are calculated differently. The idea is that each preset represents a certain performance category. For example – the Entry preset simulates the graphics load at the entry-level.

So how are the scores calculated? Let us first look at the Graphics Tests:

The Graphics score is calculated with this formula where C is two scaling constants used to bring down the score in line with traditional 3Dmark scores at launch. F are the frames per second for each Graphics test. The Graphics score is calculated with the same weight for each preset and thus you can compare the scores regardless of chosen preset or custom settings.

The CPU score is calculated in the same way:

C are scaling constants and O are the operations per second for each CPU-test. The CPU score is calculated with the same weight for each preset and thus you can compare the scores regardless of chosen preset or custom settings.

The 3DMark score however, while using the same formula for all presets, has different values for the weights while, as we mentioned above, the scaling constants remain the same for all presets.

From the values it is obvious that the CPU score affects the Entry and Performance preset more than it does the High and Extreme presets.
 

FIRST SCORES

As we only got hold of the Gold version of this benchmark a few days ago we’ve only had time to run it on a limited number of systems and video cards. The systems used were:

• Intel QX9650 @ 3.81 GHz
• ASUS P5E3 Deluxe (bios 1019)
• 4 GB of Corsair DDR3 12800@1600 MHz
• 2xSeagate 7200.11 1 TB Barracuda and 1x Seagate 7200.10 750 GB
• XFX® GeForce® 9800GX2 running ForceWare 175.12 64-bit
• PC Power and Cooling 1200Watt PSU
• Dell 2707 FPW monitor
• Windows Vista Ultimate 64-bit version with SP1

and

• Intel Q6600 @ 3 GHz and E2180 @ 2 GHz 
• ASUS Maximus Formula X38 motherboard 
• 2 GB of Corsair DDR2 @800 MHz
• ASUS 9600GT, ASUS 8800GT Sparkle 8800GTS, Sparkle 9800GTX running ForceWare 175.12
• HIS HD3850, HIS HD3870, HIS HD3870X2 running Hotfix driver 8.4
• Tuniq 1200W PSU
• Dell 24″ monitor
• Windows Vista Ultimate 32-bit version with SP1

Everything isn’t working

Both NVIDIA and AMD provided us with new drivers for the launch of 3DMark Vantage. While one might question the fact that they are releasing new drivers just for a benchmark it became pretty obvious that, at least in AMD’s case, they still have some work to do. While the NVIDIa drivers worked well on all the tested NVIDia cards, the AMD hotfix had some serious issues when running on a Crossfire-system, including a HD3870X2. The first Graphics Test had some serious flashing textures and the problem got increasingly worse as we moved up in the presets. The framerate did not seem to benefit though as you could not see any jumps in framerate when the flashing textures occured. It still seemed to render everything in the background. The Feature Test: Cloth, also had some serious issues on the Crossfire systems. Instead of a moving flag we get a melting flag which melts away. The single-GPU systems from AMD did not have any issues.

The scores

The Crossfire-systems (2xHD3870X2 and HD3870X2)
had issues with corrupted textures in the first Graphics test

With the “Entry” preset it is quite obvious how important the CPu is for the final score. When run on a E2180 Dual Core CPU @ 2GHz the 9800GTX gets a final score of only half compared to when it is  running on a Q6600 @ 3 Ghz. 

The Crossfire-systems (2xHD3870X2 and HD3870X2) had issues with corrupted textures in the
first Graphics test and thus the results should be taken with a pinch of salt.

The “Performance” preset still is affected by the CPU speed.

The Crossfire-systems (2xHD3870X2 and HD3870X2) had issues with corrupted textures in the
first Graphics test and thus the results should be taken with a pinch of salt.

At the “High” preset the CPu is basically unimportant. Instead the GPU is being pushed hard. if we look at the actual framerates for each graphics test we see that most cards score around 5-15 fps. There’s a lot of room to improve.

The Crossfire-systems (2xHD3870X2 and HD3870X2) had issues with corrupted textures in the
first Graphics test and thus the results should be taken with a pinch of salt.

Last but not least the “Extreme” preset  really pushes the cards to their limit.  It probably will take a few years before we see cards hitting 15000 or more  here.  The CPU is basically unimportant in this test.

We invite you all to post your own scores in our 3DMark Vantage launch thread.

The Rest

I will not bore you with the scores for each Feature test but the NVIDIA cards consistently performed better in Feature test 4-6 than the AMD cards. Especially the Parallax Occlusion Mapping test performed better on the NVIDIa cards than on the AMD cards. For example, the GeForce 9600GT scored 6 fps on that test while the HD3870 managed a meager 1.4 fps.
 

CONCLUSION

It feels a bit odd to write an article at the launch of a benchmark but that just shows how important 3DMark has become for the enthusiasts and gamers. While I have no doubt that this will continue also with 3DMark Vantage I cannot shake the feeling that I expected a bit more out of this new version. Just two GPU- and CPU-tests is a bit meager, especially as none of the tests exactly are blowing us away, at least not the way for example the “Matrix” inspired test in one of the earlier versions did. The fact that they also have started to charge for the benchmark might be hard to swallow. True, it is a premier program to use when boasting about the capabilities of your system but how much is the average user prepared to pay for that privilege?

At the same time it feels like Futuremark has made some wise decisions too. Adding several presets is great as it means that you now can test cards at a higher load and still get official scores. It has been a bit annoying when using 3Dmark06 with the newer high-end cards as the default setting simply hasn’t been complex enough to tax them. By adding the High and Extreme presets Futuremark has ensured that it will keep the high-end card just as busy as the low- and mid-end cards.
It is of course impossible to speculate if the testing methods are good enough to give us an accurate idea of the real-world performance of a card but just as with earlier versions at least Futuremark is trying to create a balanced set of tests that pushes the video cards in different ways. To me this is just as valid as taking a single game and trying to use the results in that game to extrapolate the performance in other games.

So there you have it. 3DMark Vantage might have its flaws but in the end I have no doubt it still will be as used as earlier versions when it comes to vide card performance testing.

Why not test it to see if it is anything to pay money for? Download the trial edition from us here. If you decide to keep it you can easily upgrade it to any of the editions by just buying a license key from Futuremark.