Futuremark PCMark Vantage

PCMark Vantage not only continues to offer tests that test the whole system, it also has been updated to test more varied situations than before. Whether you want to find out how well your system can handle high definition video or how fast it can catalogue your music collection, PCMark Vantage has tests for that.

INTRODUCTION

Even since the dawn of time mankind has been chasing more power and performance. I’m pretty sure that right after pre-historic man figured out how to make spear-heads out of flint, someone started to look for a way to make it bigger and more pointed. And when the wheel was invented and put on a cart, the next day someone started to add spoilers to the cart and pimp it to look cooler and go faster. Most of us know that Charles Babbage designed “the Analytical Engine” in 1837. What most do not know is that he then started to plan the water cooling and overclocking of his design the day after. Let’s face it, regardless how fast our computers are, how many cores our CPU have and whatever monster video card we buy, we are never satisfied and want more speed.

This is where the need for good benchmarking software comes in. Futuremark is a company that is almost synonymous with benchmarking. While they are known primarily for their video card benchmarks, 3DMark, they also have a wide variety of other benchmark programs like PCMark, MobileMark and SysMark.

Today Futuremark is releasing their latest PCMark benchmark, PCMark Vantage. This article will go through the various tests as well as add some scores that we’ve got from running the test on a couple of different systems. You can then (or already now) go and download the benchmark from us and try it once before deciding which (if any) version you want to buy.

INTRODUCTION TO PCMARK

While 3DMark is aimed at testing the video card exclusively, PCMark is a benchmark that has a broader goal of testing the whole system. The latest version before PCMark Vantage was PCMark 05, which as the name implies, came out several years ago.

PCMark®05 is everything you need to reliably and easily measure the performance of your PC and determine its strengths and weaknesses. With PCMark05, you will be able to select the optimal upgrades for your existing PC, or choose the right new PC that fits your specific needs. This easy-to-use product gives you the same tools and knowledge that virtually every professional tester in the industry uses.

This new version not only continues to offer tests that test the whole system, it also has been updated to test more varied situations than before. Whether you want to find out how well your system can handle high definition video or how fast it can catalogue your music collection, PCMark Vantage has tests for that.

WHAT DO YOU NEED TO RUN PCMARK VANTAGE?

With PCMark Vantage Futuremark is taking a big step by requiring Vista to run it. We already know that the next 3Dmark (3DMark NEXT) will do the same so it should not be a big surprise.

The requirements

PCMark Vantage package requires approx. 800 MB of hard disk space and running the tests requires additional 800 MB space.

System Requirements

• Processor
  • Modern processor with SSE2 support
  • Intel Pentium 4 at 3.0GHz or equivalent AMD processor
  • Multi-core processors recommended
• Graphics Card
  • Shader Model 2.0 compliant graphics card.
  • Shader Model 3.0 compliant graphics card for the GPU image manipulation test
  • 256MB of graphics memory for the GPU gaming test recommended
• Hard Disk
  • 1.7GB of hard disk space
  • NTFS File System
• System Memory
  • 1024MB
• Operating System
  • Microsoft® Windows Vista® Ultimate / Home Premium / Home Basic / Enterprise / Business
• Other
  • Windows Media Player 11
  • Latest DirectX Update for Windows Vista
  • Microsoft Internet Explorer 7

One other thing you need is time. Running the whole benchmark takes approximately 1 ½ hours.

PCMARK SUITE

The first suite of tests that we are going to look at is the PCMark Suite. This suite is a collection of various single- and multi-threaded CPU, Graphics and HDD test sets with the focus on Windows Vista application tests.

PCMark Suite includes the following tests:

Suite Tests (more about these later in the article)
• Memories 1 & 2
• TV and Movies 1 & 2
• Gaming 1 & 2
• Music 1 & 2
• Communications 1 & 2
• Productivity 1 & 2

CPU Test
• Data encryption, Data decryption, Data compression, Data decompression
• CPU image manipulation (compression/decompression/resize)
• Audio transcoding
• Video transcoding
• Text editing
• Web page rendering
• Windows Mail
• Windows Contacts
• CPU game test

Graphics Test
• Video playback
• GPU game test

HDD Test
• 6 HDD Tests total
• HDD Tests use new 4K block size*
• All HDD Tests use RankDisk using Intel’s IPEAK Storage Performance Toolkit
– WinTrace is used to record the HDD traces
– AnalyzeTrace is used to analyze precisely what the traces are doing
– RankDisk: a Futuremark modified version that it is non-destructive and runs in a temporary sandbox file.

Some results

System 1: [email protected] GHz, 3 GB 800 MHz DDR2, Vista Business, GeForce 7300, Seagate 160 GB EIDE
System 2: [email protected] GHz, 2 GB 800 MHz DDR2, Vista Ultimate, AMD X1950XT, Maxtor 80 GB SATA
System 3: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 4: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 5: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA

My system at work (system 1) failed this suite because it failed in the CPU Gaming test. I’m not sure why as it cleared the Gaming suite later.

This suite tests the systems in a wide variety of ways and as expected the Q6600 manages to put out the best score. I was surpirsed that the E6850 got basically the same score regardless if it was running at 3.0 GHz (defualt) or underclocked to 2.4 GHz.

MEMORIES SUITE

My first reaction to this suite was that it had everything to do with the memory in your computer. However – that is not what this suite is about. Instead it deals with the “memories” you store in digital form; images and video. This is how Futuremark descrbes the suite:

Our memories are often kept in digital form. Here, large digital photos in HD Photo format are stretched, flipped and rotated using the CPU. Plenty of system memory is highly beneficial for manipulating large images. Importing digital photos to Windows Photo Gallery is where a high performance HDD shines. More and more image manipulation is being done using the GPU, enabling instantaneous color correction, sharpening and softening of images. Home video editing with Windows Movie Maker can be very time-consuming – unless you have a high performance HDD. Home videos recorded on digital video cameras are sometimes transcoded and transferred to a portable media player. High definition videos are often archived in media servers. It may, however, be handy to have them transcoded and transferred to a portable media player. A fast CPU with many cores can handle transcoding swiftly

This suite thus tests the CPU, the GPU (if the GPU is SM3.0 capable) and the HDD.

Memories Suite includes the following tests:

Memories 1 (also included in PCMark Suite)
• Two simultaneous threads
• CPU image manipulation
• HDD picture import

Memories 2 (also included in PCMark Suite)
• Two simultaneous threads
• GPU image manipulation
• HDD video editing

Memories 3
• Video Transcoding: DV to portable device

Memories 4
• Video Transcoding: media server archive to portable device

Some results

System 1: [email protected] GHz, 3 GB 800 MHz DDR2, Vista Business, GeForce 7300, Seagate 160 GB EIDE
System 2: [email protected] GHz, 2 GB 800 MHz DDR2, Vista Ultimate, AMD X1950XT, Maxtor 80 GB SATA
System 3: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 4: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 5: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA

Having a Quad-core CPU definitely helps in this test. It can manipulate those images and transcode those videos far quicker than the Dual-core CPU running at the same frequency. A higher frequency helps the Dual-core but can’t make up for the fewer cores.

TV AND MOVIES SUITE

One growing application for PC’s is using it for watching and recording TV as well as HD-material. Futuremark has acknowledged this and included a suite that will test your system thoroughly in this regard.

High definition TV broadcasts and movies have arrived. Playing an HD DVD with additional HD content, a Blu-ray movie, or watching HDTV smoothly (while making a backup of an HD DVD by transcoding to a media server or transcoding from a media server archive to a portable media player) requires lots of computing and graphical power. Windows Media Center with a high performance HDD can handle simultaneous video recording, time-shifting, and streaming to an Extender for Windows Media Center, such as Xbox 360™.

TV and Movies Suite includes the following tests:

TV and Movies 1 (also included in PCMark Suite)
• Two simultaneous threads
• Video transcoding: HD DVD to media server archive
• Video playback: HD DVD w/ additional lower bitrate HD content from HDD, as downloaded from net

TV and Movies 2 (also included in PCMark Suite)
• Two simultaneous threads
• Video transcoding: HD DVD to media server archive
• Video playback, HD MPEG-2: 19.39 Mbps terrestrial HDTV playback

TV and Movies 3
• HDD Media Center

TV and Movies 4
• Video transcoding: media server archive to portable device
• Video playback, HD MPEG-2: 48 Mbps Blu-ray playback

Some results:

System 1: [email protected] GHz, 3 GB 800 MHz DDR2, Vista Business, GeForce 7300, Seagate 160 GB EIDE
System 2: [email protected] GHz, 2 GB 800 MHz DDR2, Vista Ultimate, AMD X1950XT, Maxtor 80 GB SATA
System 3: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 4: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 5: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA

The frequency of the CPU does not seem to have much effect on the final performance of this suite. Instead it is the extra cores that seem to make the most difference. I’m sure the HD2900XT, and even the X1950XT, help enough with the video playback so that the CPU’s do not run at full load anyway. The exception is the slow GeForce 7300 in my work system which cannot really assist the CPU. I wonder how much a HD2400 would help here?

GAMING SUITE

Since Futuremark already have a dedicated gaming-benchmark it is not a surprise that a few gaming tests have been included in PCMark Vantage. However, instead of concentrating on the 3D-performance, the gaming tests in PCMark Vantage instead look at other aspects in gaming performance: how good the system is streaming data from a HDD, how good the CPU is running several simultaneous threads as well as data decrompressions.

Gaming is one of the most popular forms of entertainment for all ages. Today’s games demand high performance graphics cards and CPUs to avoid delays and sluggish performance while playing. Loading screens in games are yesterday’s news. Streaming data from an HDD in games – such as Alan Wake™ – allows for massive worlds and riveting non-stop action. CPUs with many cores give a performance advantage to gamers in real-time strategy and massively multiplayer games.

Gaming Suite includes the following tests:

Gaming 1 (also included in PCMark Suite)
• GPU game test

Gaming 2 (also included in PCMark Suite)
• HDD: game HDD

Gaming 3
• Two simultaneous threads
• CPU game test
• Data decompression: level loading

Gaming 4
• Three simultaneous threads
• GPU game test
• CPU game test
• HDD: game HDD

Some results:

System 1: [email protected] GHz, 3 GB 800 MHz DDR2, Vista Business, GeForce 7300, Seagate 160 GB EIDE
System 2: [email protected] GHz, 2 GB 800 MHz DDR2, Vista Ultimate, AMD X1950XT, Maxtor 80 GB SATA
System 3: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 4: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 5: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA

As mentioned above, this suite does not necessarily test the actual video card so much. Instead it concentrates on other stuff like how fast your HDD can stream data or how well the CPU can decompress data.

The difference in this test between the dual-core and the quad-core CPU systems are not that big. In fact it is smaller than I thought. The difference between system 2 (E6400) and system 4(E6850 at 2.4 GHz) is far larger than the difference in frequency (13% difference in core frequency but a 26% difference in result) indicating that the faster HDD and the faster video card also has a big effect.

MUSIC SUITE

Just as users now use their computer to store and watch videos and images, they are doing the same with music. Futuremark again tries to mimic how a regular user will do music related tasks on the computer. This includes transcoding of music between various formats as well as cataloguing music on the HDD.

Online music shops have changed the way we purchase music, letting us buy exactly the tracks we want, right from home. Cataloguing your music library is a breeze for fast and powerful HDDs. The most common audio file formats decrease your music’s audio quality which is undesirable. Luckily, lossless audio file formats are becoming more popular. Transcoding from non-compressed audio to a lossless format is heavily taxing on the CPU. Transcoding your audio files from one format to another is much quicker and easier using high performance CPUs.

Music Suite includes the following tests:

Music 1 (also included in PCMark Suite)
• Three simultaneous threads
• Web page rendering – w/ music shop content
• Audio transcoding: WAV -> WMA lossless
• HDD: Adding music to Windows Media Player

Music 2 (also included in PCMark Suite)
• Audio transcoding: WAV -> WMA lossless

Music 3
• Audio transcoding: MP3 -> WMA

Music 4
• Two simultaneous threads
• Audio transcoding: WMA -> WMA
• HDD: Adding music to Windows Media Player

Some results:

System 1: [email protected] GHz, 3 GB 800 MHz DDR2, Vista Business, GeForce 7300, Seagate 160 GB EIDE
System 2: [email protected] GHz, 2 GB 800 MHz DDR2, Vista Ultimate, AMD X1950XT, Maxtor 80 GB SATA
System 3: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 4: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 5: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA

This is a suite that mainly is affected by the CPU frequency and the number of cores. I’m not sure though why the E6850 manages to perform better when I underclock it compared to when it is clocked at the default frequency. I probably have to run a bunch of follow-up tests to see if there was something wrong when those numbers where collected but since each run takes 1.5 hours to do it will have to wait until the article has been released.

COMMUNICATIONS SUITE

The communications suite does not test your broadband, instead it tests how well your system can handle displaying webpages, encrypting data and run Windows Defender.

Security is more important today than ever before. To compress and encrypt all personal information is vital for safe computing. Emails are the most important type of communication, whether it is personal or business. To keep the workflow smooth and enjoyable, high performance CPUs and HDDs are recommended. Reading news online while having your cup of coffee is quality-time. Often one site isn’t enough, so tabbed browsing is a perfect solution for news-hungry people. Spyware is very common on systems without protection against it, letting Windows Defender scan & protect your system is recommended. Voice over IP – with Skype™ or Windows Live Messenger – is very popular these days. Encrypted messaging for home and workplace gives additional security.

Communications Suite includes the following tests:

Communications 1 (also included in PCMark Suite)
• Three simultaneous threads
• Data encryption: CNG AES CBC
• Data compression
• Web page rendering: graphics content, 1024×768, windowed

Communications 2 (also included in PCMark Suite)
• Three simultaneous threads
• Web page rendering: open various news pages from IE 7 Favorites in separate tabs, close them one by one
• Data decryption: CNG AES CBC
• HDD: Windows Defender

Communications 3
• Windows Mail: Search

Communications 4
• Two simultaneous threads
• Data encryption: CNG AES CBC
• Audio transcoding: WMA -> WMA – to simulate VOIP

Some results:

System 1: [email protected] GHz, 3 GB 800 MHz DDR2, Vista Business, GeForce 7300, Seagate 160 GB EIDE
System 2: [email protected] GHz, 2 GB 800 MHz DDR2, Vista Ultimate, AMD X1950XT, Maxtor 80 GB SATA
System 3: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 4: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 5: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA

The added number of cores does not really seem to have a large affect in this suite.  The faster HDD helps in the Windows Defender and the Windows Mail:Search test.

PRODUCTIVITY SUITE

I guess we cannot be playing all the time (or why not?) and thus need to test how good our systems are when we try to be productive. This suite tests everything from how fast Vista starts to how well the system works when working on several things at the same time.

Starting various applications can take a long time – unless you have a high performance HDD. Editing text with WordPad is a breeze when done with fast CPUs and graphics cards. Often one site isn’t enough, so tabbed browsing is a perfect solution for highly productive people. Spyware is very common on systems without protection against it, letting Windows Defender scan & protect your system is recommended. Starting Windows Vista is a rather demanding task for the storage device, but a fast HDD will notably decrease the loading time. Our busy lives find us hard at work, balancing multiple tasks; with little time for breaks. It’s the same for our computers. Multiple tasks, running simultaneously, put your system under a lot of stress. Having a modern, up-to-date CPU, HDD, graphics card and board full of system memory increases your computer’s productivity and reduces your stress.

Productivity Suite includes the following tests:

Productivity 1 (also included in PCMark Suite)
• Two simultaneous threads
• Text editing
• HDD: application loading

Productivity 2 (also included in PCMark Suite)
• Two simultaneous threads
• Windows Contacts: search
• HDD: Windows Defender

Productivity 3
• HDD: Windows Vista start-up

Productivity 4
• Three simultaneous threads
• Windows Contacts: search
• Windows Mail: Run Message Rules
• Web page rendering: simultaneously open various pages from IE7 Favorites in separate tabs, close them one by one

Some results:

System 1: [email protected] GHz, 3 GB 800 MHz DDR2, Vista Business, GeForce 7300, Seagate 160 GB EIDE
System 2: [email protected] GHz, 2 GB 800 MHz DDR2, Vista Ultimate, AMD X1950XT, Maxtor 80 GB SATA
System 3: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 4: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 5: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA

We see the same pattern in this suite as in the previous communications suite. I’m not sure exactly what is the most important component here. The difference between the two groups of results are DDR2vsDDR3, slower EIDE/SATA disk vs fast SATA disk and a bit faster CPU frequencies. In this case you need to go down a lot deeper into the detailed results to find a pattern.

HDD SUITE

The last suite is the HDD Suite which tests your hard drive. If you have looked at the other suites you notice that this suite takes tests from them and combine them into a separate HDD test suite.

Spyware is very common on systems without protection against it, letting Windows Defender scan & protect your system is recommended. Streaming data from an HDD in games – such as Alan Wake™ – allows for massive worlds and riveting non-stop action. Importing digital photos to Windows Photo Gallery is where a high performance HDD shines. Starting Windows Vista is a rather demanding task for the storage device, but a fast HDD will notably decrease the loading time. Home video editing with Windows Movie Maker can be very time-consuming – unless you have a high performance HDD. Windows Media Center with a high performance HDD can handle simultaneous video recording, time-shifting, and streaming to an Extender for Windows Media Center, such as Xbox 360™. Cataloguing your music library is a breeze for fast and powerful HDDs. Starting various applications can take a long time – unless you have a high performance HDD.

HDD Suite includes the following tests:

HDD 1
• HDD: Windows Defender

HDD 2
• HDD: game HDD

HDD 3
• HDD: importing pictures

HDD 4
• HDD: Windows Vista start-up

HDD 5
• HDD: video editing

HDD 6
• HDD: Media Center

HDD 7 
•  HDD: Adding music to Windows Media Player

HDD 8
• HDD: application loading

Some results:

System 1: [email protected] GHz, 3 GB 800 MHz DDR2, Vista Business, GeForce 7300, Seagate 160 GB EIDE
System 2: [email protected] GHz, 2 GB 800 MHz DDR2, Vista Ultimate, AMD X1950XT, Maxtor 80 GB SATA
System 3: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 4: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA
System 5: [email protected] GHz, 2 GB 1066MHz DDR3, Vista Home Premium, AMD HD2900XT, WD 36 GB Raptor SATA

As expected the 10k RPM Raptor disk performs almost exactly the same on system 3-5 regardless of the speed and cores of the CPU. Apparently my computer at work (system 1) has a better drive than the one I have in my main system. Hmm, maybe time to update it.

SUBMITTING RESULTS

Just as with 3DMark, PCMark Vantage uses Futuremarks ORB (Online Results Browser) to share the results with other users. ORB lets you compare your results in many ways and find systems that resembles your system as closely as possible making it possible to spot scores that are lower than expected.

FINAL THOUGHTS

There are two types of benchmark programs that test your whole system. The first type tests each component separately and usually does this by using methods that are not necessarily indicative of real-life performance. Examples of this type of programs are SiSoft Sandra and Everest. These programs are very useful for quickly testing a separate component to make sure that it works and that its theoretical performance is correct. You often also can use these separate tests to stress-test the component.

PCMark Vantage is an example of the second type of benchmarks, the one that tries to use tests that are as close to real-life as possible. Other examples are Worldbench and Sysmark. This type of benchmark is useful to see how the components in the system work together. The downside is of course that you cannot see how good a specific component is unless you compare two similar systems and just vary one of the components.

I am quite impressed by PCMark Vantage. Futuremark has worked hard to find all sorts of relevant tests and overall I think they succeeded. I must admit I would never have come up with the idea of testing how good a computer is of cataloguing music or images but that is a real useful test that could tell you that the reason your computer is slow is not because the CPU is crappy but because that old hard drive is being the bottleneck. Overall Futuremark has managed to put together test suites that make sense.

The main problem with PCMark Vantage, and indeed in my opinion with all Futuremarks products, is that it is hard to dig deeper behind the generated scores. While you can choose to see the detailed scores and also upload them to the ORB at Futuremark’s webpage, I still feel they are not easily accessed and understood. I would like to be able to save my results in a local database and generate reports that plot various sub-scores from various test runs. My dream would be a way to take the scores for various systems and then be able to “ask” questions like “what difference did the different hard drive to too the score” or “what exactly did the CPU-swap help?”. Maybe in the next version?

The benchmark is available in several editions:

PCMark Vantage Basic
• $6.95 (download only – limited free trial edition also available)
• PCMark Test Suite
• Network connection required to view results
• Licensed for non-commercial and Personal Use only

PCMark Vantage Advanced
• $19.95 (download)
• $29.95 (CD-ROM)
• All Test Suites enabled
• Advanced online services
• Licensed for non-commercial and Personal Use only

PCMark Vantage Professional
• $495.00/seat (download version)
• $495.00/seat (CD-ROM)
• All Advanced edition features
• Full benchmark settings
• Technical support
• View results without network connection
• Benchmark automation with command line options
• Export results to Microsoft Excel
• Licensed for Full Commercial Use

I personally would go for the Advanced edition as this includes all the different suites and not just the PCMark suite. This edition seems also to let you view the results without having to be online, something I think is almost necessary when testing a system. Also note that the Basic edition is not free and that the trial version that you can download from our site allows you to run it once before you have to buy a licence key.

PCMark Vantage still is an excellent tool not only for us hardware geeks but also for regular users that want to make sure that their system is working well for the different tasks.

You can go and download the benchmark from us and try it once before deciding which (if any) version you want to buy.