15 Most Recent [RSS]
More...
|
How (not) to run hardware tests?
Ruth just pointed me at a performance assessment of the new Intel iMac which confused me a little. As I've mentioned in this place a couple times, I'm a software person. As such, I have a basic understanding of hardware, and I also took a systems programming class as an elective, and from my Dad I learned which way to hold a soldering iron, but that's pretty much the extent of my hardware knowledge. So, I'm asking my two readers: Is this really a representative test? And, more importantly, how would one set up a more representative hardware test?
The first thing that jumped out at me is that he's comparing a dual G5 (obviously a PowerMac) against the new Intel iMac. Another oddity is that the iMac has 1/8th of the RAM of the G5. And there's no info on hard disk speeds. In addition, the GCC tests are "compiling for the appropriate platform". So, essentially, he is...
- ... comparing a high-end computer from a year ago (at most) against a just-released low-end computer.
- ... comparing two computers with wildly differing RAM amounts, so the low-end machine is also constantly swapping, and we all know what a speed hit that can be.
- ... comparing compilation for two completely different architectures, RISC and CISC. So, the iMac runs through an entirely different compiler pipeline.
- ... has one entry where results "varied wildly" and he took the best of 5.
That can't be representative of anything, can it? And I'm not even beginning to try and make up excuses like "the transcoding exercises probably mean that QuickTime isn't optimised yet." Still, surprisingly the iMac came out ahead in general, which is either proof of Steve Jobs's claims how good the Core Duo is, or how badly the G5 has aged. Or maybe, we're just proving Moore's Law?
Update: Amount of instructions isn't a good benchmark, so I've re-worded the GCC part a tad.
 S�ren Kuklau writes:
"And I'm not even beginning to try and make up excuses like "the transcoding exercises probably mean that QuickTime isn't optimised yet.""
There is some merit to this, actually. Transcoding is one of the areas where AltiVec really helped out greatly and made a huge difference, and Intel's SSE3 simply can't compete. So, take the "yet" away from the sentence, and you vaguely have some truth in there.
Other than that, you are of course perfectly right. The two systems can't be reasonably compared at all.
|
 Aaron Ballman writes:
Yeah, it's apples and oranges (pun only slightly intended).
I've always been led to believe that SSE3 was on-par with AltiVec and that it was SSE2 that was lightyears behind the times. Have I been hearing wrong?
|
 Sam Pullara writes:
... As I stated in the comment to your comment the iMac was not swapping during the tests. If it were you would get extremely poor performance and would definitely not beat a dual 2.5 ghz G5.
... It is straight forward comparison between two systems. If you don't care about performing these tasks on these 2 systems its not interesting to you.
... The comparison is between tasks being done on the two machines. A user doesn't care if a task takes longer on one because of RISC vs. CISC or whatever. Just that one does the equivalent task faster.
I still get the feeling that you don't understand the results at all and don't see that the "low-end iMac" trounces the "high-end PowerMac" in almost every test. There are only a few PowerMacs that are faster than this one. 1) the new dual core 2.5 ghz G5 (only slightly), 2) the old dual proc 2.7 ghz G5, and 3) the 2.5 ghz Quad G5.
So in summary, I am not doing a detailed comparison of the hardware. I'm testing two development systems and comparing how well they perform some fairly typical developer tasks. If this isn't the kind of thing you would have me test, what would be?
|
 Jon H writes:
You seem to be leaping to something's defense when no offense was offered. You're writing as if the comparison were done to "debunk" Apple's claims about the Intel Macs' performance.
What I see is someone doing an informal benchmark of their machines, out of personal curiosity, to get a general idea of how the two machines compare for the tasks he does. That's pretty natural for a geek to do. No points being scored, no aspersions being cast. No grand, sweeping conclusions being offered.
|
 Roland Fines writes:
I agree with Jon H.
The guy did a bunch of test with the computers he have, the tests are very interesting, and I'd rather have that than nothing. His conclusion is also perfectly correct, this machine could replace his G5 for development.
Although 512Mg would not support development very well. As a Java dev, only Netbeans runs on the Intel iMac (through Rosetta, Eclipse doesn't launch), and as soon as you start opening many java files, switching from an app to another is Pain. I just upgraded it to 1Gig, and it performs muuuch better now.
|
| |
![[ Zathras.de - Uli's Web Site ]](http://www.zathras.de/angelweb/themes/barsnstripes/img/masthead_blog.jpg)
