Dear Chess Friends,
First of all,
I've spent nearly 15 years (since 2002) over computer chess benchmarks!
In that period of time,
I've created many benchmark rankings, based on hundreds of hardware users...
And without your support, all these benchmarks wouldn't exist..thanks again!
To be honest (even after a such experience),
So far, I could not find any perfect way of acurate benchmarking!
But anyhow,
I've decided to run several hardware benchmarks more...
To see, to compare...which way of measuring is best...?!
Btw, why especially Komodo is used...why not any other chess engine ?
Because I found that Komodo is one of best in comparing various machines
Note: I've tested many other engines too, before starting over the new project
Just one example,
Stockfish 030717 32-bit did not work on E5-2686 machine...really strange!
Plus, the latest SF 64-bit suffers in 'Automatic Analysis' under Arena 2.01
I mean, Stockfish oftenly skipping many of the positions (without analysing)
Even under Arena 3.5.1, Stockfish crashes (sometimes, not very often, but...)
Plus, why the old Arena 2.01 GUI is used in Automatic Analysis (not v3.5.1) ?
Simğly because Komodo crashes under Arena 3.5.1, but not under Arena 2.01...
And here are my used 3 benchmark methods (by Komodo 9.02)
Conditions:
Arena 2.01 GUI, Hyper-Threading OFF, 1024 MB hash, Syzygy OFF
The bench results of 1st and 2nd methods are analyzed on start position
Where the 3rd method's results are based on latest Komodo Benchmarks
1st Method - kN/s Benchmarks:
kN/s Processors GHz Threads EXE
14520 2x E5-2686 v3 2.00 32 w32
13224 2x E5-2686 v3 2.00 12 x64
8205 i7-980X 3.33 6 x64
Here we see that the kN/s values of 2x E5-2686 32 cores w32 are higher...
And higher kN/s values do not mean better performance in Computerchess
To be sure, please check 2x E5-2686 32c w32's results (in the 3rd method)
----------------------------------------------------------
2nd Method - Time to Depth:
Depth 28 Processors GHz Threads EXE
05:57 i7-980X 3.33 6 x64
07:39 2x E5-2686 v3 2.00 12 x64
12:42 2x E5-2686 v3 2.00 32 w32
On the 2nd method (time to depth),
i7-980X performed much better... !
But in chess performance, can we say that is it true... ?!
For example, in solving positions:
2x E5-2686 v3 12 cores x64 seems to be better...
*Confirmed by other engines too (in case of solving positions)
Plus, time to depth measuring is not so useful... because oftenly
The engines perform different time values (on same depth and PC)
---------------------------------------------------------
3rd Method - Position Solving (100 pos./30 sec per move):
Points Processors GHz Threads EXE
49 2x E5-2686 v3 2.00 12 x64
48 i7-980X 3.33 6 x64
40 2x E5-2686 v3 2.00 32 w32
The 3rd method seems to be not a perfect bench way too...because,
Sometimes on high-end CPUs there can be 1-5 points difference,
But on slow machines, usually the peformance is almost same!
Btw (as far as I know)
There is no any bench tool to produce 100% exactly same values !)
But however (in case of comparing all these benchmark methods),
I strongly believe that the 3rd method (position solving) is best !!
Anyone interested,
2x E5-2686's overall bench results are available on this link: Results
In short (for best hardware chess speed measuring),
We should concentrate over performance (position solving)
Instead of kN/s or time to depth measuring values...!
And who knows perhaps (with the current available data),
Some of the chess friends will change their minds and all these
Methods will be useful in case of comparing which way is better...
Hopes helps...
Best,
Sedat