Latest Update: 22.04.2023
Hello Chess Friends,
As usual, I'm very pleased to announce that,Here are another new series of GUI testings...
1st of all,I realized to run many GUI testings...you know,More better...and now we have more new data,Sure I mean if comparing various Chess GUIs...
And here are the latest played new test resultsNote: 5 different testings, via Bullet + Blitz
1st Test: 30s+0.5s, Balsa, 64MB hashTo be more clear same conditions for all GUIs: Banksia + Cutechess's concurrent option: 64 Engines 64-bit, AVX2 is used where available...And they are played with the default EvalfilesEach engine's move overhead arranged:100Exc. Houdini and Komodo are arranged:200All GUIs used/played from diff.engine copiesAnd Balsa's each opening is played as twice
By the way, Fritz, Shredder GUI out, e.g no way to use 30s+0.5sAnd sorry that I did not test all other rest Chess GUIs,Due to no free time for all... thanks for understanding
Arena 3.5.1 GUI: 24 Elo Difference 1 Stockfish 15.1 +93/-25/=882 53.40% 534.0/10002 Raubfisch X48c3 SC +25/-93/=882 46.60% 466.0/1000
Note:This test was complicated: many Arena GUIs played in same time And via current Arena GUI test, Stockfish is lost 1 game on time!--------------------------------------------------------Banksia 0.58 GUI: 26 Elo Difference 1 Stockfish 15.1 +104/-28/=868 53.80% 538.0/10002 Raubfisch X48c3 SC +28/-104/=868 46.20% 462.0/1000
Note: No any problem etc. is recorded, it seems all right!---------------------------------------------------------Cutechess EXPER-3.5.0: 25 Elo Difference 1 Stockfish 15.1 +94/-22/=884 53.60% 536.0/10002 Raubfisch X48c3 SC +22/-94/=884 46.40% 464.0/1000
Note: In this GUI test, Stockfish lost 5 games on time!!It seems, Stockfish 15.1 suffers under Cutechess EXPER..--------------------------------------------------------Cutechess Qt v5.1.11 GUI: 26 Elo Difference 1 Stockfish 15.1 +106/-30/=864 53.80% 538.0/10002 Raubfisch X48c3 SC +30/-106/=864 46.20% 462.0/1000
Note: No any problem is recorded, it seems all stable!--------------------------------------------------------Cutechess 1.3.0. beta 4: 30 Elo Difference 1 Stockfish 15.1 +105/-18/=877 54.35% 543.5/10002 Raubfisch X48c3 SC +18/-105/=877 45.65% 456.5/1000
Note: No any serious issue etc. is recorded, exception Under current test, the Elo difference is highest...-------------------------------------------------------------------------------------------------------------------------------------2nd Test:Another new GUI testings and close conditions as above..E.g just for comparing, the same engines tested: ResultsAnd as we see (on faster CPUs) less Elo differences..e.gIn older GUI testings, the difference was close to 38 Elo
Note also that (about the below Bullet GUI testings):Here (CF vs SF12) seems to be stable, no games lost on time!Plus, GUIs Elo differences are quite close to each other...great!
Banksia 0.58 GUI: 22 Elo Difference 1 Cfish 031120 +101/-38/=861 53.15% 531.5/10002 Stockfish 12 +38/-101/=861 46.85% 468.5/1000------------------------------------------------------Cutechess EXPER 3.5.0 GUI: 20 Elo Difference 1 Cfish 031120 +96/-39/=865 52.85% 528.5/10002 Stockfish 12 +39/-96/=865 47.15% 471.5/1000------------------------------------------------------Cutechess Qt v5.1.11: 24 Elo Difference 1 Cfish 031120 +98/-30/=872 53.40% 534.0/10002 Stockfish 12 +30/-98/=872 46.60% 466.0/1000-----------------------------------------------------Cutechess 1.3.0. beta 4: 22 Elo Difference 1 Cfish 031120 +102/-39/=859 53.15% 531.5/10002 Stockfish 12 +39/-102/=859 46.85% 468.5/1000-------------------------------------------------------------------------------------------------------------------------------------3rd Test: Bullet 30s+0.5s, Balsa, 1 Core, 64 MB Hash
Banksia 058 GUI: 19 Elo Difference 1 Houdini 6.03 +218/-163/=619 52.75% 527.5/10002 Komodo 13.3 +163/-218/=619 47.25% 472.5/1000--------------------------------------------------------Cutechess Qt v5.1.11: 16 Elo Difference 1 Houdini 6.03 +222/-175/=603 52.35% 523.5/10002 Komodo 13.3 +175/-222/=603 47.65% 476.5/1000-------------------------------------------------------------------------------------------------------------------------------------4rd Test: Blitz 2m+1s, Balsa, 1 Core, 128 MB Hash
Banksia 0.58 GUI: 17 Elo Difference 1 Houdini 6.03 +104/-79/=317 52.50% 262.5/5002 Komodo 13.3 +79/-104/=317 47.50% 237.5/500
Cutechess Qt v5.1.11: 15 Elo Difference 1 Houdini 6.03 +86/-65/=349 52.10% 260.5/5002 Komodo 13.3 +65/-86/=349 47.90% 239.5/500-------------------------------------------------------------------------------------------------------------------------------------5th Test: Blitz 2m+1s, Balsa, 1 Core, 128 MB Hash
Banksia 0.58: 20 Elo Difference
1 Stockfish 15.1 +72/-14/=914 52.90% 529.0/10002 Raubfisch X48c3 SC +14/-72/=914 47.10% 471.0/1000-------------------------------------------------------Cutechess Qt v5.1.11 GUI: 18 Elo Difference 1 Stockfish 15.1 +63/-12/=925 52.55% 525.5/10002 Raubfisch X48c3 SC +12/-63/=925 47.45% 474.5/1000-------------------------------------------------------Fritz 13 GUI: 18 Elo Difference 1 Stockfish 15.1 +28/-2/=470 52.60% 263.0/5002 Raubfisch X48c3 SC +2/-28/=470 47.40% 237.0/500
Note: Another complicated test: many Fritz GUIs played in same time.. And in this test, Engines are used to play via Perfect 2023 ctg Note also, here the played openings are not repeated as twice-------------------------------------------------------------------------------------------------------------------------------------
GAMES
As Final Words,All these GUI testings can be counted as not so bad data!E.g without these kind of testings..we would not know thatStockfish 15.1 will suffer or not under Cutechess EXPERFor anyone missed, you can check the 1st Test's results...E.g Stockfish lost 5 games on time / 1 game under ArenaBut I admit also that no any other game is lost on time..!!
Plus we would not know that,Almost all current GUIs Elo differences are close to each other!
By the way,Another proof (via Balsa, Perfect book): Elo Error margin is less!E.g about 1000 games (per player) enough data (for determining)Even 500 games (not repeated as twice) can be enough, see last testBut I can't say same words, if using critical weak opening lines!For more info (related over opening testings): Results / Results2
Ok...that's all for now... and many thanks for your interest )
Best,Sedat
Hello Chess Friends,
As usual, I'm very pleased to announce that,Here are another new series of GUI testings...
1st of all,I realized to run many GUI testings...you know,More better...and now we have more new data,Sure I mean if comparing various Chess GUIs...
And here are the latest played new test resultsNote: 5 different testings, via Bullet + Blitz
1st Test: 30s+0.5s, Balsa, 64MB hashTo be more clear same conditions for all GUIs: Banksia + Cutechess's concurrent option: 64 Engines 64-bit, AVX2 is used where available...And they are played with the default EvalfilesEach engine's move overhead arranged:100Exc. Houdini and Komodo are arranged:200All GUIs used/played from diff.engine copiesAnd Balsa's each opening is played as twice
By the way, Fritz, Shredder GUI out, e.g no way to use 30s+0.5sAnd sorry that I did not test all other rest Chess GUIs,Due to no free time for all... thanks for understanding
Arena 3.5.1 GUI: 24 Elo Difference 1 Stockfish 15.1 +93/-25/=882 53.40% 534.0/10002 Raubfisch X48c3 SC +25/-93/=882 46.60% 466.0/1000
Note:This test was complicated: many Arena GUIs played in same time And via current Arena GUI test, Stockfish is lost 1 game on time!--------------------------------------------------------Banksia 0.58 GUI: 26 Elo Difference 1 Stockfish 15.1 +104/-28/=868 53.80% 538.0/10002 Raubfisch X48c3 SC +28/-104/=868 46.20% 462.0/1000
Note: No any problem etc. is recorded, it seems all right!---------------------------------------------------------Cutechess EXPER-3.5.0: 25 Elo Difference 1 Stockfish 15.1 +94/-22/=884 53.60% 536.0/10002 Raubfisch X48c3 SC +22/-94/=884 46.40% 464.0/1000
Note: In this GUI test, Stockfish lost 5 games on time!!It seems, Stockfish 15.1 suffers under Cutechess EXPER..--------------------------------------------------------Cutechess Qt v5.1.11 GUI: 26 Elo Difference 1 Stockfish 15.1 +106/-30/=864 53.80% 538.0/10002 Raubfisch X48c3 SC +30/-106/=864 46.20% 462.0/1000
Note: No any problem is recorded, it seems all stable!--------------------------------------------------------Cutechess 1.3.0. beta 4: 30 Elo Difference 1 Stockfish 15.1 +105/-18/=877 54.35% 543.5/10002 Raubfisch X48c3 SC +18/-105/=877 45.65% 456.5/1000
Note: No any serious issue etc. is recorded, exception Under current test, the Elo difference is highest...-------------------------------------------------------------------------------------------------------------------------------------2nd Test:Another new GUI testings and close conditions as above..E.g just for comparing, the same engines tested: ResultsAnd as we see (on faster CPUs) less Elo differences..e.gIn older GUI testings, the difference was close to 38 Elo
Note also that (about the below Bullet GUI testings):Here (CF vs SF12) seems to be stable, no games lost on time!Plus, GUIs Elo differences are quite close to each other...great!
Banksia 0.58 GUI: 22 Elo Difference 1 Cfish 031120 +101/-38/=861 53.15% 531.5/10002 Stockfish 12 +38/-101/=861 46.85% 468.5/1000------------------------------------------------------Cutechess EXPER 3.5.0 GUI: 20 Elo Difference 1 Cfish 031120 +96/-39/=865 52.85% 528.5/10002 Stockfish 12 +39/-96/=865 47.15% 471.5/1000------------------------------------------------------Cutechess Qt v5.1.11: 24 Elo Difference 1 Cfish 031120 +98/-30/=872 53.40% 534.0/10002 Stockfish 12 +30/-98/=872 46.60% 466.0/1000-----------------------------------------------------Cutechess 1.3.0. beta 4: 22 Elo Difference 1 Cfish 031120 +102/-39/=859 53.15% 531.5/10002 Stockfish 12 +39/-102/=859 46.85% 468.5/1000-------------------------------------------------------------------------------------------------------------------------------------3rd Test: Bullet 30s+0.5s, Balsa, 1 Core, 64 MB Hash
Banksia 058 GUI: 19 Elo Difference 1 Houdini 6.03 +218/-163/=619 52.75% 527.5/10002 Komodo 13.3 +163/-218/=619 47.25% 472.5/1000--------------------------------------------------------Cutechess Qt v5.1.11: 16 Elo Difference 1 Houdini 6.03 +222/-175/=603 52.35% 523.5/10002 Komodo 13.3 +175/-222/=603 47.65% 476.5/1000-------------------------------------------------------------------------------------------------------------------------------------4rd Test: Blitz 2m+1s, Balsa, 1 Core, 128 MB Hash
Banksia 0.58 GUI: 17 Elo Difference 1 Houdini 6.03 +104/-79/=317 52.50% 262.5/5002 Komodo 13.3 +79/-104/=317 47.50% 237.5/500
Cutechess Qt v5.1.11: 15 Elo Difference 1 Houdini 6.03 +86/-65/=349 52.10% 260.5/5002 Komodo 13.3 +65/-86/=349 47.90% 239.5/500-------------------------------------------------------------------------------------------------------------------------------------5th Test: Blitz 2m+1s, Balsa, 1 Core, 128 MB Hash
Banksia 0.58: 20 Elo Difference
1 Stockfish 15.1 +72/-14/=914 52.90% 529.0/10002 Raubfisch X48c3 SC +14/-72/=914 47.10% 471.0/1000-------------------------------------------------------Cutechess Qt v5.1.11 GUI: 18 Elo Difference 1 Stockfish 15.1 +63/-12/=925 52.55% 525.5/10002 Raubfisch X48c3 SC +12/-63/=925 47.45% 474.5/1000-------------------------------------------------------Fritz 13 GUI: 18 Elo Difference 1 Stockfish 15.1 +28/-2/=470 52.60% 263.0/5002 Raubfisch X48c3 SC +2/-28/=470 47.40% 237.0/500
Note: Another complicated test: many Fritz GUIs played in same time.. And in this test, Engines are used to play via Perfect 2023 ctg Note also, here the played openings are not repeated as twice-------------------------------------------------------------------------------------------------------------------------------------
GAMES
As Final Words,All these GUI testings can be counted as not so bad data!E.g without these kind of testings..we would not know thatStockfish 15.1 will suffer or not under Cutechess EXPERFor anyone missed, you can check the 1st Test's results...E.g Stockfish lost 5 games on time / 1 game under ArenaBut I admit also that no any other game is lost on time..!!
Plus we would not know that,Almost all current GUIs Elo differences are close to each other!
By the way,Another proof (via Balsa, Perfect book): Elo Error margin is less!E.g about 1000 games (per player) enough data (for determining)Even 500 games (not repeated as twice) can be enough, see last testBut I can't say same words, if using critical weak opening lines!For more info (related over opening testings): Results / Results2
Ok...that's all for now... and many thanks for your interest )
Best,Sedat