Latest Update: 01.07.2019
# PLAYER : RATING POINTS GAMES (%)1 Lc0 42648 GTX1050ti : 3516.4 90.5 160 56.6%2 Cfish 190619 MP : 3516.4 323.5 557 58.1%3 Stockfish 10 MP : 3509.4 326.5 573 57.0%4 Houdini 6.03 MP : 3431.3 244.5 573 42.7%5 Komodo 13.01 MP : 3420.0 232.0 571 40.6%GAMES
Some Notes about LCZero (Leela)Really impressive...just in a very short period of time,Lc0 v0.21.2 ID 42648 is one of the Top chess programs!Many thanks to Leela testers and congrats to Lc0 team!What is behind of this MYSTERY? and why Leela is so much strong?As you may know, Network (learned) files play one of the BIG roles!Many Leela testers produce millions of games, where Lc0 benefits...It seems, a hard work has been spending over the new LCZero project!Plus, I should admit: GPU's speed is another very very important factor!
Btw, so far unfortunately I could not find any right speed formulaAbout testing in fair conditions: the CPU chess engines vs Lc0And for me it's still not much clear about equalizing GPU vs CPU There are some useful available data, but I could not see any proof...!A few comments about the well-known 'Leela Ratio formula',LeR = 875 * leela nps/sf9 npsThe formula looks like not so bad and just to be in standards!)I am still following it...e.g my played conditions are adapted especially to Leela Ratio formula: 0.98 means quite close to 1.0Also as another alternative (not sure how much is true for chess)I check VRAY Benchmark Results: GPU vs CPU
But however,I think this issue is a little bit complicated...the reasons:First, we can not measure AlphaZero nps vs Stockfish npsDue to all engines produce different nps, it will be more right:If we measure two identical engines on different hardwares...2nd, I assume DeepMind played SF with Hyper-Threading ONWhere mostly of us (including me) are testing with HT OFFWhen HT is enabled, MP nps values are approx. 20-25% higher..That means (in my case):.LeR = approx. 700* leela nps/sf9 nps
And according to my experience,HT ON's higher nps do not mean better performance in chess!3rd, there are some right ways to determine the hardware speeds:Running many games by same or close in Elo engines via Auto232 modeOr via seral cable etc... or testing the chess engines in Analyzing modeFor example, I mean about the measuring method: Komodo BenchmarksMeanwhile, in the past: I ran many Hardware Speed Tests (mainly Auto232)In other words, the above system ways are not so good idea...regarding Lc0Because Leela is a neural network engine and here is another MYSTERY!)
Actually it will be great, If we will see a chess bench tool about measuring the speed: GPU vs CPUAnd in this way, the match conditions will be fair...no hardware advantages...Otherwise (as nowadays) the played results can lead to misunderstandings?!And I can give you several examples... one of them, just imagine, Schumacher is on slow vehicle, where his opponent on much faster...
One thing more,A few months ago, I purchased a new ASUS Dual GeForce RTX 2060But this graphic card did not work properly on my machine (2x E5-2686)The problem was not clear exactly... in my PSU or Motherboard etcEven the local PC service stated: the problem can be in RTX 2060 tooLater I realized that it's not necessary to spend more time and money...And I went for a cheaper card: GTX 1050ti, which worked flawlessly!
It's true also that (with LCZero engine),I can't produce many rapid games, no way of several Lc0 multiple matchesFor example, with CPU engines: I can run separately 8 GUIs in same time...Plus, I've decided to test CFish engine, the Champion of CS II 3500 Elo+And to be honest, I expected slightly better performance by CFish 190619But anyhow, we should satisfy with that...and performed just like Lc0
Thanks for reading...
Best Wishes,Sedat Canbaz