Schachcomputer.info Community
  #1  
Alt 03.12.2023, 11:51
Benutzerbild von Ray
Ray Ray ist offline
MACH IV
 
Registriert seit: 07.12.2019
Ort: United Kingdom
Beiträge: 159
Abgegebene Danke: 49
Erhielt 311 Danke für 99 Beiträge
Aktivitäten Langlebigkeit
5/20 5/20
Heute Beiträge
1/3 ssssss159
Chessnut Evo - My Thoughts

After more testing of the Chessnut Evo here are my findings and opinions:

I still like the machine as a tool for learning and practising chess. As stated in my previous post the integration with Chess.com for example is very convenient. I like that the board positions are automatically recognised from the web site and I can try them out over a real board and my moves register in Chess.com. Of course, from such web sites I can play their chess bots over the real board and this works well.

I must admit to having not been familiar enough with the Maia project to know that it makes its moves instantly on purpose. There is no form of node searching such as Monte Carlo. I understand this is attempting to simulate humans playing by intuition. However, on testing Maia on the Evo I discovered that Maia 1600 (supposedly 1600 ELO) actually plays like a complete beginner blundering pieces all over the place! I then tried Maia 1900 against mobile shredder set to 1798 ELO(I was trying for 1800 but the slider control in Shredder does not give precise control) and the Maia was crushed blundering a piece to a pawn fork! I include the game below.

However, this is not a fault of the Evo’s implementation. I downloaded Lc0 on my PC and used the same weight files as the Evo. I had to set the Lc0’s(v0.24) UCI setting ‘Slowmover’ to a value of 0 disabling any node search and therefore it moves instantly as per the Evo. This played in exactly the same way, no surprises. I pitted Lc0 Maia 1900 against Evo Maia 1900 for two games and the result was a game each, but certainly not at the level of play of a human rated 1900 ELO! I have e-mailed Chessnut about this and must say they are excellent at responding and have stated they are going to improve the Maia weight files with their own training.

Here is the game between Evo Maia 1900 v Shredder Mobile set to 1798 ELO:
[Event "Evo Maia 1900 v Shredder Mobile 1798 ELO"]
[Site "Shredder Android"]
[Round "1"]
[White "Evo Maia 1900"]
[Black "Shredder Mobile 1798"]
[Result "0-1"]

1.e4 e6 2.d4 d5 3.e5 Ne7 4.f4 c5 5.c3 Nf5 6.Nf3 Nc6 7.Bd3 g6
8.O-O cxd4 9.cxd4 Nfxd4 10.Nc3 Nxf3+ 11.Qxf3 Be7 12.Be3?? d4 13.Ne4
dxe3 14.Qxe3 Qb6 15.Qxb6 axb6 16.a3 g5 17.fxg5 Nxe5 18.Bb5+ Bd7
19.Bxd7+ Nxd7 20.Rac1 Nc5 21.Nf6+ Bxf6 22.gxf6 Rg8 23.b4 Nd7
24.Rc7 Rg6 25.Rxb7 O-O-O 26.Rc1+ Kxb7 27.a4 Ne5 28.a5 Rd2 29.g3
Nf3+ 30.Kf1 Nxh2+ 31.Ke1 Nf3+ 32.Kf1 Rh6 33.axb6 Rh1# 0-1


White’s move 12.Be3?? is shocking. Losing either a Night or Bishop! I seriously doubt a player rated 1900 would make this mistake.

So at the moment playing against Maia may be fun but is probably not going to be challenging enough for many. I think that in many positions we need to deduct several hundred ELO to get a more accurate level of play. Don’t bother pitting it against another dedicated machine.

I am also disappointed with the SF16 implementation. All that power available for analysis but restricted to moving instantly. It really should be possible for the user to set thinking times for the engine. I have queried this with Chessnut and they intend to increase the thinking time for better analysis and have said they will consider my suggestions for adding playing levels and infinite analysis. We will have to see. That being said, SF16 moving instantly is going to give many of us a hard time! Also, as mentioned in a previous post installing DroidFish provides us with everything needed.

The Maia project is interesting but instant moves without any MCTS currently lets it down. The Evo has the Lc0 engine of course, but as this also moves instantly must also have zero search. This means that Evo’s feature of creating your own personalities by training with 200 games is limited. At best we may get for example a Magnus Carlson personality but moving instantly throughout a game and therefore I seriously doubt playing at his strength. Maybe if Magnus had zero time per move for every move in a game it may be a more accurate simulation but this is hardly a realistic form of play! I’ve not confirmed this yet by training because at the moment it’s too cumbersome to put 200 games into the Evo.

Games played on the Evo using the built in bots are not stored locally but in the cloud somewhere. This is not great and I have also queried this with Chessnut because if and when their servers are shutdown or people may not have internet for some reason, they won’t have access to their games! Chessnut are developing a web site where user’s can log in and gain access to their games. At the moment only a single game per PGN is possible although this is going to be improved. This means to transfer a PGN with 200 games for training purposes won’t work, I would have to transfer one game at a time! I will wait until the website can handle PGN files with multiple games before testing the training feature. It may be easier via one of the chess web sites, but I’m wanting to pick games from my own databases.

Am I disappointed in my purchase? No, I still like the concept and don’t treat the Evo as a normal dedicated machine. I think it has much potential but still offers a great way of learning the game. For beginners in chess the Maia engine will be fun, but for more serious play then there is SF built in (virtually no thinking time) and of course playing online through chess.com for example, and of course installing Android chess apps such as DroidFish with the option of playing against Komodo 9 or 10 or some other free engines from the App store (via APKPure).

Ray
Mit Zitat antworten
Folgende 8 Benutzer sagen Danke zu Ray für den nützlichen Beitrag:
BHGP (09.12.2023), Findel64 (03.12.2023), German Eagle (03.12.2023), kamoj (04.12.2023), Lindwurm (03.12.2023), Mapi (04.12.2023), RetroComp (03.12.2023), Tibono (03.12.2023)
  #2  
Alt 04.12.2023, 16:32
Benutzerbild von Ray
Ray Ray ist offline
MACH IV
 
Registriert seit: 07.12.2019
Ort: United Kingdom
Beiträge: 159
Abgegebene Danke: 49
Erhielt 311 Danke für 99 Beiträge
Aktivitäten Langlebigkeit
5/20 5/20
Heute Beiträge
1/3 ssssss159
Re: Chessnut Evo - My Thoughts

I'm conducting further tests to try to establish the actual ELO rating of Maia 1900 on the Evo. For this I'm using the Shredder for Android app which calculates the ELO of the opponent over several games and based on the result of each game adjusts its own ELO for the next game. In this way it targets the ELO of the opponent plus or minus a margin of error, but I believe gives a very good idea of the player's rating. So far the Maia 1900 bot has an ELO of 1465 but I think this will drop more as I get it to play more games. I'm not bothering to save these games, just playing as many as I have time for.

I have noticed that sometimes Maia appears to play reasonable moves but often it tends to be just a matter of time before it starts blundering pieces. I would think this is because it just does not have enough information for all positions in a particular game. Maybe if the neural networks were trained with a number of games a magnitude higher it may fair better, but I wish they would have implemented some node searching. Just looking 4 or 5 ply ahead would make quite a difference I think.

Regards
Ray
Mit Zitat antworten
Folgender Benutzer sagt Danke zu Ray für den nützlichen Beitrag:
BHGP (09.12.2023)
  #3  
Alt 09.12.2023, 12:27
BHGP BHGP ist offline
Revelation
 
Registriert seit: 04.02.2008
Ort: Nicht im Norden
Beiträge: 665
Abgegebene Danke: 514
Erhielt 422 Danke für 242 Beiträge
Member Photo Albums
Aktivitäten Langlebigkeit
1/20 17/20
Heute Beiträge
0/3 ssssss665
AW: Chessnut Evo - My Thoughts

Ich kann die Aussagen von Ray alle so unterstreichen. Bin mal gespannt, ob die nächsten Wochen/Monate nun an der Software gebastelt wird, nachdem die Produktion nun so langsam zur Routine werden sollte. Baustellen bzw. Verbesserungsbedarf gibt es ja durchaus in recht großer Menge, wenn ich mal so auf die anderen Threads verweisen darf.

Viele Grüße
Bernd
Mit Zitat antworten
Folgender Benutzer sagt Danke zu BHGP für den nützlichen Beitrag:
Ray (10.12.2023)
Antwort

Themen-Optionen
Ansicht

Forumregeln
Du bist nicht berechtigt, neue Themen zu erstellen.
Du bist nicht berechtigt, auf Beiträge zu antworten.
Du bist nicht berechtigt, Anhänge hochzuladen.
Du bist nicht berechtigt, deine Beiträge zu bearbeiten.

BB code ist An
Smileys sind An.
[IMG] Code ist An.
HTML-Code ist An.

Gehe zu

Ähnliche Themen
Thema Erstellt von Forum Antworten Letzter Beitrag
News: New Chessnut Evo Bryan Whitby Die ganze Welt der Schachcomputer / World of chess computers 409 18.04.2024 09:00
Frage: Neues Chessnut Brett synthese Die ganze Welt der Schachcomputer / World of chess computers 12 16.04.2024 16:43
News: Neues Kickstarter Projekt Chessnut dsommerfeld Die ganze Welt der Schachcomputer / World of chess computers 384 05.02.2024 16:59
News: New Chessnut Air + Bryan Whitby Die ganze Welt der Schachcomputer / World of chess computers 4 02.01.2024 17:53
Info: Chessnut driver (Linux) GONeill Die ganze Welt der Schachcomputer / World of chess computers 0 09.12.2022 23:19


Alle Zeitangaben in WEZ +2. Es ist jetzt 22:12 Uhr.



Powered by vBulletin (Deutsch)
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
©Schachcomputer.info