Schachcomputer.info Community

Zurück   Schachcomputer.info Community > Schachcomputer / Chess Computer: > Teststellungen und Elo Listen / Test positions and Elo lists


Antwort
 
Themen-Optionen Ansicht

  #1  
Alt 03.02.2023, 22:40
Tibono Tibono ist offline
Resurrection
 
Registriert seit: 22.05.2018
Ort: Frankreich
Alter: 63
Land:
Beiträge: 601
Abgegebene Danke: 3.723
Erhielt 1.407 Danke für 478 Beiträge
Aktivitäten Langlebigkeit
5/20 7/20
Heute Beiträge
1/3 ssssss601
AW: Khmelnitsky test: Mephisto IIIS Glasgow

Of course I cannot leave without a synthesis: all in one graph!
A larger one to better zoom on the skills profiles:



There is a striking oddity in this graph, but I think it can be explained: not only the strategic skills did not step forward with the acceleration, as previously observed, but the graph displays a regression! The good strategic guidelines embedded in the Glasgow program weight higher with short thinking time; deeper calculations steer the score away because of tactical connsiderations, that is my opinion. Maybe strategic elements are mainly computed at root level?
Another small oddity lies in the calculations, where the fast IIIS performs less then the slow one; but otherwise the calculations scores look to range accordingly to expectations. The fast IIIS must have been on a wrong track with some position, there is of course some alea related to the current score at the time the thinking process is stopped.
Opening, middlegame, tactics, threat recognition and sacrifice progress rather evenly and significantly along with speed.
As already pointed out, defense skills were stuck without any significant progress, until the large acceleration provided by the Mephisto Phoenix. Counterattack is sort of similar, with unleashed power starting with the Revelation. And the Glasgow program kept brakes on attacking skills, but that is not uncommon with our dear chess computers (a bit passive, lacking any attacking plan and awaiting tactical opportunities).

Feel welcome to comment further!
I'll be back very soon with additional data to share, but enough for today.
MfG, Eric
Miniaturansicht angehängter Grafiken
Klicke auf die Grafik für eine größere Ansicht

Name:	All_in_one.jpg
Hits:	395
Größe:	41,5 KB
ID:	5851  
Mit Zitat antworten
Folgende 4 Benutzer sagen Danke zu Tibono für den nützlichen Beitrag:
Beeco76 (04.02.2023), Chessguru (04.02.2023), Egbert (04.02.2023), Roberto (24.07.2023)
  #2  
Alt 04.02.2023, 04:31
Beeco76 Beeco76 ist offline
Mephisto Montreux
 
Registriert seit: 23.03.2020
Beiträge: 254
Abgegebene Danke: 1.313
Erhielt 409 Danke für 179 Beiträge
Aktivitäten Langlebigkeit
0/20 5/20
Heute Beiträge
0/3 ssssss254
AW: Khmelnitsky test: Mephisto IIIS Glasgow

Hi Eric,

Thank you very much for your detailed analysises.

Like your excellent analysis of the Chessmaster Diamond on your website, they provide valuable information about the strength and weaknesses of the engines.

I've a question regarding the test setting though:

I found a description in German language, how the Khmelnitsky test distinguishes between strategy, tactics, opening, endgame, etc.
(For those, who are interested:
https://www.freechess.info/ueebersic...ing-guide.html)

But where are the human player profiles taken from?

Best regards
Markus
Mit Zitat antworten
Folgende 3 Benutzer sagen Danke zu Beeco76 für den nützlichen Beitrag:
Chessguru (04.02.2023), Egbert (04.02.2023), Tibono (04.02.2023)
  #3  
Alt 04.02.2023, 05:03
Benutzerbild von Egbert
Egbert Egbert ist offline
Lebende Foren Legende
 
Registriert seit: 20.12.2009
Ort: Dreieich
Alter: 61
Land:
Beiträge: 10.221
Abgegebene Danke: 15.628
Erhielt 18.058 Danke für 6.947 Beiträge
Member Photo Albums
Aktivitäten Langlebigkeit
19/20 15/20
Heute Beiträge
2/3 ssss10221
AW: Khmelnitsky test: Mephisto IIIS Glasgow

Hello Eric,

Thank you very much for this very informative test series. Especially the improved defensive capability of the Mephisto Glasgow on the Mephisto Phoenix I could also observe in the context of my competitions! The overall weak endgame performances, even under Mephisto Phoenix also correspond to my expectations and are also present in practice. I also share your assessments, which are based on your test. Really great work Eric!

Greetings
Egbert

Geändert von Egbert (04.02.2023 um 07:50 Uhr) Grund: Ergänzung
Mit Zitat antworten
Folgende 2 Benutzer sagen Danke zu Egbert für den nützlichen Beitrag:
Chessguru (04.02.2023), Tibono (04.02.2023)
  #4  
Alt 04.02.2023, 08:58
Tibono Tibono ist offline
Resurrection
 
Registriert seit: 22.05.2018
Ort: Frankreich
Alter: 63
Land:
Beiträge: 601
Abgegebene Danke: 3.723
Erhielt 1.407 Danke für 478 Beiträge
Aktivitäten Langlebigkeit
5/20 7/20
Heute Beiträge
1/3 ssssss601
AW: Khmelnitsky test: Mephisto IIIS Glasgow

 Zitat von Beeco76 Beitrag anzeigen
But where are the human player profiles taken from?
Hi Markus, they are extracted from the data of the very same Khmelnitsky's test. The Elo values come from a data sheet in the book, providing two columns per domain (attack, counterattack,...); one column being the % of your score in the domain, the other the corresponding Elo observed per statistics across a number of players. 99% score is achieved by 2500+ players, 3% by 815 or so players. In between, the distribution vary according to each domain. Elo values for intermediate % values (they are provided per 3% step, so 3%, 6%, 9%... up to 96%, 99%) rely on interpolation.

Well, the data sheet can be red the other way around: I am a 1400 Elo player, which score (%) am I supposed to achieve per domain of chess knowledge? An interpolation can be performed between two Elo values in order to get an accurate percentage within the 3% spread.
Of course, the computer-player comparison cannot usefully rely on an Elo graph (this would just add a circle to the basic Elo-graph, plotting the player's strength). So the comparison graph is based on %, as well displaying usual, average strengths and weaknesses of the human player.
By the way, time ago I used this idea (not mentioned within the book!) to provide a graph displaying how the skills develop along to the Elo, and posted about in the Hiarcs forum. Here is the graph:



and my comments:
Code:
- skills develop rather evenly from 830 to 900 player (very same shape), with natural ability for calculations and no, to very little, knowledge about opening phase. Of course, standard endgame positions are likely to be unknown.
- 1000 player gets some clues at tactics and related stuff (attack, defense, recognizing threats)
- 1200 player has developped clues in strategy and standard endgame positions
- 1400 player is a better attacker and went on developping knowledge about standard endgame positions
- 1600 player developped tactical skills including sacrifice
- 1800 player went on with tactics and better endgame knowledge
- 2000 player is a better defender and attacker
- 2200 player shows evidence of a breakthrough with strategy, plus reinforced endgame mastery
- 2400 player is very balanced, with strategy, opening and middlegame as most enhanced areas. Beware the strong defenser and counterattacker!
Miniaturansicht angehängter Grafiken
Klicke auf die Grafik für eine größere Ansicht

Name:	KT_Players_eng.jpg
Hits:	254
Größe:	33,7 KB
ID:	5855  
Mit Zitat antworten
Folgende 2 Benutzer sagen Danke zu Tibono für den nützlichen Beitrag:
Chessguru (04.02.2023), Egbert (04.02.2023)
  #5  
Alt 04.02.2023, 09:25
Tibono Tibono ist offline
Resurrection
 
Registriert seit: 22.05.2018
Ort: Frankreich
Alter: 63
Land:
Beiträge: 601
Abgegebene Danke: 3.723
Erhielt 1.407 Danke für 478 Beiträge
Aktivitäten Langlebigkeit
5/20 7/20
Heute Beiträge
1/3 ssssss601
AW: Khmelnitsky test: Mephisto IIIS Glasgow

Now back to my mainstream, the Glasgow program.
Code:
SMhz	Ratio/7.2Mhz    Log(Ratio)     KT Elo
7.2	   1.00           0.000000       1374
12	   1.67           0.221849       1441
38	   5.28           0.722451       1541
480	  66.67           1.823909       1755
This table is quite simple: Speed in Mhz, ratio based on the 7.2 basic speed, log(base 10) of the ratio, and achieved KT Elo from the test. The base of the log function has no influence on the below graph pattern:



The curve's closeness to a straight line is obvious. This means the Elo growth follows the usual rule (a constant value gain per doubling in speed) even when reaching sky-high acceleration. I was expecting more of a drop in the curve from the Mephisto Phoenix, it is not so, well done from the Glasgow program!
I added a polynomial regression curve to get the displayed formula we will use later on.
Miniaturansicht angehängter Grafiken
Klicke auf die Grafik für eine größere Ansicht

Name:	KT_SMhz.jpg
Hits:	253
Größe:	21,8 KB
ID:	5856  

Geändert von Tibono (04.02.2023 um 09:28 Uhr) Grund: typo
Mit Zitat antworten
Folgende 3 Benutzer sagen Danke zu Tibono für den nützlichen Beitrag:
Chess Monarch (15.02.2023), Chessguru (04.02.2023), Egbert (04.02.2023)
  #6  
Alt 04.02.2023, 09:46
Tibono Tibono ist offline
Resurrection
 
Registriert seit: 22.05.2018
Ort: Frankreich
Alter: 63
Land:
Beiträge: 601
Abgegebene Danke: 3.723
Erhielt 1.407 Danke für 478 Beiträge
Aktivitäten Langlebigkeit
5/20 7/20
Heute Beiträge
1/3 ssssss601
AW: Khmelnitsky test: Mephisto IIIS Glasgow

Next step is an easy one: let's find out the Elo gain value per doubling. No need for hard calculations, the worksheet does it for us: just scale the speed according to number of doubling, and add a linear regression line - the slope factor is the value:



The gain is 61.48 Elo points per doubling (on average, according to the Khmelnitsky test).
Miniaturansicht angehängter Grafiken
Klicke auf die Grafik für eine größere Ansicht

Name:	KT_doubling.jpg
Hits:	449
Größe:	19,3 KB
ID:	5857  
Mit Zitat antworten
Folgende 2 Benutzer sagen Danke zu Tibono für den nützlichen Beitrag:
Chessguru (04.02.2023), Egbert (04.02.2023)
  #7  
Alt 04.02.2023, 10:45
Tibono Tibono ist offline
Resurrection
 
Registriert seit: 22.05.2018
Ort: Frankreich
Alter: 63
Land:
Beiträge: 601
Abgegebene Danke: 3.723
Erhielt 1.407 Danke für 478 Beiträge
Aktivitäten Langlebigkeit
5/20 7/20
Heute Beiträge
1/3 ssssss601
AW: Khmelnitsky test: Mephisto IIIS Glasgow

So far we "only" leveraged the Khmelnitsky test data, in a rather straightforward way. Let's now explore a more hazardous area: try to standardize the data, introducing the knowledge we have thanks to the Wiki-Elo-List.
As the KT results do comply with the rule of thumb (constant Elo gain per doubling), to my understanding we only need a shift in the KT score to better align with the Wiki-Elo-List (meaning: add a constant value).
Do you remember the polynomial formula displayed above? We juste need to substitute a better value to the final constant (and keep the x² and x part of the formula unchanged).
So far, the scope of the test only includes two devices for which the Wiki-Elo-List provides values: the fast MephiIIIS and the Ph.Revelation. But we have a third device with a known Elo within the list: the Ph.Revelation II. Let's use it as well, as the polynomial formula can provide an estimate for its KT Elo:

Code:
                                SMhz	   Constant-free          Wiki Elo       Delta       Standardized KT Elo
IIIS Glasgow                    7.2              0.0                                                 1701
IIIS Glasgow 12Mhz              12              53.2                1707        1653.8               1754
Phoenix Revelation              38             165.6                1900        1734.4               1866
Phoenix Revelation II           109            259.0                1973        1714.0               1960
Phoenix Revelation II AE        196            307.2                                                 2008
Phoenix Reflexion 1,5Ghz        244            324.4                                                 2025
Phoenix Reflexion 1,7Ghz        272            332.8                                                 2034
Mephisto Phoenix                480            375.3                                                 2076
                                                               average delta:   1700.7
The "constant-free" formula is -21.3015036253151*LOG10(SMhz/7.2)^2+244.626198127018*LOG10(SMhz/7.2)
(see above formula within the graph)
The delta is the Wiki-Elo minus the constant-free value; from which an average value is calculated.
The standardized KT Elo is of course the same polynomial pattern with the average delta as a new constant: -21.3015036253151*LOG10(SMhz/7.2)^2+244.626198127018*LOG10(SMhz/7.2)+1700.7

This provides this new graph with "better" Elo values (better meaning standardized according to the Wiki-Elo-List data available so far):



'x' is still a bit of a complicated value (log of the speed ratio), so let's expand the x axis to the SMhz (speed in Mhz) value, and add a logarithmic regression curve to get a less cumbersome formula:



New formula displayed: 89.7821030247115*LN(SMhz)+1531.25499276664
or, should you prefer the log function: 206.730932042356*LOG10(SMhz)+1531,25499276664

Resulting tab, using either of the above:
Code:
IIIS Glasgow                1708
IIIS Glasgow 12Mhz          1754
Phoenix Revelation          1858
Phoenix Revelation II       1952
Phoenix Revelation II AE    2005
Phoenix Reflexion 1,5Ghz    2025
Phoenix Reflexion 1,7Ghz    2035
Mephisto Phoenix            2086
Miniaturansicht angehängter Grafiken
Klicke auf die Grafik für eine größere Ansicht

Name:	Std_KT.jpg
Hits:	427
Größe:	21,0 KB
ID:	5858   Klicke auf die Grafik für eine größere Ansicht

Name:	Std_KT2.jpg
Hits:	432
Größe:	20,0 KB
ID:	5859  
Mit Zitat antworten
Folgende 5 Benutzer sagen Danke zu Tibono für den nützlichen Beitrag:
Chessguru (04.02.2023), Egbert (04.02.2023), Roberto (24.07.2023), Thomas J (04.02.2023), udo (14.02.2023)
  #8  
Alt 04.02.2023, 11:05
Benutzerbild von Egbert
Egbert Egbert ist offline
Lebende Foren Legende
 
Registriert seit: 20.12.2009
Ort: Dreieich
Alter: 61
Land:
Beiträge: 10.221
Abgegebene Danke: 15.628
Erhielt 18.058 Danke für 6.947 Beiträge
Member Photo Albums
Aktivitäten Langlebigkeit
19/20 15/20
Heute Beiträge
2/3 ssss10221
AW: Khmelnitsky test: Mephisto IIIS Glasgow

Hello Eric,

very nice, the Elo numbers should now harmonise very well with our existing lists. I am curious, should Mephisto Glasgow on Mephisto Phoenix actually get to approx. 2080 Elo in my competitions, that would be a real surprise for me. My experience has shown that the performance of Mephisto Glasgow is extremely dependent on the opponent. Very nice to see in the modest result against the Revelation II AE CXG Sphinx 40 with approx. 1983 Elo. Against the Novag Super Expert C, however, it was already around 2070 Elo and currently against Mephisto Dallas 68000, 12 MHz it looks like an even better result than 2070 Elo could come out. But of course after 3 games it is not yet possible to make a reliable forecast. I hope that all my results can contribute to calibrating your highly interesting Khmelnitsky test.

Greetings
Egbert
Mit Zitat antworten
Folgende 3 Benutzer sagen Danke zu Egbert für den nützlichen Beitrag:
Chess Monarch (15.02.2023), Chessguru (04.02.2023), Tibono (04.02.2023)
  #9  
Alt 04.02.2023, 11:09
Tibono Tibono ist offline
Resurrection
 
Registriert seit: 22.05.2018
Ort: Frankreich
Alter: 63
Land:
Beiträge: 601
Abgegebene Danke: 3.723
Erhielt 1.407 Danke für 478 Beiträge
Aktivitäten Langlebigkeit
5/20 7/20
Heute Beiträge
1/3 ssssss601
AW: Khmelnitsky test: Mephisto IIIS Glasgow

As a conclusion to the outcomes of the test, the KT figures (for those who prefer figures to graphs ):

Code:
 
                        Score  Endgame  Midgame   Opening  Calcu. Std.Endg. Strategy Tactics R.Threats Attack Ct.attack  Defense Sacrifice
IIIS Slow (waits)       1374     1328     1410     1598     1375     1057     1727     1237     1550     1310     1339     1475     1086
IIIS Fast (w/o waits)   1441     1328     1502     1909     1241     1057     1808     1374     1592     1401     1339     1446     1216
Phoenix Revelation      1541     1328     1806     2155     1683     1123     1472     1731     1784     1501     1767     1475     1520
Mephisto Phoenix        1755     1506     1890     2308     1824     1212     1521     2088     2022     1501     1980     1835     1691

MfG, Eric.
Mit Zitat antworten
Folgende 6 Benutzer sagen Danke zu Tibono für den nützlichen Beitrag:
applechess (14.02.2023), Chessguru (04.02.2023), chessman68 (04.02.2023), Egbert (04.02.2023), Mapi (04.02.2023), Thomas J (04.02.2023)
  #10  
Alt 14.02.2023, 07:53
Benutzerbild von Egbert
Egbert Egbert ist offline
Lebende Foren Legende
 
Registriert seit: 20.12.2009
Ort: Dreieich
Alter: 61
Land:
Beiträge: 10.221
Abgegebene Danke: 15.628
Erhielt 18.058 Danke für 6.947 Beiträge
Member Photo Albums
Aktivitäten Langlebigkeit
19/20 15/20
Heute Beiträge
2/3 ssss10221
AW: Khmelnitsky test: Mephisto IIIS Glasgow

Good morning Eric,

tournament practice with the Mephisto Glasgow shows that the programme characteristics do indeed correspond to the profile you determined in the "Khmelnitsky test". I am impressed by the significance of this test procedure! If Mephisto Glasgow were improved in terms of endgame theory and endgames in general, it could rise to even higher playing strength regions, especially under Mephisto Phoenix.

Greetings
Egbert
Mit Zitat antworten
Antwort


Forumregeln
Du bist nicht berechtigt, neue Themen zu erstellen.
Du bist nicht berechtigt, auf Beiträge zu antworten.
Du bist nicht berechtigt, Anhänge hochzuladen.
Du bist nicht berechtigt, deine Beiträge zu bearbeiten.

BB code ist An
Smileys sind An.
[IMG] Code ist An.
HTML-Code ist An.

Gehe zu

Ähnliche Themen
Thema Erstellt von Forum Antworten Letzter Beitrag
Test: Mephisto IIIS Glasgow Phoenix mclane Partien und Turniere / Games and Tournaments 108 01.04.2024 14:35
Frage: Mephisto Glasgow applechess Technische Fragen und Probleme / Tuning 8 06.01.2019 12:31
Frage: Mephisto III Glasgow hebel Technische Fragen und Probleme / Tuning 1 25.06.2013 19:40


Alle Zeitangaben in WEZ +2. Es ist jetzt 10:40 Uhr.



Powered by vBulletin (Deutsch)
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
©Schachcomputer.info