AW: Khmelnitsky test: Mephisto IIIS Glasgow
 

Hi, time for the outcome... I shall split the data across several posts, for an easyer reading.

Let's start with a first graph displaying the skills domains of the slow Mephisto IIIS Glasgow, the one set with CPU waits (7.2 Mhz equivalent):

https://www.schachcomputer.info/foru...1&d=1675451664

The peak strength is strategy, a usually weak domain for oldies, but known to have a large impact (with tactics) on the overall strength. That is a much uncommon feature, revealing deeply thought chess software!

Opening management is strong, then middlegame and endgame loose some strength, a usual trend with our chess computers. Level of play is very weak in standard endgame positions (lack of theoretical knowledge), and the program is rather blind or reluctant to sacrifices. Tactics and calculation ability are limited, whilst threats recognition and defense are fine.

The overall KT score is 1374, to be taken with a grain of salt: the test has not been designed nor standardized for chess computers (nevertheless, some kind of standardization can be applied, I shall do my best in a later post). So, let's compare profiles to a 1374 Elo player:

https://www.schachcomputer.info/foru...1&d=1675452939

Playing level is roughly on par with regards to tactical skills (tactics, recognizing threats, attack, counterattack and defense), and as well about calculations ability, middlegame and endgame. This provides a very human-like distribution of skills, highlighted by the better strategy management. The lack of endgame theory (standard positions) is the most obvious weakness compared to the human player.

AW: Khmelnitsky test: Mephisto IIIS Glasgow
 

Let's continue with the fast version of the Mephisto IIIS Glasgow: without wait states, thus 12Mhz efficient.

https://www.schachcomputer.info/foru...1&d=1675454153

The profile is a bit more round, with slight enhancements for some weak points of the slow version: tactics, attack, sacrifice. Opening (playing skills once out of book) and strategy, already strong, keep on progressing. No hope for endgame theory, the slight increase in speed cannot make for the lack of knowledge.

The overall KT score is 1441, so let's check with a puny human:

https://www.schachcomputer.info/foru...1&d=1675454801

Still this much unusual domination as far as strategy is concerned! The player should better wait for the endgame. Tactical domains keep on being on par. An interesting opponent for the human player!

AW: Khmelnitsky test: Mephisto IIIS Glasgow
 

Let's speed up the device, using the Phoenix Revelation (simulated as explained top of the thread); 38Mhz equivalent:

https://www.schachcomputer.info/foru...1&d=1675455331

The overall pattern looks different. The strategy is no more the peak strength, the tactical domains now prevail, undoubtly with regards to far better calculations ability. Leveraging calculations, the Glasgow program can now strongly counterattack, and even commit or recognize sacrifice opportunities. Opening to middlegame phase is very strong.

The overall KT score is 1541, 100 Elo points stronger than the previous 12Mhz.
Compared to a similar strength human:

https://www.schachcomputer.info/foru...1&d=1675456077

No more any advantage in strategy, the human is slightly stronger at that level. Nevertheless, he will have a hard time until late middlegame!

AW: Khmelnitsky test: Mephisto IIIS Glasgow
 

Hi Tibono,

your Tests are very interesting.

Thank You

best regards

Markus

AW: Khmelnitsky test: Mephisto IIIS Glasgow
 

Now the long awaited cherry on the cake, the Mephisto Phoenix profile, with an overwhelming 480Mhz speed equivalence:

https://www.schachcomputer.info/foru...1&d=1675456577

Well, the bad news is strategy is now one weak point amongst few others at similar level (like attack and endgame). Not that surprising, more speed is not more knowledge! For the very same reason, standard engame positions do not progress much. On another hand, tactical capabilities reach a significantly higher level. Still not a great attacker, nevertheless now stronger defender and counterattacker. Defense is commonly strong with other chess computers, a quality the Glasgow program did not highlight until this large acceleration.

What about a human opponent? The KT score being 1755, here you are:

https://www.schachcomputer.info/foru...1&d=1675457391

The endgame knowledge of the human player is overwhelming. The Glasgow must rely on its strong opening and middlegame phase, then expect some tactical opportunity to avoid the endgame, at least get some concrete advantage before entering it.

AW: Khmelnitsky test: Mephisto IIIS Glasgow
 

Of course I cannot leave without a synthesis: all in one graph!
A larger one to better zoom on the skills profiles:

https://www.schachcomputer.info/foru...1&d=1675458166

There is a striking oddity in this graph, but I think it can be explained: not only the strategic skills did not step forward with the acceleration, as previously observed, but the graph displays a regression! The good strategic guidelines embedded in the Glasgow program weight higher with short thinking time; deeper calculations steer the score away because of tactical connsiderations, that is my opinion. Maybe strategic elements are mainly computed at root level?
Another small oddity lies in the calculations, where the fast IIIS performs less then the slow one; but otherwise the calculations scores look to range accordingly to expectations. The fast IIIS must have been on a wrong track with some position, there is of course some alea related to the current score at the time the thinking process is stopped.
Opening, middlegame, tactics, threat recognition and sacrifice progress rather evenly and significantly along with speed.
As already pointed out, defense skills were stuck without any significant progress, until the large acceleration provided by the Mephisto Phoenix. Counterattack is sort of similar, with unleashed power starting with the Revelation. And the Glasgow program kept brakes on attacking skills, but that is not uncommon with our dear chess computers (a bit passive, lacking any attacking plan and awaiting tactical opportunities).

Feel welcome to comment further!
I'll be back very soon with additional data to share, but enough for today.
MfG, Eric

AW: Khmelnitsky test: Mephisto IIIS Glasgow
 

Hi Eric,

Thank you very much for your detailed analysises.

Like your excellent analysis of the Chessmaster Diamond on your website, they provide valuable information about the strength and weaknesses of the engines.

I've a question regarding the test setting though:

I found a description in German language, how the Khmelnitsky test distinguishes between strategy, tactics, opening, endgame, etc.
(For those, who are interested:
https://www.freechess.info/ueebersic...ing-guide.html)

But where are the human player profiles taken from?

Best regards
Markus

AW: Khmelnitsky test: Mephisto IIIS Glasgow
 

Hello Eric,

Thank you very much for this very informative test series. Especially the improved defensive capability of the Mephisto Glasgow on the Mephisto Phoenix I could also observe in the context of my competitions! The overall weak endgame performances, even under Mephisto Phoenix also correspond to my expectations and are also present in practice. I also share your assessments, which are based on your test. Really great work Eric! :top:

Greetings
Egbert

AW: Khmelnitsky test: Mephisto IIIS Glasgow
 

Hi Markus, they are extracted from the data of the very same Khmelnitsky's test. The Elo values come from a data sheet in the book, providing two columns per domain (attack, counterattack,...); one column being the % of your score in the domain, the other the corresponding Elo observed per statistics across a number of players. 99% score is achieved by 2500+ players, 3% by 815 or so players. In between, the distribution vary according to each domain. Elo values for intermediate % values (they are provided per 3% step, so 3%, 6%, 9%... up to 96%, 99%) rely on interpolation.

Well, the data sheet can be red the other way around: I am a 1400 Elo player, which score (%) am I supposed to achieve per domain of chess knowledge? An interpolation can be performed between two Elo values in order to get an accurate percentage within the 3% spread.
Of course, the computer-player comparison cannot usefully rely on an Elo graph (this would just add a circle to the basic Elo-graph, plotting the player's strength). So the comparison graph is based on %, as well displaying usual, average strengths and weaknesses of the human player.
By the way, time ago I used this idea (not mentioned within the book!) to provide a graph displaying how the skills develop along to the Elo, and posted about in the Hiarcs forum. Here is the graph:

https://www.schachcomputer.info/foru...1&d=1675497228

and my comments:

AW: Khmelnitsky test: Mephisto IIIS Glasgow
 

Now back to my mainstream, the Glasgow program.
This table is quite simple: Speed in Mhz, ratio based on the 7.2 basic speed, log(base 10) of the ratio, and achieved KT Elo from the test. The base of the log function has no influence on the below graph pattern:

https://www.schachcomputer.info/foru...1&d=1675498435

The curve's closeness to a straight line is obvious. This means the Elo growth follows the usual rule (a constant value gain per doubling in speed) even when reaching sky-high acceleration. I was expecting more of a drop in the curve from the Mephisto Phoenix, it is not so, well done from the Glasgow program!
I added a polynomial regression curve to get the displayed formula we will use later on.