Home

RK 2003 Tournament
8 engines, 10 rounds, 280 games, Athlon 1.3/64 MB hash each engine
Time control: 120'/60 plus 60'/30 and 30', ponder=off, 3-/4-men EGTB
All programs with own (write protected) books, no book learning
 Book options=Use books, tournament book, optimal, minimum games 3
TK 3.23 SKR with general.ctg (write protected), optimal, weights reset
All games under ChessBase-GUI Hiarcs8 (old, readable PGN format)
Tester: Rolf Bühler, Zurich / Kurt Utzinger, Wetzikon (Schweiz)

 Program 
                                       Score     %    Av.Op.  Elo    +   -    Draws

  1 Shredder 7.04                  :  43.5/ 70  62.1   2489   2576   72  58   44.3 %
  2 The King 3.23 skr              :  43.0/ 70  61.4   2490   2571   72  56   45.7 %
  3 Junior 8                       :  41.5/ 70  59.3   2492   2557   74  61   38.6 %
  4 Fritz 8                        :  39.5/ 70  56.4   2495   2539   77  60   38.6 %
  5 Deep Sjeng 1.5                 :  31.5/ 70  45.0   2505   2470   61  79   35.7 %
  6 Chess Tiger 15.0               :  29.5/ 70  42.1   2507   2452   51  76   50.0 %
  7 Ruffian 1.0.5                  :  28.0/ 70  40.0   2509   2439   55  74   45.7 %
  8 Hiarcs 8                       :  23.5/ 70  33.6   2515   2396   57  68   47.1 %

Comments
The winner of our tournament RK 2003 is Shredder 7.04 of Stefan Meyer-Kahlen. It was a close win but considering the many other tournaments and results, it is a further sign that the first place of Shredder 7.04 at SSDF is fully justified. Only one match, against Junior 8, went lost. Chess Tiger 15 was defeated by 8,5-1,5 and the question remains if Shredder 7.04 is the most dreadful opponent of the strong French program. My comments to this can be found in the chapter about Chess Tiger 15. Why is Shredder 7.04 such strong? We believe it knows perhaps best of all programs to deal with the middle game. The endgame is still good enough but other engines are not worse. What can however be seen in the endgame is that Shredder 7.04 uses the EGTB’s quite early and in a very efficient way.

The second place takes The King 3.23 SKR with only half a point behind Shredder 7.04 and using the opening book general.ctg. Please note that the same engine with the remis.ctg got 10,5 points less out of 70 games and this would have been 5th place only. The influence of the opening books seems to be much higher than we have imagined. We were pleased with the good result of The King 3.23 SKR and hope that nobody will have any doubts that this setting belongs (at longer time controls) to the absolute top in computerchess. It’s a pity that Johan de Koning will not participate the World Chess Computer Champtionship. And one thing remains to be mentioned: The King has no access to the EGTB’s during its search and this - as per our experience - leads to a loss in performance of about five percent.

For quite a long time Junior 8 made attempts to keep the pool position but eventually both Shredder 7.04 and The King 3.23 SKR slightly overrun the program written by the Israelis Amir Ban and Shay Bushinsky. For many chessplayers Junior 8 is somewhat a mystery but of course a must to have. The dynamic playing style often leads to very exciting winning games but also to games with faults that can hardly be seen with other top engines. We are still unable to give an objective appraisement of this most interesting program. It might be that Junior 8 needs pondering to bring full strenght. What however can already be said is that Junior 8 depends more than other programs on its well optimized opening book. This book surely contributes a great deal to the overall strength of Junior 8. This can also be seen from our interrupted tournament RK 2003 where all engines had to use the book remis.ctg. At the time we stopped this tournament, Junior 8 had played 21 games and took last place with not more than 7,5 points http://www.utzingerk.com/rk_2003_old.htm. Also the close match at SSDF between Ruffian 1.0.1 and Junior 8 could give some evidence that no other engine depends so much on its good book. In this match Junior 8 was often out of book at an early stage and the final result of 21,5-18,5 in favour of Junior 8 is unusual against an amateur engine.

Less luck in the tournament had Fritz 8 (v 8.0.0.8) whereby it was in particular the low score of 35 percent or 3,5 points out of 10 games vs the Dutch opponent The King 3.23 SKR to be considered.  And the close loss of 4,5-5,5 vs Deep Sjeng 1.5 was also not what was expected wheras the 40 % vs Shredder 7.04 looks quite normal. Despite of this all we think that Fritz 8 is an allaround engine with a very balanced style. Only a pity that Fritz 8 in the analysis mode quite often only shows one move and no variation. A lot of people dislike this rarely seen behaviour and for this reason even avoid to start Fritz 8 for analysis purposes. People want the see the thinking process and the main variation. For us this is not very important as the further shown moves are in most cases of less importance.

One thing is sure: the performance of Deep Sjeng 1.5 is a great surprise. Before starting our tournament, in plenty of messages was argued that the Belgium amateur engine would not have much chances in the strong field. Responsible for the good result of Deep Sjeng 1.5 is the clear win of 7-3 vs Hiarcs 8, the win vs Fritz 8 and the draw vs Chess Tiger 15. The score of only 30 % against Shredder 7.04 and Junior 8 and a tight lost match (4,5-5,5) vs Ruffian 1.0.5 prevented an even better place for Deep Sjeng 1.5. To describe the playing style of this engine is not so easy. It prefers or plays stronger in closed and half-open positions. The classical e4/e5 opening does not suit very well to Deep Sjeng 1.5 and obvious improvements should be made in the endgame.

A score of only 42.1 % for Chess Tiger 15 is surely a disappointment for all fans of the strong French program. In particular the heavy loss of 1,5-8,5 vs Shredder 7.04 looks strange and the question is if the latter engine could be a dreaded opponent for Chess Tiger 15. This is however not the case when we consider the rating lists and results at lower time controls of Klaus Wlotzka http://www.computerschach.de/rangliste/ and also Heinz Walz http://www.beepworld.de/members38/heinzwalz-computerschach/index.htm. On the other hand when we go to the SSDF list http://ssdf.bosjo.net/list.htm with games at longer time controls (tournament level 120’/40) Shredder 7.04 won 30-10. We have the impression that Chess Tiger 15 finds good moves rather early (within a time limit of one minute on fast machines) and that playing strength does not considerably increase with more thinking time. This would explain the good results of Klaus Wlotzka and Heinz Walz and the less good results at SSDF. You can observe this behaviour in the analysis mode too: Chess Tiger 15 needs a lot of time to find alternative good moves. The defeat against Shredder 7.04 should of course not be overestimated as Chess Tiger 15 also lost vs The King 3.23 SKR and Junior 8. Maybe just an unlucky tournament as it happens from time to time.

Out of 70 games Ruffian 1.0.5 obtained a reasonable score of 40 %. The amateur engine of Perola Valfridsson plays all phases of the game rather well and is a dangerous opponent for all other engines. And for a long time it seemed as if Ruffian 1.0.5 would even get a better place but the 2,5-7,5 losses vs The King 3.23 SKR and Junior 8 prevented this. The version 1.0.5 has not been released for public but it is perhaps not stronger than 1.0.1 although it is of course difficult to give an estimation on the basis of only one tournament.

It was Hiarcs 8 being unable to win a single match. One reason for this - we noticed this in many games - the opening book that is not well adapted to the engine for fighting other engines. With a score of only 25 % vs Shredder 7.04 and a heavy and unexpected loss of 3-7 vs Deep Sjeng 1.5 it was of course not possible to get enough points. But who is interested in Hiarcs 8 when a much stronger version, namely Hiarcs 9, has come out. And there must be a loser in each tournament. And a final observation: Please note that it was just one tournament among others and it is not wise to draw wrong conclusions. In this respect we would draw your attention to the chapter “Some statistics” to understand what we mean.

In the meantime we have of course also tested Hiarcs 9 and worked out a crosstable (see below) where all games of Hiarcs 8 have been replaced by the latest version of Mark Uniacke’s fine program. And instead of 25.0/70 (score of 35,7 %) for Hiarcs 8, the engine Hiarcs 9 got a result of 33.5/70 (score 47,8 %) thus giving a clear prove of the improvements between the old and new version. Whilst Hiarcs 8 took last place, Hiarcs 9 ended on 5th place. Frankly spoken we had even expected a somewhat better result but the program (in our opinion a very strong one) missed to please us. We however hope that Hiarcs 9 will achieve a better performance in one of our next tournaments. At  http://www.utzingerk.com/rk_2003_h9.zip can all the games of our “RK 2003 with Hiarcs 9” tournament be downloaded.

Openings

Games 
                       :    280 (finished)

White Wins   :     99 (35.4 %)
Black Wins   :     60 (21.4 %)
Draws        :    121 (43.2 %)
Unfinished   :      0

White Perf.  : 57.0 %
Black Perf.  : 43.0 %

ECO A =      9 Games ( 3.2 %)
ECO B =    137 Games (48.9 %)
ECO C =     39 Games (13.9 %)
ECO D =     43 Games (15.4 %)
ECO E =     52 Games (18.6 %)

It’s interesting to divide the games into White wins, Black wins, and Draws. The drawing score with 43,2 percent seems rather high and is probably the consequence of the quality of the opening books.  The advantage for White to start the game with 35,4 percent is not much whilst Black could win 21,4 percent. And considering the openings played we see at first glance that the chess computer opening books clearly prefer open (1.e4) and half-open opening lines. Openings with code ECO A are represented in only nine games.

And as already mentioned the choice of the opening books is very important. The same tournament would have gone completely different had The King 3.23 SKR used the remis.ctg instead of the general.ctg (see cross table below) although Shredder 7.04 would still have won the tournament but: a tied second/third place for Fritz 8 and Junior 8 whereas The King 3.23 SKR would only have taken 5th place. The games of this so called RK 2003n tournament are ready for download at http://www.utzingerk.com/rk_2003n.zip .

A further example and a proof of the importance of the opening book demonstrated Junior 8 in the interrupted tournament where all engines had to use the book remis.ctg. In this tournament, after the third serie and 21 games Junior 8 took the last place with 7,5 points: see report at http://www.utzingerk.com/rk_2003_old.htm and the games can be downloaded at http://www.utzingerk.com/rk_2003_old.zip.

Some statistics

Please let us again have a look at the cross table of our tournament:

 Program 
 
                                                      Score     %    Av.Op.  Elo    +   -    Draws

  1 Shredder 7.04                  :  43.5/ 70  62.1   2489   2576   72  58   44.3 %
  2 The King 3.23 skr              :  43.0/ 70  61.4   2490   2571   72  56   45.7 %
  3 Junior 8                       :  41.5/ 70  59.3   2492   2557   74  61   38.6 %
  4 Fritz 8                        :  39.5/ 70  56.4   2495   2539   77  60   38.6 %
  5 Deep Sjeng 1.5                 :  31.5/ 70  45.0   2505   2470   61  79   35.7 %
  6 Chess Tiger 15.0               :  29.5/ 70  42.1   2507   2452   51  76   50.0 %
  7 Ruffian 1.0.5                  :  28.0/ 70  40.0   2509   2439   55  74   45.7 %
  8 Hiarcs 8                       :  23.5/ 70  33.6   2515   2396   57  68   47.1 %

All seems to be clear, with Shredder 7.04 we have a fine winner, Hiarcs 8 is the great loser and Ruffian 1.0.5 on the second last place. But all this is nonsense because the above cross table shows a margin of error given with 95 % confidence. In other words does the performance of Shredder 7.04 (2576 Elo) include a margin of error of -58 points which would in the worst case give a performance of only 2439 Elo and this would be 5th place only. On the other hand, Ruffian 1.0.5 got 2439 Elo and considering the margin of error of +55 points, the program could also have a performance of 2494 Elo and this would correspond with 5th place too. What we want to say with these details is that the results/cross table is a rather relative value. And do we not forget: each engine had to play 10 games against all other programs and it is well known what often happens: 8-2 and then 2-8. And a bad result in such short tournaments like RK 2003 does very much influence the outcome. It also occurred in this tournament that some engines were defeated by others: Shredder 7.04 vs Chess Tiger 15 (8,5-1,5) / The King 3.23 SKR vs Ruffian 1.0.5 (7,5-2,5), Junior 8 vs Ruffian 1.05 (7,5-2,5) und Fritz 8 vs Ruffian 1.0.5 (8,0-2,0), and who knows if the situation would not be quite the contrary? And eventually the indidual results of the engines:

Individual statistics:

(1) Shredder 7.04 
                            :  70 (+ 28,= 31,- 11), 62.1 %

Chess Tiger 15.0              :  10 (+  7,=  3,-  0), 85.0 %
The King 3.23 skr             :  10 (+  3,=  5,-  2), 55.0 %
Deep Sjeng 1.5                :  10 (+  4,=  6,-  0), 70.0 %
Fritz 8                       :  10 (+  5,=  2,-  3), 60.0 %
Ruffian 1.0.5                 :  10 (+  2,=  6,-  2), 50.0 %
Hiarcs 8                      :  10 (+  6,=  3,-  1), 75.0 %
Junior 8                      :  10 (+  1,=  6,-  3), 40.0 %


(2) The King 3.23 skr         :  70 (+ 27,= 32,- 11), 61.4 %

Chess Tiger 15.0              :  10 (+  3,=  6,-  1), 60.0 %
Deep Sjeng 1.5                :  10 (+  4,=  5,-  1), 65.0 %
Shredder 7.04                 :  10 (+  2,=  5,-  3), 45.0 %
Fritz 8                       :  10 (+  3,=  7,-  0), 65.0 %
Ruffian 1.0.5                 :  10 (+  6,=  3,-  1), 75.0 %
Hiarcs 8                      :  10 (+  5,=  4,-  1), 70.0 %
Junior 8                      :  10 (+  4,=  2,-  4), 50.0 %


(3) Junior 8                  :  70 (+ 28,= 27,- 15), 59.3 %

Chess Tiger 15.0              :  10 (+  3,=  6,-  1), 60.0 %
The King 3.23 skr             :  10 (+  4,=  2,-  4), 50.0 %
Deep Sjeng 1.5                :  10 (+  6,=  2,-  2), 70.0 %
Shredder 7.04                 :  10 (+  3,=  6,-  1), 60.0 %
Fritz 8                       :  10 (+  2,=  2,-  6), 30.0 %
Ruffian 1.0.5                 :  10 (+  6,=  3,-  1), 75.0 %
Hiarcs 8                      :  10 (+  4,=  6,-  0), 70.0 %


(4) Fritz 8                   :  70 (+ 26,= 27,- 17), 56.4 %

Chess Tiger 15.0              :  10 (+  4,=  5,-  1), 65.0 %
The King 3.23 skr             :  10 (+  0,=  7,-  3), 35.0 %
Deep Sjeng 1.5                :  10 (+  3,=  3,-  4), 45.0 %
Shredder 7.04                 :  10 (+  3,=  2,-  5), 40.0 %
Ruffian 1.0.5                 :  10 (+  6,=  4,-  0), 80.0 %
Hiarcs 8                      :  10 (+  4,=  4,-  2), 60.0 %
Junior 8                      :  10 (+  6,=  2,-  2), 70.0 %


(5) Deep Sjeng 1.5            :  70 (+ 19,= 25,- 26), 45.0 %

Chess Tiger 15.0              :  10 (+  3,=  4,-  3), 50.0 %
The King 3.23 skr             :  10 (+  1,=  5,-  4), 35.0 %
Shredder 7.04                 :  10 (+  0,=  6,-  4), 30.0 %
Fritz 8                       :  10 (+  4,=  3,-  3), 55.0 %
Ruffian 1.0.5                 :  10 (+  3,=  3,-  4), 45.0 %
Hiarcs 8                      :  10 (+  6,=  2,-  2), 70.0 %
Junior 8                      :  10 (+  2,=  2,-  6), 30.0 %


(6) Chess Tiger 15.0          :  70 (+ 12,= 35,- 23), 42.1 %

The King 3.23 skr             :  10 (+  1,=  6,-  3), 40.0 %
Deep Sjeng 1.5                :  10 (+  3,=  4,-  3), 50.0 %
Shredder 7.04                 :  10 (+  0,=  3,-  7), 15.0 %
Fritz 8                       :  10 (+  1,=  5,-  4), 35.0 %
Ruffian 1.0.5                 :  10 (+  3,=  5,-  2), 55.0 %
Hiarcs 8                      :  10 (+  3,=  6,-  1), 60.0 %
Junior 8                      :  10 (+  1,=  6,-  3), 40.0 %


(7) Ruffian 1.0.5             :  70 (+ 12,= 32,- 26), 40.0 %

Chess Tiger 15.0              :  10 (+  2,=  5,-  3), 45.0 %
The King 3.23 skr             :  10 (+  1,=  3,-  6), 25.0 %
Deep Sjeng 1.5                :  10 (+  4,=  3,-  3), 55.0 %
Shredder 7.04                 :  10 (+  2,=  6,-  2), 50.0 %
Fritz 8                       :  10 (+  0,=  4,-  6), 20.0 %
Hiarcs 8                      :  10 (+  2,=  8,-  0), 60.0 %
Junior 8                      :  10 (+  1,=  3,-  6), 25.0 %


(8) Hiarcs 8                  :  70 (+  7,= 33,- 30), 33.6 %

Chess Tiger 15.0              :  10 (+  1,=  6,-  3), 40.0 %
The King 3.23 skr             :  10 (+  1,=  4,-  5), 30.0 %
Deep Sjeng 1.5                :  10 (+  2,=  2,-  6), 30.0 %
Shredder 7.04                 :  10 (+  1,=  3,-  6), 25.0 %
Fritz 8                       :  10 (+  2,=  4,-  4), 40.0 %
Ruffian 1.0.5                 :  10 (+  0,=  8,-  2), 40.0 %
Junior 8                      :  10 (+  0,=  6,-  4), 30.0 %

Tournament course

We give the details about the course of the RK 2003 tournament (total 280 games) that went over 10 series at 28 games. Each serie consisted of seven rounds at four games. And all participants - with the exception of Chess Tiger 15, Ruffian 1.0.5 und Hiarcs 8 - were able to at least win one serie. Again a good example to demonstrate that it is very important to play as much games as possible. The less games are played the more increases the probabilitiy that we have a winner by mere accident.

Serie 10  (games 253-280)
Winner = Fritz 8.0.0.8

Serie 9  (games 225-252)
Winner = The King 3.23 srk

Serie 8  (games 197-224)
Winner = Junior 8.0.0.2

Serie 7  (games 169-196)
Winner = Shredder 7.04

Serie 6  (games 141-168)
Winner = Junior 8.0.0.2

Serie 5  (games 113-140)
Winner = Shredder 7.04

Serie 4  (games 85-112)
Winner = The King 3.23 skr

Serie 3  (games 57-84)
Winner = Deep Sjeng 1.5

Serie 2  (games 29-56)
Winner = Fritz 8

Serie 1 (games 1 - 28)
Winner = Junior 8.0.0.2

Home