rating and depth ? #20
Comments
|
Hi @tissatussa thanks for this issue. Regarding this issue, I personally find blunders like these hard to debug. I suspect it is related to the bug fixed with v3.1 (I forgot to update the Cargo.toml for the uci wrapper so you probably pulled the one with the engine at v3.0, I'll update it later today) Regarding the rating, you can checkout the bot's lichess. It's around 1700-1800 against the other bots. There's quite a lot of bots on lichess so it's the best benchmark I can get. Blunders like this definitely make it look pretty silly, but it doesn't really know much about chess, just about doing a minimax search! I did build a NNUE evaluation function to give it more insight into tactics but this greatly slowed the search down and reduced its rating to around 1300. As for myself, I'm actually a pretty poor chess player (not great at visualizing what is being attacked), I actually made ShallowRed because my friends are all quite good players. Looking forward to working with you @tissatussa I'm sure we can get the rating up a bit 🙂. (Regarding multiPV, let me get back to you once I've thought about it) |
|
another strange move appeared .. i was just playing a 15m10s game against v0.3.1 and this position occured (i had the Black pieces) :
in fact i'm lost here : i'll lose the Rook for its kNight and i will probably not survive .. but here Shallow-Red played the 'friendly' 36.Qxc6+ ??, so i captured the Queeen and won this game. what happened ? feels like a joke :-) |
|
MultiPV is nice but not needed in the first place, however, it may give insight during development. But only if we look at practical play, not the statistics .. -do you count on those? i guess i will not be making PRs .. i'm into programming but not Rust, i can instruct though .. i often tried to play such LiChess bot but they almost always are offline .. |
|
btw. your binary takes 1.5 Gb memory, which is much : what's done with it? |
|
the fact that ShallowRed often starts a game by moving the 2 kNights onto their natural squares indicates to me some PST function may induce that, but playing like this needs good understanding of pawns-in-the-center : often the Nc3 is a worse move, because it blocks the c-pawn which can be crucial at the following moves. i know minimax but HCE (evaluation) should do the pruning, isn't it? And is it easy for you to implement that UCI info string? |
|
do you know "Test Suites" ? a nice and exiting way to see how strong the engine is and what mistakes it can make. I just found this README at https://github.com/Tearth/Inanis/tree/v1.3.0 , see also README.md i have some experience with this and i can help .. are you interested ? a puzzle can have multiple 'solutions'. Programs exist to perform such Tests, and many Suites exist. However, it's up to the programmer to interpret the results and adjust some code accordingly. EPD puzzles exist with only (/also) 'am' : 'Avoid Move' - as opposed to 'bm' : 'Best Move'. |
Hi @tissatussa, could you clarify what you mean by practical play vs statistics? For additions to the bot, there's a couple small clean up changes I'm happy to make, but it's not my personal main focus right now. I might work on a nicer NNUE eval function (see this repo for the old one), or some other additions like endgame tables, but I likely myself won't come back to performance improvements for a while. I would be completely happy to help you out however, if you'd like to tinker with the engine. I don't have too much free time, but I hope I could get back to your PRs and issues in a reasonable amount of time. I understand you're deterred by rust, but especially with an existing project it should be a nice language to pickup. If you are interested, changes like this with uci parsing would be good quick changes to get started. |
This is cache allocation, the amount is somewhat arbitrary, and the default value is here. That is just a default though, whatever is controlling the cache (the uci-wrapper layer for example) has the real control over it. |
The engine has a test suite for puzzles here. I believe |
Yes a PST is the cause of that move, I suggest taking a look through the src folder, it should provide lots of details about the engine. For example there is a The evaluation function is not responsible for the pruning itself, it instead is only responsible for returning the evaluation. The search function performs alpha-beta pruning based on the evaluations coming in. More uci info could possibly be added, but only at the first few moves of the search for speed reasons. The engine searches millions of moves and printing out the same amount of strings would grind it to a halt. Even the check for whether or not our depth is too great to print the string might be unfavorable. |
many chess engine programmers only use the results of bullet self-play, they don't eximine / study real (longer) games, because they're not a chess player themself.
i didn't know ShallowRed has an NNUE, does it really ?
i was not planning to do so, but i might.
Rust must be a nice language, indeed this might be a good start.
but why that large ? Many engines use upto 250 Mb max ..
OK, but i don't know what you mean by 'regression checks', although i've seen that term often ..
i'm aware of this mechanism, for me this comes down to the evaluation values determining the pruning, although indirectly - as you explain .. nevermind.
somehow it's possible to output UCI info during search, most engines do : they output this info (only) when a depth is fully finished. |
about this position :
here Shallow-Red -White- played 14.Nxd4, which is a blunder .. at first glance we might understand the idea, White wants to win this pawn, covered by a combination : when Black captures the uncovered Nd4 with the Bf6, White can do Qd5 with a double attack on Ra8 and Bd4, which seems to ensure recapturing one of these pieces, but then Black can play Bg4!, also with dual purpose : this counter move covers the Ra8 and threatens mate on e2 .. White can avoid that loss, but Black will do Rd8, again with dual purpose : this attacks the Qd5 and covers the Bd4 .. all this is a bit unusual but the moves are forced and rather easy to see (for an experienced chess player, like me -ahum..) but Shallow-Red fails, it even didn't play Qd5 after BxNd4, it did Bf4 and lost this game, simply being a piece down.
i don't know upto which depth the engine goes, there's no display of that in the CuteChess GUI .. i propose you try to implement this - it might also give you more insight what the engine's doing : after how many ms will a first (often) bad move be discarded and other ones appear as bestmove, changing, by using MultiPV this calculation process is even more visible, although normally not used in CuteChess games .. you'd add such UCI info line to the output.
the combinations i showed are not very deep - i'm surpised Shallow-Red made this mistake, being just a tactic .. do you have an idea of its rating? - this was a 3m2s game.
are you a (good) chess player yourself? i'm a club player for years (but still below 1900).
how do you judge the played games by your engine? (only) by statistics? -rating can be determined against many (weak) engines.
i guess its rating is no more then 1200.
respect though - it's all about the idea :-)
a minor code change may easily result in a much higher rating .. can i help? -i'm mainly thinking about special positions, to test 'behaviour' .. i'm into chess engine programming, though only in general, i've not created a serious engine myself upto now.
i await your answer !
Replay the game
[ i'm on Xubuntu 22.04 ]
The text was updated successfully, but these errors were encountered: