Problem using python-chess #1

djdekker · 2016-06-12T07:15:55Z

I'm using K2 0.83 with PicoChess using python-chess by @niklasf. I compiled K2 on Raspberry Pi 3 with -march=armv6 to ensure backwards compatibility with RPi2.

The problem is: when K2 leaves the opening book it stops responding. What could be the cause?

There was another engine, Cinnamon by @gekomad, which also had this error. It was related to an infinite loop when closing stdin and to the use of isready/readyok. Cinnamon has been fixed now (pull requests #3 and #4).

Traceback (most recent call last):
File "./picochess.py", line 841, in
main()
File "./picochess.py", line 788, in main
stop_search_and_clock()
File "./picochess.py", line 205, in stop_search_and_clock
stop_search()
File "./picochess.py", line 198, in stop_search
engine.stop()
File "/opt/picochess/engine.py", line 230, in stop
self.engine.stop()
File "/usr/local/lib/python3.4/dist-packages/chess/uci.py", line 1249, in stop
return self._queue_command(command, async_callback)
File "/usr/local/lib/python3.4/dist-packages/chess/uci.py", line 890, in _queue_command
return future.result()
File "/usr/lib/python3.4/concurrent/futures/_base.py", line 397, in result
self._condition.wait(timeout)
File "/usr/lib/python3.4/threading.py", line 290, in wait
waiter.acquire()

niklasf · 2016-06-12T10:20:03Z

I can get k2 to communicate nicely with python-chess 0.14.1 on my computer:

import chess
import chess.uci
import sys
import logging

logging.basicConfig(level=logging.DEBUG)

e = chess.uci.popen_engine([sys.argv[1]])
e.uci()

e.position(chess.Board("rnbqkbnr/ppp2ppp/8/3pp3/4P3/5N2/PPPP1PPP/RNBQKB1R w KQkq d6 0 3"))

print(e.go(movetime=2000))

[...]
DEBUG:chess.uci:<PopenProcess at 0x7f070ce9b208 (pid=25499)> >> info nodes 827259 nps 437863 currmove f3d4 currmovenumber 24 hashfull 65
DEBUG:chess.uci:<PopenProcess at 0x7f070ce9b208 (pid=25499)> >> info nodes 827737 nps 437841 currmove f1a6 currmovenumber 25 hashfull 65
DEBUG:chess.uci:<PopenProcess at 0x7f070ce9b208 (pid=25499)> >> info nodes 827971 nps 437803 currmove e1e2 currmovenumber 26 hashfull 65
DEBUG:chess.uci:<PopenProcess at 0x7f070ce9b208 (pid=25499)> >> info nodes 828235 nps 437766 currmove f3g1 currmovenumber 27 hashfull 65
DEBUG:chess.uci:<PopenProcess at 0x7f070ce9b208 (pid=25499)> >> info nodes 828868 nps 437782 currmove f3h4 currmovenumber 28 hashfull 65
DEBUG:chess.uci:<PopenProcess at 0x7f070ce9b208 (pid=25499)> >> info nodes 829387 nps 437617 currmove f3e5 currmovenumber 1 hashfull 65
DEBUG:chess.uci:<PopenProcess at 0x7f070ce9b208 (pid=25499)> >> bestmove f3e5 ponder d5e4
BestMove(bestmove=Move.from_uci('f3e5'), ponder=Move.from_uci('d5e4'))

djdekker · 2016-06-27T17:51:39Z

Hi @serg-meus and @niklasf,

Thank you for your attention and for your time and efforts to solve this issue.

I compiled the latest version of K2, but it still shows the same behaviour when using it with PicoChess (by means of the python-chess library). In the mean time I've found the same issue with the Rodent_II engine by @nescitus. Because many other engines are not affected, I assume (for now) that PicoChess is not at fault here.

Could it perhaps have anything to do with python-chess sending 'isready' during search and then output strings getting mixed up?

Greetings,
DJ

serg-meus · 2016-06-28T15:45:05Z

Hello, @djdekker, thank you for your interest.

Are you sure that K2 has been compiled successfully and executable file has permissions to execute? Can you connect your RPi to display, run the terminal and execute the engine in terminal? It may be done by following commands:

cd path_to_k2
./k2
go

Best regards,
Sergey Meus

djdekker · 2016-06-28T17:11:01Z

Hi Sergey,
K2 runs on my RPi3: I made a screencast to show you. It seems, however, that K2 stops calculating at depth 11.
Best regards,
Dirk-Jan

serg-meus · 2016-06-28T17:36:14Z

@djdekker, everything is correct, engine seems to work fine. By default K2 use time control 5 minutes per game and the calculation stop is caused by timer.
I am going to install PicoChess on my laptop and try it, but it may take a lot of time, I am not an expert in installation of Python packages and my little children will not accelerate the process :)

Best regards,
Sergey Meus

djdekker · 2016-06-28T18:40:41Z

pi@raspberrypi:/opt/picochess $ sudo python3 ./picochess.py -c -e /opt/picochess/engines/armv7l/k2

If you run picochess like this in console mode (so that you don't need the physical DGT board) with K2 selected as its engine, you will see that picochess never receives the PV or score from the engine when search has started. Just enter a2a4 as the first move and enter go to start the search. There's no answer coming from the engine.

Trying exactly the same procedure with, for example, Stockfish or Gull is successful (replace /opt/picochess/engines/armv7l/k2 with /opt/picochess/engines/armv7l/a-stockf).

serg-meus · 2016-06-29T04:59:33Z

@djdekker, is it possible to get all commands sent from picochess to K2 from begin of session to the moment where K2 is not responding?

djdekker · 2016-06-29T06:18:43Z

Hi Sergey, it's the python-chess library that handles the chess playing part of PicoChess. I don't know the internals of python-chess, but @niklasf does.

serg-meus · 2016-06-29T06:27:52Z

Hi Dirk-Jan, thanks for your response.
I will try to look at python-chess sources, too.

serg-meus · 2016-06-30T17:32:40Z

@djdekker, @niklasf, @nescitus
Gentlemen, I got it.

My chess engine does not support completely the UCI protocol, in particular, command 'go ... searchmoves ...' is not supported, and this is a bug. After 'searchmoves' there must be a list of moves that engine must to take into account, all the rest moves must be skipped.
My chess engine freezes if not supported command inside 'go' command comes, and this is a bug.
The picochess via python-chess sends to engine 'go ... searchmoves ...' command with list of all legal moves, I don't know why, but this is not a bug.

Bug number 2 fixed by a474d64, picochess with K2 is working now on my laptop

nescitus · 2016-06-30T17:51:07Z

thank You very much! fixing it will take a while, but since I think that
Rodent needs a root move list anyhow, this is not a problem.

2016-06-30 19:32 GMT+02:00 serg-meus [email protected]:

@djdekker https://github.com/djdekker, @niklasf
https://github.com/niklasf, @nescitus https://github.com/nescitus
Gentlemen, I got it.

My chess engine does not support completely the UCI protocol, in
particular, command 'go ... searchmoves ...' is not supported, and this is
a bug. After 'searchmoves' there must be a list of moves that engine must
to take into account, all the rest moves must be skipped.

My chess engine freezes if not supported command inside 'go' command
comes, and this is a bug.

The picochess via python-chess sends to engine 'go ... searchmoves ...'
command with list of all legal moves, I don't know why, but this is not a
bug.

Bug #2 #2 fixed by a474d64
a474d64

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/ABEHaDDAP85palhnvBMV5hxGxy8pNeKpks5qQ_24gaJpZM4Izub6
.

djdekker · 2016-06-30T18:07:56Z

Thank you very much, Sergey! I just made a fresh K2 compile and it works fine now. 😃

djdekker · 2016-06-30T19:19:39Z

Sorry Sergey, there still seems to be an issue with strings passed from the K2 engine to python-chess. In the log here below it's an invalid cp value, but sometimes it's an exception parsing the pv.

2016-06-30 20:40:45.223 ERROR uci - _info: exception parsing score cp value
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/chess/uci.py", line 762, in _info
score_cp = int(token)
ValueError: invalid literal for int() with base 10: 'readyok'
2016-06-30 20:40:45.224 DEBUG dgtdisplay - run: Search started
2016-06-30 20:40:45.229 ERROR uci - _info: exception parsing pv
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/chess/uci.py", line 747, in _info
pv.append(board.push_uci(token))
File "/usr/local/lib/python3.4/dist-packages/chess/init.py", line 2795, in push_uci
move = self.parse_uci(uci)
File "/usr/local/lib/python3.4/dist-packages/chess/init.py", line 2773, in parse_uci
move = Move.from_uci(uci)
File "/usr/local/lib/python3.4/dist-packages/chess/init.py", line 776, in from_uci
raise ValueError("expected uci string to be of length 4 or 5: {0}".format(repr(uci)))
ValueError: expected uci string to be of length 4 or 5: '!'
2016-06-30 20:40:45.230 DEBUG picochess - main: received event from evt_queue: EVT_NEW_SCORE
2016-06-30 20:40:45.235 ERROR uci - _info: exception parsing pv
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/chess/uci.py", line 747, in _info
pv.append(board.push_uci(token))
File "/usr/local/lib/python3.4/dist-packages/chess/init.py", line 2795, in push_uci
move = self.parse_uci(uci)
File "/usr/local/lib/python3.4/dist-packages/chess/init.py", line 2773, in parse_uci
move = Move.from_uci(uci)
File "/usr/local/lib/python3.4/dist-packages/chess/init.py", line 776, in from_uci
raise ValueError("expected uci string to be of length 4 or 5: {0}".format(repr(uci)))
ValueError: expected uci string to be of length 4 or 5: '!'
2016-06-30 20:40:45.237 DEBUG picochess - main: received event from evt_queue: EVT_NEW_PV
2016-06-30 20:40:45.237 DEBUG dgtdisplay - run: received message from msg_queue: MSG_NEW_SCORE
2016-06-30 20:40:45.250 DEBUG dgtdisplay - run: received message from msg_queue: MSG_NEW_PV
2016-06-30 20:40:45.249 ERROR uci - _info: exception parsing pv
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/chess/uci.py", line 747, in _info
pv.append(board.push_uci(token))
File "/usr/local/lib/python3.4/dist-packages/chess/init.py", line 2795, in push_uci
move = self.parse_uci(uci)
File "/usr/local/lib/python3.4/dist-packages/chess/init.py", line 2773, in parse_uci
move = Move.from_uci(uci)
File "/usr/local/lib/python3.4/dist-packages/chess/init.py", line 776, in from_uci
raise ValueError("expected uci string to be of length 4 or 5: {0}".format(repr(uci)))
ValueError: expected uci string to be of length 4 or 5: '?'

djdekker · 2016-06-30T20:05:16Z

After 1. f3 e5 2. g4 Qh4# I'm getting this error immediately after putting the black Queen on h4:

Traceback (most recent call last):
File "./picochess.py", line 826, in
main()
File "./picochess.py", line 542, in main
process_fen(event.fen)
File "./picochess.py", line 245, in process_fen
move = list(game.legal_moves)[legal_fens.index(fen)]
IndexError: list index out of range

LocutusOfPenguin · 2016-07-01T11:34:50Z

@djdekker
what version of picochess & engine you are using?

@nescitus
not being able to use rodent_II after the book left, blocks abit my "personality" update for picochess.
I hope y can find the time to fix this. Thank you.

serg-meus · 2016-07-01T17:03:12Z

@djdekker, I cant reproduce the mentioned errors, on my laptop picochess and K2 plays correctly, and f2f3 e7e5 g2g4 is also looks fine.

I think it would be better if you change one line in picochess.py to make log on interaction between engine and python-chess. Then run picochess with keys '-l debug -lf logfile.log' and send me log file with erroneous behavior, if any.

The original line is:
logging.getLogger("chess.uci").setLevel(logging.INFO) # don't want to get so many python-chess uci messages

The changed line is:
logging.getLogger("chess.uci").setLevel(logging.DEBUG) #want to get so many python-chess uci messages

djdekker · 2016-07-01T18:14:33Z

@serg-meus This was not a bug in your engine, but probably some PicoChess bug. The latest PicoChess updates by @LocutusOfPenguin have solved it!

@LocutusOfPenguin: I was using your 'level' branch (last week's version) and K2 version 0.83. After switching back to your 'master' branch (today's version, 72ab3e5) there is no error anymore. Then I switched back to your 'level' branch again, updated it with git to today's version (f4eabc8), and the error was also gone there too.

LocutusOfPenguin · 2016-07-01T20:09:37Z

well...today i (internally) released v0.68. This is alot more stable as the older version (solving some bug issues on our list).
But right now, its not tested enough, so i prob have to wait until beginning of next week.

djdekker · 2016-07-02T05:22:22Z

@serg-meus Here is a log file (with python-chess log level set to DEBUG) to show you what's happening with the invalid pv / score strings:
https://dl.dropboxusercontent.com/u/10122745/20160702_picochess.log

serg-meus · 2016-07-03T19:30:17Z

@djdekker, please try 7eed29b.
Regards,
SM

djdekker · 2016-07-04T19:09:01Z

Hello Sergey,
Thank you for your commit! I rebuilt the K2 engine from source. The invalid pv/score strings are gone 😃, but now I found that K2 sometimes sends corrupt 'readyok' messages. In one log file I saw it sending 'readyok152' instead of 'readyok'. Mysterious!
Greetings,
DJ

LocutusOfPenguin · 2016-07-05T13:34:47Z

Perhaps same as: niklasf/python-chess#83 ?

serg-meus · 2016-07-05T17:09:44Z

@djdekker, @LocutusOfPenguin, thank you very much!
I never thought that it would be problem there.
I hope cee2289 will solve this, because it's hard to reproduce multi-threading errors.

serg-meus added the bug label Jun 12, 2016

serg-meus closed this as completed Jun 12, 2016

serg-meus reopened this Jun 12, 2016

serg-meus closed this as completed Jul 10, 2016

LocutusOfPenguin mentioned this issue Aug 13, 2018

Go searchmoves #9

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Problem using python-chess #1

Problem using python-chess #1

djdekker commented Jun 12, 2016 •

edited

Loading

niklasf commented Jun 12, 2016 •

edited

Loading

djdekker commented Jun 27, 2016

serg-meus commented Jun 28, 2016

djdekker commented Jun 28, 2016

serg-meus commented Jun 28, 2016

djdekker commented Jun 28, 2016

serg-meus commented Jun 29, 2016

djdekker commented Jun 29, 2016

serg-meus commented Jun 29, 2016

serg-meus commented Jun 30, 2016 •

edited

Loading

nescitus commented Jun 30, 2016

djdekker commented Jun 30, 2016

djdekker commented Jun 30, 2016

djdekker commented Jun 30, 2016

LocutusOfPenguin commented Jul 1, 2016

serg-meus commented Jul 1, 2016 •

edited

Loading

djdekker commented Jul 1, 2016

LocutusOfPenguin commented Jul 1, 2016

djdekker commented Jul 2, 2016

serg-meus commented Jul 3, 2016

djdekker commented Jul 4, 2016

LocutusOfPenguin commented Jul 5, 2016

serg-meus commented Jul 5, 2016

Problem using python-chess #1

Problem using python-chess #1

Comments

djdekker commented Jun 12, 2016 • edited Loading

niklasf commented Jun 12, 2016 • edited Loading

djdekker commented Jun 27, 2016

serg-meus commented Jun 28, 2016

djdekker commented Jun 28, 2016

serg-meus commented Jun 28, 2016

djdekker commented Jun 28, 2016

serg-meus commented Jun 29, 2016

djdekker commented Jun 29, 2016

serg-meus commented Jun 29, 2016

serg-meus commented Jun 30, 2016 • edited Loading

nescitus commented Jun 30, 2016

djdekker commented Jun 30, 2016

djdekker commented Jun 30, 2016

djdekker commented Jun 30, 2016

LocutusOfPenguin commented Jul 1, 2016

serg-meus commented Jul 1, 2016 • edited Loading

djdekker commented Jul 1, 2016

LocutusOfPenguin commented Jul 1, 2016

djdekker commented Jul 2, 2016

serg-meus commented Jul 3, 2016

djdekker commented Jul 4, 2016

LocutusOfPenguin commented Jul 5, 2016

serg-meus commented Jul 5, 2016

djdekker commented Jun 12, 2016 •

edited

Loading

niklasf commented Jun 12, 2016 •

edited

Loading

serg-meus commented Jun 30, 2016 •

edited

Loading

serg-meus commented Jul 1, 2016 •

edited

Loading