Let's support Windows

[Alexander-Shukaev]

I finally had time to test everything on Windows. First of all, in Vim YCM works just fine here, so the following problems are most likely related only to your package and not YCM on Windows in general.

I use company-ycmd and I keep getting:

Ycmd completion unavailable while parsing is in progress.

Do you have any ideas why?

ycmd-extra-conf-handler documentation says that it will by default ask whether to load .ycm_extra_conf.py, but it never does like if it cannot find it. In Vim it searches for it bottom-up starting from the directory of the currently edited source file. Is the same true in your case? If yes, then why is it not locating .ycm_extra_conf.py?

[abingham]

It sounds like you're experiencing two independent problems (though I may be wrong.) First, the perpetual parsing problem...

One possibility is that ycmd.el is not properly determining when parsing completes. In ycmd-notify-file-ready-to-parse we set the variable ycmd--notification-in-progress to t when we send he HTTP request to ycmd, and we reset it to nil when the request ends. If that variable never gets reset for some reason, then we'll always think that ycmd is busy parsing some buffer, and we'll print the message you report.

It's also possible, of course, that ycmd is just legitimately always busy parsing. This might mean that ycmd.el is asking it to parse too often. You can fiddle with ycmd-parse-conditions to change the conditions under which ycmd.el will send a parse request. That might make it easier to debug this issue. You can also watch ycmd's activity in the *ycmd-server* buffer.

[abingham]

For the ycmd-extra-conf-handler issue, the first thing to check is the actual value of ycmd-extra-conf-handler. If it's set to 'ask then it should be asking you, but if it's set to 'ignore then it won't.

It's also possible that all of the .ycm_extra_conf.pys that ycmd is encountering are in your whitelist, ycmd-extra-conf-whitelist. In that case, ycmd will simply load them without asking as well. Check the value of that list and see if perhaps that's the reason you're seeing any requests.

If you check everything and it still look like you should be getting load requests, the next thing to check is the HTTP traffic to ycmd. Set ycmd--log-enabled to non-nil and open a file with an associated .ycm_extra_conf.py. Then in the buffer *ycmd-content-log* you should see the contents of all of the HTTP requests and responses between emacs and ycmd. In the response to /event_notification check to see if you see anything about loading extra-conf files.

[abingham]

But just to be clear, it's ycmd itself – not ycmd.el – that does the actual searching for .ycm_extra_conf.py. ycmd.el just sends over the whitelist and provides a means to tell ycmd (possibly after querying the user) when to load non-whitelisted extra-confs.

[Syndim]

Hi, I found that on Windows emacs will freeze when opening any cpp file...

I added the following lines into my config:

(custom-set-variables
`(ycmd-server-command ("C:/Python27/python.exe" "C:/Users/username/.emacs.d/repos/ycmd/ycmd")))

It seems that the python process is started and listening to a local port but the ycmd server is not started correctly(If it started correctly I will get a 404 when hitting localhost:port but I can't hit it when it's stared by emacs). I already put the pyd and libclang.dll files into the ycmd folder, do you know what's happening here?

[abingham]

Interesting. If you run that command (i.e. C:/Python27/python.exe C:/Users/username/.emacs.d/repos/ycmd/ycmd) manually from the command prompt, what does it print?

[Syndim]

Sorry I just check the behavior again, the ycmd server does started correctly by emacs(I must have seen the wrong port yesterday), however the emacs window freezes...

I did some experiments yesterday and it seems that the buffer is always empty so I guess there might be the problem with the buffer?

[abingham]

OK, thanks for following up on that. I don't have a windows machine available for testing, so I'll have to work through you a bit to try to get this sorted out.

First, when you say that "the buffer is always empty", do you mean the *ycmd-server* buffer?

Second, when you say that emacs freezes, do you mean that it it becomes completely unresponsive and that you have to kill it? Or are you able to use C-g to terminate the ycmd commands?

If you're able to terminate the ycmd command, then we might be able to debug this. One simple thing you can do is to print out information to the message buffer. (You mentioned that you're not familiar with elisp, so the command for printing like that is (message "some useful information"). If you could start by instrumenting ycmd-open and ycmd--start-server, that would help. It will be especially helpful to know if we ever exit those functions.

[abingham]

@r4nt Have any of your users tried running this on windows?

[Syndim]

Hi, thanks for your help.

The buffer I mean is the "proc-buff" in line 806: (proc (apply #'start-process ycmd--server-process proc-buff server-program+args)), I think it's the same as *ycmd-server*, right?

I followed your instructions to add (message ...) to those 2 functions, and also print the server-program+args and the proc-buff, below is the result:

ad-handle-definition: `evil-mode' got redefined
Loading paren...done
For information about GNU Emacs and the GNU system, type C-h C-a.
ycmd-open start
ycmd--start-server start
server-program+args:
(C:/Python27/python.exe C:/Users/username/.emacs.d/repos/ycmd/ycmd --options_file=c:/Users/usernmae/AppData/Local/Temp/ycmd-options9820YK0 --log=debug --keep_logfile --idle_suicide_seconds=10800)
proc-output: [3 times]
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

proc-output:
2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload

Quit

As you can see the "serving on..." line is missing.

I also did a search in ymcd source code, and found that the "DEBUG" line is printed by log.debug, but the "serving" line is printed by the print function, so I think they may be printed to different places on Windows.

[r4nt]

@abingham - the only people I know who develop on Windows use VS.

[abingham]

@Syndim Yes, proc-buff and *ycmd-server* are the same.

Since the proc-output you're printing seems to contain normal ycmd.el output, it seems that ycmd is being started and that emacs is getting responses from it. It's strange that proc-output has content but that *ycmd-server* is empty since – as I understand these things – proc-output is supposed to just contain the contents of *ycmd-server*.

I'm grasping at straws a bit here, but one thing you could try is to take the last argument off of the call to accept-process-output. Make it into:

(accept-process-output proc 0 100)

I'm just guessing here, but perhaps there's some windows magic happening here.

In any event, I'll keep thinking about this. Without a windows machine this is quite hard to debug, so thanks for working with me on this!

[Syndim]

@abingham Sorry I didn't make it clearly, the proc-output prints exactly the same content as the *ycmd-server*, I said it's empty because at that time I provided the global config path so ycmd didn't print the debug message.(For now if I run the command in the cmd.exe, it will print 2 lines:

2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload
serving on: http:https://127.0.0.1:47389

)

Now I know why emacs window is freezing(since it tries to read the non-exist output for 3000 times). I think the problem is in reading the output from ycmd server.

I changed code as you mentioned but it still not working :(

I will have more investigation today when I'm off work.

Thanks a lot to create the plugin, this plugin is the reason I want to use emace(again), I'd like to do everything I can to help resolve this problem:)

[abingham]

Ah ha! I think I might see a problem :) The "serving on" line in your *ycmd-server* buffer seems to have an extra colon. That is, you're seeing:

serving on: http:https://127.0.0.1:47389

but ycmd.el is looking for:

serving on http:https://127.0.0.1:47389

i.e. without the colon after "on".

We can try testing this by updating ycmd.el. In ycmd--start-server, try changing this call:

(string-match "^serving on http:https://.*:\\\([0-9]+\\\)$" proc-output)

to

(string-match "^serving on: http:https://.*:\\\([0-9]+\\\)$" proc-output)

If that fixes things, then we need to figure out why there's an extra colon in your ycmd output!

[Syndim]

I was typing the message by hand so I accidentally added the colon... Sorry that I mislead you...

Let me summarize the problem here:

when ycmd is started by hand, it works fine and prints the following lines:

2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload
serving on http:https://127.0.0.1:47389

However when it's started by emacs, the serving on http:https://127.0.0.1:47389 line is missing in buffer for some reason

So ycmd.el keeps trying to read the buffer and search for that string for 3000 times, and so the emacs window is freezing for a long time and then prints the "Time out" error message.

The 2015-01-21 14:09:28,769 - DEBUG - No global extra conf, not calling method YcmCorePreload line is printed by log.debug (link to source code)

And the serving on http:https://127.0.0.1:47389 line is printed by the print function(link to source code), so my guess is that the accept-process-output function can't read the output printed by the print function.

I'll do some more experimentation today, hopefully I can find some useful information...

[Syndim]

I just did a quick experimentation, I changed the code from using the print function to use logging.debug, and the ycmd.el can read the serving on http:https://127.0.0.1:47389 output now...

[abingham]

You mean that you changed the code in ycmd's third-party/waitress? That's interesting. It definitely sounds like you're right about accept-process-output. I assume print() is writing to stdout and that logging.debug() is writing to stderr. I wonder if the stdout on windows is simply not being flushed.

I wonder: if you put sys.stdout.flush() after the print("serving on...") call in ycmd, does that make thing work?

[Syndim]

After adding sys.stdout.flush(), ycmd.el starts normally. You are right that stdout is not being flushed on Windows :(

[abingham]

We might be able to work around this for the short term. According to this stackoverflow answer, you might be able to make Python flush more quickly with this elisp code:

(setenv "PYTHONUNBUFFERED" "x")

I think if you put that near the front of your emacs configuration, the problem might go away. Or you might just be able to set that in your outer environment.

[abingham]

@Valloric To bring you up to speed: We're facing a problem on Windows where waitress is not printing it's "serving on http:https://host:port" in a timely manner. This appears to be related to flushing of stdout, block vs. line buffering, and plumbing like that. If that line isn't printed, then ycmd.el doesn't know how to talk to ycmd. Do you have any insight/input on this?

[Syndim]

I found that I can also add -u to the ycmd-server-command to disable python's buffering for stdout, (stackoverflow link)

Thank you again for creating this tool!

[abingham]

Ah, brilliant! I should make that the default behavior, I suppose. I'll also add note to the README. Does this get you to a state where you can use the tool then?

[Syndim]

Yes, the tool is working fine after I provided the global config file.

But I found that the company mode is slower than vim + ycm :(

[Valloric]

We're facing a problem on Windows where waitress is not printing it's "serving on http:https://host:port" in a timely manner.

I don't have a clue. I only boot into Windows to play Steam games. :D