Force launchd plist file to be reloaded when launching watchman #358
Comments
|
I solved it by deleting the |
|
Thx! Works like a charm. |
|
Reopening, leaving it to watchman maintainers to decide if it's a bug or not... |
|
Unfortunately, removing the state dir deleted any logs that we could have used to figure this out, so we have no idea what happened. I'm going to close this because there isn't anything to go on. If this happens for anyone else, please attach the log file from the state dir and we can start troubleshooting from there. |
|
I hit this yesterday with our React Native project - start-packager was hanging and I narrowed it down to watchman. Moving the state dir fixed the problem. If it helps, I'd been changing the system time into the future and back again to test something when the problem started happening, but I have no idea if that's actually connected. |
|
@roefunc yes, please provide the logs as I asked in my prior comment! The excerpt that you included is just how a shutdown request from launchd (pid=1) manifests, so it doesn't tell us anything about why watchman was stopped. It will probably also be useful for you to open Console.app and filter by |
|
This is happening for me (in the context of Ember CLI—see the issue referenced directly above this comment on GitHub); I've attached logs below. Curiously, as you'll note, it's reporting Watchman 4.6.0 in the logs even though I have 4.7.0 installed. There are no messages from watchman in my console logs. Happy to provide any other info you need as well. Running:
/usr/local/var/run/watchman/chris-state/log (fair warning: 190 lines) |
|
I ran into this issue as well. Here's some logs I could gather, hope it has some useful info. Had to mask some of the data, but if any of the maintainers need the actual log I can send it over other channels. |
|
Follow-up after looking at that log: it hasn't stored any data there since I upgraded watchman; did you folks move where it logs to? Or is it just not logging anything on this? |
|
Maybe the problem is that launchd just isn't even starting the 4.7 binary? What are the contents of your |
|
Here's that plist content: <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "https://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.github.facebook.watchman</string>
<key>Disabled</key>
<false/>
<key>ProgramArguments</key>
<array>
<string>/usr/local/Cellar/watchman/4.7.0/libexec/bin/watchman</string>
<string>--foreground</string>
<string>--logfile=/usr/local/var/run/watchman/chris-state/log</string>
<string>--log-level=1</string>
<string>--sockname=/usr/local/var/run/watchman/chris-state/sock</string>
<string>--statefile=/usr/local/var/run/watchman/chris-state/state</string>
<string>--pidfile=/usr/local/var/run/watchman/chris-state/pid</string>
</array>
<key>Sockets</key>
<dict>
<key>sock</key>
<dict>
<key>SockPathName</key>
<string>/usr/local/var/run/watchman/chris-state/sock</string>
<key>SockPathMode</key>
<integer>384</integer>
</dict>
</dict>
<key>KeepAlive</key>
<dict>
<key>Crashed</key>
<true/>
</dict>
<key>RunAtLoad</key>
<true/>
<key>EnvironmentVariables</key>
<dict>
<key>PATH</key>
<string><![CDATA[/Users/chris/.nodenv/versions/5.9.0/bin:/usr/local/Cellar/nodenv/1.0.0/libexec:/Users/chris/.rbenv/shims:/Users/chris/.nodenv/shims:/Users/chris/.pyenv/shims:/Users/chris/bin:/Users/chris/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin]]></string>
</dict>
<key>ProcessType</key>
<string>Interactive</string>
<key>Nice</key>
<integer>-5</integer>
</dict>
</plist>And I found the relevant console logs in the Looks like launchd is trying to hit the old one. Nothing stands out as to why in the launchd plist, though. Could it be an issue with the socket? Aside: I haven't rebooted in that interval. |
|
ok, please try: |
|
|
I think you're now all set. I think we need to more aggressively unload and reinstate the service when we try to launch it via launchd. |
|
@chriskrycho thanks for working through this with me! |
|
No problem! Happy to be able to help! |
|
@chriskrycho I'm trying to figure out how you ended up in that broken state. The spawning logic in watchman is to Does the last timestamp in your 4.6 log (2016-09-09T21:08:17) correspond to roughly when you upgraded watchman? Do you happen to recall running any specific watchman or ember commands around the time that things stopped working? |
No idea whether relevant: the |
wez
added a commit
to wez/watchman
that referenced
this issue
Sep 14, 2016
Summary: in facebook#358 we ended up in a state where the contents of the plist were correct but launchd still thought that it should be monitoring the older 4.6 binary. I don't know how we ended up in such a state, as we unload before we write out the new plist. My working theory is that the unload failed for some unknown reason but we carried on anyway. Also, this nugget from the man page: > NOTE: Due to bugs in the previous implementation and long-standing > client expectations around those bugs, the load and unload subcommands > will only return a non-zero exit code due to improper usage. > Otherwise, zero is always returned. This is a speculative diff to add the `-F` force flag in. I have no way to prove whether this will solve this, only that it doesn't break spawning watchman for me today. Test Plan: with watchman not running and my plist referencing the internal FB build of watchman, build and run watchman from the checkout on my mac as `./watchman version`. Inspect the plist and observe that the executable path has changed from the installed location of the binary to the path to my checkout and that we got a valid watchman version response.
|
Over in #361 @onpaws asked a couple of questions that I think are worth answering, but over here so that we keep a single thread going. I'm paraphrasing the questions to make them easier to answer :-) I'm also going to ask and answer a couple of other questions that give some more context. Why doesn't brew uninstall remove the launchd and other state?Watchman has a slightly unusual model in that it runs a separate server process for each user. This is important because it allows multiple users to be isolated from each other both from a resource usage and also a security perspective. Launchd registration happens for you on-demand and is out of the sight of what homebrew can see today, so uninstallation just doesn't know about them. In addition, the state dir is intended to survive across upgrades: it holds persistent information such as the set of watches that you had before the upgrade so that those can be re-started and recrawled for you when you next start it, and also when you first log in so that those are ready to be queried. Why not keep the state in
|
ghost
pushed a commit
that referenced
this issue
Sep 16, 2016
Summary: in #358 we ended up in a state where the contents of the plist were correct but launchd still thought that it should be monitoring the older 4.6 binary. I don't know how we ended up in such a state, as we unload before we write out the new plist. My working theory is that the unload failed for some unknown reason but we carried on anyway. Also, this nugget from the man page: > NOTE: Due to bugs in the previous implementation and long-standing > client expectations around those bugs, the load and unload subcommands > will only return a non-zero exit code due to improper usage. > Otherwise, zero is always returned. This is a speculative diff to add the `-F` force flag in. I have no way to prove whether this will solve this, only that it doesn't break spawning watchman for me today. Closes #360 Reviewed By: sid0 Differential Revision: D3874674 Pulled By: wez fbshipit-source-id: daa7c1581940d286d601546d1ab892c2f1dbddcc
|
Thank you for the insights. This was causing my Atom Helper process to crash on exit (even with plugins disabled including Nuclide), and also to have high CPU usage when idle. |
|
I see the issue is still open, I experienced this and |
|
Just wanted to chime in and say that Thanks! |
|
Anybody have idea how to fix Details in #371 |
|
@amurchick - Yeah, I'm having the same issue |
|
I've been a bit quiet on this issue. The game plan is to remove socket activation from the launchd registration in the next release to homebrew. We're not quite in a position to do that right now (in the middle of a big refactor), so if you happen to hit this, the recommendation is to run the launchctl unload command to resolve it: |
This was referenced Dec 3, 2016
This was referenced Dec 21, 2016
This was referenced Jan 3, 2017
Closed
|
Note that the -F flag above forces the loading of the plist, ignoring the disable key. I ran the command without -F and it worked fine, just like it did for others. Opinions on what's better? |
facebook-github-bot
pushed a commit
that referenced
this issue
Jan 23, 2017
Summary: This will help prevent a situation like #358 from having a similar impact in the future. Launchd will still launch watchman at startup and keep it running if it falls over, but should get out of the way when we upgrade in the future. I'll look at adding some explicit unloading of the plist at installation time in the homebrew recipe. Reviewed By: sid0 Differential Revision: D4417672 fbshipit-source-id: b4d4597107f26f2e4e51ecd540dfb36586cfa44c
wez
added a commit
to wez/homebrew-core
that referenced
this issue
Jan 30, 2017
This is taking a run at: facebook/watchman#358 Watchman installs and updates its own launchd.plist file on behalf of the user; it is not integrated with `brew services`. In the linked issue, watchman would stop working for some homebrew users. The problem was that the launchd.plist used socket activation to set up the service, and because launchd is maintaining the socket, and the socket connection attempt just hangs, watchman never had a chance to fix up the plist and things stayed broken. To help solve this, we're forcing an unload in the post-install portion of the recipe.
4 tasks
MikeMcQuaid
pushed a commit
to Homebrew/homebrew-core
that referenced
this issue
Feb 10, 2017
This is taking a run at: facebook/watchman#358 Watchman installs and updates its own launchd.plist file on behalf of the user; it is not integrated with `brew services`. In the linked issue, watchman would stop working for some homebrew users. The problem was that the launchd.plist used socket activation to set up the service, and because launchd is maintaining the socket, and the socket connection attempt just hangs, watchman never had a chance to fix up the plist and things stayed broken. To help solve this, we're forcing an unload in the post-install portion of the recipe.
|
I still have this issue - no solution so far. |
|
This issue seems to be present when you have your home folder on a different (external?) drive from your system. At least with me macOS High Sierra selected "Ignore ownership on this volume" for the external drive. By disabling this odd feature the "Caller specified a plist with bad ownership/permissions" issue was solved. Links |
|
I am facing the same issue with latest build I have this Version And Facing This Issue 2020-12-16T12:11:33,518: [cli] Failed to spawn watchman server; it exited with code 3. Watchman: watchman --no-pretty get-sockname returned with exit code=1, signal=null, stderr= 2020-12-16T12:11:33,518: [cli] Failed to spawn watchman server; it exited with code 3. |
|
I'm also running into this problem. I'm on watchman version I'm using the latest Followed all steps shown here: https://create-react-app.dev/docs/troubleshooting none of them resolve my problem. |
|
Had the same problem with watchman hanging, I noticed that it started happening randomly and my react-native project stopped building, it was frozen, it took me forever to figure out that watchman was the culprit, even though I've reinstalled watchman multiple times. Version: 2021.06.07.00 I am lucky it only took 4 hours to find the problem & solution, the fix was to delete the state folder: /usr/local/var/run/watchman/chris-state/ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
I'm facing a bug with watchman.
Whenever I'm starting watchman and sending a command (such as
watchman watch-listor eventwatchman versionor creating a trigger, it's getting stuck.watchman --help & watchman -v work though.
I've been using watchman for months without troubles.
Yesterday I've enabled filevault on my mac and I upgraded watchman to 4.7.0 using brew.
I've reinstalled 4.6.0 from the sources, I'm observing the same problem.
Any advices on how I could provide you more info / logs on this issue ?
Regards,
Florent
The text was updated successfully, but these errors were encountered: