Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[k8s] Add checks for shuf dependency #3009

Open
romilbhardwaj opened this issue Jan 21, 2024 · 4 comments
Open

[k8s] Add checks for shuf dependency #3009

romilbhardwaj opened this issue Jan 21, 2024 · 4 comments
Labels
good first issue Good for newcomers k8s Kubernetes related items

Comments

@romilbhardwaj
Copy link
Collaborator

shuf is used for avoiding concurrent socat connections (#2628). However, some environments may not have shuf installed:

Tailing logs of job 1 on cluster 'redacted'...
/Users/redacted/.sky/port-forward-proxy-cmd.sh: line 48: shuf: command not found
usage: sleep seconds
Warning: Permanently added '127.0.0.1' (ED25519) to the list of known hosts.
Warning: Permanently added '10.42.56.141' (ED25519) to the list of known hosts.
I 01-21 17:51:18 log_lib.py:431] Start streaming logs for job 1.
INFO: Tip: use Ctrl-C to exit log streaming (task will not be killed).

We should add dependency checks for shuf just like we have for nc and socat.

@romilbhardwaj
Copy link
Collaborator Author

Also, shuf may get installed as gshuf on some macos environments. Since shuf and gshuf are fungible, we should auto-detect and use whichever is available.

@romilbhardwaj romilbhardwaj added k8s Kubernetes related items good first issue Good for newcomers labels Jan 21, 2024
@ghost ghost mentioned this issue Jan 28, 2024
5 tasks
Copy link

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.

@dongreenberg
Copy link
Contributor

Bumping, ran into the shuf issue today (on Mac)

@romilbhardwaj
Copy link
Collaborator Author

Thanks @dongreenberg - our dependency on shuf will be removed after #3657.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers k8s Kubernetes related items
Projects
None yet
Development

No branches or pull requests

2 participants