CPU Launcher fails available check with venv

[ElfoLiNk]

🐛 Describe the bug

cpu launcher doesn't work

Error logs

2024-03-28T08:00:31,792 [DEBUG] W-9000-embeddings_1.1 org.pytorch.serve.wlm.WorkerLifeCycle - launcherAvailable cmdline: [python, -m, torch.backends.xeon.run_cpu, --use_logical_core, --enable_jemalloc, --no_python, hostname]
2024-03-28T08:00:31,821 [WARN ] W-9000-embeddings_1.1 org.pytorch.serve.wlm.WorkerLifeCycle - torch.backends.xeon.run_cpu is not available. Proceeding without worker core pinning. For better performance, please make sure torch.backends.xeon.run_cpu is available.

launcherAvailable cmdline is running outside the virtual env of torchserve like the below command which fails

python -m torch.backends.xeon.run_cpu --use_logical_core --no_python hostname
/usr/bin/python: Error while finding module specification for 'torch.backends.xeon.run_cpu' (ModuleNotFoundError: No module named 'torch')

Instead using the virtual env where the torchserve is running works

[rocky@hostname ~]$ source /opt/venv-torchserve/bin/activate
(venv-torchserve) [rocky@hostname ~]$ python -m torch.backends.xeon.run_cpu --use_logical_core --no_python hostname
2024-03-28 08:04:22,935 - __main__ - INFO - Use TCMalloc memory allocator
2024-03-28 08:04:22,936 - __main__ - INFO - OMP_NUM_THREADS=4
2024-03-28 08:04:22,936 - __main__ - INFO - Using Intel OpenMP
2024-03-28 08:04:22,936 - __main__ - INFO - KMP_BLOCKTIME=1
2024-03-28 08:04:22,936 - __main__ - INFO - LD_PRELOAD=/opt/venv-torchserve/lib/libiomp5.so:/usr/lib64/libtcmalloc.so
2024-03-28 08:04:22,936 - __main__ - INFO - numactl -C 0-3 -m 0 hostname

Installation instructions

Torchserve 0.10.0 installed in a virtual env with pip

Model Packaging

not applicable

config.properties

ipex_enable=true
cpu_launcher_enable=true
cpu_launcher_args=--use_logical_core

Versions

TorchServe 0.10.0
Python 3.9

Repro instructions

Start a torchserve from a virtual env with cpu_launcher_enable true

Possible Solution

i think https://github.com/pytorch/serve/blob/v0.10.0/frontend/server/src/main/java/org/pytorch/serve/wlm/WorkerLifeCycle.java#L69 should be updated to use python environment used by the running torchserve

[lxning]

@ElfoLiNk ipex_enable=true should be set. Please check https://github.com/pytorch/serve/blob/master/examples/intel_extension_for_pytorch/README.md#torchserve-with-launcher

[ElfoLiNk]

Hi @lxning, I already have the ipex_enable=true. I updated the bug adding more info

[agunapal]

cc @min-jean-cho