[MetaSchedule] Restore num_threads parameter in tuning API

[masahi]

num_threads parameter in the Relay tuning API was (accidentally?) removed in #12895. This PR restores this parameter and also uses it consistently for XGB model training and builder / runner as well.

@zxybazh

[tvm-bot]

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

• cc @Hzfengsy, @elvin-n, @junrushao _{See #10317 for details}
• Built docs for commit 71452b5 can be found here.

_{Generated by tvm-bot}

[junrushao]

I am not sure it should be a top-level parameter as it’s less frequently used by introductory-level users, but don’t have much strong opinions.

In the meantime, I’m not sure if using the same num_threads for xgb and for evo search is a good idea because I assume xgb prefers physical cores? I don’t have numbers handy so I’m not 100% sure.

Removed this because @spectrometerHBH @Hzfengsy @jinhongyii @MasterJH5574 suggested to, so I’d love to hear more from you guys.

[spectrometerHBH]

I would suggest using a better name because num_threads can be confusing, especially when the TIR programs to be tuned are using parallelization.

[masahi]

The main use case of this param is for tuning on a high-core system shared by many users. Currently if one user starts tuning, it occupies all CPU resources, which disrupts other users. So the goal is to limit the number of cores used by MS throughout the tuning process (evo search, post order apply, XGB training, builder / runner).

Also, tune_tir API has num_threads param as well.

tvm/python/tvm/meta_schedule/tir_integration.py

Line 57 in 6780c9f

num_threads: Union[Literal["physical", "logical"], int] = "physical",

I would suggest using a better name because num_threads can be confusing

Agreed, but that's what TuneContext calls... I can replace num_threads in the high-level API with max_workers or something, and initialize TuneContext by num_threads=max_workers. If people think this is better I can do that, otherwise I'd keep the existing convention.

[zxybazh]

I'm kinda in favor of this change because it allows more customization. We may want to find a good naming and default value for num_threads though.

[junrushao]

I don't have a clear idea (very bad at naming). Perhaps @tqchen you could suggest?

[masahi]

I'm going to go with num_tuning_cores per discussion with @zxybazh, if there is no objection. For now I'll keep num_threads in TuneContext.

[junrushao]

The reason that I don’t like “cores” is that they are threads, which are not necessarily related to physical cpu cores. Perhaps num_threads makes more sense at the moment.

[zxybazh]

Just want to add that Runner & Builder are using new processes, that's why we may want to consider using other terminology. On the other hand, if we don't have very good idea on new naming, I don't mind just sticking to the current naming nun_threads before that.

[masahi]

I think "cores" is better because we are also using this parameter to set the number of workers used by builder / runner. Also, when a user wants to limit the amount of CPU resources used by MS, they would think in terms of the number of "cores", not "threads". So as an API, "core" sounds more intuitive to me.

[junrushao]

The argument that builder/runner workers are using processes rather than threads makes sense to me. Thanks for the explanation! Then I'm happy with the "core" terminology

[masahi]

Replaced num_threads with num_tuning_cores in the API.

[zxybazh]

LGTM

[junrushao]

LGTM