question about bits_pattern feature

[harrysyz99]

Feature request

The bits_pattern function is crucial for assigning different quantization levels to each layer of a neural network. This flexibility is essential for optimizing the performance and efficiency of models, especially in resource-constrained environments, I want to know how you plan to achieve such a feature.

Motivation

I propose implementing the bits_pattern function to allow users to specify different quantization bits for each layer in a neural network.

Your contribution

I could help with implementing the function

[BenjaminBossan]

Could you give further references to this feature? I haven't seen it anywhere.

It seems like you talk about weight quantization. Note that in general, only the weights of the base model are quantized. The adapter weights used by PEFT are not quantized, since it is intended that they be trained. Therefore, this method you describe should be implemented on the level of the base model, so e.g. on libraries like transformers.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.