Skip to content

Commit

Permalink
Add ability to use importance matrix for all k-quants (#4930)
Browse files Browse the repository at this point in the history
Co-authored-by: Iwan Kawrakow <[email protected]>
  • Loading branch information
ikawrakow and Kawrakow committed Jan 14, 2024
1 parent bb0c139 commit 467a882
Show file tree
Hide file tree
Showing 4 changed files with 462 additions and 16 deletions.
2 changes: 1 addition & 1 deletion examples/quantize/quantize.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ static void usage(const char * executable) {
printf(" --allow-requantize: Allows requantizing tensors that have already been quantized. Warning: This can severely reduce quality compared to quantizing from 16bit or 32bit\n");
printf(" --leave-output-tensor: Will leave output.weight un(re)quantized. Increases model size but may also increase quality, especially when requantizing\n");
printf(" --pure: Disable k-quant mixtures and quantize all tensors to the same type\n");
printf(" --imatrixfile_name: use data in file_name as importance matrix for quant optimizations\n");
printf(" --imatrix file_name: use data in file_name as importance matrix for quant optimizations\n");
printf(" --include-weights tensor_name: use importance matrix for this/these tensor(s)\n");
printf(" --exclude-weights tensor_name: use importance matrix for this/these tensor(s)\n");
printf("Note: --include-weights and --exclude-weights cannot be used together\n");
Expand Down
Loading

0 comments on commit 467a882

Please sign in to comment.