Commits

Commits on Sep 11, 2023

metal : PP speedup (ggerganov#3084 )

committedSep 11, 2023

Commits on Sep 8, 2023

metal : Q3_K speedup (ggerganov#2995 )

ikawrakow
and
Kawrakow
committedSep 8, 2023

Commits on Sep 7, 2023

metal : parallel RoPE on Metal (ggerganov#3024 )

ikawrakow
and
Kawrakow
committedSep 7, 2023
metal : correct fix of kernel_norm (ggerganov#3060 )

committedSep 7, 2023

Commits on Sep 5, 2023

Guard against all weights in a super-block being zero (ggerganov#3010 )

ikawrakow
and
Kawrakow
committedSep 5, 2023

Commits on Sep 3, 2023

metal : more optimizations (ggerganov#2959 )

committedSep 3, 2023

Commits on Sep 1, 2023

metal: somewhat faster f16 x f32 matrix multiply kernel (ggerganov#2951 )

ikawrakow
and
Kawrakow
committedSep 1, 2023

Commits on Aug 29, 2023

Tell users attmepting to run perplexity with too few tokens to use more (ggerganov#2882 )

ikawrakow
and
Kawrakow
committedAug 29, 2023
10X faster BPE tokenizer (ggerganov#2876 )

ikawrakow
and
Kawrakow
committedAug 29, 2023

Commits on Aug 27, 2023

llama : speedup tokenization (ggerganov#2831 )

ikawrakow
and
Kawrakow
committedAug 27, 2023
k_quants tuning for Falcon-7b (ggerganov#2816 )

ikawrakow
and
Kawrakow
committedAug 27, 2023

Commits on Aug 26, 2023

Better perplexity for 2- and 3-bit quantization for LLaMA-v2-70B (ggerganov#2807 )

ikawrakow
and
Kawrakow
committedAug 26, 2023
Fix HellaSwag (ggerganov#2805 )

ikawrakow
and
Kawrakow
committedAug 26, 2023

Commits on Aug 25, 2023

Faster perplexity computation (ggerganov#2786 )

ikawrakow
and
Kawrakow
committedAug 25, 2023

Commits on Aug 23, 2023

Fix values shown in the quantize tool help (ggerganov#2735 )

ikawrakow
and
Kawrakow
committedAug 23, 2023
Strided perplexity (ggerganov#2714 )

ikawrakow
and
Kawrakow
committedAug 23, 2023

Commits on Aug 22, 2023

Quantization imrovements for k_quants (ggerganov#2707 )

ikawrakow
and
Kawrakow
committedAug 22, 2023

Commits on Aug 21, 2023

HellaSwag: split token evaluation into batches if needed (ggerganov#2681 )

ikawrakow
and
Kawrakow
committedAug 21, 2023

Commits on Aug 20, 2023

More efficient Hellaswag implementation (ggerganov#2677 )

ikawrakow
and
Kawrakow
committedAug 20, 2023

Commits on Jul 25, 2023

Add LLAMA_DEFAULT_RMS_EPS so we can change the default (ggerganov#2384 )

ikawrakow
and
Kawrakow
committedJul 25, 2023
Another speed gain for Q4_0 and Q4_1 on Metal (ggerganov#2375 )

ikawrakow
and
Kawrakow
committedJul 25, 2023
Fix Q4_K and Q5_K for QK_K = 64 on CUDA (ggerganov#2359 )

ikawrakow
and
Kawrakow
committedJul 25, 2023

Commits on Jul 24, 2023

Fix scalar version of Q5_K when QK_K = 64 (ggerganov#2362 )

ikawrakow
and
Kawrakow
committedJul 24, 2023

Commits on Jul 23, 2023

Some more Q4_K and Q5_K speedup on CUDA (ggerganov#2346 )

ikawrakow
and
Kawrakow
committedJul 23, 2023
Speed up Q4_K (ggerganov#2322 )

ikawrakow
and
Kawrakow
committedJul 23, 2023

Commits on Jul 21, 2023

Custom RoPE + bettter memory management for CUDA (ggerganov#2295 )

ikawrakow
and
Kawrakow
committedJul 21, 2023
Faster Q3_K implementation on Metal (ggerganov#2307 )

ikawrakow
and
Kawrakow
committedJul 21, 2023
Faster Q2_K on Metal (ggerganov#2297 )

ikawrakow
and
Kawrakow
committedJul 21, 2023

Commits on Jul 20, 2023

Faster Q5_K and Q6_K on Metal (ggerganov#2294 )

ikawrakow
and
Kawrakow
committedJul 20, 2023
Faster Q4_K on Metal (ggerganov#2290 )

ikawrakow
and
Kawrakow
committedJul 20, 2023

Commits on Jul 14, 2023

Metal: faster Q4_0 and Q4_1 matrix x vector kernels (ggerganov#2212 )

ikawrakow
and
Kawrakow
committedJul 14, 2023

Commits on Jun 26, 2023

k-quants : support for super-block size of 64 (ggerganov#2001 )

ikawrakow
and
Kawrakow
committedJun 26, 2023

Commits on Jun 19, 2023

llama : only use Q6_K for output weights if tensor size is multiple of 256 (ggerganov#1932 )

ikawrakow
and
Kawrakow
committedJun 19, 2023
cuda : faster k-quants on older GPUs (ggerganov#1930 )

ikawrakow
and
Kawrakow
committedJun 19, 2023

Commits on Jun 18, 2023

llama : prevent usage of k-quants when tensor size is not a multiple of 256 (ggerganov#1921 )

ikawrakow
and
Kawrakow
committedJun 18, 2023