karpathy llm.c · Discussions · GitHub

llm.c discussions
General karpathy

Discussions

You must be logged in to vote

Reproducing GPT-2 (124M) in llm.c in 90 minutes for $20
notable
karpathy started May 28, 2024 in General

117
You must be logged in to vote

LLMs on any GPU: True Open Source to run LLMs

githubuser-64 started Jul 5, 2024 in Polls

5
You must be logged in to vote

Intra-document masking - cross contamination and distractions during pre-training

biuq started Jul 16, 2024 in Ideas

0
You must be logged in to vote

Let's reproduce GPT-2 (1.6B): one 8XH100 node, 24 hours, $672, in llm.c

karpathy started Jul 11, 2024 in General

27
You must be logged in to vote

How to train a little dataset ? What format to use and how to call train_gpt2 executable to generate the LLM ? Then, how to test it ?

FiveTechSoft asked Jul 13, 2024 in Q&A · Unanswered

3
You must be logged in to vote

No dropout in this version

Phil-U-U asked Jul 9, 2024 in Q&A · Unanswered

0
You must be logged in to vote

Surpassing(?) FineWeb-edu with the DCLM-Baseline Dataset

robbiegwald started Jul 1, 2024 in Show and tell

1
You must be logged in to vote

Understand this entire codebase with the help of a custom GPT [LLM-C by Andrej Karpathy]

ehzawad started Apr 17, 2024 in General

4
You must be logged in to vote

running CUDA on mac or any non-nvidia gpu

vinayakarsh asked Apr 22, 2024 in Q&A · Closed · Unanswered

1
You must be logged in to vote

Intel SYCL standard runtime support with oneAPI

abhilash1910 started Jun 7, 2024 in Ideas

6
You must be logged in to vote

GPT-2 (774M) reproduced

karpathy started Jun 12, 2024 in General

9
You must be logged in to vote

Translator layer to cut on vocab size

IlyaGazman started Jun 13, 2024 in General

1
You must be logged in to vote

How to solve the dx of the /doc/layernorm.py?

slothfull asked Jun 18, 2024 in Q&A · Unanswered

0
You must be logged in to vote

Self-attention in O(N) while still letting all the tokens to talk to each other

IlyaGazman started Jun 13, 2024 in General

0
You must be logged in to vote

CPU: llm.c vs Pytorch Benchmarks

dagelf started Apr 25, 2024 in General

3
You must be logged in to vote

Memory, tokens per sec, and MFU behavior in train_gpt2cu

chinthysl started May 31, 2024 in Show and tell

1
You must be logged in to vote

PyTorch vs. llm.c cross-checks
notable
karpathy started May 24, 2024 in General

2
You must be logged in to vote

About amd gpu Issue #527

Lookforworld started Jun 3, 2024 in General · Closed

3
You must be logged in to vote

Is that possible to modify this model to do language translation?

code-cp started May 30, 2024 in General

1
You must be logged in to vote

How to move from a continuation model to a chat model?

antonkratz started May 30, 2024 in General

2
You must be logged in to vote

State of the Union [April 22, 2024]

karpathy started Apr 22, 2024 in General

2
You must be logged in to vote

llm.c discussions

karpathy started Apr 12, 2024 in General

40
You must be logged in to vote

Compared to academic Python or CUDA, pure C need more attention and development

bugwang started May 23, 2024 in General · Closed

0
You must be logged in to vote

Made a video walkthrough on how to run the CUDNN version from scratch

ChrisDryden started May 10, 2024 in Show and tell

1
You must be logged in to vote

Electro-swing version of train_gpt2.c 🎵 🎶

dagelf started May 21, 2024 in Show and tell

0