-
Notifications
You must be signed in to change notification settings - Fork 335
Insights: pytorch/torchtune
Overview
Could not load contribution data
Please try again later
12 Pull requests merged by 6 people
-
Allow snapshot.pickle to be saved for all recipes with profiler support
#1384 merged
Aug 22, 2024 -
Update torch checks for preprocess
#1377 merged
Aug 22, 2024 -
Remove torchao-nightly
#1389 merged
Aug 22, 2024 -
enable customize activation functions in clip vision encoder
#1385 merged
Aug 21, 2024 -
When profile_memory, also export and save the snapshot.pickle for lora_finetune_single_device.py
#1382 merged
Aug 21, 2024 -
Undo commenting out NF4/bitsandbytes tests
#1370 merged
Aug 20, 2024 -
Add system prompt
#1366 merged
Aug 20, 2024 -
Fix max_seq_len bug in Qwen2 tokenizer
#1375 merged
Aug 20, 2024 -
[STOPGAP] Remove public APIs for vision utils
#1367 merged
Aug 19, 2024 -
Adding validation to
column_map
in messages.py#1368 merged
Aug 19, 2024 -
Adding classifier checkpointing utils - renaming
peft/peft_utils.py
#1313 merged
Aug 19, 2024 -
Update CodeLlama configs (and fix a couple Phi3 ones)
#1358 merged
Aug 17, 2024
7 Pull requests opened by 6 people
-
Improve training UX with TPS, GPU peak mem %, cudaMalloc retries, and do it in color for pizzaz
#1360 opened
Aug 17, 2024 -
Adds max_seq_len to recipes and updates unit tests
#1363 opened
Aug 18, 2024 -
Fix KV cache
#1364 opened
Aug 19, 2024 -
Fix dtype error during quantized generation
#1371 opened
Aug 19, 2024 -
expose rope scale_factor to the builder
#1386 opened
Aug 21, 2024 -
chunked cross entropy
#1390 opened
Aug 22, 2024 -
Update quantization to use tensor subclasses
#1403 opened
Aug 23, 2024
30 Issues closed by 8 people
-
Gemma2B missing lm head weight?
#1062 closed
Aug 23, 2024 -
FlashAttention for `TransformerDecoder`
#1380 closed
Aug 23, 2024 -
Enable compile for batched RLHF generation utils
#1402 closed
Aug 23, 2024 -
Add back in `lm_head.weight` in Qwen2 after training
#1381 closed
Aug 21, 2024 -
Is Custom Eleuther Eval supported?
#1376 closed
Aug 21, 2024 -
CUDA illegal memory access exception
#1112 closed
Aug 21, 2024 -
Missing non-LoRA key tok_embeddings.weight from base model dict
#1110 closed
Aug 21, 2024 -
How to provide our own daaset.json or csv file to the llama2:7B finetune using tune run
#1099 closed
Aug 21, 2024 -
Can I just get the loss on validation and test set?
#1066 closed
Aug 21, 2024 -
[Feature Request] Add lr_scheduler for full_finetune (single_device/distributed)
#1060 closed
Aug 21, 2024 -
No lr_scheduler in full_finetune_distributed
#952 closed
Aug 21, 2024 -
Refactor InstructTemplate and ChatFormat
#1044 closed
Aug 21, 2024 -
Configurable sharding strategy for distributed finetune
#1014 closed
Aug 21, 2024 -
Llama-3 Inference and Uploading to Huggingface
#931 closed
Aug 21, 2024 -
Test directory structure for models doesn't match corresponding implementation files
#905 closed
Aug 21, 2024 -
ImportError: cannot import name 'return_and_correct_aliasing' on Kaggle notebook
#806 closed
Aug 21, 2024 -
Out of CUDA on 15 GB colab . Just trying to train Mistral 7. v1
#665 closed
Aug 21, 2024 -
Finetuning whisper
#556 closed
Aug 21, 2024 -
Reproducing benchmark numbers
#1365 closed
Aug 21, 2024 -
following tutorial for llama3 - missing key device in eleuther_eval
#1379 closed
Aug 20, 2024 -
Allow user to easily add a system message to each message
#1328 closed
Aug 20, 2024 -
`InstructDataset` currently does not take into account `max_seq_len`
#1374 closed
Aug 20, 2024 -
Add validation to `ToMessages` classes `column_map` params
#1327 closed
Aug 19, 2024 -
Rename `peft/_peft_utils.py` to `peft/_utils.py`
#1300 closed
Aug 19, 2024 -
Can't do tune run with QWEN 2 model
#1359 closed
Aug 17, 2024 -
Rework code-llama configs to follow standard format
#1332 closed
Aug 17, 2024 -
Modify ChatML chat format to support generation
#1194 closed
Aug 16, 2024 -
Qwen2 chat template
#1345 closed
Aug 16, 2024 -
Decouple dataset from prompt template
#1343 closed
Aug 16, 2024 -
`Qwen2Tokenizer` does not work with `SFTDataset`
#1353 closed
Aug 16, 2024
19 Issues opened by 12 people
-
Improve the torchtune docs index page
#1400 opened
Aug 23, 2024 -
error saving checkpoint when model is compiled
#1399 opened
Aug 23, 2024 -
This QLoRA config makes the model initialize with 15GB, instead of 6GB
#1398 opened
Aug 23, 2024 -
Dataset class for pre-tokenized data with `input_ids` and `labels` keys.
#1397 opened
Aug 23, 2024 -
Add filter_fn to text_completion_dataset
#1396 opened
Aug 22, 2024 -
[RFC] RLHF follow-ups
#1395 opened
Aug 22, 2024 -
text_completion_dataset: RuntimeError: each element in list of batch should be of equal size
#1394 opened
Aug 22, 2024 -
Add configurable system prompt in generate
#1393 opened
Aug 22, 2024 -
Use PromptTemplate in generate recipe
#1392 opened
Aug 22, 2024 -
Message validation in SFTDataset
#1391 opened
Aug 22, 2024 -
Streamline/better documentation for torchtune -> transformers workflow
#1388 opened
Aug 21, 2024 -
Error for inference LoRA Llama3-8b in Python Script
#1387 opened
Aug 21, 2024 -
Tracker for Multimodal requests
#1383 opened
Aug 21, 2024 -
In place operations in backward pass cause errors in asynchronous concurrent training in FSDP
#1378 opened
Aug 20, 2024 -
Include converter from OpenAI message format
#1373 opened
Aug 20, 2024 -
custom llm
#1372 opened
Aug 19, 2024 -
Improve documentation on custom datasets finetune
#1369 opened
Aug 19, 2024 -
Can I Finetune Llama3 Without Creating CustomDataset Function?
#1362 opened
Aug 18, 2024 -
add pretraining code and multi modal support?
#1361 opened
Aug 17, 2024
37 Unresolved conversations
Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.
-
[6/7] SFTDataset: revamp instruct/chat
#1286 commented on
Aug 22, 2024 • 25 new comments -
Add CPU offload optimizer from torchao
#1351 commented on
Aug 22, 2024 • 20 new comments -
Flamingo Model Components
#1357 commented on
Aug 23, 2024 • 12 new comments -
[RFC][DOCS] Recipe [DOCS] ([DOC]umentation)
#1230 commented on
Aug 21, 2024 • 11 new comments -
[PoC] Prevent OOM during checkpoint save on colab
#1315 commented on
Aug 22, 2024 • 8 new comments -
DoRA
#1115 commented on
Aug 23, 2024 • 7 new comments -
Adding Intel XPU backend support using device-agnostic way
#1280 commented on
Aug 21, 2024 • 6 new comments -
fix QAT version dependency
#1333 commented on
Aug 22, 2024 • 1 new comment -
create import protection for torchvision
#1352 commented on
Aug 19, 2024 • 0 new comments -
Fix import of `torchao` now that `torchao-nightly` does not exist
#1355 commented on
Aug 21, 2024 • 0 new comments -
Fix generation for bsz > 1
#1250 commented on
Aug 22, 2024 • 0 new comments -
[7/7] Multimodal datasets (The Cauldron, LLaVA-Instruct-150K)
#1158 commented on
Aug 22, 2024 • 0 new comments -
Integrate flex attention
#1193 commented on
Aug 20, 2024 • 0 new comments -
MPS Support
#1233 commented on
Aug 22, 2024 • 0 new comments -
Delete torchtune's inplace copy definition for NF4
#1294 commented on
Aug 23, 2024 • 0 new comments -
Setting `expandable_segments:True` in our recipes.
#1309 commented on
Aug 22, 2024 • 0 new comments -
Update QAT recipe to match full_finetune_distributed
#1330 commented on
Aug 23, 2024 • 0 new comments -
Add LR Scheduler to single device full finetune
#1350 commented on
Aug 21, 2024 • 0 new comments -
[regression test] make nightlies and stable independent from each other
#1356 commented on
Aug 20, 2024 • 0 new comments -
Add more coding models
#1336 commented on
Aug 19, 2024 • 0 new comments -
RuntimeError: Index put requires the source and destination dtypes match, got BFloat16 for the destination and Float for the source.
#1349 commented on
Aug 19, 2024 • 0 new comments -
Tracker for dataset improvements
#1337 commented on
Aug 20, 2024 • 0 new comments -
How to save a trained model so it can be loaded with HF `from_pretrained()`?
#832 commented on
Aug 21, 2024 • 0 new comments -
Gradient accumulation is not efficiently implemented for distributed recipes
#1275 commented on
Aug 21, 2024 • 0 new comments -
Validate all paths before doing any expensive work
#861 commented on
Aug 21, 2024 • 0 new comments -
[FR] (Q)DoRA
#893 commented on
Aug 21, 2024 • 0 new comments -
lm harness distributed evaluation?
#930 commented on
Aug 21, 2024 • 0 new comments -
Input embeddings directly in `TransformerDecoder`
#968 commented on
Aug 21, 2024 • 0 new comments -
Support for Intel GPUs
#999 commented on
Aug 21, 2024 • 0 new comments -
Apply QLoRA to output projections and token embedding
#1000 commented on
Aug 21, 2024 • 0 new comments -
Make objects in notebooks importable in config
#1003 commented on
Aug 21, 2024 • 0 new comments -
how to support huawei Ascend npu device
#1006 commented on
Aug 21, 2024 • 0 new comments -
Running LoRA + QLoRA on Colab Notebooks
#828 commented on
Aug 21, 2024 • 0 new comments -
Add LR Scheduler to full finetune recipes
#1308 commented on
Aug 21, 2024 • 0 new comments -
Validation and early stopping during training
#883 commented on
Aug 21, 2024 • 0 new comments -
No param to control save checkpoints every N steps ?
#988 commented on
Aug 21, 2024 • 0 new comments -
text_completion_dataset removed?
#1140 commented on
Aug 21, 2024 • 0 new comments