Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor sequence parallel #823

Merged

Conversation

tastelikefeet
Copy link
Collaborator

@tastelikefeet tastelikefeet commented Apr 27, 2024

PR type

  • Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

  1. Refactor the code of sequence_parallel
  2. remove to_hf_dataset call
  3. Update doc
  4. Add phi3-mini model
  5. Fix a bug in long-alpaca-12k

Experiment results

Paste your experiment result here(if needed).

@tastelikefeet tastelikefeet changed the title [WIP]sequence parallel [WIP]Refactor sequence parallel May 3, 2024
…ctor_sequence_parallel

* commit '88ae9ed5dc7c28160875fd4389b5e55c125c36ea':
  fix
  fix script
  fix
  add temp file
  add metric
* main: (24 commits)
  fix pre-commit
  traindataset异常提示 (modelscope#859)
  Feat/pack (modelscope#881)
  fix swift cli exit code if subprocess is failed (modelscope#879)
  support Deepseek-V2-Chat and InternVL-Chat-V1.5-int8 model  (modelscope#876)
  add llava-llama (modelscope#873)
  support ORPO algorithm (modelscope#854)
  fix dataset info deepcopy (modelscope#871)
  fix dataset_test_ratio=1 (modelscope#869)
  Refactor dataset (modelscope#802)
  Feat/loras (modelscope#865)
  Update tuner docs (modelscope#853)
  update docs (modelscope#850)
  Fix code format and docs (modelscope#847)
  update (modelscope#846)
  fix xcomposer device_map (modelscope#844)
  fix merge_lora_dtype (modelscope#842)
  Fix infer default dtype (modelscope#834)
  fix ui (modelscope#830)
  support Internvl-chat-v1.5 model (modelscope#824)
  ...

# Conflicts:
#	docs/source/LLM/自定义与拓展.md
#	docs/source_en/LLM/Customization.md
#	examples/pytorch/llm/custom.py
#	scripts/benchmark/exp_utils.py
#	scripts/utils/run_dataset_info.py
#	swift/aigc/diffusers/train_controlnet.py
#	swift/aigc/diffusers/train_controlnet_sdxl.py
#	swift/aigc/diffusers/train_text_to_image.py
#	swift/aigc/diffusers/train_text_to_image_lora.py
#	swift/aigc/diffusers/train_text_to_image_lora_sdxl.py
#	swift/aigc/diffusers/train_text_to_image_sdxl.py
#	swift/llm/__init__.py
#	swift/llm/deploy.py
#	swift/llm/dpo.py
#	swift/llm/export.py
#	swift/llm/infer.py
#	swift/llm/sft.py
#	swift/llm/tuner.py
#	swift/llm/utils/__init__.py
#	swift/llm/utils/argument.py
#	swift/llm/utils/client_utils.py
#	swift/llm/utils/dataset.py
#	swift/llm/utils/model.py
#	swift/llm/utils/preprocess.py
#	swift/llm/utils/template.py
#	swift/llm/utils/utils.py
#	swift/trainers/trainers.py
#	swift/tuners/base.py
#	swift/ui/llm_infer/llm_infer.py
#	swift/ui/llm_infer/runtime.py
#	swift/ui/llm_train/dataset.py
#	swift/ui/llm_train/llm_train.py
#	tests/llm/test_run.py
* feat/phi3_mini:
  add phi3-mini
  enable longlora and adalora merge
…ctor_sequence_parallel

* commit '400c0a3f6b1f3c09598d160508469a93b1c95784':
  replace dataset name with modelscope dataset id (modelscope#899)
  replace dataset name with dataset path from modelscope (modelscope#897)
  fix doc link
  enable longlora and adalora merge (modelscope#892)
  fix lisa show bug (modelscope#891)
  update doc (modelscope#888)
  fix list index out of range bugs of model_name and model_author (modelscope#885)
  support epoch based saving (modelscope#887)
  Fix/doc (modelscope#882)
@tastelikefeet tastelikefeet changed the title [WIP]Refactor sequence parallel Refactor sequence parallel May 10, 2024
@tastelikefeet tastelikefeet merged commit 1fc148a into modelscope:main May 10, 2024
2 checks passed
tastelikefeet added a commit to tastelikefeet/swift that referenced this pull request May 14, 2024
* commit 'bdc8f54848daad335e513183482e16cc5da17c88': (36 commits)
  fix export self-cognition (modelscope#929)
  fix deepseek2(modelscope#924)
  Add 34b quantized model (modelscope#920)
  yi1.5 quantized model (modelscope#917)
  update readme&doc (modelscope#916)
  init (modelscope#915)
  fix unsloth import (modelscope#912)
  add more models local repo support (modelscope#911)
  lint
  DeepseekVL add local_repo_path argument AND infer support delete truncation_strategy (modelscope#883)
  Support Hqq and Eetq quantization  (modelscope#900)
  fix val_sample (modelscope#909)
  Add val_dataset argument (modelscope#906)
  Refactor sequence parallel (modelscope#823)
  replace dataset name with modelscope dataset id (modelscope#899)
  replace dataset name with dataset path from modelscope (modelscope#897)
  fix doc link
  enable longlora and adalora merge (modelscope#892)
  fix lisa show bug (modelscope#891)
  update doc (modelscope#888)
  ...

# Conflicts:
#	swift/llm/utils/dataset.py
hjh0119 pushed a commit to hjh0119/swift that referenced this pull request Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants