Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support falcon model #762

Merged
merged 30 commits into from
Jun 17, 2023
Merged

support falcon model #762

merged 30 commits into from
Jun 17, 2023

Conversation

xinhaoc
Copy link
Collaborator

@xinhaoc xinhaoc commented Jun 12, 2023

Description of changes:

this PR support the multi-query attention model falcon in incr_decoding mode.

Related Issues:

Linked Issues:

  • Issue #

Issues closed by this PR:

  • Closes #

Before merging:

  • Did you update the flexflow-third-party repo, if modifying any of the Cmake files, the build configs, or the submodules?

@xinhaoc xinhaoc requested a review from lambda7xx June 12, 2023 19:03
@xinhaoc
Copy link
Collaborator Author

xinhaoc commented Jun 12, 2023

the weight layout is at falcon

include/flexflow/ffconst.h Outdated Show resolved Hide resolved
include/flexflow/model.h Outdated Show resolved Hide resolved
src/ops/inc_multiquery_attention.cc Outdated Show resolved Hide resolved
src/ops/inc_multiquery_attention.cc Outdated Show resolved Hide resolved
src/ops/inc_multiquery_attention.cu Outdated Show resolved Hide resolved
src/runtime/ffconst_utils.cc Outdated Show resolved Hide resolved
src/runtime/graph.cc Outdated Show resolved Hide resolved
src/runtime/graph.cc Outdated Show resolved Hide resolved
@xinhaoc
Copy link
Collaborator Author

xinhaoc commented Jun 14, 2023

Give three tips for staying healthy.

  1. Eat a balanced diet.
  2. Get plenty of exercise.
  3. Get enough sleep.
    What are the three most important things you can do to stay healthy?
  4. Eat a balanced diet.
  5. Get plenty of exercise.
  6. Get enough sleep.
    What are the three most important things you can do to stay healthy?
  7. Eat a balanced diet.
  8. Get plenty of exercise.
  9. Get enough sleep.
    What are the three most important things you can do to stay healthy?
  10. Eat a balanced diet

@xinhaoc xinhaoc marked this pull request as ready for review June 15, 2023 12:26
@xinhaoc xinhaoc requested a review from goliaro June 15, 2023 12:48
Copy link
Collaborator

@goliaro goliaro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great to see this PR complete! Thanks for the hard work! LGTM after the small changes

.github/workflows/build.yml Outdated Show resolved Hide resolved
.github/workflows/gpu-ci.yml Outdated Show resolved Hide resolved
.gitignore Outdated Show resolved Hide resolved
inference/models/falcon.cc Outdated Show resolved Hide resolved
inference/utils/download_opt_weights.py Show resolved Hide resolved
tests/inference_tests.sh Show resolved Hide resolved
@xinhaoc xinhaoc requested a review from goliaro June 15, 2023 20:13
Copy link
Collaborator

@goliaro goliaro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Let's get this merged as soon as the CI test passes

@goliaro goliaro enabled auto-merge (squash) June 17, 2023 13:11
@goliaro goliaro merged commit 3969a67 into flexflow:inference Jun 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants