Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sync: use encodec's latest version as a submodule #124

Merged
merged 69 commits into from
Feb 13, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
2302881
rm encodec
PABannier Oct 26, 2023
450a606
add git submodules
PABannier Oct 26, 2023
4ece5fa
removed bark util
PABannier Oct 26, 2023
9c9f7e8
updated CMakeLists
PABannier Oct 26, 2023
e2b350a
rm build scripts
PABannier Oct 26, 2023
d491cc7
mv dr_wav in examples
PABannier Oct 26, 2023
38c2e49
common cpp
PABannier Oct 26, 2023
6b32b3b
moved def constants
PABannier Oct 26, 2023
ab9b528
text encoder loaded with the latest ggml API
PABannier Oct 27, 2023
6a9b50a
pulled ggml upstream
PABannier Oct 27, 2023
efbdd56
temporarily removed subdirectory encodec.cpp
PABannier Oct 27, 2023
753d5cf
clean forward pass text encoder
PABannier Oct 27, 2023
e4e712f
compiling
PABannier Oct 27, 2023
d8fc378
fix issue definition
PABannier Oct 27, 2023
b258c08
clean
PABannier Oct 28, 2023
6642e75
remove codec parsing functions
PABannier Oct 28, 2023
33d186e
kinda works
PABannier Oct 28, 2023
83a21ec
bias is stored in hparams
PABannier Oct 28, 2023
6cad888
working text encoder
PABannier Oct 28, 2023
242e7c5
cln tests
PABannier Oct 28, 2023
c1d0edd
coarse working?
PABannier Oct 28, 2023
94cd5e2
override bias
PABannier Oct 29, 2023
acf9dfa
working fine encoder
PABannier Oct 29, 2023
c1def75
rename quantize.cpp into main.cpp
PABannier Oct 29, 2023
cfaa59c
included quantize as a target
PABannier Oct 29, 2023
05ef89d
exposed quantization function
PABannier Oct 29, 2023
6172381
minor
PABannier Oct 29, 2023
6d0db93
update CIs
PABannier Oct 29, 2023
a978908
updated CIs
PABannier Oct 29, 2023
8ae7dc5
passing tokenizer test
PABannier Oct 29, 2023
7ad8cd5
Merge branch 'main' of https://github.com/PABannier/bark.cpp into enc…
PABannier Oct 29, 2023
7c2ae84
fast text encoder
PABannier Dec 11, 2023
d3971c2
Merge branch 'main' of https://github.com/PABannier/bark.cpp into enc…
PABannier Dec 11, 2023
5874a87
`bark.cpp` -> `bark`
PABannier Dec 11, 2023
5312577
server abides by latest API
PABannier Dec 12, 2023
e7b7d75
rm fast-text-encoder example
PABannier Dec 12, 2023
2aaf7b2
pass `-O3` release flag
PABannier Dec 12, 2023
79ed551
rm fast_text_encoder from CMakeLists
PABannier Dec 12, 2023
4f72d56
restructured
PABannier Dec 12, 2023
f13498a
CMakeLists arranged
PABannier Dec 13, 2023
f517570
update CIs
PABannier Dec 13, 2023
b8bdd76
add encodec.cpp in the loop
PABannier Dec 14, 2023
5319d26
add verbosity level
PABannier Dec 15, 2023
11c3f9a
Fix CIs (#128)
AlexHayton Dec 30, 2023
da3cc56
Merge branch 'encodec_as_submodule' of https://github.com/PABannier/b…
PABannier Jan 2, 2024
07a322c
fix coarse encoder internal pass
PABannier Jan 3, 2024
3002698
`VerbosityLevel` -> `bark_verbosity_level`
PABannier Jan 5, 2024
747345c
updated examples
PABannier Jan 5, 2024
a3e3e92
populated time per token
PABannier Jan 5, 2024
19e1683
remove whitespace
PABannier Jan 5, 2024
fa6975c
BarkProgressBar implemented
PABannier Jan 6, 2024
b9e2109
verbosity level controlled for cleaner output
PABannier Jan 6, 2024
4401975
removed params as macros and moved them into default constructor
PABannier Jan 6, 2024
38846ec
updated README
PABannier Jan 6, 2024
59d5352
removed useless `n_predict` in params
PABannier Jan 6, 2024
07e92de
removed old tests
PABannier Jan 6, 2024
ec677fb
fix wrong return type, quantization works again
PABannier Jan 6, 2024
035ef16
Added Metal and CUDA backend
PABannier Jan 6, 2024
6e4ac9a
updated docs
PABannier Jan 6, 2024
ac327a9
cosmit
PABannier Jan 6, 2024
d347134
rm submodule
PABannier Jan 7, 2024
d7e9661
added encodec submodule
PABannier Jan 7, 2024
1fbe29d
remove mem_per_token
PABannier Jan 7, 2024
b3d9179
more verbose errors
PABannier Jan 7, 2024
94fea82
clean
PABannier Jan 7, 2024
bec8547
reset allocr to reduce memory footprint
PABannier Jan 7, 2024
df7c22a
add tests
PABannier Jan 7, 2024
6fbc184
expose forward passes
PABannier Jan 7, 2024
87a102b
enhanced README.md
PABannier Feb 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix CIs (#128)
Co-authored-by: root <[email protected]>
Co-authored-by: Alex Hayton <[email protected]>
  • Loading branch information
3 people committed Dec 30, 2023
commit 11c3f9a9b7605d5f5dc4a2ef2018a45b609472ee
29 changes: 25 additions & 4 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,31 @@ on:
push:
branches:
- main
paths: ['.github/workflows/**', '**/CMakeLists.txt', '**/Makefile', '**/*.h', '**/*.hpp', '**/*.c', '**/*.cpp', '**/*.cu']
- encodec-submodule-fix-ci
paths:
[
".github/workflows/**",
"**/CMakeLists.txt",
"**/Makefile",
"**/*.h",
"**/*.hpp",
"**/*.c",
"**/*.cpp",
"**/*.cu",
]
pull_request:
types: [opened, synchronize, reopened]
paths: ['**/CMakeLists.txt', '**/Makefile', '**/*.h', '**/*.hpp', '**/*.c', '**/*.cpp', '**/*.cu', ".github/workflows/**"]
paths:
[
"**/CMakeLists.txt",
"**/Makefile",
"**/*.h",
"**/*.hpp",
"**/*.c",
"**/*.cpp",
"**/*.cu",
".github/workflows/**",
]

env:
BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
Expand All @@ -24,7 +45,7 @@ jobs:
id: checkout
uses: actions/checkout@v4
with:
submodules: true
submodules: recursive

- name: Dependencies
id: depends
Expand All @@ -49,7 +70,7 @@ jobs:
id: checkout
uses: actions/checkout@v4
with:
submodules: true
submodules: recursive

- name: Dependencies
id: depends
Expand Down
4 changes: 0 additions & 4 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
[submodule "ggml"]
path = ggml
url = https://github.com/ggerganov/ggml.git

[submodule "encodec.cpp"]
path = encodec.cpp
url = https://github.com/PABannier/encodec.cpp.git
22 changes: 12 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,15 +158,16 @@ Here are the steps for the bark model.
```bash
git clone --recursive https://github.com/PABannier/bark.cpp.git
cd bark.cpp
git submodule update --init --recursive
```

### Build

In order to build bark.cpp you must use `CMake`:

```bash
mkdir build
cd build
mkdir bark/build
cd bark/build
cmake ..
cmake --build . --config Release
```
Expand All @@ -175,20 +176,19 @@ cmake --build . --config Release

```bash
# install Python dependencies
python3 -m pip install -r requirements.txt
python3 -m pip install -r bark/requirements.txt

# obtain the original bark and encodec weights and place them in ./models
python3 download_weights.py --download-dir ./models
python3 bark/download_weights.py --download-dir ./models

# convert the model to ggml format
python3 convert.py \
python3 bark/convert.py \
--dir-model ./models \
--codec-path ./models \
--vocab-path ./ggml_weights/ \
--out-dir ./ggml_weights/

# run the inference
./main -m ./ggml_weights/ -p "this is an audio"
./bark/build/examples/main/main -m ./ggml_weights/ -p "this is an audio"
```

### (Optional) Quantize weights
Expand All @@ -199,9 +199,11 @@ Note that to preserve audio quality, we do not quantize the codec model. The bul
computation is in the forward pass of the GPT models.

```bash
./quantize ./ggml_weights/ggml_weights_text.bin ./ggml_weights_q4/ggml_weights_text.bin q4_0
./quantize ./ggml_weights/ggml_weights_coarse.bin ./ggml_weights_q4/ggml_weights_coarse.bin q4_0
./quantize ./ggml_weights/ggml_weights_fine.bin ./ggml_weights_q4/ggml_weights_fine.bin q4_0
mkdir ggml_weights_q4
cp ggml_weights/*vocab* ggml_weights_q4
./bark/build/examples/quantize/quantize ./ggml_weights/ggml_weights_text.bin ./ggml_weights_q4/ggml_weights_text.bin q4_0
./bark/build/examples/quantize/quantize ./ggml_weights/ggml_weights_coarse.bin ./ggml_weights_q4/ggml_weights_coarse.bin q4_0
./bark/build/examples/quantize/quantize ./ggml_weights/ggml_weights_fine.bin ./ggml_weights_q4/ggml_weights_fine.bin q4_0
```

### Seminal papers and background on models
Expand Down
1 change: 0 additions & 1 deletion ggml
Submodule ggml deleted from d5b778