Skip to content
/ AlpacaChat Public

A Swift library that runs Alpaca-LoRA prediction locally to implement ChatGPT like app on Apple platform devices.

License

Notifications You must be signed in to change notification settings

niw/AlpacaChat

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

27 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

AlpacaChat

A Swift library that runs Alpaca-LoRA prediction locally to implement ChatGPT like app on Apple platform devices.

AlpacaChat

It is basically a wrapper for alpaca.cpp that provides a simple Swift API for it.

import AlpacaChat

// Load model and instantiate a chat.
let model = try await Model.load(from: URL(fileURLWithPath: "model.bin"))
let chat = Chat(model: model)

// Ask users to get prompt.
let prompt = readLine()!

// Run prediction and print tokens.
for try await token in chat.predictTokens(for: prompt) {
    print(token)
}

Model

Read alpaca.cpp, alpaca-lora, and llma.cpp, then create 4-bits quantized ggml model bin file.

Place it in /Applications/AlpacaChatApp/Resouces/model.bin for example, and build app and run it.

Usage

See actual command line and SwiftUI application for usages.

Applications

/Applications/AlpacaChatCLI

A command line chat app that can run on macOS.

To build, use Xcode or simply use swift command.

$ cd Applications/AlpacaChatCLI
$ swift build -c release
$ .build/release/AlpacaChatCLI -m /path/to/model.bin

/Applications/AlpacaChatApp.xcodeproj

A SwiftUI chat app that can run on iOS devices.

To build app runs on actual device, you need to create your own AppID and provisioning profile that allows extended memory usage with an entitlement.

Place /Applications/AlpacaChatApp/Configurations/Local.xcconfig to provide these your local development configurations for signing.

You may want to change scheme to use Release configuration for Run, or it may be seriously slow.

About

A Swift library that runs Alpaca-LoRA prediction locally to implement ChatGPT like app on Apple platform devices.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published