This is a self-hosted server for the refact.ai coding assistant.
With Refact you can run high-quality AI code completions on-premise and use a number of functions for code transformation.
Refact is currently available as a plugin for JetBrains products and VS Code IDE.
This server allows you to run AI coding models on your hardware, your code doesn't go outside your control.
At the moment, you can choose between 2 of our own models that support 20+ languages and are state-of-the-art in size and latency. In the future, we plan to add support to other models.
We recommend using this server with Nvidia GPU. Another option is to use ıt wıth CPU, but it'll be slower. Check system requrements below before you choose the model:
| Model | GPU (VRAM) | CPU (RAM) | |
|---|---|---|---|
| CONTRASTcode/medium/multi | 3Gb | 3Gb | |
| CONTRASTcode/3b/multi | 8Gb | 12Gb | |
| starcoder/15b/base4bit | 12Gb | - | Available ~May 5 |
| starcoder/15b/base8bit | 24Gb | - | Available ~May 5 |
Install plugin for your IDE: JetBrains or VSCode and sign up or login in to your account.
The recommended way to run server is a pre-build Docker image.
Install Docker with NVidia GPU support. On Windows you need to install WSL 2 first, one guide to do this.
Docker tips & tricks
Add your yourself to docker group to run docker without sudo (works for Linux):
sudo usermod -aG docker {your user}
List all containers:
docker ps -a
Create a new container:
docker run
Start and stop existing containers (stop doesn't remove them):
docker start
docker stop
Remove a container and all its data:
docker rm
Model weights are saved inside the container. If you remove the container, it will download the weights again.
Shows messages from the container:
docker logs -f
Run docker container with following command:
docker run -p 8008:8008 --gpus 0 --name refact_self_hosting smallcloud/refact_self_hosting --env MODEL=MODEL
If you don't have a suitable GPU run it on CPU:
docker run -p 8008:8008 --name refact_self_hosting smallcloud/refact_self_hosting
Next time you can start it with following command:
docker start -i refact_self_hosting
After start, container will automatically check for updates and download the chosen model (see in your account).
To run server manually, install this repo first (this might install a lot of packages on your computer):
pip install git+https://github.com/smallcloudai/code-contrast.git
pip install git+https://github.com/smallcloudai/refact-self-hosting.git
Now you can run server with following command:
python -m refact_self_hosting.server --workdir /workdir --model MODEL
Go to plugin settings and set up a custom inference url:
https://localhost:8008
JetBrains
Settings > Tools > Refact.ai > Advanced > Inference URLVSCode
Extensions > Refact.ai Assistant > Settings > InfurlNow it should work, just try to write some code! If it doesn't, please report your experience to GitHub issues.
Remote server
If you run server on remote host, you should add it to /etc/hosts (or C:\Windows\System32\drivers\etc\hosts on Windows) on client. Do not forget to replace {server ip address} to real server ip address.
{server ip address} inference.smallcloud.local
and set up this inference url in plugin:
https://inference.smallcloud.local:8008
Join our Discord server and follow our Twitter to get the latest updates.
We are open for contributions. If you have any ideas and ready to implement this, just: