Assembled | Disassembled |
---|---|
Friend is an AI wearable device that records everything you say, gives you proactive feedback and advice.
- Real-Time AI Audio Processing: Leverage powerful on-device AI capabilities for real-time audio analysis.
- Low-powered Bluetooth: Capture audio for 24h+ on a small button battery
- Open-Source Software: Access and contribute to the pin's software stack, designed with openness and community collaboration in mind.
- Wearable Design: Experience unparalleled convenience with ergonomic and lightweight design, perfect for everyday wear.
graph TD
A[Device] -- Streams Audio --> B[Phone App]
B -- Saves Audio --> C[Phone Storage]
C -- After X mins --> D[Send Audio to Whisper API]
D -- Returns Transcript --> B[Phone App]
B -- Saves Transcript --> F[Phone Storage]
style A fill:#FFFFFF, stroke:#FF4136, stroke-width:2px, color:#FFFFFF
style B fill:#FFFFFF, stroke:#1ABC9C, stroke-width:2px, color:#FFFFFF
style C fill:#FFFFFF, stroke:#FFCC00, stroke-width:2px, color:#1F2020
style D fill:#FFFFFF, stroke:#2ECC40, stroke-width:2px, color:#1F2020
style F fill:#FFFFFF, stroke:#FF6B6B, stroke-width:2px, color:#FFFFFF
linkStyle 0 stroke:#FF4136, stroke-width:2px
linkStyle 1 stroke:#1ABC9C, stroke-width:2px
linkStyle 2 stroke:#FFCC00, stroke-width:2px
linkStyle 3 stroke:#2ECC40, stroke-width:2px
linkStyle 4 stroke:#FF6B6B, stroke-width:2px
There are 2 different apps in these repositories located in different branches and folders. Our goal is to merge them into one big project.
-
Standalone Branch or Folder "AppStandalone": Standalone version of the app that doesn't require any hardware to use it.
-
AppWithWearable Branch or Folder "AppWithWearable": Wearable-connected version of the app that requires the "Friend necklace" to use it.
-
[Main Branch] Branch that contains firmware, hardware, designs and both apps
-
Board: Seeed Studio XIAO nRF52840 Sense | $15 on Seedstudio and $24 on Amazon | <$15/unit
-
Rechargeable Coin Battery| $13 for 5 units on Amazon and also link for europe but you only need one | $2.5/unit
-
Slider switch | $7 for 30+ units on Amazon but you only need one | <$1/unit
-
3D-print design of the case is located here
Follow these steps to get started with your Friend. Note, we tested everything on a mac + iphone. Currently trying to make it work for Android
-
Clone the repo
git clone https://github.com/BasedHardware/friend.git
-
Choose which version of the app you want to install (see Structure).
- Don't have the device? run
cd AppStandalone
in terminal - Have the device/NRF Board? run
cd AppWithWearable
in terminal
- Don't have the device? run
-
Install your environment variables
- For AppWithWearable, open file api_calls.dart located in
AppWithWearable/lib/backend/api_requests
Find "Whisper" and instead of "key", provide your own api-key for openai whisper for transcriptions to work
- For AppStandalone, update variables in in .env.template file
- For AppWithWearable, open file api_calls.dart located in
-
iOS: Install XCode then navigate to the iOS folder. You might need to launch Xcode to select a team and specify a bundle identifier. Android: Download/install android Studio then navigate to the Android folder Don't run in web/simulator: Bluetooth will not work
-
Run
flutter clean
thenflutter pub get
thenpod install
-
When everything is installed, run
flutter run
, this should run your app on a selected device
No-Code Alternative:
- Don't have the device? Clone this Flutterflow Project
- Have the wearable device? Copy this Flutterflow Project
-
Run
cd src/BluetoothDeviceDriver
in your home repository and Open Arduino .ino file, go to "Settings" and paste these 2 links in additional Boards Manager URLshttps://adafruit.github.io/arduino-board-index/package_adafruit_index.json https://files.seeedstudio.com/arduino/package_seeeduino_boards_index.json
-
Go to Boards Manager and download these 2 Boards
-
Connect NRF52840 board via USB cable to your computer
-
Go to Tools > Board >
and select "Seeed nRF52 mbed-enabled Boards (you need board that has Sense)
-
Go to Sketch => Include Library => Add .zip library and upload a library which you should download from here, make sure you download the
Seeed_Arduino_Mic
repository itself as a .zip file and not use the .zip from its releases section. -
Install Arduino BLE and Seeed Arduino LSM6DS3 libraries which can be found in Arduino's menu
-
Click "Upload" and then open Serial Monitor to see logs
Follow these steps to test audio recording on your computer using a Python script:
-
Open your terminal and navigate to your home directory.
-
Change to the "src" folder:
cd src
-
Install the required Python modules:
pip install -r requirements.txt
-
Run the Python script:
python local_laptop_client.py
This script will list the available audio devices and their corresponding IDs.
-
Copy the ID of the desired audio device.
-
Open the
local_laptop_client.py
file in a text editor and locate the following line:DEVICE_ID = "564A72F4-4552-8CE8-719D-8D5CB2E5D43D"
Replace
"564A72F4-4552-8CE8-719D-8D5CB2E5D43D"
with the ID you copied in step 5. -
Save the changes to the
local_laptop_client.py
file. -
Run the script again:
python local_laptop_client.py
or
python3 local_laptop_client.py
-
You can now control the audio recording:
- Double-tap the device to start recording.
- Double-tap the device again to stop recording.
The recorded audio files will be stored periodically in the src/recordings
directory.
That's it! You have now set up and tested audio recording on your computer.
Step 0: Make sure you have bought everything from the buying guide above
Step 1: You need to design the case using 3D printer. Find .stl file here. If you don't know how to do it, send this file to someone who has a 3d printer
Step 2: Solder everything together like on the picture below. using a soldering kit. Don't have it? buy this one for $9
Step 3: Fit everything in the case. Biggest hole is for the usb port. In my example, I put the battery first, then the board and then the switch, however it's not an ideal design. If you will figure out a better solution, please contribute!
Step 4: Use hot glue to attach the lid to the case. You can also use a scotch tape first for testing purposes. Last, on the USB-port side, you'll find 2 small round holes. This is where the thread should go through.
Congratulations! you now have a fully working and assembled device!
Join our Discord! We welcome contributions from the community! If you're interested in improving Friend, our current biggest goal is to combine both apps together (AppStandalone with AppWithWearable).
- Standalone App brings great prompts and rich structure
- AppWithWearable brings simple bluetooth connecting functionality
For open-source support, please open an issue on GitHub and/or ask in our Discord Community. For commercial support, license inquiries, or any other questions, please contact us directly at [email protected].
Please note that the Friend is a prototype project and is provided "as is", without warranty of any kind. Use of the device should comply with all local laws and regulations concerning privacy and data protection.
Thank you for your interest in Friend, the open-source AI wearable. We're excited to see what you'll build with it!
Friend is available under dual licensing options:
-
GNU General Public License (GPL): For open-source projects and community development, Friend is available under the GPL. This strong copyleft license ensures that all modifications and derived works are also open-source, fostering a collaborative development environment.
-
Commercial License: For individuals or entities wishing to use Friend in closed-source projects or who require more flexible licensing terms than those offered by the GPL, a commercial license is available. The commercial license permits private modification, use, and distribution, as well as commercial support and warranty.
- If you wish to contribute to or use Friend in open-source projects, you are free to do so under the terms of the GPL, as detailed in the LICENSE file.
- If you require a commercial license for your project or enterprise, please contact us at [email protected] to discuss your needs and obtain licensing information.