Spaces:
Build error
Build error
repo config
Browse files
README.md
CHANGED
@@ -1,276 +1,9 @@
|
|
1 |
-
|
2 |
-
|
3 |
-
|
4 |
-
|
5 |
-
|
6 |
-
|
7 |
-
|
8 |
-
|
9 |
-
|
10 |
-
|
11 |
-
<div align="center">
|
12 |
-
<a href="https://discord.gg/e4AYNnFg2F">
|
13 |
-
<img src="https://img.shields.io/badge/discord-join%20chat-blue.svg?style=for-the-badge" alt="Join our Discord" height="20">
|
14 |
-
</a>
|
15 |
-
<a href="https://twitter.com/agishaun">
|
16 |
-
<img alt="Twitter Follow" src="https://img.shields.io/twitter/follow/agishaun?style=for-the-badge" height="20">
|
17 |
-
<a href="https://github.com/Shaunwei/RealChar">
|
18 |
-
<img alt="GitHub" src="https://img.shields.io/github/stars/Shaunwei/RealChar?style=for-the-badge&color=gold" height="20">
|
19 |
-
</a>
|
20 |
-
<a href="https://github.com/Shaunwei/RealChar/commits/main">
|
21 |
-
<img alt="GitHub" src="https://img.shields.io/github/last-commit/Shaunwei/RealChar/main?style=for-the-badge" height="20">
|
22 |
-
</a>
|
23 |
-
<a href="https://github.com/Shaunwei/RealChar/blob/main/README.md" target="_blank">
|
24 |
-
<img src="https://img.shields.io/static/v1?label=license&message=MIT&color=green&style=for-the-badge" alt="License" height="20">
|
25 |
-
</a>
|
26 |
-
<a href="https://hub.docker.com/repository/docker/shaunly/real_char/general" target="_blank">
|
27 |
-
<img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/shaunly/real_char?style=for-the-badge" height="20">
|
28 |
-
</a>
|
29 |
-
</div>
|
30 |
-
|
31 |
-
## β¨ Demo
|
32 |
-
Try our site at [RealChar.ai](https://realchar.ai/)
|
33 |
-
|
34 |
-
(We are also beta-testing our iOS mobile appπ±! Sign up [here](https://testflight.apple.com/join/JA6p9sZQ))
|
35 |
-
|
36 |
-
### Demo 1 - with AI Elon about cage fight!
|
37 |
-
|
38 |
-
https://github.com/Shaunwei/RealChar/assets/5101573/5de0b023-6cf3-4947-84cb-596f429d109e
|
39 |
-
|
40 |
-
### Demo 2 - with AI Raiden about AI and "real" memory
|
41 |
-
|
42 |
-
https://github.com/Shaunwei/RealChar/assets/5101573/62a1f3d1-1166-4254-9119-97647be52c42
|
43 |
-
|
44 |
-
|
45 |
-
|
46 |
-
__Demo settings: Web, GPT4, ElevenLabs with voice clone, Chroma, Google Speech to Text__
|
47 |
-
|
48 |
-
## π― Key Features
|
49 |
-
- **Easy to use**: No coding required to create your own AI character.
|
50 |
-
- **Customizable**: You can customize your AI character's personality, background, and even voice
|
51 |
-
- **Realtime**: Talk to or message your AI character in realtime
|
52 |
-
- **Multi-Platform**: You can talk to your AI character on web, terminal and mobile(Yes. we open source our mobile app)
|
53 |
-
- **Most up-to-date AI**: We use the most up-to-date AI technology to power your AI character, including OpenAI, Anthropic Claude 2, Chroma, Whisper, ElevenLabs, etc.
|
54 |
-
- **Modular**: You can easily swap out different modules to customize your flow. Less opinionated, more flexible. Great project to start your AI Engineering journey.
|
55 |
-
|
56 |
-
## π¬ Tech stack
|
57 |
-
<div align="center">
|
58 |
-
<img src="https://storage.googleapis.com/assistly/static/realchar/techstack.png" alt="RealChar-tech-stack" width="100%" style="padding: 20px"/>
|
59 |
-
</div>
|
60 |
-
|
61 |
-
- β
**Web**: [Vanilla JS](http://vanilla-js.com/), [WebSockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API)
|
62 |
-
- β
**Mobile**: [Swift](https://developer.apple.com/swift/), [WebSockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API)
|
63 |
-
- β
**Backend**: [FastAPI](https://fastapi.tiangolo.com/), [SQLite](https://www.sqlite.org/index.html), [Docker](https://www.docker.com/)
|
64 |
-
- β
**Data Ingestion**: [LlamaIndex](https://www.llamaindex.ai/), [Chroma](https://www.trychroma.com/)
|
65 |
-
- β
**LLM Orchestration**: [LangChain](https://langchain.com/), [Chroma](https://www.trychroma.com/)
|
66 |
-
- β
**LLM**: [OpenAI GPT3.5/4](https://platform.openai.com/docs/api-reference/chat), [Anthropic Claude 2](https://docs.anthropic.com/claude/docs/getting-started-with-claude)
|
67 |
-
- β
**Speech to Text**: [Local Whisper](https://github.com/openai/whisper), [OpenAI Whisper API](https://platform.openai.com/docs/api-reference/audio), [Google Speech to Text](https://cloud.google.com/speech-to-text/docs#docs)
|
68 |
-
- β
**Text to Speech**: [ElevenLabs](https://beta.elevenlabs.io/)
|
69 |
-
- β
**Voice Clone**: [ElevenLabs](https://beta.elevenlabs.io/voice-lab)
|
70 |
-
|
71 |
-
## π Comparison with existing products
|
72 |
-
<div align="center">
|
73 |
-
<img src="https://storage.googleapis.com/assistly/static/realchar/compare.png">
|
74 |
-
</div>
|
75 |
-
|
76 |
-
|
77 |
-
## π¨βπ Prerequisites
|
78 |
-
|
79 |
-
Before you begin setting up this project, please ensure you have completed the following tasks:
|
80 |
-
|
81 |
-
### 0. Setup Tutorial
|
82 |
-
|
83 |
-
- [Tutorial - YouTuBe](https://www.youtube.com/watch?v=Q16ZH3kJWxw)
|
84 |
-
|
85 |
-
### 1. LLM - OpenAI API Token
|
86 |
-
<details><summary>πclick me</summary>
|
87 |
-
This application utilizes the OpenAI API to access its powerful language model capabilities. In order to use the OpenAI API, you will need to obtain an API token.
|
88 |
-
|
89 |
-
To get your OpenAI API token, follow these steps:
|
90 |
-
|
91 |
-
1. Go to the [OpenAI website](https://beta.openai.com/signup/) and sign up for an account if you haven't already.
|
92 |
-
2. Once you're logged in, navigate to the [API keys page](https://beta.openai.com/account/api-keys).
|
93 |
-
3. Generate a new API key by clicking on the "Create API Key" button.
|
94 |
-
4. Copy the API key and store it safely.
|
95 |
-
5. Add the API key to your environment variable, e.g. `export OPENAI_API_KEY=<your API key>`
|
96 |
-
|
97 |
-
(Optional) To use Azure OpenAI API instead, refer to the following section:
|
98 |
-
|
99 |
-
1. Set API type
|
100 |
-
`export OPENAI_API_TYPE=azure`
|
101 |
-
|
102 |
-
If you want to use the earlier version `2023-03-15-preview`:
|
103 |
-
|
104 |
-
`export OPENAI_API_VERSION=2023-03-15-preview`
|
105 |
-
|
106 |
-
2. To set the base URL for your Azure OpenAI resource.
|
107 |
-
You can find this in the Azure portal under your Azure OpenAI resource.
|
108 |
-
|
109 |
-
`export OPENAI_API_BASE=https://your-base-url.openai.azure.com`
|
110 |
-
|
111 |
-
3. To set the OpenAI model deployment name for your Azure OpenAI resource.
|
112 |
-
|
113 |
-
`export OPENAI_API_MODEL_DEPLOYMENT_NAME=gpt-35-turbo-16k`
|
114 |
-
|
115 |
-
4. To set the OpenAIEmbeddings model deployment name for your Azure OpenAI resource.
|
116 |
-
|
117 |
-
`export OPENAI_API_EMBEDDING_DEPLOYMENT_NAME=text-embedding-ada-002`
|
118 |
-
|
119 |
-
</details>
|
120 |
-
|
121 |
-
### 1.1 (Optional) Prepare LLM - Anthropic(Claude 2) API Token
|
122 |
-
<details><summary>πclick me</summary>
|
123 |
-
|
124 |
-
To get your Anthropic API token, follow these steps:
|
125 |
-
|
126 |
-
1. Go to the [Anthropic website](https://docs.anthropic.com/claude/docs/getting-started-with-claude) and sign up for an account if you haven't already.
|
127 |
-
2. Once you're logged in, navigate to the [API keys page](https://console.anthropic.com/account/keys).
|
128 |
-
3. Generate a new API key by clicking on the "Create Key" button.
|
129 |
-
4. Copy the API key and store it safely.
|
130 |
-
5. Add the API key to your environment variable, e.g. `export ANTHROPIC_API_KEY=<your API key>`
|
131 |
-
</details>
|
132 |
-
|
133 |
-
### 2. (Optional) Prepare Speech to Text - Google Cloud API
|
134 |
-
<details><summary>πclick me</summary>
|
135 |
-
|
136 |
-
To get your Google Cloud API credentials.json, follow these steps:
|
137 |
-
|
138 |
-
1. Go to the [GCP website](https://cloud.google.com/speech-to-text/docs/before-you-begin) and sign up for an account if you haven't already.
|
139 |
-
2. Follow the guide to create a project and enable Speech to Text API
|
140 |
-
3. Put `google_credentials.json` in the root folder of this project. Check [GCP website](https://cloud.google.com/speech-to-text/docs/before-you-begin#set_your_authentication_environment_variable)
|
141 |
-
4. Change `SPEECH_TO_TEXT_USE` to use `GOOGLE` in your `.env` file
|
142 |
-
</details>
|
143 |
-
|
144 |
-
|
145 |
-
### 3. Prepare Text to Speech - ElevenLabs API Key
|
146 |
-
<details><summary>πclick me</summary>
|
147 |
-
1. Creating an ElevenLabs Account
|
148 |
-
Visit [ElevenLabs](https://beta.elevenlabs.io/) to create an account. You'll need this to access the text to speech and voice cloning features.
|
149 |
-
|
150 |
-
2. In your Profile Setting, you can get an API Key. Save it in a safe place.
|
151 |
-
|
152 |
-
3. Set API key in your .env file:
|
153 |
-
```
|
154 |
-
ELEVEN_LABS_API_KEY=<api key>
|
155 |
-
```
|
156 |
-
</details>
|
157 |
-
|
158 |
-
## πΏ Installation via Python
|
159 |
-
- **Step 1**. Clone the repo
|
160 |
-
```sh
|
161 |
-
git clone https://github.com/Shaunwei/RealChar.git && cd RealChar
|
162 |
-
```
|
163 |
-
- **Step 2**. Install requirements
|
164 |
-
- Install [portaudio](https://people.csail.mit.edu/hubert/pyaudio/) and [ffmpeg](https://ffmpeg.org/download.html) for audio
|
165 |
-
```sh
|
166 |
-
# for mac
|
167 |
-
brew install portaudio
|
168 |
-
brew install ffmpeg
|
169 |
-
```
|
170 |
-
```sh
|
171 |
-
# for ubuntu
|
172 |
-
sudo apt update
|
173 |
-
sudo apt install portaudio19-dev
|
174 |
-
sudo apt install ffmpeg
|
175 |
-
```
|
176 |
-
- Then install all python requirements
|
177 |
-
```sh
|
178 |
-
pip install -r requirements.txt
|
179 |
-
```
|
180 |
-
- **Step 3**. Create an empty [sqlite](https://www.sqlite.org/index.html) database if you have not done so before
|
181 |
-
```sh
|
182 |
-
sqlite3 test.db "VACUUM;"
|
183 |
-
```
|
184 |
-
- **Step 4**. Run db upgrade
|
185 |
-
```sh
|
186 |
-
alembic upgrade head
|
187 |
-
```
|
188 |
-
- **Step 5**. Setup `.env`: update API keys and select module
|
189 |
-
```sh
|
190 |
-
cp .env.example .env
|
191 |
-
```
|
192 |
-
- **Step 6**. Run server with `cli.py` or use uvicorn directly
|
193 |
-
```sh
|
194 |
-
python cli.py run-uvicorn
|
195 |
-
# or
|
196 |
-
uvicorn realtime_ai_character.main:app
|
197 |
-
```
|
198 |
-
- **Step 7**. Run client:
|
199 |
-
- Use **GPT4** for better conversation and **Wear headphone** for best audio(avoid echo)
|
200 |
-
- There are two ways to access the web client:
|
201 |
-
- **Option 1**: Open your web browser and navigate to http://localhost:8000 (NOT 0.0.0.0:8000)
|
202 |
-
- **Option 2**: Running the client in React.
|
203 |
-
```sh
|
204 |
-
cd client/web
|
205 |
-
npm install
|
206 |
-
npm start
|
207 |
-
```
|
208 |
-
After running these commands, a local development server will start, and your default web browser will open a new tab/window pointing to this server (usually http://localhost:3000).
|
209 |
-
- (Optional) Terminal client: Run the following command in your terminal
|
210 |
-
```sh
|
211 |
-
python client/cli.py
|
212 |
-
```
|
213 |
-
- (Optional) mobile client: open `client/mobile/ios/rac/rac.xcodeproj/project.pbxproj` in Xcode and run the app
|
214 |
-
- **Step 8**. Select one character to talk to, then start talking
|
215 |
-
|
216 |
-
|
217 |
-
## (Optional) π Installation via Docker
|
218 |
-
<details><summary>πclick me</summary>
|
219 |
-
|
220 |
-
1. Docker image: you can use our docker image directly
|
221 |
-
```sh
|
222 |
-
docker pull shaunly/real_char:latest
|
223 |
-
```
|
224 |
-
(Or you want build yourself) Build docker image
|
225 |
-
```sh
|
226 |
-
python cli.py docker-build
|
227 |
-
```
|
228 |
-
If you have issues with docker (especially on a non-Linux machine), please refer to https://docs.docker.com/get-docker/ (installation) and https://docs.docker.com/desktop/troubleshoot/overview/ (troubleshooting).
|
229 |
-
2. Run docker image with `.env` file
|
230 |
-
```sh
|
231 |
-
python cli.py docker-run
|
232 |
-
```
|
233 |
-
|
234 |
-
3. Go to http://localhost:8000 (NOT 0.0.0.0:8000) to start talking or use terminal client
|
235 |
-
```sh
|
236 |
-
python client/cli.py
|
237 |
-
```
|
238 |
-
|
239 |
-
</details>
|
240 |
-
|
241 |
-
<br/>
|
242 |
-
|
243 |
-
## π! LangSmith integration
|
244 |
-
<details><summary>πclick me</summary>
|
245 |
-
|
246 |
-
If you have access to LangSmith, you can edit these environment variables to enable:
|
247 |
-
```
|
248 |
-
LANGCHAIN_TRACING_V2=false # default off
|
249 |
-
LANGCHAIN_ENDPOINT=https://api.smith.langchain.com
|
250 |
-
LANGCHAIN_API_KEY=YOUR_LANGCHAIN_API_KEY
|
251 |
-
LANGCHAIN_PROJECT=YOUR_LANGCHAIN_PROJECT
|
252 |
-
```
|
253 |
-
And it should work out of the box.
|
254 |
-
|
255 |
-
</details>
|
256 |
-
|
257 |
-
<br/>
|
258 |
-
|
259 |
-
## π Roadmap
|
260 |
-
- [x] Launch v0.0.1 and build a community
|
261 |
-
- [x] Move away from Vanilla JS
|
262 |
-
- [x] Launch mobile app (iOS TestFlight Beta link: https://testflight.apple.com/join/JA6p9sZQ)
|
263 |
-
- [ ] Add authentication for customization
|
264 |
-
- [ ] Allow selecting different LLM
|
265 |
-
- [ ] Add ability to add community characters
|
266 |
-
|
267 |
-
## π«Ά Contribute to RealChar
|
268 |
-
Please check out our [Contribution Guide](contribute.md)!
|
269 |
-
|
270 |
-
## πͺ Contributors
|
271 |
-
<a href="https://github.com/Shaunwei/RealChar">
|
272 |
-
<img src="https://contrib.rocks/image?repo=Shaunwei/RealChar" />
|
273 |
-
</a>
|
274 |
-
|
275 |
-
## π² Community
|
276 |
-
- Join us on [Discord](https://discord.gg/e4AYNnFg2F)
|
|
|
1 |
+
---
|
2 |
+
title: RealChar - Your Realtime AI Character
|
3 |
+
emoji: ποΈπ€
|
4 |
+
colorFrom: blue
|
5 |
+
colorTo: indigo
|
6 |
+
sdk: docker
|
7 |
+
sdk_version: 1.24.0
|
8 |
+
pinned: true
|
9 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|