pycui commited on
Commit
641369d
β€’
1 Parent(s): babeaf6

repo config

Browse files
Files changed (1) hide show
  1. README.md +9 -276
README.md CHANGED
@@ -1,276 +1,9 @@
1
- # <img src="./realtime_ai_character/static/realchar.svg" height="24px" style="padding-top:4px"/>RealChar. - Your Realtime AI Character
2
- <br/>
3
- <div align="center">
4
- <img src="./realtime_ai_character/static/logo.png" alt="RealChar-logo" width="80%" style="padding: 40px"/>
5
- </div>
6
- <br/>
7
- <p align="center">
8
- πŸŽ™οΈπŸ€–<em>Create, customize and talk to your AI Character/Companion in realtime</em>πŸŽ™οΈπŸ€–
9
- </p>
10
-
11
- <div align="center">
12
- <a href="https://discord.gg/e4AYNnFg2F">
13
- <img src="https://img.shields.io/badge/discord-join%20chat-blue.svg?style=for-the-badge" alt="Join our Discord" height="20">
14
- </a>
15
- <a href="https://twitter.com/agishaun">
16
- <img alt="Twitter Follow" src="https://img.shields.io/twitter/follow/agishaun?style=for-the-badge" height="20">
17
- <a href="https://github.com/Shaunwei/RealChar">
18
- <img alt="GitHub" src="https://img.shields.io/github/stars/Shaunwei/RealChar?style=for-the-badge&color=gold" height="20">
19
- </a>
20
- <a href="https://github.com/Shaunwei/RealChar/commits/main">
21
- <img alt="GitHub" src="https://img.shields.io/github/last-commit/Shaunwei/RealChar/main?style=for-the-badge" height="20">
22
- </a>
23
- <a href="https://github.com/Shaunwei/RealChar/blob/main/README.md" target="_blank">
24
- <img src="https://img.shields.io/static/v1?label=license&message=MIT&color=green&style=for-the-badge" alt="License" height="20">
25
- </a>
26
- <a href="https://hub.docker.com/repository/docker/shaunly/real_char/general" target="_blank">
27
- <img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/shaunly/real_char?style=for-the-badge" height="20">
28
- </a>
29
- </div>
30
-
31
- ## ✨ Demo
32
- Try our site at [RealChar.ai](https://realchar.ai/)
33
-
34
- (We are also beta-testing our iOS mobile appπŸ“±! Sign up [here](https://testflight.apple.com/join/JA6p9sZQ))
35
-
36
- ### Demo 1 - with AI Elon about cage fight!
37
-
38
- https://github.com/Shaunwei/RealChar/assets/5101573/5de0b023-6cf3-4947-84cb-596f429d109e
39
-
40
- ### Demo 2 - with AI Raiden about AI and "real" memory
41
-
42
- https://github.com/Shaunwei/RealChar/assets/5101573/62a1f3d1-1166-4254-9119-97647be52c42
43
-
44
-
45
-
46
- __Demo settings: Web, GPT4, ElevenLabs with voice clone, Chroma, Google Speech to Text__
47
-
48
- ## 🎯 Key Features
49
- - **Easy to use**: No coding required to create your own AI character.
50
- - **Customizable**: You can customize your AI character's personality, background, and even voice
51
- - **Realtime**: Talk to or message your AI character in realtime
52
- - **Multi-Platform**: You can talk to your AI character on web, terminal and mobile(Yes. we open source our mobile app)
53
- - **Most up-to-date AI**: We use the most up-to-date AI technology to power your AI character, including OpenAI, Anthropic Claude 2, Chroma, Whisper, ElevenLabs, etc.
54
- - **Modular**: You can easily swap out different modules to customize your flow. Less opinionated, more flexible. Great project to start your AI Engineering journey.
55
-
56
- ## πŸ”¬ Tech stack
57
- <div align="center">
58
- <img src="https://storage.googleapis.com/assistly/static/realchar/techstack.png" alt="RealChar-tech-stack" width="100%" style="padding: 20px"/>
59
- </div>
60
-
61
- - βœ…**Web**: [Vanilla JS](http://vanilla-js.com/), [WebSockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API)
62
- - βœ…**Mobile**: [Swift](https://developer.apple.com/swift/), [WebSockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API)
63
- - βœ…**Backend**: [FastAPI](https://fastapi.tiangolo.com/), [SQLite](https://www.sqlite.org/index.html), [Docker](https://www.docker.com/)
64
- - βœ…**Data Ingestion**: [LlamaIndex](https://www.llamaindex.ai/), [Chroma](https://www.trychroma.com/)
65
- - βœ…**LLM Orchestration**: [LangChain](https://langchain.com/), [Chroma](https://www.trychroma.com/)
66
- - βœ…**LLM**: [OpenAI GPT3.5/4](https://platform.openai.com/docs/api-reference/chat), [Anthropic Claude 2](https://docs.anthropic.com/claude/docs/getting-started-with-claude)
67
- - βœ…**Speech to Text**: [Local Whisper](https://github.com/openai/whisper), [OpenAI Whisper API](https://platform.openai.com/docs/api-reference/audio), [Google Speech to Text](https://cloud.google.com/speech-to-text/docs#docs)
68
- - βœ…**Text to Speech**: [ElevenLabs](https://beta.elevenlabs.io/)
69
- - βœ…**Voice Clone**: [ElevenLabs](https://beta.elevenlabs.io/voice-lab)
70
-
71
- ## πŸ“š Comparison with existing products
72
- <div align="center">
73
- <img src="https://storage.googleapis.com/assistly/static/realchar/compare.png">
74
- </div>
75
-
76
-
77
- ## πŸ‘¨β€πŸš€ Prerequisites
78
-
79
- Before you begin setting up this project, please ensure you have completed the following tasks:
80
-
81
- ### 0. Setup Tutorial
82
-
83
- - [Tutorial - YouTuBe](https://www.youtube.com/watch?v=Q16ZH3kJWxw)
84
-
85
- ### 1. LLM - OpenAI API Token
86
- <details><summary>πŸ‘‡click me</summary>
87
- This application utilizes the OpenAI API to access its powerful language model capabilities. In order to use the OpenAI API, you will need to obtain an API token.
88
-
89
- To get your OpenAI API token, follow these steps:
90
-
91
- 1. Go to the [OpenAI website](https://beta.openai.com/signup/) and sign up for an account if you haven't already.
92
- 2. Once you're logged in, navigate to the [API keys page](https://beta.openai.com/account/api-keys).
93
- 3. Generate a new API key by clicking on the "Create API Key" button.
94
- 4. Copy the API key and store it safely.
95
- 5. Add the API key to your environment variable, e.g. `export OPENAI_API_KEY=<your API key>`
96
-
97
- (Optional) To use Azure OpenAI API instead, refer to the following section:
98
-
99
- 1. Set API type
100
- `export OPENAI_API_TYPE=azure`
101
-
102
- If you want to use the earlier version `2023-03-15-preview`:
103
-
104
- `export OPENAI_API_VERSION=2023-03-15-preview`
105
-
106
- 2. To set the base URL for your Azure OpenAI resource.
107
- You can find this in the Azure portal under your Azure OpenAI resource.
108
-
109
- `export OPENAI_API_BASE=https://your-base-url.openai.azure.com`
110
-
111
- 3. To set the OpenAI model deployment name for your Azure OpenAI resource.
112
-
113
- `export OPENAI_API_MODEL_DEPLOYMENT_NAME=gpt-35-turbo-16k`
114
-
115
- 4. To set the OpenAIEmbeddings model deployment name for your Azure OpenAI resource.
116
-
117
- `export OPENAI_API_EMBEDDING_DEPLOYMENT_NAME=text-embedding-ada-002`
118
-
119
- </details>
120
-
121
- ### 1.1 (Optional) Prepare LLM - Anthropic(Claude 2) API Token
122
- <details><summary>πŸ‘‡click me</summary>
123
-
124
- To get your Anthropic API token, follow these steps:
125
-
126
- 1. Go to the [Anthropic website](https://docs.anthropic.com/claude/docs/getting-started-with-claude) and sign up for an account if you haven't already.
127
- 2. Once you're logged in, navigate to the [API keys page](https://console.anthropic.com/account/keys).
128
- 3. Generate a new API key by clicking on the "Create Key" button.
129
- 4. Copy the API key and store it safely.
130
- 5. Add the API key to your environment variable, e.g. `export ANTHROPIC_API_KEY=<your API key>`
131
- </details>
132
-
133
- ### 2. (Optional) Prepare Speech to Text - Google Cloud API
134
- <details><summary>πŸ‘‡click me</summary>
135
-
136
- To get your Google Cloud API credentials.json, follow these steps:
137
-
138
- 1. Go to the [GCP website](https://cloud.google.com/speech-to-text/docs/before-you-begin) and sign up for an account if you haven't already.
139
- 2. Follow the guide to create a project and enable Speech to Text API
140
- 3. Put `google_credentials.json` in the root folder of this project. Check [GCP website](https://cloud.google.com/speech-to-text/docs/before-you-begin#set_your_authentication_environment_variable)
141
- 4. Change `SPEECH_TO_TEXT_USE` to use `GOOGLE` in your `.env` file
142
- </details>
143
-
144
-
145
- ### 3. Prepare Text to Speech - ElevenLabs API Key
146
- <details><summary>πŸ‘‡click me</summary>
147
- 1. Creating an ElevenLabs Account
148
- Visit [ElevenLabs](https://beta.elevenlabs.io/) to create an account. You'll need this to access the text to speech and voice cloning features.
149
-
150
- 2. In your Profile Setting, you can get an API Key. Save it in a safe place.
151
-
152
- 3. Set API key in your .env file:
153
- ```
154
- ELEVEN_LABS_API_KEY=<api key>
155
- ```
156
- </details>
157
-
158
- ## πŸ’Ώ Installation via Python
159
- - **Step 1**. Clone the repo
160
- ```sh
161
- git clone https://github.com/Shaunwei/RealChar.git && cd RealChar
162
- ```
163
- - **Step 2**. Install requirements
164
- - Install [portaudio](https://people.csail.mit.edu/hubert/pyaudio/) and [ffmpeg](https://ffmpeg.org/download.html) for audio
165
- ```sh
166
- # for mac
167
- brew install portaudio
168
- brew install ffmpeg
169
- ```
170
- ```sh
171
- # for ubuntu
172
- sudo apt update
173
- sudo apt install portaudio19-dev
174
- sudo apt install ffmpeg
175
- ```
176
- - Then install all python requirements
177
- ```sh
178
- pip install -r requirements.txt
179
- ```
180
- - **Step 3**. Create an empty [sqlite](https://www.sqlite.org/index.html) database if you have not done so before
181
- ```sh
182
- sqlite3 test.db "VACUUM;"
183
- ```
184
- - **Step 4**. Run db upgrade
185
- ```sh
186
- alembic upgrade head
187
- ```
188
- - **Step 5**. Setup `.env`: update API keys and select module
189
- ```sh
190
- cp .env.example .env
191
- ```
192
- - **Step 6**. Run server with `cli.py` or use uvicorn directly
193
- ```sh
194
- python cli.py run-uvicorn
195
- # or
196
- uvicorn realtime_ai_character.main:app
197
- ```
198
- - **Step 7**. Run client:
199
- - Use **GPT4** for better conversation and **Wear headphone** for best audio(avoid echo)
200
- - There are two ways to access the web client:
201
- - **Option 1**: Open your web browser and navigate to http://localhost:8000 (NOT 0.0.0.0:8000)
202
- - **Option 2**: Running the client in React.
203
- ```sh
204
- cd client/web
205
- npm install
206
- npm start
207
- ```
208
- After running these commands, a local development server will start, and your default web browser will open a new tab/window pointing to this server (usually http://localhost:3000).
209
- - (Optional) Terminal client: Run the following command in your terminal
210
- ```sh
211
- python client/cli.py
212
- ```
213
- - (Optional) mobile client: open `client/mobile/ios/rac/rac.xcodeproj/project.pbxproj` in Xcode and run the app
214
- - **Step 8**. Select one character to talk to, then start talking
215
-
216
-
217
- ## (Optional) πŸ“€ Installation via Docker
218
- <details><summary>πŸ‘‡click me</summary>
219
-
220
- 1. Docker image: you can use our docker image directly
221
- ```sh
222
- docker pull shaunly/real_char:latest
223
- ```
224
- (Or you want build yourself) Build docker image
225
- ```sh
226
- python cli.py docker-build
227
- ```
228
- If you have issues with docker (especially on a non-Linux machine), please refer to https://docs.docker.com/get-docker/ (installation) and https://docs.docker.com/desktop/troubleshoot/overview/ (troubleshooting).
229
- 2. Run docker image with `.env` file
230
- ```sh
231
- python cli.py docker-run
232
- ```
233
-
234
- 3. Go to http://localhost:8000 (NOT 0.0.0.0:8000) to start talking or use terminal client
235
- ```sh
236
- python client/cli.py
237
- ```
238
-
239
- </details>
240
-
241
- <br/>
242
-
243
- ## πŸ†•! LangSmith integration
244
- <details><summary>πŸ‘‡click me</summary>
245
-
246
- If you have access to LangSmith, you can edit these environment variables to enable:
247
- ```
248
- LANGCHAIN_TRACING_V2=false # default off
249
- LANGCHAIN_ENDPOINT=https://api.smith.langchain.com
250
- LANGCHAIN_API_KEY=YOUR_LANGCHAIN_API_KEY
251
- LANGCHAIN_PROJECT=YOUR_LANGCHAIN_PROJECT
252
- ```
253
- And it should work out of the box.
254
-
255
- </details>
256
-
257
- <br/>
258
-
259
- ## πŸ“ Roadmap
260
- - [x] Launch v0.0.1 and build a community
261
- - [x] Move away from Vanilla JS
262
- - [x] Launch mobile app (iOS TestFlight Beta link: https://testflight.apple.com/join/JA6p9sZQ)
263
- - [ ] Add authentication for customization
264
- - [ ] Allow selecting different LLM
265
- - [ ] Add ability to add community characters
266
-
267
- ## 🫢 Contribute to RealChar
268
- Please check out our [Contribution Guide](contribute.md)!
269
-
270
- ## πŸ’ͺ Contributors
271
- <a href="https://github.com/Shaunwei/RealChar">
272
- <img src="https://contrib.rocks/image?repo=Shaunwei/RealChar" />
273
- </a>
274
-
275
- ## 🎲 Community
276
- - Join us on [Discord](https://discord.gg/e4AYNnFg2F)
 
1
+ ---
2
+ title: RealChar - Your Realtime AI Character
3
+ emoji: πŸŽ™οΈπŸ€–
4
+ colorFrom: blue
5
+ colorTo: indigo
6
+ sdk: docker
7
+ sdk_version: 1.24.0
8
+ pinned: true
9
+ ---