OpenAI/ChatGPT Backend with conversation and API
- Import documents of knowledge base from a table (CSV), save them into PostgreSQL
- Based on the title and content of the document, generate and fill in QA items in the document using some APIs
- Summarize the questions and generate corresponding vectors
- Implement high-quality Q&A using vector search
- Welcome message and preset messages
- Chat History for Conversation (based on redis)
- RESTful API
- Support text/event-stream
- Login with OAuth2 client for general Security Provider
GET /api/welcome
None
http code content-type response 200application/json{"message": "welcome message", "id": "new-cid"}
GET /api/me
None
http code content-type response 200application/json{"data": {"avatar": "", "name": "name", "uid": "uid"}}401application/json{"error": "", "message": ""}
POST /api/chat-sse or /api/chat with {stream: true}
name type data type description csidoptional string conversation ID promptrequired string message for ask streamoptional bool enable event-stream, force on /api/chat-sse
http code content-type response 200text/event-stream{"delta": "message fragments", "id": "conversation ID"}401application/json{"status": "Unauthorized", "message": ""}
POST /api/chat-process for chatgpt-web only
name type data type description promptrequired string message for ask optionsoptional object { conversationId: "" }
http code content-type response 200application/octet-stream{"delta": "message fragments", "text": "message", "conversationId": ""}401application/json{"status": "Unauthorized", "message": ""}
test -e .env || cp .env.example .env
# Edit .env and change api key of OpenAI
# Embedding and replacing frontend resources
make deps
forego start
# or
make dist
welcome:
content: Hello, I am your virtual assistant. How can I help you?
role: assistant
messages:
- content: You are a helpful assistant.
role: system
- content: When is my birthday?
role: user
- content: How would I know?
role: assistant
# more messages
CREATE USER morrigan WITH LOGIN PASSWORD 'mydbusersecret';
CREATE DATABASE morrigan WITH OWNER = morrigan ENCODING = 'UTF8';
GRANT ALL PRIVILEGES ON DATABASE morrigan to morrigan;
\c morrigan
-- optional: install extension from https://github.com/pgvector/pgvector
CREATE EXTENSION vector;
USAGE:
morrigan [global options] command [command options] [arguments...]
COMMANDS:
usage, env show usage
initdb init database schema
import import documents from a csv
fill-qa, fillQAs fill QA in documents
export-qa, exportQAs export QA from documents
embedding, embedding-pormpt read prompt documents and embedding
web, run run a web server
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--help, -h show help
go run . usageExample:
MORRIGAN_OPENAI_API_KEY=oo-xx
MORRIGAN_HTTP_LISTEN=:3002
# optional preset data
MORRIGAN_PRESET_FILE=./data/preset.yaml
# optional OAuth2 login
MORRIGAN_AUTH_REQUIRED=true
OAUTH_PREFIX=https://portal.my-company.xyz
# optional proxy
HTTPS_PROXY=socks5://proxy.my-company.xyz:1081
- Prepare a CSV file for the corpus document.
- Import documents.
- Generate Questions and Anwsers from documents with Completion.
- Generate Prompts and vector from QAs with Embedding
- Done and go to chat
| title | heading | content |
|---|---|---|
| my company | introducion | A great company stems from a genius idea. |
go run . initdb
go run . import mycompany.csv
go run . fill-qa
go run . embedding- Go to frontend project directory
- Build frontend pages and accompanying static resources.
- Copy them into ./htdocs
Example:
cd ../chatgpt-svelte
npm run build
rsync -a --delete dist/* ../morrigan/htdocs/
cd -During the development and debugging phase, you can still use with proxy to collaborate with the front-end project.