This project demonstrates a basic Retrieval-Augmented Generation (RAG) system that loads documents from text files, websites, and PDF files.

View Live Site here

The implementation can be found in rag/simplerag.ipynb and follows the steps outlined below:

• Text Files: Loaded using TextLoader from langchain_community.document_loaders, with an example file located at rag/robotics.txt.
  

• Web Content: Loaded from Wikipedia using WebBaseLoader.
  

• PDF Files: Loaded using PyPDFLoader from the same library, with the PDF located at rag/robotics.pdf.
  

• Used RecursiveCharacterTextSplitter from langchain.text_splitter to split documents into 1,000-character chunks with 200-character overlap.
  

• Tokenized and embedded document chunks using the sentence-transformers/all-MiniLM-L6-v2 model from HuggingFace.
  

• Implemented a vector store using ChromaDB to manage embeddings.
  

• Created a document chain using a custom prompt that takes user queries as input and relevant document chunks as context, powered by create_stuff_documents_chain from langchain.chains.combine_documents and the ChatGroq model.
  

• Built a retriever using db.as_retriever() and constructed a retrieval chain via create_retrieval_chain from langchain.chains.retrieval.
  

• Create virtual environment: python -m venv venv
• Activate virtual environment: call venv/Scripts/activate.bat in cmd
• Install dependencies: pip install -r requirements.txt
• Create environment variables LANGCHAIN_API_KEY and GROQ_API_KEY. You can get your langchain api key from here, and your groq api key from here.

• Langchain
• Langchain Groq
• Streamlit
• Python-Dotenv
• Langchain
• PyPDF
• bs4
• chromadb
• transformers
• torch

• LinkedIn: Natan Asrat
• Gmail: nathanyilmaasrat@gmail.com
• Telegram: Natan Asrat
• Youtube: Natville