Skip to content

Using Docker Compose

Follow the instructions to run a complete GenAI Stack using Docker Compose

Table of Contents

Prereq

Step 1. Install Ollama

Visit this link to download and install Ollama on your preferred operating system.

Image2

Choose your preferrable operating system.

Image3

Step 2. Create OpenAI Secret API Keys [optional]

Visit this link to create your new OpenAI Secret API Keys.

Step 3. Sign Up for LangChain Beta for API Keys [optional if you want to enable Langchain Smith API]

Visit this link in order to create Langchain Endpoint and API Keys. You will need the following information

LANGCHAIN_ENDPOINT="https://api.smith.langchain.com"
LANGCHAIN_TRACING_V2=true # false
LANGCHAIN_PROJECT=default
LANGCHAIN_API_KEY=ls__cbabccXXXXXX

Step 4. Clone the repository

 git clone https://github.com/docker/genai-stack
 cd genai-stack

Step 5. Create .env file as copy of env.example file

Create a copy of env.example file and rename it as .env. Make sure to update the optional parameters for OpenAI and Langchain if you wish to use them in the project.

cat .env 
OPENAI_API_KEY=sk-EsNJzI5uMBCXXXXXXXX
OLLAMA_BASE_URL=http://host.docker.internal:11434
NEO4J_URI=neo4j://database:7687
NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=password
LLM=llama2 #or any Ollama model tag, or gpt-4 or gpt-3.5
EMBEDDING_MODEL=sentence_transformer #or openai or ollama

LANGCHAIN_ENDPOINT="https://api.smith.langchain.com"
LANGCHAIN_TRACING_V2=true # false
LANGCHAIN_PROJECT=default
LANGCHAIN_API_KEY=ls__cbabccXXXXXX

Step 6. Bring up Compose services

 docker compose up -d --build

..
..
genai-stack-bot-1         |   You can now view your Streamlit app in your browser.
genai-stack-bot-1         |
genai-stack-bot-1         |   URL: http://0.0.0.0:8501
genai-stack-bot-1         |
genai-stack-pdf_bot-1     |
genai-stack-pdf_bot-1     |   You can now view your Streamlit app in your browser.
genai-stack-pdf_bot-1     |
genai-stack-pdf_bot-1     |   URL: http://0.0.0.0:8503
genai-stack-pdf_bot-1     |
genai-stack-loader-1      |
genai-stack-loader-1      |   You can now view your Streamlit app in your browser.
genai-stack-loader-1      |
genai-stack-loader-1      |   URL: http://0.0.0.0:8502
genai-stack-loader-1      |

Step 7. Viewing the Services on Docker Dashboard

Image9

Step 8. Accessing the StackOverflow Loader app

Visit http://0.0.0.0:8502 to access the following:

Image11

Click "Import". It will take a minute or two to run the import. Most of the time is spent generating the embeddings. After or during the import you can click the link to http://localhost:7474 and log in with username “neo4j” and password “password” as configured in docker compose. There, you can see an overview in the left sidebar and show some connected data by clicking on the “pill” with the counts.

The data loader will import the graph using the following schema.

Image12

Result:

Image13

The graph schema for Stack Overflow consists of nodes representing Questions, Answers, Users, and Tags. Users are linked to Questions they’ve asked via the “ASKED” relationship and to Answers they’ve provided with the “ANSWERS” relationship. Each Answer is also inherently associated with a specific Question. Furthermore, Questions are categorized by their relevant topics or technologies using the “TAGGED” relationship connecting them to Tags.

Step 9. Accessing the Neo4j

As instructed, open http://localhost:7474 and log in with username “neo4j” and password “password” as configured in docker compose.

Image14

This application server on http://localhost:8501 has the classic LLM chat UI and lets the user ask questions and get answers.

There’s a switch called RAG mode where the user can rely either completely on the LLMs trained knowledge (RAG: Disabled), or the more capable (RAG: Enabled) mode where the application uses similarity search using text embedding and graph queries to find the most relevant questions and answers in the database.

Click "Highly ranked questions"

Step 11. Accessing GenAI Stack PDF Bot

Open http://0.0.0.0:8503/ on the browser to access the PDF Bot that allows you to chat with your PDF file.

Image15

In order to test the app, one of us (Ajeet) has uploaded his latest resume and asked a quick question. It responded back with the right answer. Amazing !!