Local Genai Lab

Local GenAI Lab

[](https://github.com/jrodolfo/local-genai-lab/actions/workflows/ci.yml) [](https://github.com/jrodolfo/local-genai-lab/blob/main/LICENSE)

Local-first GenAI lab for building and testing tool-assisted chat workflows. Not just a chatbot UI.

This project combines a React frontend, a Spring Boot orchestration backend, local and remote model providers, persistent session memory, MCP-backed AWS tooling, and a RAG workspace over the project documentation in one full-stack repository.

Fastest Path

If you want the shortest path to a running local setup:

cp .env.example .env
./scripts/start.sh

Then open:

• frontend: http://localhost:5173
• backend health: http://localhost:8080/actuator/health

Notes:

• By default, the backend starts on http://localhost:8080 with APP_MODEL_PROVIDER=ollama.
• If you want Bedrock or Hugging Face available in the provider selector, add their config to .env first.
• For the default Ollama path, make sure llama3:8b is installed locally.
• The RAG workspace is enabled by default and uses the local docs/ corpus.
• The lifecycle scripts store PID files and logs under .run/.
• Use ./scripts/status.sh or make status to inspect the local runtime.

Why This Matters

Most LLM demos stop at chat. This project explores how to connect models to real systems.

It focuses on:

• tool-assisted chat with backend-side orchestration instead of direct frontend-to-model calls
• provider abstraction with Ollama by default plus Amazon Bedrock and Hugging Face as optional remote provider APIs
• persistent session memory with resume, search, filter, import, and export flows
• MCP-backed local tool execution for AWS audits, reports, and artifact generation
• a separate RAG workspace for asking questions against the local documentation corpus with cited source chunks
• structured report rendering, artifact preview, streaming responses, and API observability

Architecture

High-level interaction flows:

For the full system-level view, see [docs/architecture.md](./docs/architecture.md).

React -> Spring Boot
          |-> MCP -> Shell Scripts -> AWS CLI -> Report artifacts
          |-> Prompt enrichment with tool result -> Ollama / Bedrock

In the successful tool-assisted path, the backend:

1. receives the user message
2. decides a tool is needed
3. calls the MCP-backed tool
4. gets structured tool output back
5. builds an augmented prompt with that tool context
6. sends that enriched prompt to Ollama, Bedrock, or Hugging Face

Primary chat path:

React Frontend -> Spring Boot Backend -> Ollama, Bedrock, or Hugging Face

Tool-assisted chat path:

React Frontend -> Spring Boot Backend -> Local MCP Server -> Shell Scripts -> AWS CLI / report artifacts -> Spring Boot Backend prompt enrichment -> Ollama, Bedrock, or Hugging Face

RAG path:

React RAG Workspace -> Spring Boot RAG API -> local docs corpus -> chunk retrieval -> Ollama, Bedrock, or Hugging Face -> answer with cited source chunks

Immediate backend response path:

React Frontend -> Spring Boot Backend -> clarification or tool-failure response

Project Structure

local-genai-lab/
├── Makefile
├── scripts/
│   ├── start.sh
│   ├── stop.sh
│   ├── restart.sh
│   ├── status.sh
│   ├── build.sh
│   ├── docker-start.sh
│   ├── docker-stop.sh
│   ├── docker-restart.sh
│   ├── docker-status.sh
│   ├── docker-check.sh
│   ├── docker-verify.sh
│   ├── docker-scan.sh
│   ├── docker-full-check.sh
│   └── README.md
├── ops/
│   ├── lib/
│   ├── tests/
│   └── README.md
├── backend/
│   ├── src/
│   ├── pom.xml
│   └── README.md
├── frontend/
│   ├── src/
│   ├── package.json
│   └── README.md
├── agents/
│   ├── reports/
│   ├── tests/
│   ├── dependency-freshness.sh
│   ├── Makefile
│   └── README.md
├── mcp/
│   ├── src/
│   ├── package.json
│   └── README.md
├── docker-compose.yml
└── README.md

Script separation:

• scripts/ contains human-facing app lifecycle commands, including host-run commands and full Docker Compose commands
• ops/ contains internal local runtime helpers such as backend-only startup and stack smoke checks
• agents/ contains MCP/tool-facing shell scripts, report generators, and their shell tests

Docker note:

• the backend Docker image intentionally includes the built MCP server, Node 20.19+, the MCP tool scripts, and empty report directories
• this keeps Docker mode aligned with host-run mode and allows /actuator/health to validate MCP and storage paths correctly
• generated report artifacts are not copied into the image; Docker starts with empty report directories

Prerequisites

• Java 21. The backend build enforces Java 21; confirm with java -version before building or running backend tests.
• Maven 3.9+. The backend build enforces this minimum Maven version.
• Node 20.19+
• Ollama installed locally for the default provider
• Docker + Docker Compose, optional
• AWS CLI v2 + jq + valid AWS credentials, only for AWS shell tools and local MCP-backed report flows

Quick Start

1. Pull a local model

ollama pull llama3:8b
ollama run llama3:8b

Ollama should be reachable at http://localhost:11434.

2. Optional: create a local environment file

cp .env.example .env

Fill in only the providers you want available in the running backend process. The backend helper script will auto-load .env if it exists.

3. Start the app

./scripts/start.sh

This starts both:

• the Spring Boot backend
• the Vite frontend dev server

If you want Bedrock or Hugging Face as the default backend provider instead of Ollama, set APP_MODEL_PROVIDER=bedrock or APP_MODEL_PROVIDER=huggingface in .env or the shell before starting the app. Provider configuration details live in [docs/providers.md](./docs/providers.md).

If port 8080 is already in use, choose another backend port explicitly:

SERVER_PORT=8081 ./scripts/start.sh

The frontend dev proxy follows that override automatically when you start the app this way.

Backend URLs:

• API root: http://localhost:8080
• OpenAPI: http://localhost:8080/v3/api-docs
• Swagger UI: http://localhost:8080/swagger-ui/index.html
• Health: http://localhost:8080/actuator/health
• Info: http://localhost:8080/actuator/info

GET /actuator redirects to /actuator/health. Swagger excludes /actuator/** so the generated API docs stay focused on the application API.

4. Stop, restart, or inspect the app

./scripts/stop.sh
./scripts/restart.sh
./scripts/status.sh

./scripts/stop.sh stops processes managed by this repo's PID files. ./scripts/restart.sh uses ./scripts/stop.sh --all, so it also clears processes occupying the configured backend and frontend ports before starting the app again.

If you want to build generated artifacts before restarting, run:

./scripts/build.sh
./scripts/restart.sh

./scripts/build.sh builds the backend package, frontend production build, and MCP server build without starting or stopping the app. Use ./scripts/build.sh --skip-tests for a faster local build when you have already run tests.

Expected build output includes Maven and npm progress plus any JVM/native-access warnings emitted by Java dependencies during tests. Application/controller stack traces from expected negative-path tests should not appear in normal build output. If they do, treat that as test-log noise to investigate rather than as an expected part of the build.

Verification Commands

Use these commands depending on what you need to verify:

| Command | Use when | Requires running app? | | --- | --- | --- | | make start | Start backend and frontend in the background. | No | | make stop | Stop managed backend/frontend processes. | Useful when running | | make restart | Stop then start the local app. | No | | make status / ./scripts/status.sh | Inspect local processes, health URLs, RAG mode, Ollama readiness, and Qdrant readiness. | Useful when running | | make build | Build backend, frontend, and MCP artifacts without changing the running app. | No | | make check-app | Smoke-check the live backend/frontend stack after startup. | Yes | | make docker-start | Start backend, frontend, and Qdrant with Docker Compose. | No | | make docker-stop | Stop the Docker Compose stack. | Useful when running | | make docker-restart | Restart the Docker Compose stack. | No | | make docker-status | Show Docker Compose service status and expected URLs. | Useful when running | | make docker-check | Smoke-check the running Docker Compose stack. | Yes | | make docker-verify | Restart, inspect, and smoke-check Docker mode. | No | | make docker-scan | Scan Docker images for known vulnerabilities. | No, but images should exist | | make docker-full-check | Run Docker verification and Docker image scan. | No | | make dependency-freshness | Report Maven, npm, and Docker dependency freshness without modifying files. | No | | make release-check | Run the local pre-release validation gate. | No | | make release-check-docker | Run release check with early Docker preflight and Docker verification/scan. | No | | make clean-ds-store | Remove local macOS .DS_Store files from the repo tree. | No | | make test | Normal local pre-commit suite for ops, backend, and frontend tests. | No | | make verify | Broader CI-aligned verification, including frontend build, MCP tests/build, and MCP tool script lint/tests. | No | | make test-rag-qdrant-smoke | Verify the live Ollama embeddings plus Qdrant RAG path. | Yes, in Qdrant vector mode |

Use make test when you only need normal verification. Use make verify before larger pushes or broad changes. Use make build or ./scripts/build.sh when you also want fresh generated backend, frontend, and MCP artifacts before restarting the local app.

Use make dependency-freshness as a maintenance radar. It reports Maven parent/dependency/plugin updates, npm outdated packages for frontend/ and mcp/, Docker image references, and moving Docker tags such as latest. It is report-only and does not upgrade or rewrite dependency files. The final triage summary suggests possible follow-up branch names; treat those as planning hints, not as automatic upgrade instructions.

Use make release-check when you want one local gate before wrapping a larger batch of work. It runs tests, broader verification, dependency freshness, and git diff --check. Use make release-check-docker when you also want Docker verification and image scanning. The Docker-inclusive target preflights Docker, Docker Compose, and Trivy before running expensive tests:

make release-check-docker

For the full testing matrix, see [docs/testing.md](./docs/testing.md).

The frontend provider and model selectors now load from the backend's /api/models endpoint. You can switch between supported providers at runtime without restarting the backend. For Ollama, the UI only offers locally installed models. If no local models are installed, the UI shows a clear pull hint instead of failing only after submit.

The provider selector only shows providers configured in the running backend process. The provider status banner is cached briefly to avoid excessive live checks, shows Last checked, and includes a manual Refresh status action when you want to re-fetch the current status explicitly.

For tool-assisted streaming chat, the UI now shows explicit tool lifecycle phases while the request is in flight. Completed assistant replies also show compact tool provenance, and generated summaries, reports, and file lists can be inspected through the artifact inspector panel.

The separate RAG workspace is enabled by default. It queries a fixed local corpus rooted at docs/ and returns answers with cited source chunks. If you want to hide it, start the backend with RAG_ENABLED=false.

Lexical retrieval remains the default. The RAG question form can select Lexical, Vector - In Memory, or Vector - Qdrant per question. Vector retrieval embeds the same docs corpus with RAG_EMBEDDING_PROVIDER=ollama and RAG_EMBEDDING_MODEL=nomic-embed-text; RAG_RETRIEVAL_MODE and RAG_VECTOR_STORE still define the backend default target at startup.

Evaluation-only RAG docs are excluded from the indexed corpus by default so manual test prompts do not become misleading retrieval sources.

The RAG index is built automatically on the first question. You do not need to click Rebuild Index before normal first use. Use Rebuild Index after changing docs, switching retrieval settings, or troubleshooting stale results.

For a plain-language explanation of RAG page terms such as Index, Rebuild Index, Sources, and Technical Details, see [docs/rag-troubleshooting.md#rag-page-mental-model](./docs/rag-troubleshooting.md#rag-page-mental-model).

If RAG or vector retrieval does not behave as expected, run ./scripts/status.sh first. It reports RAG mode, Ollama readiness, whether the configured embedding model is installed, and Qdrant reachability plus collection point count when the Vector - Qdrant comparison target is available. Common fixes and the RAG answer Technical Details fields are documented in [docs/rag-troubleshooting.md](./docs/rag-troubleshooting.md).

Good first RAG test prompts:

• How does provider selection work?
• Why is MCP separate from the backend?
• How are sessions persisted?
• What ADR explains the Mermaid architecture diagram?

5. Optional: build the local MCP server

cd mcp
npm install
npm run build

MCP is enabled by default in the backend. To run without it, set MCP_ENABLED=false.

Docker

Keep Ollama running on the host first, then use the Docker lifecycle wrappers:

./scripts/docker-restart.sh
./scripts/docker-status.sh
./scripts/docker-check.sh
./scripts/docker-stop.sh

For the functional Docker verification workflow, use:

./scripts/docker-verify.sh

This script is not read-only. It stops host-run backend/frontend processes, restarts the Docker Compose stack, prints Docker status, and runs the Docker smoke check.

For the broadest Docker check, including the advisory security scan, use:

./scripts/docker-full-check.sh

This runs ./scripts/docker-verify.sh first and then ./scripts/docker-scan.sh.

Equivalent Make targets are available:

make docker-restart
make docker-status
make docker-check
make docker-stop
make docker-verify
make docker-scan
make docker-full-check
make release-check
make release-check-docker

The direct Compose command also works:

docker compose up --build

• frontend: http://localhost:3000
• backend: http://localhost:8080
• qdrant: http://localhost:6333

Docker backend containers reach host Ollama through http://host.docker.internal:11434 by default. If your Docker runtime needs a different address, set DOCKER_OLLAMA_BASE_URL; keep host-run OLLAMA_BASE_URL separate because localhost inside a container means the container itself.

The existing ./scripts/start.sh, ./scripts/stop.sh, ./scripts/restart.sh, and ./scripts/status.sh scripts run the backend and frontend directly on the host. The docker-* scripts run the full Docker Compose stack. Keep those workflows separate to avoid accidentally mixing host-run processes with containerized services.

Use ./scripts/docker-status.sh when you want to know what is running and where to look. It is diagnostic and prints Compose status, readiness, URLs, log commands, port checks, and recovery hints.

Use ./scripts/docker-check.sh when you want to know whether the Docker app is usable enough to trust. It is a read-only smoke check and exits non-zero if backend health, frontend, Qdrant, /api/models, or /api/rag/status is unavailable. After ./scripts/docker-start.sh or ./scripts/docker-restart.sh, run ./scripts/docker-check.sh befor

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: jrodolfo
• Source: jrodolfo/local-genai-lab
• License: MIT

Install and usage instructions live in the source repository linked above.