Github Backup Architecture

GitHub Backup Automation System — Architecture Guide

This skill provides an overview of the system architecture, data models, communication protocols, and deployment environments.

1. Branch-First Development

> [!IMPORTANT] > CREATE A LOCAL BRANCH FIRST: Always start by creating a local branch from main: > ``bash > git switch -c MishraShardendu22/main/ > ` > Never make changes directly on main`.

---

2. System Topology

┌─────────────────────────────────────────────────────────────┐
│                    Next.js 16 Frontend                      │
│                  (Deployed on Vercel)                       │
└───────────────┬─────────────────────────────┬───────────────┘
                │ REST / SSE                  │ REST / WebSocket
                ▼                             ▼
┌─────────────────────────────┐ ┌─────────────────────────────┐
│     Python Observatory      │ │         Go Backend          │
│    (Deployed on Vercel)     │ │    (Deployed on Render)     │
│   FastAPI · LangChain AI    │ │  Fiber v2 · Live WS Stream  │
└───────────────┬─────────────┘ └─────────────┬───────────────┘
                │                             │
                │     ┌─────────────────┐     │
                ├────►│ Neon PostgreSQL │◄────┤
                │     │ (pgvector + FTS)│     │
                │     └────────┬────────┘     │
                │              ▲              │
                │              │ Sync         │
                │     ┌────────┴────────┐     │
                │     │ Backup Worker   │     │
                │     │ (CLI / Cron)    │     │
                │     └─────────────────┘     │
                ▼                             ▼
┌─────────────────────────────┐ ┌─────────────────────────────┐
│     OpenRouter AI APIs      │ │     SMTP Email Service      │
│ (Multi-Key Failover Pool)   │ │  (Human-In-The-Loop Alerts) │
└─────────────────────────────┘ └─────────────────────────────┘

---

2. Service Responsibilities

Next.js Frontend (frontend/)

• Framework: Next.js 16 App Router with Turbopack, Tailwind CSS, Biome linter.
• Responsibilities: Unified Dashboard, AI Chat Interface, Vector Search Playground, Real-time WebSocket Log Streaming, Human-in-the-Loop Action Approvals.
• Config: [frontend/src/config/env.ts](file:///home/ms22/Coding_stuff/Personal-Projects/github-backup-automation-system/frontend/src/config/env.ts).

Python Observatory (agentic-observatory/)

• Framework: FastAPI, LangChain, asyncpg, SQLAlchemy, httpx, Jinja2.
• Responsibilities:
• Multi-turn AI Agent reasoning loop (invoke_agent, stream_agent).
• Hybrid Search: PostgreSQL Full-Text Search + pgvector cosine similarity + Reciprocal Rank Fusion (RRF).
• Automated and interactive embedding generation pipeline (embedding_generations, embedding_jobs, embedding_chunks).
• Human-in-the-loop report generation and SMTP dispatch (send_report_email).
• JWT Authentication for dashboard chat.
• Config: [agentic-observatory/config/settings.py](file:///home/ms22/Coding_stuff/Personal-Projects/github-backup-automation-system/agentic-observatory/config/settings.py).

Go Backend (backend/)

• Framework: Go Fiber v2, pgxpool.
• Responsibilities:
• Ingesting backup execution runs, repository results, and structured logs from the worker.
• Serving real-time WebSocket hub for active backup runs (/ws).
• Exposing database metrics and telemetry.
• Config: [backend/config/config.go](file:///home/ms22/Coding_stuff/Personal-Projects/github-backup-automation-system/backend/config/config.go).

Backup Worker (backup-worker/)

• Framework: Go CLI (backup-worker/main.go).
• Responsibilities:
• Discovering repositories from GitHub Organizations & Personal accounts.
• Cloning / pulling mirrors locally into backup-worker/_Repos/.
• Caching remote HEAD commit hashes in backup-worker/app.db.
• Recording telemetry, logs, and failure fixes to PostgreSQL.
• Config: [backup-worker/config/data.config.go](file:///home/ms22/Coding_stuff/Personal-Projects/github-backup-automation-system/backup-worker/config/data.config.go).

---

3. Database Schema

1. backup_runs: Stores each backup batch (ID, status, total repos, duration, timestamps, error_message).
2. backup_results: Per-repository outcome (status, errormessage, sizes, commithash).
3. execution_logs: Structured step-by-step logs.
4. analytics_snapshots: Aggregated metrics and commit snapshots over time (1-to-1 unique with backup_runs).
5. backup_fixes & backup_run_fixes: Historical failure resolutions and commit tags.
6. ai_chat_sessions & ai_session_metadata: Normalized conversation sessions and key-value metadata.
7. ai_chat_messages & ai_tool_calls: Chat history and granular tool execution telemetry.
8. investigations: Saved agent investigation traces, tool calls, and results.
9. embedding_generations, embedding_jobs, embedding_chunks: Vector index and chunk storage with pgvector and deterministic blue-green lifecycle management.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: MishraShardendu22
• Source: MishraShardendu22/agent-skills
• License: MIT
• Homepage: https://github.mishrashardendu22.is-a.dev

Install and usage instructions live in the source repository linked above.