Starrocks Upgrade

StarRocks Upgrade Skill

Compares two local branches of a StarRocks repository to identify upgrade risks. Four-phase analysis: data collection, per-commit diff analysis, deep impact investigation, and report synthesis. Runs 11 scanners + per-commit diff tier classification for comprehensive compatibility checking with unified impact assessment.

Requires local StarRocks source code. This tool operates entirely on local git history — no network calls needed (except optional --fetch-prs).

Prerequisites

• Python 3 (standard library only)
• git (for branch diff)
• gh (GitHub CLI, authenticated — only needed for --fetch-prs)
• PyYAML (optional — only needed for --cluster-profile)
• StarRocks official documentation (in local repo): docs/zh/ directory. The following docs MUST be referenced when generating the upgrade report:
• docs/zh/deployment/upgrade.md — Upgrade procedure (upgrade order, compatibility settings, caveats)
• docs/zh/deployment/deployment_prerequisites.md — Deployment prerequisites (JDK version, etc.)
• docs/zh/release_notes/ — Release notes for each version

Cluster Profile (Optional but Highly Recommended)

The cluster profile provides your production environment context so the analysis can produce cluster-specific risk assessment instead of generic findings.

What the profile enables:

• Config conflict detection: Find removed/changed configs that exist in your fe.conf/be.conf
• Deployment-aware guidance: K8s rolling restart triggers MV re-activation and leader transfer;

VM deployments follow a different upgrade order

• Scale-aware risk: A cluster with 120 MVs faces higher MV-compatibility risk than one with 5
• Targeted upgrade checklist: Only includes items relevant to your configuration

Profile Collection Flow

When the user triggers this skill, the agent should:

1. Check if cluster-profile.yaml exists in skills/starrocks-upgrade/
2. If it exists: Show a brief summary and ask if anything needs updating
3. If it doesn't exist: Proactively ask the user for the following information through conversation

What to Collect

Ask the user for these items (one by one or let them paste all at once):

1. Cluster name and deployment method: K8s / VM / mixed?

• K8s: rolling upgrade restarts pods → triggers MV re-activation, leader transfer
• VM: manual restart order → different upgrade sequence guidance

1. fe.conf content: Paste the full fe.conf from production FE node

• Used to: detect removed/invalid configs, check if overrides conflict with new defaults,

generate targeted checklist items

1. be.conf content: Paste the full be.conf from production BE node

• Same purpose as fe.conf but for BE side

1. Cluster scale (optional but helps prioritize):

• How many FE/BE nodes?
• Roughly how many tables and MVs?
• Using async MV, sync MV (rollup), or both?

How the Agent Assembles the YAML

After collecting info through conversation, the agent creates skills/starrocks-upgrade/cluster-profile.yaml:

cluster:
  name: prod-cn-east
  deployment: k8s               # k8s | vm | mixed
  scale:
    fe_nodes: 3
    be_nodes: 12
    tables: 800
    mvs: 120
    has_async_mv: true
    has_sync_mv: true

fe_conf: |
  LOG_DIR = ${STARROCKS_HOME}/log
  DATE = %Y%m%d-%H%M%S
  mysql_server_version = 5.1.0
  metadata_failure_recovery = false
  ...

be_conf: |
  LOG_DIR = ${STARROCKS_HOME}/log
  sys_log_level = INFO
  storage_root_path = /data/starrocks/storage
  mem_limit = 0.8
  max_tablet_version_count = 5000
  ...

Agent behavior:

• If the user pastes conf content directly, parse it into the YAML fe_conf: / be_conf: block literals
• If the user says "skip" or "I don't have it", proceed without profile

(analysis still works but won't have cluster-specific risk assessment)

• Save the assembled profile to skills/starrocks-upgrade/cluster-profile.yaml

so it persists across sessions

• When loading an existing profile, show a brief summary and ask if anything needs updating

Example Conversation

User: I want to do a StarRocks upgrade analysis, from 3.3.16-cj-0708 to 3.5.17-cj-0604

Agent: OK, let me check the cluster configuration first...
       No cluster-profile.yaml found. I need some cluster information:

       1. What is the cluster name and deployment method? (K8s / VM / mixed)
       2. Can you paste the production fe.conf content?
       3. Can you paste the production be.conf content?
       4. Cluster scale: how many FE/BE nodes? Roughly how many tables and MVs?
          Using async MV, sync MV (rollup), or both?

       You can provide all information at once, or tell me one by one.

User: Cluster name prod-cn-east, K8s deployment, 3 FE 12 BE, about 800 tables 120 MVs,
      both async MV and sync rollup.
      fe.conf: (paste full content)
      be.conf: (paste full content)

Agent: [Auto-parsed and saved cluster-profile.yaml]
       Saved cluster configuration profile:
       - Deployment: K8s
       - FE configs: 25 items
       - BE configs: 18 items
       - Scale: 800 tables, 120 MVs
       Starting upgrade analysis...

Config Conflict Detection Logic

When a cluster profile is provided, the tool automatically:

| Scenario | What happens | Risk | |----------|-------------|------| | Config removed + exists in your conf | Will cause startup warning/error | HIGH | | Config default changed + you use old default | You'll get the new default after upgrade | MEDIUM | | Config default changed + you have custom override | Your override takes precedence | LOW | | Config default changed (high-risk) + not in your conf | You'll get the new default | HIGH/CRITICAL |

This means: if you already override a config, default value changes are low risk. But if you rely on the default and it flips, that's a real risk the tool will flag.

Usage

Recommended workflow

Switch to the target branch, then compare against production:

cd ~/starrocks && git checkout 3.5.17-cj-0604
python3 skills/starrocks-upgrade/starrocks_upgrade.py --against 3.3.16-cj-0708

Explicit branch specification (both required)

python3 skills/starrocks-upgrade/starrocks_upgrade.py --branch-a  --branch-b 

With full PR details from GitHub (slow when there are many PRs)

python3 skills/starrocks-upgrade/starrocks_upgrade.py --against  --fetch-prs

With cluster profile for config conflict detection

python3 skills/starrocks-upgrade/starrocks_upgrade.py --against  --cluster-profile skills/starrocks-upgrade/cluster-profile.yaml

What the tool does

1. git log branchA..branchB to find commits only in B (batch, single call)
2. git log branchB..branchA to find commits only in A (batch, single call)
3. Extract PR numbers from commit messages (commit subject already contains PR title)
4. Categorize commits by type (feat/fix/refactor/perf/etc.)
5. Run 11 compatibility scanners (see below)
6. Classify all findings by risk level (critical/high/medium/low) with unified impact model
7. Cross-reference with local release notes from the target branch
8. Optionally fetch full PR details from GitHub with --fetch-prs (body, labels, files)

Compatibility Scanners

The tool runs 11 scanners to detect upgrade risks:

| Scanner | What it scans | Impact focus | |---------|--------------|--------------| | config | FE Config.java default value changes | Operational | | session_variables | SessionVariable.java session var defaults | Behavior | | system_variables | GlobalVariable.java system var defaults | Behavior | | be_config | BE config.h default value changes | Operational, Data | | protocol | .thrift / .proto IDL changes (removed fields, enum changes) | Rolling upgrade | | parser | SQL parser grammar/token changes | Behavior | | auth | Authentication and privilege manager changes | Operational | | storage_format | BE storage format, tablet metadata, encoding | Data, Rolling upgrade | | charset_collation | Charset and collation behavior changes | Data, Behavior | | type_system | Type conversion, varchar handling, schema changes | Data | | mv | Materialized view refresh, rewrite, partition, schema | Data, Behavior |

Unified Impact Model

Each finding includes an impact assessment across four dimensions:

• Data: Affects existing data (storage format, encoding, charset)
• Behavior: Changes query results for the same SQL
• Operational: Requires config changes, restarts, or object re-creation
• Rolling Upgrade: Breaks mixed-version cluster during rolling upgrade

Risk levels: critical > high > medium > low. Critical findings are always flagged prominently.

Options

• --output : Output directory (default: ./upgrade-report)
• --repo : Path to StarRocks repo (default: current directory)
• --skip-diff-detail: Skip per-commit diff generation (faster, but no commit-level diff analysis possible)
• --diff-stat-only: Only save --stat per commit, not full diffs (quick preview mode)
• --cluster-profile : Path to cluster profile YAML for config conflict detection

Output Structure

upgrade-report/
├── prs/                        # Individual PR details (only with --fetch-prs)
│   ├── 73237.json
│   └── ...
├── commits/                    # Commit details
│   ├── only-in-3.5.17-cj-0604.json
│   ├── only-in-3.3.16-cj-0708.json
│   ├── tiered-3.5.17-cj-0604.json   # Per-commit tier metadata (HIGH/MEDIUM/LOW/SKIP)
│   ├── tiered-3.3.16-cj-0708.json   # Per-commit tier metadata
│   └── detail/                      # Per-commit diff files (HIGH/MEDIUM only)
│       ├── abc123-diff.txt
│       └── ...
├── categories/                 # Categorized commits
│   ├── feat-in-3.5.17-cj-0604.json
│   ├── fix-in-3.5.17-cj-0604.json
│   └── ...
├── pr-diff.json                # PR number diff
├── incompatibilities.json      # All scanner results (config, session vars, BE config, protocol, parser, auth, storage, charset, type system, MV)
├── cluster-config-conflicts.json # Config conflict detection results (only with --cluster-profile)
├── release-notes-cross-ref.json # Release notes cross-reference
└── summary.json                # Overall summary with scanner counts, impact breakdown, and tier counts

Generating the Upgrade Report

After the script collects data, the agent should follow a four-phase analysis:

Phase 1: Collect (single agent)

1. Run the Python script — collects commits, scanner results, tier classifications
2. Read summary.json to understand overall scope, scanner counts, impact breakdown, and tier distribution
3. Read incompatibilities.json for all scanner findings, categorized by scanner and risk level
4. Read pr-diff.json to see which PRs are only in each branch
5. Read commits/tiered-*.json to get the per-commit tier metadata and identify HIGH/MEDIUM commits
6. Read each PR JSON in prs/ (if --fetch-prs was used) for detailed impact analysis
7. Read cluster-config-conflicts.json (if --cluster-profile was used) for config conflict detection

results, deployment-specific risks, and scale assessment

1. Read official upgrade documentation from the StarRocks repo:

• docs/zh/deployment/upgrade.md — Get the correct upgrade procedure (upgrade order, compatibility config steps)
• docs/zh/deployment/deployment_prerequisites.md — Get prerequisites for the target version (JDK version, etc.)

1. Identify all findings requiring deep analysis:

• Scanner HIGH/CRITICAL findings
• Cluster config conflicts (HIGH risk: removed configs in your conf)
• Deployment-specific risks (K8s: pod restart triggers; VM: upgrade order)
• HIGH tier commits with diffs
• MEDIUM tier commits that may have compatibility impact

Phase 2: Commit Diff Analysis (parallel subagents)

This phase analyzes per-commit diffs for HIGH and MEDIUM tier commits. This is critical because the 11 scanners only cover specific file patterns — changes to core modules like the optimizer, executor, or catalog may introduce incompatibilities that scanners miss.

Group commits by module for subagent batching:

Subagent A: Optimizer/Planner commits (5-8 commits per subagent)
Subagent B: Storage engine commits (5-8 commits)
Subagent C: Protocol/RPC commits (5-8 commits)
Subagent D: MV/refresh/rewrite commits (5-8 commits)
Subagent E: Catalog/metadata commits (5-8 commits)
Subagent F: Other MEDIUM tier commits (10-15 commits, summary analysis)
...

Target: 3-8 parallel subagents

Subagent prompt template for commit diff analysis:

You are a StarRocks upgrade compatibility analyst. Analyze the diff of the following commits and assess upgrade risks.

## Upgrade Context
- Source branch: {branch_a}
- Target branch: {branch_b}
- Your assigned module: {module_name}

## Commits to Analyze

### Commit 1: {subject}
- Hash: {hash}
- PR: #{pr_number}
- Tier: HIGH
- Tier reason: {tier_reason}
- Changed files: {file_list}

Diff:
{diff_content}

---

### Commit 2: ...

## Analysis Requirements

For each commit, output the following structured result:

1. **compatibility_impact**: Are there incompatible changes? [YES/NO]
2. **impact_type**: [API_BREAKING | BEHAVIOR_CHANGE | DATA_FORMAT | CONFIG_REQUIRED |
   ROLLING_UPGRADE_RISK | ERROR_MESSAGE_CHANGE | DEPRECATION | NONE]
3. **severity**: [CRITICAL | HIGH | MEDIUM | LOW]
4. **summary**: One-sentence description of the change and its risk
5. **incompatible_detail**:
   - Which interface/behavior/data format changed
   - What happens to old-version clients/old data after upgrade
   - Whether it causes issues in a mixed-version cluster
6. **error_scenario**: If incompatible, the specific error that may appear after upgrade (include the exact error message text)
7. **reproduction**: Reproduction steps, format:
   - Precondition: which version, what objects to create
   - Action: what operation to perform (upgrade/restart/DDL/DML)
   - Expected result: behavior before upgrade
   - Actual result: behavior/error after upgrade
   - Verify fix: how to verify the fix (config rollback/restart/expected result)
8. **affected_callers**: Affected callers (key call sites to confirm via grep)
9. **rollback**: Can it be rolled back? Is it a one-way migration?

## Evaluation Principles
- Prefer false positives over false negatives: if unsure whether compatible, mark as HIGH
- Watch for indirect impacts: a method signature change may break all callers
- Key focus areas: type system changes, null handling changes, default value flips, exception type changes, serialization format changes, SQL semantics changes
- Any deleted public method/class = CRITICAL
- Any method signature change without backward compatibility = HIGH
- Any error message format change = MEDIUM (may break monitoring/alerting)
- Watch for K8s restart scenarios: will FE/BE pod restart trigger issues?
  - MV re-activation via AlterJobMgr.java
  - FE leader transfer via GlobalStateMgr.transferToLeader()
  - BE startup via StorageEngine.open()
  - Metadata reload via GlobalStateMgr.loadImage()

Subagent output format (JSON):

{
  "module": "optimizer",
  "commits_analyzed": 6,
  "findings": [
    {
      "commit_hash": "abc123",
      "subject": "fix: handle null in varchar type comparison",
      "pr_number": 73237,
      "compatibility_impact": "YES",
      "impact_type": "BEHAVIOR_CHANGE",
      "severity": "HIGH",
      "summary": "ScalarType.isTypeCompatible() logic changed for VARCHAR(NULL), may cause schema check failure during MV re-activation",
      "incompatible_detail": "Old version treated VARCHAR(10) and VARCHAR(NULL) as compatible types; new version no longer allows this. On FE restart, MV re-activation calls Column.isSchemaCompatible() — if the MV definition contains VARCHAR columns, schema check failure causes the MV to become inactive",
      "error_scen

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [crossoverJie](https://github.com/crossoverJie)
- **Source:** [crossoverJie/skills](https://github.com/crossoverJie/skills)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.