Skill Feedback Adaptation

Skill Feedback Adaptation

Track when users push back on agent answers or skill-driven behavior, surface inefficiency in the Usage Report dashboard, and propose skills when a new task starts.

AUTO-START (new session / new task)

Run immediately — before any other task work — when any of these is true:

• Session hook injects [Claude Skills] NEW SESSION (Claude Code SessionStart,

Cursor sessionStart, Kiro sessionStart, Copilot SessionStart via profile-init-watch.js).

• User opens a new chat or describes a new task (feature, bug, review,

assessment) in their first message.

• User asks which skills fit the current task.

Do this first:

1. Read this skill (you are here).
2. Check .claude/learning/task-skill-proposals.json — if generatedAt is **` — it will appear in the

Usage Report inefficiency panel.

CLI helper (from project root):

py record_feedback.py ci-pipeline-debug --signal "no" --user-text "no wrong job" --context "Suggested wrong CI stage"

2. Dashboard inefficiency (extension)

The VS Code Usage Report reads skill-feedback.jsonl and shows an Inefficient skills panel:

• Inefficiency % — scales with negative feedback count (more feedback →

higher % and deeper red heat).

• Update suggestion — short actionable hint per skill.
• Skills with 3+ negative reports are prioritized.

You do not need to regenerate this manually — the extension computes it on report open. After recording feedback, tell the user they can open Claude Skills: Show Usage Report to see updated scores.

High token usage notification

When a git branch or active task (from task-skill-proposals.json) uses more than the configured share of monthly credits (default 50%, setting claudeSkills.skillFeedback.monthlyCreditThresholdPercent), the extension shows a popup offering to Apply suggested skills from the proposal set.

Settings (claudeSkills.skillFeedback.*):

• promptOnHighUsage — enable/disable the popup (default on).
• monthlyCreditThresholdPercent — threshold % (default 50).
• monthlyCreditsUsd — monthly budget baseline; 0 uses daily budget × 30 or

30-day workspace spend.

Manual apply: Claude Skills: Apply Suggested Skills for Current Task.

3. Propose skills for a new task

When to run: AUTO-START (above), user starts a clearly new task (new feature, bug area, refactor scope) — especially the first message describing what they want to build or fix — or asks "which skills should I use for this?".

Steps:

1. Read the user's task prompt (goal, files mentioned, stack).
2. Scan the repo: Glob/Grep for file types, CI configs, infra, docs.
3. Read the skill library manifest.json (extension bundled

skills_library/manifest.json or ~/.claude/skills/ catalog / .claude/learning/skills-catalog.json).

1. Cross-reference:

• detect_globs matches in the workspace
• Task keywords vs skill names/descriptions
• Already installed skills in .claude/skills/
• Past negative feedback — deprioritize skills with high inefficiency in

skill-feedback.jsonl unless no alternative exists

1. Write .claude/learning/task-skill-proposals.json:

{
  "version": 1,
  "generatedAt": "2026-06-13T10:05:00.000Z",
  "taskSummary": "Fix GitLab CI deploy stage failing on feature branch",
  "promptExcerpt": "deploy job fails after terraform plan...",
  "proposals": [
    {"name": "ci-pipeline-debug", "reason": "GitLab CI failure debugging", "confidence": 90, "installed": true},
    {"name": "terraform-plan-review", "reason": "Task mentions terraform plan errors", "confidence": 75, "installed": false}
  ]
}

1. Present the top 3–5 proposals in chat. When autoApplyTaskProposals is on (default), say

skills are installed locally — do not suggest manual install commands unless auto-apply is disabled or a skill shows "installed": false. When taskSkillFocus is on (default), non-proposed installed skills are on the task ignore list (skillOverrides: off) — do not load their SKILL.md files.

Regenerate this file when the task scope changes materially — overwrite the previous proposals.

Task scope drift (extension auto-refresh)

When claudeSkills.features.taskDriftReproposal is on (default), the extension may overwrite task-skill-proposals.json without waiting for a new chat:

• Off-profile use — runs.jsonl hook rows with metadata.not_in_active_profile: true

reach claudeSkills.skillFeedback.taskDriftMinOffProfileInvokes (default 2).

• Large session — session-watch.json reports warn or critical transcript size

(threshold: taskDriftSessionSizeLevel, default warn).

On drift the extension re-applies task focus, may auto-apply proposals, and injects a one-time message via task-drift-watch (Claude UserPromptSubmit, Cursor beforeSubmitPrompt, Kiro promptSubmit, Copilot UserPromptSubmit) or on the next session start via profile-init-watch when a prompt was queued while the IDE was idle.

Agent behavior after drift inject:

1. Follow the refreshed active skill set — do not reload ignored skills.
2. Refine task-skill-proposals.json only if the user's stated goal changed.
3. Prefer /compact when the inject mentions a large session.

4. Integration with other skills

• [[self-learning]] — run outcomes (runs.jsonl); feedback is complementary

(user sentiment vs command exit codes).

• [[skill-usage-insights]] — KPI report; include inefficiency from

skill-feedback.jsonl when analyzing skill health.

• [[profile-init]] — branch profile picks skills; task proposals are

narrower and session-scoped.

5. Reporting

On request ("skill feedback status", "inefficient skills"):

• Count entries per skill in skill-feedback.jsonl.
• Summarize latest task-skill-proposals.json if present.
• Point to Usage Report for visual heat map.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: sergeyitaly
• Source: sergeyitaly/claude-skill-deployer
• License: MIT

Install and usage instructions live in the source repository linked above.