Paper Scout

Paper Scout: Daily Auto-Skill & Auto-Rubric Paper Finder

When to Use

• Daily check for new papers in the Auto-Skill / Auto-Rubric domain
• When you want to update the awesome list with recent publications
• When you want a summary of noteworthy new papers on self-evolving agents or rubric learning

Search Topics

Search arxiv for papers matching these keyword groups. Combine multiple queries to maximize coverage.

Auto-Skill Keywords

• "self-evolving" agent skill
• "skill evolution" LLM agent
• "skill creation" agent
• "skill library" agent
• "skill discovery" agent
• "agentic evolution"
• "self-improving" agent skill
• "harness evolution"
• "skill reuse" LLM

Auto-Rubric Keywords

• "Auto-Rubric" reward
• "rubric" "preference" reward LLM
• "rubric learning" alignment
• "rubric-based" reward
• "rubric generation" LLM
• "criteria" "preference" reward modeling

Procedure

Step 1: Search for New Papers

Search arxiv for recent papers using the keyword groups above. Use the fetch_webpage tool to query:

https://arxiv.org/search/?query=KEYWORDS&searchtype=all&order=-announced_date_first

Run multiple searches across both Auto-Skill and Auto-Rubric keyword groups. Focus on papers from the last 7-14 days.

Step 2: Filter and Evaluate

For each candidate paper found:

1. Fetch the arxiv abstract page to get full details (title, authors, date, abstract)
2. Evaluate relevance — must be directly about:

• Self-evolving agent skills / skill libraries / skill creation / skill evolution, OR
• Rubric learning from preferences / rubric-based reward modeling / auto-rubric generation

1. Skip papers that only tangentially mention these topics

Step 3: Present Recommendations

Present findings to the user in this format:

## New Papers Found (Date Range)

### Auto-Skill
1. **Paper Name** (arXiv:XXXX.XXXXX, Date)
   - TL;DR: One-sentence summary
   - Why it's relevant: Brief note

### Auto-Rubric
1. **Paper Name** (arXiv:XXXX.XXXXX, Date)
   - TL;DR: One-sentence summary
   - Why it's relevant: Brief note

### Verdict
- 🔥 Must-add: [list]
- 👀 Worth watching: [list]
- ⏭️ Skip: [list with reasons]

Step 4: Update the Repo (upon user confirmation)

After the user confirms which papers to add:

1. Read the current README.md at c:\Users\t-ihchen\Downloads\IHChen\Github\Awesome-AutoSkill-AutoRubric\README.md
2. Determine the correct section for each paper:

• Auto-Skill > Skill Learning & Evolution: Papers about skill creation, evolution, reuse
• Auto-Skill > Benchmarks & Evaluation: Benchmarks for skill evaluation
• Auto-Skill > Surveys & Frameworks: Survey papers, frameworks
• Auto-Rubric > Rubric Learning from Preference Data: Papers on learning rubrics from preferences
• Auto-Rubric > Rubric-Guided RL & Reward Modeling: Papers using rubrics for RL/reward
• Auto-Rubric > Rubric-Based Evaluation & Judges: Papers on rubric-based evaluation

1. Insert the new entry in chronological order within the section
2. Follow the table format: | **Name** | Date | [arXiv:ID](link) | [Code](link) or - | TL;DR |
3. Commit and push:

`` git add . git commit -m "Add: Paper1, Paper2, ..." git push ``

Important Notes

• Always check if a paper is already in the README before adding
• Use the paper's v1 submission date for the "Date" column
• If a paper has a GitHub repo, include it; otherwise use -
• Note venue acceptance if mentioned (e.g., ICML 2026, ACL 2026)
• TL;DR should be 1-2 sentences, focusing on the key contribution

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: AaronCIH
• Source: AaronCIH/Awesome-AutoSkill-AutoRubric
• License: MIT

Install and usage instructions live in the source repository linked above.