Ai Researcher

AI Researcher

When to Use

• Surveying state-of-the-art models, methods, or benchmarks
• Comparing model families or techniques with fair experimental design
• Designing ablation studies with controlled variables
• Writing research memos or technical reports for stakeholder decisions
• Critiquing methodology in papers or internal experiments
• Planning novel experiments with falsifiable hypotheses
• Reproducing published results and verifying claims

When NOT to Use

• Shipping production LLM features, RAG, or agent systems → ai-engineer
• Enterprise AI policy, regulation, or risk tiering → ai-risk-governance
• Adversarial product testing or jailbreak campaigns → ai-redteam
• Classical ML pipelines, A/B testing, or statistical analysis → data-scientist

Related skills

| Need | Skill | |---|---| | Production RAG, agents, deployment | ai-engineer | | Prompt and agent implementation detail | prompt-engineer | | Classical ML and A/B statistics | data-scientist | | Governance, regulation, risk registers | ai-risk-governance | | Red-team attacks on deployed systems | ai-redteam | | Token/context efficiency research | research-engineer-scientist-tokens | | Safeguard ML benchmarks and classifiers | ml-research-engineer-safeguards | | RL distributed training infrastructure | ml-systems-engineer-rl-engineering |

Core Workflows

1. Research question framing

1. Convert vague ask into falsifiable question
2. Define scope: task, data regime, compute budget, timeline
3. List baselines that must be beaten or matched
4. Specify primary and secondary metrics
5. Document assumptions and out-of-scope items

See references/research_framing.md for question templates and hypothesis types.

2. Literature review

Process:

1. Search: arXiv, ACL Anthology, OpenReview, major labs' blogs
2. Screen by relevance, recency, citation quality
3. Extract: problem, method, data, metrics, limitations
4. Synthesize themes and open gaps
5. Cite primary sources; avoid over-relying on secondary summaries

See references/literature_review.md for screening matrix and synthesis outline.

3. Experimental design

| Element | Requirement | |---|---| | Baselines | Strong and fair (same data, tuning budget) | | Ablations | One change at a time | | Seeds | Multiple runs for stochastic methods | | Stats | Confidence intervals, not single-point luck | | Reproducibility | Config, data version, code commit logged |

See references/experiment_design.md for power analysis pointers and leakage checks.

4. Benchmarking and analysis

• Use public benchmarks when task-aligned; document train/test contamination risk
• Report compute cost (GPU hours) alongside accuracy
• Separate in-distribution vs stress tests
• Visualize failure modes, not only aggregate scores

See references/benchmarking.md for leaderboard caveats and custom eval sets.

5. Research communication

Deliverable types: memo (1–3 pages), technical report, slide deck for decision meeting.

Include: question, method summary, results table, limitations, recommended next step.

See references/research_writing.md for memo structure and peer-review checklist.

When to load references

• Question and hypothesis → references/research_framing.md
• Literature survey → references/literature_review.md
• Experiments → references/experiment_design.md
• Benchmarks → references/benchmarking.md
• Writing → references/research_writing.md

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: daemon-blockint-tech
• Source: daemon-blockint-tech/Agentic-Enteprises-Skill
• License: MIT

Install and usage instructions live in the source repository linked above.