Blog Topic Research

blog-topic-research

Generates topic candidates for a blog with documented user demand. The skill exists to fight hallucinated SEO ideas: every topic it proposes must point to a URL that proves someone is asking about it.

research  topics [for cluster ] [--append-to ]

• N - number of topics to return (default 50; cap 100)
• cluster - if the blog has cluster taxonomy, restrict to one cluster the user names
• --append-to - after presenting results, ask the user before appending accepted topics as JSON to the given path (a backlog file, a CSV, whatever the blog uses)

The skill is content-only: it does no scraping of its own. It drives the agent's WebFetch and WebSearch tools to fetch sources, and (optionally) shells out to a Python similarity script for the cannibalization step.

---

The contract

For every topic the skill emits, it captures:

| Field | What it is | |---|---| | topic | Full title shaped like a long-tail query | | cluster | A bucket the user defines for their blog (e.g. n8n, databases, react-hooks) | | format | One of how-to-fix, how-to-connect, how-to-automate, x-vs-y, what-is, use-case, listicle, migration, release-recap | | demand_signals[] | One or more, each with type, url, evidence (verbatim text), strength (1-3) | | signal_score | Sum of strength across all signals; topic accepted only if >=3 | | primary_sources[] | At least 1 vendor doc / GitHub issue / official changelog URL | | keywords[] | Primary keyword + 3-5 LSI variants extracted from source text | | commentary | 1-2 sentences on what makes this topic specific (no fluff) | | problem_summary | 1-2 sentences distilling the symptom + trigger from the highest-engagement signal's body, in factual writer-voice (no marketing). Lets the writer skip re-fetching to figure out what the problem actually is. | | confirmed_fixes[] | Each {kernel, source}: a short fix kernel (one phrase, e.g. "set N8N_PAYLOAD_SIZE_MAX=16000000", "downgrade crewai to 0.113") plus the source URL where that fix is reported. Empty list if no fix is documented yet (still-open issues count). The writer expands kernels into prose and re-verifies. | | version_context | String like "n8n 1.65+", "Cursor 0.42 only", "introduced in CrewAI 0.114", or null if no version qualifier applies. | | question_variants[] | 2-4 paraphrases of the topic that real users actually post (lifted from PAA, autocomplete depth-2, sibling forum titles). Feed the writer's FAQ block + LSI keyword spread directly. Not invented - every variant must trace to a captured signal or autocomplete completion. |

Hard rules:

• No URL, no topic. If the skill cannot cite a verifiable demand signal, the topic is dropped. No "this seems like a good topic" reasoning.
• No paraphrased evidence. The evidence field is copied byte-for-byte from the source (PAA question text, GitHub issue title, Reddit post title, forum thread title).
• No invented version numbers, prices, or stats. All numbers in the topic title must come from a source URL or be omitted entirely.
• Signal score >=3. Each signal scores 1 (exists), 2 (engaged), or 3 (heavy engagement) per the strength tiers below. One 3-star signal is enough; three 1-stars is enough.
• Specificity floor. Title must be either >=7 words OR contain a concrete qualifier: a version number, an error code/string, a named integration pair (X to Y), or a named edge case. Reject vague titles like "n8n tutorial" or "what is automation".
• Cannibalization check. Three layers when the user supplies an existing-titles cache: (1) Jaccard token overlap >=0.6 against any cached title = drop. (2) Cosine similarity >=0.85 = drop. (3) Token-dupe (shared distinctive numeric/error-code + shared tool keyword, regardless of cosine rank) = drop. Cosine 0.75-0.85 with no token-dupe = REVIEW (kept but flagged). If the user has no cache, run only Jaccard and skip the rest with a one-line note in the summary footer.
• Substance distilled, not invented. problem_summary, confirmed_fixes[], version_context, and question_variants[] are derived from fetched bodies - never hallucinated. If a body doesn't mention a fix, confirmed_fixes[] stays empty. If no version is named, version_context is null. The writer treats these as a verified scaffold and still re-fetches at least one primary source for currency.
• No padding. If the skill cannot reach N validated topics, it returns what it has and reports the shortfall.

---

Demand-signal taxonomy

A signal is valid only if its type is one of these, and the linked URL contains the verbatim evidence text:

| Type | What counts | |---|---| | paa | "People Also Ask" question on a Google SERP. URL = the parent query SERP. Evidence = the PAA question text. | | autocomplete | Google Suggest entry triggered by typing a partial query. Evidence = the suggested completion. | | reddit | A Reddit post asking the question or a close variant on a relevant subreddit. Evidence = the post title. | | stackoverflow | A Stack Overflow question with the same intent. Evidence = the question title. | | github_issue | A GitHub issue on the tool's repo describing the problem (open or closed). Evidence = the issue title. | | forum | A community-forum thread (vendor or third-party). Evidence = the thread title. | | vendor_doc | A vendor docs page that exists because users ask the question. Evidence = the page heading. | | trends | A Google Trends rising query. Evidence = the query string + the rising-percent label from Trends. | | keyword_data | Real search-volume + keyword-difficulty from DataForSEO Labs (see [Step 3d](#step-3d---dataforseo-volume--difficulty-enrichment)). URL = dataforseo:labs/keyword_overview. Evidence = "" search_volume=/mo, keyword_difficulty=, main_intent=, location=US. This is the one quantitative signal — it carries an absolute volume figure because it comes from a real API, not an estimate. |

Strength tiers (used to compute signal_score):

| Type | 1 (exists) | 2 (engaged) | 3 (heavy) | |---|---|---|---| | paa | appears once | appears across >=2 parent SERPs | appears + "More questions" expansion shows >=4 follow-ups on same intent | | autocomplete | direct suggestion | suggestion at >=4-word depth | suggestion at >=6-word depth | | reddit | post exists | >=10 comments OR >=20 upvotes | >=50 comments OR >=100 upvotes | | stackoverflow | question exists | score >=3 OR views >=500 | score >=10 OR views >=2000 | | github_issue | issue exists | >=3 reactions OR >=5 comments | >=10 reactions OR >=20 comments OR linked from changelog | | forum | thread exists | >=5 replies | >=20 replies OR pinned / marked solution | | vendor_doc | page exists | page in main nav | dedicated FAQ entry or "Common errors" section | | trends | rising query | rising >=100% | rising >=500% or labelled "Breakout" | | keyword_data | searchvolume >=30 | searchvolume >=100 AND KD =300 AND KD and r/`.

• Recent docs changes (a doc page that was edited last week often answers a recurring question).

Vertical clusters (e.g. ecommerce, customer-support, marketing-ops)

• Subreddits dedicated to the vertical (r/ecommerce, r/customerservice, etc.).
• Vendor template galleries (e.g. n8n's workflow gallery, Make's scenario library, Zapier's app directory) - each gallery entry is a verified user-demand signal because people search the named outcome.
• Forum threads asking "how do I [outcome]" with high engagement.

Always also mine

• Google Suggest autocomplete for the tool / language / vertical names.
• People Also Ask boxes on SERPs for the same.
• Google Trends rising queries for the cluster's main terms.

If the user has a specific cluster list, ask them for the source URLs before mining; if they don't, suggest a list and let them edit it.

---

Process

Step 1 - Inventory existing coverage (cannibalization prep)

If the user has an existing backlog file (backlog.json, posts.csv, an RSS feed of published posts, whatever), ask for the path and a way to enumerate titles. The cannibalization step needs an embedding cache built from these existing titles.

If a Python embedding script is available, the user can build a cache (a JSON file mapping each existing title to its OpenAI text-embedding-3-small vector) and point the cannibalization step at it. Typical cost is ~$0.02 per 1M tokens.

If no Python / no cache, fall back to Jaccard-only cannibalization (cheap, less accurate; surfaces near-identical titles but misses synonym dupes).

Also build a token-set per existing title (lowercase, alphanumeric, stopwords stripped) for the Jaccard prefilter used in Step 4.

Step 2 - Compute cluster targets

If the user has cluster weights (e.g. "30% n8n, 20% AI coding, ..."), apply them: subtract current backlog counts per cluster, then distribute N proportionally to the largest deficits.

If cluster was specified, allocate all N to that cluster.

If no cluster taxonomy at all, treat the whole blog as one cluster and aim for format diversity instead (see Step 3 format matrix).

Step 3 - Mine candidates per cluster

For each cluster, walk its source list. For each source URL:

1. Fetch via WebFetch (or WebSearch for SERP-derived signals).
2. Extract candidate query strings: GitHub issue titles, PAA questions, Reddit post titles, forum thread titles, autocomplete suggestions.
3. Record the source URL + the verbatim title text + the engagement count (reactions, comments, upvotes, replies) so a strength tier can be assigned later.
4. Mine the body, not just the title. For each high-engagement issue / thread / SO question, fetch the body and extract:

• Error strings (lines like Error:, TypeError:, Traceback, exception class names, stack-frame headers).
• Code blocks between triple-backticks (often contain the failing snippet that names the real edge case).
• Version-qualified phrases ("after upgrade to 1.65", "on Cursor 0.42", "since the v3 release").

Each extracted string becomes a candidate seed in its own right. Long-tail troubleshooting queries are usually the literal error message, which never appears in the post title.

Google-side seeds - use WebSearch with these patterns and parse for autocomplete + PAA. Run the full set per cluster, not just the troubleshooting one - long-tail diversity is what stops the corpus drifting into all-how-to-fix:

| Pattern | Format target | |---|---| | , not working, stuck, fails | how-to-fix | | how to , connect to | how-to-connect / how-to-automate | | vs , or | x-vs-y | | what is , explained, how does work | what-is | | build with , for (for ecommerce, for SaaS, for marketing, for customer support), agent for | use-case | | best , top , free , alternatives, templates for | listicle | | migrate from to , switch from to , to migration | migration | | what's new in , changelog, features, release notes | release-recap |

Repeat the matrix per tool / framework / vertical in the cluster.

Per-format source pointers (in addition to the per-cluster source list above):

• use-case - vendor template galleries are the highest-signal source. Each gallery entry is a verified user-demand signal (people search the named outcome). Reddit threads like "how do I [outcome] with [tool]" with many comments count too. Drop any seed whose only signal is an existing SEO blog's listicle - that's a competitor signal, not a user-demand signal.
• listicle - Google autocomplete depth-2 on best and top ; competitor roundup SERPs (look at what the top 3 results list). Reddit posts asking "what's the best X for Y" with many comments score high.
• migration - Reddit threads with titles starting moving from / switching from; Stack Overflow questions about exporting + re-importing between two tools; vendor migration docs (the doc exists because users ask).
• release-recap - the vendor changelogs already in the source list, but mine for specific versions shipped in the last 90 days. A release-recap post for is only worth writing if (a) that version is current or one minor behind, and (b) the changelog has at least one user-facing entry, not just internal refactors. Older recaps go stale fast.

Each extracted query becomes a candidate. The source is its first demand signal.

Step 3b - Recursive autocomplete pass

For each candidate that survived Step 3, append one more qualifier word and re-query Google Suggest. Keep any deeper completion that returns a new variant. This is what gets you actual long-tail vs mid-tail - one autocomplete pass surfaces "how to fix n8n webhook error", a second surfaces "how to fix n8n webhook error 404 after restart".

Common qualifiers to try (try several, keep what returns a real completion):

error, not working, after update, stuck, slow, timeout, free, limit, vs , `, self-hosted, cloud, docker`

Cap at 2 recursive passes to bound runtime. Each new completion inherits the parent's first signal and must still pass Step 4 on its own.

Step 3c - Distill substance from the source body

For each candidate that survived Step 3b, build the writer-facing scaffold by re-reading the body of the highest-engagement demand signal (the one that scored 2 or 3 in the strength table - usually a high-reaction GitHub issue, pinned forum thread, or popular SO question). If you already fetched the body in Step 3.4, reuse it; otherwise fetch now.

Extract four fields:

1. problem_summary - 1-2 sentences in writer-voice describing the symptom + trigger. Factual, no marketing copy. Example: "n8n's HTTP Request node returns 401 when an OAuth2 credential's access token has expired and the refresh-token grant is missing the offline_access scope." Pull verbs and nouns from the body; don't paraphrase the title.

1. confirmed_fixes[] - list of {kernel, source} entries. Each kernel is one short phrase capturing the action (env var to set, version to downgrade to, setting toggle, code edit). The source URL is where the fix is reported (forum reply, vendor doc, GitHub commit, changelog entry). Walk the thread replies, accepted-answer block, vendor "common errors" page. Skip noise: ignore "have you tried restarting" or fixes contradicted by later replies. If the issue is genuinely still open with no working fix, leave confirmed_fixes[] empty - the writer will frame the post around "what's known so far" rather than fabricating a fix.

1. version_context - extract any version-qualified phrase the body or thread uses to scope the problem ("after upgrade to n8n 1.65", "Cursor 0.42 only", "introduced in CrewAI 0.114", "Power Automate desktop V2"). If multiple versions are named, pick the one most closely tied to the failure. If the body is version-agnostic, set null.

1. question_variants[] - 2-4 near-paraphrases of the topic that surface in PAA boxes, autocomplete depth-2 completions, sibling forum thread titles, or SO related-questions. Each variant must be a real captured string from a source - do not invent paraphrases. These feed the writer's FAQ block and LSI keyword spread.

If any of these can't be filled honestly from the body, leave the field at its empty default ("", [], null) rather than fabricating. A sparsely-filled scaffold is more useful than a hallucinated one - the writer can re-research, but cannot un-trust a polluted blob.

Step 3d - DataForSEO volume & difficulty enrichment

If DATA_FOR_SEO_API_BASE64 is configured in the environment, enrich every surviving candidate with real Google search-volume + keyword-difficulty. This is the quantitative half of demand validation — the qualitative URL signals prove someone asks; this proves *

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: AutomateLab-tech
• Source: AutomateLab-tech/publishing-skills
• License: MIT-0
• Homepage: https://automatelab.tech/products/skills-and-plugins/publishing-skills/

Install and usage instructions live in the source repository linked above.