News Monitoring

News Monitoring & Digest Delivery

Set up recurring topic/news monitoring that searches the web for recent articles, compiles a headline digest, and delivers it to a messaging channel or document pipeline.

Workflow

Step 1 — Define Topic

Work with the user to define:

• Subject: what to monitor (e.g. PBSA in Hong Kong, AI regulation, self-hosting tools)
• Geography: which region(s) matter (e.g. Hong Kong, Singapore, global)
• Angles: which aspects to track — investment/deals, policy/regulation, supply/pipeline, occupancy/rental, general commentary
• Velocity: how fast-moving the field is — determines acceptable article age (see Step 5)

Step 2 — Create Search Keywords

Build search queries from the topic definition:

• Multi-angle: one query per angle (investment, policy, supply, etc.)
• Multi-language: English keywords + native language keywords for each geography
• Time-bounded: specify recency in the query where possible (e.g. "2026", "latest")
• Keep queries concise — 3-7 terms works best. Overly long queries return thin results.

Step 3 — Select Sources

Identify which sources matter for the topic and geography:

• English-language: international outlets, local English press, industry publications
• Native-language: local press in the geography's primary language(s) — these often break news first
• Industry-specific: trade publications, research houses, REIT/sector-specific sites
• Closed-ecosystem platforms (小红书, 抖音, 微信公众号): content is NOT web-indexed. Be upfront about this limitation — options are manual monitoring or third-party analytics tools.

Record the source list in the topic's reference file (see Reference Files section).

Step 4 — Run Searches

Date grounding (mandatory). Before running any search, ground the model in the real current date. LLMs default to training-cutoff years in queries — this produces stale results. Inject:

> Today's date is {current date as "DD Month YYYY"}. When a search query needs a year or refers to "latest"/"current"/"this year", use {current year} or relative wording — never a year inferred from training data.

Primary method: use the harness web_search tool.

web_search(query="Hong Kong PBSA student accommodation investment 2026", limit=10)

The tool uses whatever backend is configured in config.yaml (e.g. SearXNG → DDGS fallback). The skill is backend-agnostic — it does not prescribe which search engine to use. Run one web_search call per query from Step 2.

Advanced option: SearXNG direct curl for news-specific filters.

When you need news-category filtering, language selection, or time bounding that web_search doesn't expose:

# English news, last month
curl -sL "${SEARXNG_URL}/search?q=ENCODED_QUERY&format=json&categories=news&language=en&time_range=month"

# Chinese news, last month
curl -sL "${SEARXNG_URL}/search?q=ENCODED_ZH_QUERY&format=json&categories=news&language=zh-Hant&time_range=month"

Key SearXNG parameters:

• format=json — mandatory (omitting returns 403)
• categories=news — news-only results
• language=zh-Hant / language=en
• time_range=day|week|month|year
• Results include direct source URLs (no proxy resolution needed)

RSS feeds for curated must-track sources that search engines may not surface quickly:

curl -sL "https://example.com/feed/"

Parse ` blocks for , , , `.

Step 5 — Process Results

Go through all search results and:

1. Quality filter — before any ranking, discard low-quality results:

• Thin content: page is a landing page, aggregator stub, or has no substantive text (2-3 months old may be stale — flag them or exclude
• Slow-moving fields (real estate, infrastructure, policy): articles up to 6 months old may still be relevant
• Use the velocity defined in Step 1 to calibrate

1. Filter — remove articles outside the topic scope, outside the geography, or behind the recency threshold

Step 6 — Compare With Previous Digests

If this is a recurring digest (cron job), check for overlap:

• Use context_from on the cron job to inject the previous digest output as context (see Cron Setup section)
• Compare new results against the previous digest — match by title similarity, not exact string match
• Remove articles that already appeared in the previous run
• For niche/low-activity topics where overlap is high, consider broadening scope or reducing frequency

Step 7 — Compile Digest

Format the ranked, deduplicated results into a readable digest:

📰 [TOPIC] — Headline Digest
🗓 [Today's date as DD Mon YYYY]

1. [Date] — Source
   **Headline**
   1-2 line summary. Key stat or implication.
   🔗 Direct source URL

2. [Date] — Source
   ...

---
🔑 Key themes:
- Theme 1: brief summary
- Theme 2: brief summary

Formatting rules:

• Always include the publish date on every item
• Headline + 1-2 line summary (not a paragraph analysis)
• Sort by priority/impact (highest first)
• Source links as 🔗 + direct URL to the publisher's site — never Google News proxy URLs or Bing apiclick URLs
• Use 🆕 for articles from the last 7 days
• Max items: 5 by default (configurable per topic). Reduce frequency rather than padding with stale content.
• Key themes summary at the bottom — one-line takeaways of dominant trends

URL rules:

• Every link MUST resolve directly to the article on the publisher's website
• Google News RSS links (news.google.com/rss/articles/CBMi...) are protobuf-encoded proxy URLs — unresolvable by any method. If using Google News RSS for discovery only, resolve the actual source URL via SearXNG title-search or web_search before including.
• Bing News RSS links are apiclick.aspx redirects — extract the real URL from the url= parameter.
• If resolution fails: include the article with ⚠️ Direct link unavailable — search [SOURCE] for: [title]

Step 8 — Delivery

Deliver the digest to the user via one of:

• Chat channel (Telegram, WhatsApp) — most common. Use deliver parameter on cron jobs (e.g. deliver=telegram, deliver=whatsapp:). WhatsApp requires an explicit target — bare deliver=whatsapp fails.
• Document generation — pass the compiled digest to a document generation skill to produce a formatted PDF, slide deck, or article. The skill should output the digest content in a structured format (markdown) that the downstream skill can consume.

For one-off digests: deliver directly in the current chat. For recurring digests: set up a cron job (see below).

Step 9 — Cron Setup (Reference)

For recurring digests, create a scheduled job using the cronjob tool. This is optional — one-off digests don't need it.

Key parameters:

• schedule: cron expression (e.g. 0 9 1,15 * * for biweekly at 9am, 0 3 */3 * * for every 3 days)
• prompt: self-contained instructions including topic, search queries, source lists, digest format, output language, and output discipline
• deliver: target channel (telegram, whatsapp:, etc.)
• context_from: list of job IDs whose previous output should be injected as context (for dedup — typically the job's own ID for self-referencing)
• enabled_toolsets: restrict to ["web", "terminal"] to reduce token overhead

Cron prompt template:

You are a [TOPIC] news scout. Find recent articles (within the last [N] months ONLY) and compile a concise digest.

## Date grounding
Today's date is {current date}. When a search query needs a year or refers to "latest"/"current"/"this year", use {current year} or relative wording — never a year inferred from training data.

## Search
Run [N] searches using the web_search tool — one per angle, in both English and [native language]:

1. web_search("[English query 1]", limit=10)
2. web_search("[English query 2]", limit=10)
3. web_search("[Native language query 1]", limit=10)
...

If results need news-specific filtering (categories, language, time_range), use SearXNG directly:
curl -sL "${SEARXNG_URL}/search?q=ENCODED_QUERY&format=json&categories=news&language=[lang]&time_range=month"

If SearXNG is unreachable, the web_search tool will fall back automatically.

## Processing
1. Quality filter — discard thin content (landing pages, stubs, "]` on the cron job (using the job's own ID) so each run receives the previous digest. Add to the prompt: *"Review the previous digest output (provided as context). Do NOT include any article that already appeared — match by title similarity, not exact string match."*

**Frequency tuning:**
- High-volume topics: daily or every 2-3 days
- Medium topics: weekly
- Niche/low-activity topics: biweekly (1st and 15th) — allows more content to accumulate, reduces overlap

## Source Selection Reference

### Hong Kong Sources

**English:** SCMP, The Standard HK, Hong Kong Business, Mingtiandi, Colliers/JLL/CBRE/Savills research, ResearchAndMarkets

**Chinese (Traditional):** HK01, 香港經濟日報 HKET, 明報 Ming Pao, 文匯報 Wen Wei Po, 東方日報 on.cc, Now財經, 信報 HKEJ, 香港商報

**Industry-specific:** PBSA News, Global Student Living, Centurion announcements (SGX-listed PBSA REIT)

## Pitfalls

- **No dates in digest = rejected.** User explicitly asked for publish dates on every item.
- **Stale articles = rejected.** Filter strictly by the field's velocity — flag or exclude articles past their relevance window.
- **Over-summarizing.** User wants headline + 1-2 lines, not a paragraph analysis.
- **Too many items.** User wants max 5 items by default. Reduce frequency rather than padding with stale content.
- **Missing native-language sources.** Local-language outlets often break news first. Always search in both English and the geography's native language(s).
- **Don't promise closed-ecosystem platforms.** 小红书, 抖音, 微信公众号 are app-locked. Be upfront.
- **Google News RSS links are unresolvable.** Never include `news.google.com/rss/articles/CBMi...` URLs. Resolve to the publisher's direct URL first.
- **Output duplication — digest rendered twice.** Some models write the full digest during "thinking" then write it again. Fix: add "OUTPUT DISCIPLINE: Output EXACTLY ONE copy." and specify first/last line anchors.
- **Output language drift.** The skill contains native-language search queries (input). Multilingual models may interpret this as a signal to produce output in that language. Always specify output language explicitly — anchor at both top and bottom of the prompt.
- **"Must-include" RSS sources crowded out by discovery results.** Use quotas (e.g. 5 discovery + 5 RSS) or guarantee minimum slots per must-include source.
- **WhatsApp cron delivery requires explicit target.** `deliver=whatsapp` fails. Use `deliver=whatsapp:` or an explicit chat ID.
- **delegate_task with web toolset is unreliable for search-heavy tasks.** Run searches directly and parse output yourself.
- **SearXNG address changed.** If the NAS IP changes, update `SEARXNG_URL` in `~/.hermes/.env` and the curl commands in topic reference files.
- **DDGS installed in Hermes venv.** No `PYTHONPATH` hacks needed — `ddgs` CLI and `from ddgs import DDGS` work directly. Do not use stale paths like `/root/.hermes/ddgs_lib` or `/opt/hermes/.venv/bin/python`.

## Reference Files

- `references/web-search-fallback.md` — How the Hermes web search backend dispatch works, current config (SearXNG → DDGS fallback), runtime fallback patch details, and env var verification tips. Load when debugging search issues.
- `references/pbsa-hk-search-config.md` — PBSA Hong Kong specific: search queries, source tiers, key themes, delivery config. Load when running the PBSA HK monitoring scan.
- `references/selfhosting-digest-config.md` — Self-hosting & homelab digest: search queries, RSS must-track sources, delivery config. Load when running the Self-Hosting weekly digest.

## Evals

`evals/routing-fixtures.json` holds lightweight contract fixtures — sample
request → expected routing (including when a request should go to
`deep-research` or `youtube-topic-research` instead), required digest output
fields, and forbidden output patterns. They are specs, not run against a live
model; the repo-root `tests/test_routing_fixtures.py` validates they stay well-formed and
route to real skills.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [moonlight-lupin](https://github.com/moonlight-lupin)
- **Source:** [moonlight-lupin/agent-skills](https://github.com/moonlight-lupin/agent-skills)
- **License:** MIT
- **Homepage:** https://hermes-agent.nousresearch.com

Install and usage instructions live in the source repository linked above.