Magic Report Generation
Assemble data analysis findings into structured Markdown reports with mandatory sections (Summary, Data Provenance, Methodology, Key Findings, Caveats, Next Steps). Use when creating the final deliverable report after analysis is complete, generating an executive summary, converting findings JSON into a formatted document, or producing ckpt_07_report.md. Supports standard, executive, and technica…
Magic Data Transformation
Transform data by reshaping, aggregating, merging, deriving columns, and delivering to external destinations (database, HuggingFace Hub). Use when: (1) pivoting, melting, or unpivoting tables, (2) grouping and aggregating data, (3) joining or merging multiple datasets, (4) creating calculated or derived columns, (5) uploading/delivering/pushing data to HuggingFace Hub or database. Trigger keyword…
Magic Data Profiling
Profile datasets — run quality scoring, distribution analysis, outlier detection, and issue detection. Use when assessing data quality, running quality_score.py, getting a quality overview, or profiling before cleaning.
Magic Data Exploration
Explore data interactively and detect patterns systematically. Use when investigating a dataset — freely exploring quality issues, comparing segments, discovering correlations, or running automated pattern detection. Covers both interactive investigation (asking questions, following threads) and scripted analysis (pattern detection, segment comparison, relationship exploration).
Magic Data Loading
Load and ingest data from any source — files (CSV, TSV, Parquet, JSON, JSONL, Excel), databases (SQLite, PostgreSQL, MySQL via connection string), or remote repositories (HuggingFace Hub datasets). Auto-detects format, encoding, and delimiter for files. Use when a user mentions data, a dataset, a file, a database, a table, records, or any structured data source they want to work with — even vague…
Magic Data Lifecycle
Routing and orchestration knowledge for data processing tasks. Provides pipeline ordering (load → profile → clean → transform → validate → deliver), skill routing table (which magic-data-* skill handles which operation), quality gating guidance, and checkpoint strategy. Read this skill to understand how data processing phases connect and which skill to invoke for each step. Use when: the task inv…
Magic Data Synthesis
Synthesize, generate, and transform data using LLM-based operations via DataDesigner engine. Use when: (1) filling missing values/sentinels with contextual content, (2) translating columns, (3) converting formats (HTML→markdown), (4) annotating/labeling records, (5) extracting structured data from text, (6) generating new columns from existing context. Trigger keywords: synthesize, generate, fill…
Magic Data Validation
Validate datasets against inferred or custom schemas, check cross-column constraints, detect sentinel/placeholder values, and catch statistical pitfalls (Simpson's paradox, join explosion). Use when verifying data quality after cleaning, enforcing schemas before delivery, checking for content placeholders, or sanity-checking transformation results.
Magic Data Cleaning
Clean data by detecting issues, handling missing values, normalizing strings, and executing cleaning plans. Use when: (1) data has missing values or nulls to impute, (2) text columns need normalization or deduplication, (3) type errors or inconsistent formats need fixing, (4) planning a cleaning strategy before execution. Does NOT handle sentinel/placeholder values requiring LLM — route those to…
Magic Workspace Init
Initialize a MAGIC data processing workspace: directory scaffolding, Python environment verification, dependency installation, and LLM configuration. Use when starting a new data project or setting up the MAGIC environment for the first time.
Magic Data Visualization
Select appropriate chart types and generate publication-quality visualizations (PNG, SVG, interactive HTML). Use when creating charts, plotting distributions, comparing groups visually, visualizing correlations, or supporting findings with visuals. Covers bar, line, scatter, histogram, box, heatmap, and small multiples. Use after profiling or statistical analysis to communicate results.
Magic Linguistic Lexicon
Lexicography for ML: dictionary-building methodology, sense splitting vs lumping decisions, MWE inventories for RAG glossary injection + MT post-edit, citation-form conventions, variant handling. Use whenever the user mentions lexicography, dictionary-building, sense splitting, sense lumping, lemma, citation form, MWE inventory, glossary, Wiktionary, DBnary, Atkins & Rundell, or asks how to build…
Magic Linguistic Tokenize
Audit tokenizer fertility for a target language and recommend SentencePiece config + vocab-extension strategy (FOCUS / OFA / HyperOfa) + byte-fallback policy. Use whenever the user mentions tokenizer, BPE, SentencePiece, Unigram LM, fertility, vocab extension, byte fallback, OOV explosion, subword regularization, BPE-dropout, vocabulary expansion, or asks why a non-English model produces too many…
Magic Linguistic Syntax
Universal Dependencies (UD) treebank usage, cross-lingual parser transfer (UDify, Trankit, stanza), and agreement-probe construction for grammatical-correctness evaluation. Use whenever the user mentions UD, Universal Dependencies, treebank, dependency parsing, constituency parsing, parser transfer, agreement probe, subject-verb agreement, gender agreement, case marking, syntactic eval, or asks h…
Magic Linguistic Annotate
Design, run, and audit annotation projects: guideline authoring methodology, IAA (inter-annotator agreement) metric selection (Cohen κ, Fleiss κ, Krippendorff α, γ), adjudication workflow, active learning for sample selection. Use whenever the user mentions annotation, labeling, gold standard, IAA, kappa, alpha, Krippendorff, Fleiss, agreement, adjudication, active learning, annotator disagreemen…
Magic Data Synthesis
Synthesize, generate, and transform data using LLM-based operations via DataDesigner engine. Use when: (1) filling missing values/sentinels with contextual content, (2) translating columns, (3) converting formats (HTML→markdown), (4) annotating/labeling records, (5) extracting structured data from text, (6) generating new columns from existing context. Trigger keywords: synthesize, generate, fill…
Magic Linguistic Speech
Bridge field-linguistics annotation (ELAN, Praat, FLEx, SayMore) and audio data into ML pipelines (Lhotse, ESPnet, k2/icefall, MMS, Whisper). G2P / IPA workflows; low-resource ASR / TTS recipe selection. Use whenever the user mentions ELAN, EAF, Praat, TextGrid, FLEx, SayMore, Lhotse, CutSet, ESPnet, k2, icefall, MMS, Whisper, NeMo, SpeechBrain, G2P, grapheme-to-phoneme, IPA, phonetic transcripti…
Magic Linguistic Ethics
Apply CARE / FPIC / community-sovereignty / license-compliance / sacred-text gating across every linguistic project phase. Use whenever the user mentions FPIC, CARE principles, Indigenous data, data sovereignty, community engagement, license audit, attribution, sacred text, restricted corpus, religious text use, endangered-language data, model card ethics statement, or asks 'is it OK to use this…
Magic Linguistic Corpus
Curate monolingual corpora for a target language: catalog awareness (OLDI / CulturaX / MADLAD-400 / Glot500 / Wikipedia / Common Crawl segments), language-ID at paragraph granularity (GlotLID / FastText / CLD3), Unicode-safe deduplication with MinHash, two-sided contamination audit, register-balance analysis. Use whenever the user mentions a corpus, dataset for [language], CulturaX, MADLAD-400, O…
Magic Linguistic Historical
Historical / comparative linguistics primitives for ML data augmentation: cognate sets across related languages, Swadesh lists for cheap bilingual-lexicon bootstrapping, regular sound-correspondence rules. Use whenever the user mentions cognates, Swadesh, comparative method, sound correspondence, proto-language, language family, LingPy, NorthEuraLex, IE-CoR, BDPROTO, CogNet, or asks ''are languag…
Magic Linguistic Scope
Identify a target language precisely and set the strategic direction for any LLM/NLP project on it. Use whenever the user mentions a non-English language by name, asks 'what should I do for [language]', mentions resource class / Joshi / Glottolog / WALS / Grambank / URIEL / typology / transfer source / language vitality, or whenever a workflow needs ISO 639-3 + Glottolog disambiguation. **You sho…
Magic Linguistic Morph
Morphological analysis for the target language: UniMorph paradigm lookup, SIGMORPHON segmenters, FST/HFST analyzer recommendations, morphology-aware data augmentation. Use whenever the user mentions morphology, morpheme segmentation, UniMorph, SIGMORPHON, FST, foma, HFST, agglutinative / polysynthetic / templatic / fusional, paradigm completion, lemma + features, inflection table, morpheme-aware…
Magic Linguistic Codeswitch
Code-switching awareness for ML pipelines: Hinglish, Spanglish, Singlish, MSA + dialect Arabic, Mandarin + Cantonese alternation, and other bilingual / multilingual mixing. Use whenever the user mentions code-switching, code-switch, code-mixing, Hinglish, Spanglish, Singlish, Chinglish, Konglish, mixed-script chat, MADAR, GLUECoS, LinCE, Matrix Language Frame, or asks how to handle bilingual / mu…
Magic Linguistic Scripts
Decide Unicode normalization policy, detect script confusables, recommend romanization/transliteration, and protect diacritics for the target language. Use whenever the user mentions Unicode, NFC, NFKC, normalization, romanize, transliterate, IAST, Pinyin, Hepburn, IPA, confusable characters, mixed-script text, diacritic restoration, ZWJ/ZWNJ, BOM, or non-Latin script handling. **Use this skill B…
Magic Linguistic Bitext
Mine, align, filter, and synthesize parallel corpora for low-resource MT. Use whenever the user mentions parallel data, bitext, sentence alignment, LASER3, SONAR, Vecalign, hunalign, Bleualign, NLLB mining, CCMatrix, CCAligned, FLORES, OPUS, back-translation, dictionary substitution, MT pivoting, synthetic parallel, or asks about translation between English and a low-resource language. **Use BEFO…
Magic Linguistic Eval
Honest evaluation for low-resource LLMs: benchmark choice (FLORES+ / NTREX / Belebele / AfroBench / IndicXTREME / SEACrowd), metric choice (chrF++ / spBLEU / COMET / MetricX / GEMBA-MQM), BLiMP-style grammatical-knowledge probes, contamination check. Use whenever the user mentions evaluation, benchmark, FLORES, NTREX, Belebele, AfroBench, IndicXTREME, SEACrowd, MasakhaNER, AfriSenti, BLEU, chrF,…
Magic Data Lifecycle
Routing and orchestration knowledge for data processing tasks. Provides pipeline ordering (load → profile → clean → transform → validate → deliver), skill routing table (which magic-data-* skill handles which operation), quality gating guidance, and checkpoint strategy. Read this skill to understand how data processing phases connect and which skill to invoke for each step. Use when: the task inv…
Magic Linguistic Discourse
Discourse-level analysis for the target language: choosing a discourse framework (RST / PDTB / SDRT / GUM), coreference (incl. zero-anaphora in pro-drop languages), discourse markers, and coherence-aware evaluation for long-context LLMs. Use whenever the user mentions discourse, RST, Rhetorical Structure Theory, PDTB, Penn Discourse Treebank, GUM, SDRT, coreference, anaphora, zero-anaphora, pro-d…
Magic Linguistic Orchestrator
Start here for any linguistic / NLP / LLM-for-low-resource-language task. Coordinates the 5-phase pipeline (Scope -> Acquire -> Analyze -> Evaluate -> Release) and routes to the right magic-linguistic-* specialist skill. Use whenever the user mentions a target language, asks about training/fine-tuning/adapting an LLM for a non-English language, mentions tokenization fertility, FLORES/Belebele/Afr…
Magic Linguistic Semantics
Lexical + frame semantics for the target language: WordNet / Open Multilingual WordNet (OMW) coverage, FrameNet / PropBank-style SRL guidance, multi-word expressions (MWE / PARSEME). Use whenever the user mentions WordNet, OMW, synset, sense disambiguation, FrameNet, PropBank, semantic role labeling, SRL, MWE, idioms, multi-word expressions, light verbs, phrasal verbs, semantic equivalence, or as…
Magic Linguistic Transfer
Plan cross-lingual adaptation of pretrained LLMs: LoRA / QLoRA / DoRA config (rank scales with typological distance, not data size), MAD-X / BAD-X adapter stacks, source-language selection via URIEL, continued-pretraining-vs-LoRA decision, catastrophic-forgetting mitigation, tool selection (Unsloth vs LLaMA-Factory vs Axolotl). Use whenever the user mentions LoRA, QLoRA, DoRA, adapter, MAD-X, BAD…
Magic Statistical Analysis
Perform descriptive statistics, hypothesis testing, and correlation analysis with mandatory uncertainty communication. Use when computing statistics, testing hypotheses, comparing groups, or analyzing correlations with significance.