Nlp Text Processing

NLP Text Processing

Activate this skill when working with text data and natural language processing.

When to Use

• Text preprocessing and cleaning
• Building text classification models
• Named entity recognition (NER)
• Sentiment analysis pipelines
• Text embedding and similarity search
• Working with Hugging Face transformers

Libraries

• spaCy: Production NLP pipelines
• Hugging Face Transformers: Pre-trained models
• NLTK: Classic NLP tools
• sentence-transformers: Text embeddings

Patterns

from transformers import pipeline

# Quick inference
classifier = pipeline("text-classification", model="distilbert-base-uncased")
result = classifier("This product is amazing!")

# Custom fine-tuning
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased")

Rules

• Preprocess text consistently (lowercase, tokenize)
• Handle multilingual text explicitly
• Use pre-trained models before training from scratch
• Validate with human evaluation, not just metrics
• Consider computational cost of large language models

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: ihatesea69
• Source: ihatesea69/kiro-kit
• License: MIT
• Homepage: https://www.npmjs.com/package/kiro-kit

Install and usage instructions live in the source repository linked above.