Transformers Js

Transformers.js - Machine Learning for JavaScript

Transformers.js enables running state-of-the-art machine learning models directly in JavaScript across browsers and server-side runtimes (Node.js, Bun, Deno), with no Python server required.

When to Use This Skill

Use this skill when you need to:

• Run ML models for text analysis, generation, or translation in JavaScript
• Perform image classification, object detection, or segmentation
• Implement speech recognition or audio processing
• Build multimodal AI applications (text-to-image, image-to-text, etc.)
• Run models client-side in the browser without a backend

Installation

NPM Installation

npm install @huggingface/transformers

Browser Usage (CDN)

  import { pipeline } from 'https://cdn.jsdelivr.net/npm/@huggingface/transformers';

Core Concepts

1. Pipeline API

The pipeline API is the easiest way to use models. It groups together preprocessing, model inference, and postprocessing:

import { pipeline } from '@huggingface/transformers';

// Create a pipeline for a specific task
const pipe = await pipeline('sentiment-analysis');

// Use the pipeline
const result = await pipe('I love transformers!');
// Output: [{ label: 'POSITIVE', score: 0.999817686 }]

// IMPORTANT: Always dispose when done to free memory
await pipe.dispose();

⚠️ Memory Management: All pipelines must be disposed with pipe.dispose() when finished to prevent memory leaks. See examples in [Code Examples](./references/EXAMPLES.md) for cleanup patterns across different environments.

2. Model Selection

You can specify a custom model as the second argument:

const pipe = await pipeline(
  'sentiment-analysis',
  'Xenova/bert-base-multilingual-uncased-sentiment'
);

Finding Models:

Browse available Transformers.js models on Hugging Face Hub:

• All models: https://huggingface.co/models?library=transformers.js&sort=trending
• By task: Add pipeline_tag parameter
• Text generation: https://huggingface.co/models?pipeline_tag=text-generation&library=transformers.js&sort=trending
• Image classification: https://huggingface.co/models?pipeline_tag=image-classification&library=transformers.js&sort=trending
• Speech recognition: https://huggingface.co/models?pipeline_tag=automatic-speech-recognition&library=transformers.js&sort=trending

Tip: Filter by task type, sort by trending/downloads, and check model cards for performance metrics and usage examples.

3. Device Selection

Choose where to run the model:

// Run on CPU (default for WASM)
const pipe = await pipeline('sentiment-analysis', 'model-id');

// Run on GPU (WebGPU)
const pipe = await pipeline('sentiment-analysis', 'model-id', {
  device: 'webgpu',
});

4. Quantization Options

Control model precision vs. performance:

// Use quantized model (faster, smaller)
const pipe = await pipeline('sentiment-analysis', 'model-id', {
  dtype: 'q4',  // Options: 'fp32', 'fp16', 'q8', 'q4'
});

Supported Tasks

Note: All examples below show basic usage.

Natural Language Processing

Text Classification

const classifier = await pipeline('text-classification');
const result = await classifier('This movie was amazing!');

Named Entity Recognition (NER)

const ner = await pipeline('token-classification');
const entities = await ner('My name is John and I live in New York.');

Question Answering

const qa = await pipeline('question-answering');
const answer = await qa({
  question: 'What is the capital of France?',
  context: 'Paris is the capital and largest city of France.'
});

Text Generation

const generator = await pipeline('text-generation', 'onnx-community/gemma-3-270m-it-ONNX');
const text = await generator('Once upon a time', {
  max_new_tokens: 100,
  temperature: 0.7
});

For streaming and chat: See [Text Generation Guide](./references/TEXT_GENERATION.md) for:

• Streaming token-by-token output with TextStreamer
• Chat/conversation format with system/user/assistant roles
• Generation parameters (temperature, topk, topp)
• Browser and Node.js examples
• React components and API endpoints

Translation

const translator = await pipeline('translation', 'Xenova/nllb-200-distilled-600M');
const output = await translator('Hello, how are you?', {
  src_lang: 'eng_Latn',
  tgt_lang: 'fra_Latn'
});

Summarization

const summarizer = await pipeline('summarization');
const summary = await summarizer(longText, {
  max_length: 100,
  min_length: 30
});

Zero-Shot Classification

const classifier = await pipeline('zero-shot-classification');
const result = await classifier('This is a story about sports.', ['politics', 'sports', 'technology']);

Computer Vision

Image Classification

const classifier = await pipeline('image-classification');
const result = await classifier('https://example.com/image.jpg');
// Or with local file
const result = await classifier(imageUrl);

Object Detection

const detector = await pipeline('object-detection');
const objects = await detector('https://example.com/image.jpg');
// Returns: [{ label: 'person', score: 0.95, box: { xmin, ymin, xmax, ymax } }, ...]

Image Segmentation

const segmenter = await pipeline('image-segmentation');
const segments = await segmenter('https://example.com/image.jpg');

Depth Estimation

const depthEstimator = await pipeline('depth-estimation');
const depth = await depthEstimator('https://example.com/image.jpg');

Zero-Shot Image Classification

const classifier = await pipeline('zero-shot-image-classification');
const result = await classifier('image.jpg', ['cat', 'dog', 'bird']);

Audio Processing

Automatic Speech Recognition

const transcriber = await pipeline('automatic-speech-recognition');
const result = await transcriber('audio.wav');
// Returns: { text: 'transcribed text here' }

Audio Classification

const classifier = await pipeline('audio-classification');
const result = await classifier('audio.wav');

Text-to-Speech

const synthesizer = await pipeline('text-to-speech', 'Xenova/speecht5_tts');
const audio = await synthesizer('Hello, this is a test.', {
  speaker_embeddings: speakerEmbeddings
});

Multimodal

Image-to-Text (Image Captioning)

const captioner = await pipeline('image-to-text');
const caption = await captioner('image.jpg');

Document Question Answering

const docQA = await pipeline('document-question-answering');
const answer = await docQA('document-image.jpg', 'What is the total amount?');

Zero-Shot Object Detection

const detector = await pipeline('zero-shot-object-detection');
const objects = await detector('image.jpg', ['person', 'car', 'tree']);

Feature Extraction (Embeddings)

const extractor = await pipeline('feature-extraction');
const embeddings = await extractor('This is a sentence to embed.');
// Returns: tensor of shape [1, sequence_length, hidden_size]

// For sentence embeddings (mean pooling)
const extractor = await pipeline('feature-extraction', 'onnx-community/all-MiniLM-L6-v2-ONNX');
const embeddings = await extractor('Text to embed', { pooling: 'mean', normalize: true });

Finding and Choosing Models

Browsing the Hugging Face Hub

Discover compatible Transformers.js models on Hugging Face Hub:

Base URL (all models):

https://huggingface.co/models?library=transformers.js&sort=trending

Filter by task using the pipeline_tag parameter:

| Task | URL | |------|-----| | Text Generation | https://huggingface.co/models?pipelinetag=text-generation&library=transformers.js&sort=trending | | Text Classification | https://huggingface.co/models?pipelinetag=text-classification&library=transformers.js&sort=trending | | Translation | https://huggingface.co/models?pipelinetag=translation&library=transformers.js&sort=trending | | Summarization | https://huggingface.co/models?pipelinetag=summarization&library=transformers.js&sort=trending | | Question Answering | https://huggingface.co/models?pipelinetag=question-answering&library=transformers.js&sort=trending | | Image Classification | https://huggingface.co/models?pipelinetag=image-classification&library=transformers.js&sort=trending | | Object Detection | https://huggingface.co/models?pipelinetag=object-detection&library=transformers.js&sort=trending | | Image Segmentation | https://huggingface.co/models?pipelinetag=image-segmentation&library=transformers.js&sort=trending | | Speech Recognition | https://huggingface.co/models?pipelinetag=automatic-speech-recognition&library=transformers.js&sort=trending | | Audio Classification | https://huggingface.co/models?pipelinetag=audio-classification&library=transformers.js&sort=trending | | Image-to-Text | https://huggingface.co/models?pipelinetag=image-to-text&library=transformers.js&sort=trending | | Feature Extraction | https://huggingface.co/models?pipelinetag=feature-extraction&library=transformers.js&sort=trending | | Zero-Shot Classification | https://huggingface.co/models?pipeline_tag=zero-shot-classification&library=transformers.js&sort=trending |

Sort options:

• &sort=trending - Most popular recently
• &sort=downloads - Most downloaded overall
• &sort=likes - Most liked by community
• &sort=modified - Recently updated

Choosing the Right Model

Consider these factors when selecting a model:

1. Model Size

• Small ( 500MB): High accuracy, slower, better for Node.js or powerful devices

2. Quantization Models are often available in different quantization levels:

• fp32 - Full precision (largest, most accurate)
• fp16 - Half precision (smaller, still accurate)
• q8 - 8-bit quantized (much smaller, slight accuracy loss)
• q4 - 4-bit quantized (smallest, noticeable accuracy loss)

3. Task Compatibility Check the model card for:

• Supported tasks (some models support multiple tasks)
• Input/output formats
• Language support (multilingual vs. English-only)
• License restrictions

4. Performance Metrics Model cards typically show:

• Accuracy scores
• Benchmark results
• Inference speed
• Memory requirements

Example: Finding a Text Generation Model

// 1. Visit: https://huggingface.co/models?pipeline_tag=text-generation&library=transformers.js&sort=trending

// 2. Browse and select a model (e.g., onnx-community/gemma-3-270m-it-ONNX)

// 3. Check model card for:
//    - Model size: ~270M parameters
//    - Quantization: q4 available
//    - Language: English
//    - Use case: Instruction-following chat

// 4. Use the model:
import { pipeline } from '@huggingface/transformers';

const generator = await pipeline(
  'text-generation',
  'onnx-community/gemma-3-270m-it-ONNX',
  { dtype: 'q4' } // Use quantized version for faster inference
);

const output = await generator('Explain quantum computing in simple terms.', {
  max_new_tokens: 100
});

await generator.dispose();

Tips for Model Selection

1. Start Small: Test with a smaller model first, then upgrade if needed
2. Check ONNX Support: Ensure the model has ONNX files (look for onnx folder in model repo)
3. Read Model Cards: Model cards contain usage examples, limitations, and benchmarks
4. Test Locally: Benchmark inference speed and memory usage in your environment
5. Filter by Library: Use library=transformers.js to find compatible models: https://huggingface.co/models?library=transformers.js
6. Version Pin: Use specific git commits in production for stability:

``javascript const pipe = await pipeline('task', 'model-id', { revision: 'abc123' }); ``

Advanced Configuration

Environment Configuration (env)

The env object provides comprehensive control over Transformers.js execution, caching, and model loading.

Quick Overview:

import { env, LogLevel } from '@huggingface/transformers';

// View version
console.log(env.version); // e.g., '4.x'

// Common settings
env.allowRemoteModels = true;  // Load from Hugging Face Hub
env.allowLocalModels = false;  // Load from file system
env.localModelPath = '/models/'; // Local model directory
env.useFSCache = true;         // Cache models on disk (Node.js)
env.useBrowserCache = true;    // Cache models in browser
env.cacheDir = './.cache';     // Cache directory location
// Optional: override logging level (default is LogLevel.WARNING)
env.logLevel = LogLevel.INFO;

// Optional: custom fetch for auth headers, retries, abort signals, etc.
env.fetch = (url, options) =>
  fetch(url, {
    ...options,
    headers: {
      ...options?.headers,
      Authorization: `Bearer ${HF_TOKEN}`,
    },
  });

Configuration Patterns:

// Development: Fast iteration with remote models
env.allowRemoteModels = true;
env.useFSCache = true;

// Production: Local models only
env.allowRemoteModels = false;
env.allowLocalModels = true;
env.localModelPath = '/app/models/';

// Custom CDN
env.remoteHost = 'https://cdn.example.com/models';

// Disable caching (testing)
env.useFSCache = false;
env.useBrowserCache = false;

For complete documentation on all configuration options, caching strategies, cache management, pre-downloading models, and more, see:

→ [Configuration Reference](./references/CONFIGURATION.md)

ModelRegistry (v4)

ModelRegistry gives you visibility and control over model assets before loading a pipeline. Use it to estimate download size, check cache status, inspect available dtypes, and clear cached artifacts for a specific task/model/options tuple.

import { ModelRegistry } from '@huggingface/transformers';

const task = 'feature-extraction';
const modelId = 'onnx-community/all-MiniLM-L6-v2-ONNX';
const modelOptions = { dtype: 'fp32' };

// List required files for this pipeline
const files = await ModelRegistry.get_pipeline_files(task, modelId, modelOptions);

// Check if assets are already cached
const cached = await ModelRegistry.is_pipeline_cached(task, modelId, modelOptions);

// Inspect precision formats available for this model
const dtypes = await ModelRegistry.get_available_dtypes(modelId);

console.log({ files: files.length, cached, dtypes });

For production patterns and full API coverage, see [ModelRegistry Reference](./references/MODEL_REGISTRY.md).

Standalone Tokenization (@huggingface/tokenizers)

For tokenization-only workflows, use @huggingface/tokenizers. It is a separate lightweight package useful when you need fast tokenization/encoding without loading full model inference pipelines.

npm install @huggingface/tokenizers

import { Tokenizer } from '@huggingface/tokenizers';

Working with Tensors

import { AutoTokenizer, AutoModel } from '@huggingface/transformers';

// Load tokenizer and model separately for more control
const tokenizer = await AutoTokenizer.from_pretrained('bert-base-uncased');
const model = await AutoModel.from_pretrained('bert-base-uncased');

// Tokenize input
const inputs = await tokenizer('Hello world!');

// Run model
const outputs = await model(inputs);

Batch Processing

const classifier = await pipeline('sentiment-analysis');

// Process multiple texts
const results = await classifier([
  'I love this!',
  'This is terrible.',
  'It was okay.'
]);

Runtime-Specific Considerations

WebGPU Usage

WebGPU provides GPU acceleration in browsers and server-side runtimes (when supported):

const pipe = await pipeline('text-generation', 'onnx-communit

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [huggingface](https://github.com/huggingface)
- **Source:** [huggingface/skills](https://github.com/huggingface/skills)
- **License:** Apache-2.0
- **Homepage:** https://huggingface.co

Install and usage instructions live in the source repository linked above.