Hebrew Voice Bot Builder

Hebrew Voice Bot Builder

Build production-ready Hebrew voice bots and IVR systems for Israeli businesses. This skill covers the full voice pipeline: speech-to-text (STT), text-to-speech (TTS), IVR flow design, telephony integration, and Hebrew-specific challenges like accent handling and mixed Hebrew-English speech.

Instructions

Step 1: Choose Your Architecture

Before building, decide on the voice bot architecture based on the use case:

| Architecture | Best For | Components | |-------------|----------|------------| | IVR (keypad) | Simple menu navigation, payment lines, appointment scheduling | TTS + DTMF + telephony | | Voice bot (conversational) | Customer service, order status, FAQ handling | STT + LLM + TTS + telephony | | Voicemail transcription | Missed call handling, message routing | STT + notification pipeline | | Hybrid | Complex flows with both speech and keypad input | STT + TTS + DTMF + telephony |

Key decisions:

• STT provider: OpenAI gpt-4o-transcribe or gpt-4o-mini-transcribe (best Hebrew WER and lower latency than whisper-1, available via the OpenAI Realtime API for streaming), whisper-large-v3-turbo for self-host, ivrit-ai's Hebrew-tuned variants (ivrit-ai/whisper-large-v3-turbo-ct2) for the best open Hebrew WER, Google Cloud STT (low latency), Azure Speech (enterprise features). The legacy whisper-1 API is still supported but gpt-4o-transcribe is the current default for Hebrew.
• TTS provider — split by use case:
• Real-time / streaming (voice agents, IVR, live conversation): OpenAI Realtime API (gpt-4o-realtime) — native multilingual speech-to-speech including Hebrew, the 2026 default for sub-500ms turn-taking. Inworld TTS-1.5 / Realtime TTS-2 — explicitly lists Hebrew, sub-130ms P90 latency, designed for live conversation. Deepdub Phantom X 3.2 — Israeli company, real-time Hebrew with emotive eTTS, launched March 2026. ElevenLabs eleven_flash_v2_5 — WebSocket-capable but Hebrew quality is weak, use only when latency dominates over quality.
• Offline / max quality (audiobooks, voicemail playback, batch generation): ElevenLabs eleven_v3 — best Hebrew quality ElevenLabs offers, supports Hebrew (heb) among 70+ languages but NO WebSocket / streaming API, REST only. Deepdub Phantom X 3.2 also serves this track with emotional control.
• Fallbacks: Azure Neural TTS (he-IL-HilaNeural, he-IL-AvriNeural), Google Cloud TTS Wavenet (he-IL-Wavenet-A/B), Amazon Polly Hebrew (Avri only, no neural Hebrew voice as of Apr 2026, verify before relying on it). ElevenLabs Multilingual v2 lists Hebrew in its 29-language table but real-world Hebrew quality is mediocre; Turbo v2.5 was not designed with Hebrew in mind.
• Telephony: Twilio (largest Israeli number inventory), Vonage (competitive pricing for Israel).
• Hosting: Cloud functions for low-volume, dedicated servers for high-volume.
• Recording-consent disclosure: Israeli outbound automated calls must disclose recording at the start of the call (השיחה מוקלטת). Privacy Protection Law Amendment 13 (in force Aug 2025) tightens consent requirements for biometric voice data.

Step 2: Hebrew Speech-to-Text (STT)

OpenAI Whisper (Recommended for Accuracy)

Whisper provides the best Hebrew transcription accuracy, especially for mixed Hebrew-English speech common in Israeli tech environments.

import openai

client = openai.OpenAI()

def transcribe_hebrew(audio_file_path: str) -> str:
    """Transcribe Hebrew audio using OpenAI Whisper."""
    with open(audio_file_path, "rb") as audio_file:
        transcript = client.audio.transcriptions.create(
            model="whisper-1",
            file=audio_file,
            language="he",  # Force Hebrew language detection
            response_format="text",
        )
    return transcript

def transcribe_hebrew_with_timestamps(audio_file_path: str) -> dict:
    """Transcribe with word-level timestamps for subtitle generation."""
    with open(audio_file_path, "rb") as audio_file:
        transcript = client.audio.transcriptions.create(
            model="whisper-1",
            file=audio_file,
            language="he",
            response_format="verbose_json",
            timestamp_granularities=["word"],
        )
    return transcript

Whisper Hebrew tips:

• Set language="he" explicitly to avoid misdetecting Hebrew as Arabic
• For mixed Hebrew-English, let Whisper auto-detect (omit the language parameter) and post-process
• Whisper handles niqqud-free text well (standard for modern Hebrew)
• Audio quality matters: 16kHz+ sample rate, mono channel, WAV or FLAC preferred
• Maximum file size: 25MB. For longer recordings, split into segments

Google Cloud Speech-to-Text

Lower latency than Whisper, suitable for real-time voice bots.

from google.cloud import speech_v1

def transcribe_hebrew_google(audio_content: bytes) -> str:
    """Transcribe Hebrew audio using Google Cloud STT."""
    client = speech_v1.SpeechClient()

    audio = speech_v1.RecognitionAudio(content=audio_content)
    config = speech_v1.RecognitionConfig(
        encoding=speech_v1.RecognitionConfig.AudioEncoding.LINEAR16,
        sample_rate_hertz=16000,
        language_code="he-IL",
        # Enable automatic punctuation for Hebrew
        enable_automatic_punctuation=True,
        # Model optimized for phone calls
        model="phone_call",
        # Enable word-level confidence scores
        enable_word_confidence=True,
    )

    response = client.recognize(config=config, audio=audio)

    results = []
    for result in response.results:
        results.append(result.alternatives[0].transcript)

    return " ".join(results)

def stream_transcribe_hebrew(audio_generator):
    """Real-time streaming transcription for live phone calls."""
    client = speech_v1.SpeechClient()

    config = speech_v1.StreamingRecognitionConfig(
        config=speech_v1.RecognitionConfig(
            encoding=speech_v1.RecognitionConfig.AudioEncoding.MULAW,
            sample_rate_hertz=8000,  # Standard phone audio
            language_code="he-IL",
            model="phone_call",
            enable_automatic_punctuation=True,
        ),
        interim_results=True,  # Get partial results for faster response
    )

    streaming_config = speech_v1.StreamingRecognizeRequest(
        streaming_config=config
    )

    def request_generator():
        yield streaming_config
        for chunk in audio_generator:
            yield speech_v1.StreamingRecognizeRequest(audio_content=chunk)

    responses = client.streaming_recognize(requests=request_generator())

    for response in responses:
        for result in response.results:
            if result.is_final:
                yield result.alternatives[0].transcript

Azure Speech Services

Enterprise-grade with custom model training for domain-specific Hebrew vocabulary.

import azure.cognitiveservices.speech as speechsdk

def transcribe_hebrew_azure(audio_file_path: str) -> str:
    """Transcribe Hebrew audio using Azure Speech Services."""
    speech_config = speechsdk.SpeechConfig(
        subscription="YOUR_AZURE_KEY",
        region="westeurope",  # Closest region to Israel
    )
    speech_config.speech_recognition_language = "he-IL"

    audio_config = speechsdk.AudioConfig(filename=audio_file_path)
    recognizer = speechsdk.SpeechRecognizer(
        speech_config=speech_config,
        audio_config=audio_config,
    )

    result = recognizer.recognize_once()

    if result.reason == speechsdk.ResultReason.RecognizedSpeech:
        return result.text
    elif result.reason == speechsdk.ResultReason.NoMatch:
        return ""
    else:
        raise RuntimeError(f"Speech recognition failed: {result.reason}")

Consult references/hebrew-stt-models.md for a detailed comparison of STT providers with accuracy benchmarks.

Step 3: Hebrew Text-to-Speech (TTS)

Google Cloud TTS (Recommended for Natural Sound)

from google.cloud import texttospeech

def synthesize_hebrew(text: str, output_path: str, voice_gender: str = "female") -> None:
    """Convert Hebrew text to speech using Google Cloud TTS."""
    client = texttospeech.TextToSpeechClient()

    input_text = texttospeech.SynthesisInput(text=text)

    # Available Hebrew voices
    voice_name_map = {
        "female": "he-IL-Wavenet-A",  # Female, high quality
        "male": "he-IL-Wavenet-B",    # Male, high quality
        "female_standard": "he-IL-Standard-A",  # Female, lower cost
        "male_standard": "he-IL-Standard-B",    # Male, lower cost
    }

    voice = texttospeech.VoiceSelectionParams(
        language_code="he-IL",
        name=voice_name_map.get(voice_gender, "he-IL-Wavenet-A"),
    )

    audio_config = texttospeech.AudioConfig(
        audio_encoding=texttospeech.AudioEncoding.MP3,
        speaking_rate=1.0,   # 0.5 to 2.0, adjust for clarity
        pitch=0.0,           # -20.0 to 20.0 semitones
    )

    response = client.synthesize_speech(
        input=input_text, voice=voice, audio_config=audio_config
    )

    with open(output_path, "wb") as out:
        out.write(response.audio_content)

Amazon Polly Hebrew

Cost-effective for high-volume TTS needs.

import boto3

def synthesize_hebrew_polly(text: str, output_path: str) -> None:
    """Convert Hebrew text to speech using Amazon Polly."""
    polly = boto3.client("polly", region_name="eu-west-1")

    response = polly.synthesize_speech(
        Text=text,
        OutputFormat="mp3",
        VoiceId="Avri",  # Hebrew male standard voice (no neural Hebrew voice exists)
        Engine="standard",   # Standard engine (neural not available for Hebrew)
        LanguageCode="he-IL",
    )

    with open(output_path, "wb") as out:
        out.write(response["AudioStream"].read())

Azure Neural TTS

Highest quality Hebrew voices with SSML support for fine-grained control.

import azure.cognitiveservices.speech as speechsdk

def synthesize_hebrew_azure(text: str, output_path: str) -> None:
    """Convert Hebrew text to speech using Azure Neural TTS."""
    speech_config = speechsdk.SpeechConfig(
        subscription="YOUR_AZURE_KEY",
        region="westeurope",
    )
    # Hebrew neural voices
    speech_config.speech_synthesis_voice_name = "he-IL-HilaNeural"  # Female
    # Alternative: "he-IL-AvriNeural" for male voice

    audio_config = speechsdk.AudioConfig(filename=output_path)
    synthesizer = speechsdk.SpeechSynthesizer(
        speech_config=speech_config,
        audio_config=audio_config,
    )

    result = synthesizer.speak_text(text)

    if result.reason != speechsdk.ResultReason.SynthesizingAudioCompleted:
        raise RuntimeError(f"Speech synthesis failed: {result.reason}")

def synthesize_hebrew_ssml(ssml: str, output_path: str) -> None:
    """
    Synthesize Hebrew speech with SSML for fine control.

    Example SSML for IVR prompt:
    
        
            
                ברוכים הבאים לשירות הלקוחות.
            
            
            לתמיכה טכנית, הקישו 1.
            
            למכירות, הקישו 2.
        
    
    """
    speech_config = speechsdk.SpeechConfig(
        subscription="YOUR_AZURE_KEY",
        region="westeurope",
    )
    audio_config = speechsdk.AudioConfig(filename=output_path)
    synthesizer = speechsdk.SpeechSynthesizer(
        speech_config=speech_config,
        audio_config=audio_config,
    )

    result = synthesizer.speak_ssml(ssml)
    if result.reason != speechsdk.ResultReason.SynthesizingAudioCompleted:
        raise RuntimeError(f"SSML synthesis failed: {result.reason}")

Step 4: IVR Menu Design for Israeli Businesses

Israeli IVR systems have specific conventions that differ from US/European patterns.

Business Hours Routing

Israeli business week is Sunday through Thursday. IVR systems must account for this:

from datetime import datetime
import pytz

ISRAEL_TZ = pytz.timezone("Asia/Jerusalem")

def get_business_status() -> dict:
    """Determine current business status for IVR routing."""
    now = datetime.now(ISRAEL_TZ)
    day = now.weekday()  # 0=Monday, 6=Sunday
    hour = now.hour

    # Israeli business days: Sunday (6) through Thursday (3)
    # Friday (4): half day until ~13:00
    # Saturday (5): closed (Shabbat)

    if day == 5:  # Saturday (Shabbat)
        return {
            "status": "closed",
            "reason": "shabbat",
            "message_he": "שלום, אנחנו סגורים בשבת. נחזור אליכם ביום ראשון.",
            "next_open": "Sunday 9:00",
        }
    elif day == 4:  # Friday
        if hour  dict:
    """
    Process a voicemail recording: transcribe, classify, and route.

    Args:
        audio_path: Path to the voicemail audio file
        caller_number: Caller's phone number (+972...)

    Returns:
        Processed voicemail with transcript and routing info
    """
    # Step 1: Transcribe using Whisper (best Hebrew accuracy)
    transcript = transcribe_hebrew(audio_path)

    # Step 2: Detect language (Hebrew, English, or mixed)
    language = detect_voicemail_language(transcript)

    # Step 3: Classify intent
    intent = classify_voicemail_intent(transcript)

    # Step 4: Extract key entities
    entities = extract_voicemail_entities(transcript)

    result = {
        "caller": caller_number,
        "timestamp": datetime.now().isoformat(),
        "transcript": transcript,
        "language": language,
        "intent": intent,
        "entities": entities,
        "audio_path": audio_path,
        "duration_seconds": get_audio_duration(audio_path),
    }

    # Step 5: Route based on intent
    result["routing"] = route_voicemail(intent, entities)

    return result

def detect_voicemail_language(text: str) -> str:
    """Detect whether voicemail is Hebrew, English, or mixed."""
    hebrew_chars = sum(1 for c in text if "\u0590"  0.7:
        return "hebrew"
    elif hebrew_ratio  str:
    """Classify voicemail intent based on Hebrew keywords."""
    for intent, keywords in VOICEMAIL_INTENTS.items():
        if any(keyword in transcript for keyword in keywords):
            return intent
    return "general"

Step 6: Mixed Language Handling (Hebrew-English)

Israeli tech professionals frequently switch between Hebrew and English mid-sentence (code-switching). Voice bots must handle this gracefully.

def handle_mixed_speech(audio_path: str) -> dict:
    """
    Handle mixed Hebrew-English speech common in Israeli tech.

    Strategy: Use Whisper without language hint for auto-detection,
    then post-process to normalize mixed output.
    """
    client = openai.OpenAI()

    with open(audio_path, "rb") as f:
        # Omit language parameter to let Whisper handle code-switching
        transcript = client.audio.transcriptions.create(
            model="whisper-1",
            file=f,
            response_format="verbose_json",
        )

    segments = []
    for segment in transcript.segments:
        text = segment["text"]
        lang = detect_segment_language(text)
        segments.append({
            "text": text,
            "language": lang,
            "start": segment["start"],
            "end": segment["end"],
        })

    return {
        "full_transcript": transcript.text,
        "segments": segments,
        "detected_languages": list(set(s["language"] for s in segments)),
    }

# Common Hebrew-English tech phrases that Whisper may mishandle
HEBREW_ENGLISH_CORRECTIONS = {
    "דיפלוי": "deploy",     # Hebrew-accented English
    "פושׁ": "push",
    "קומיט": "commit",
    "סרבר": "server",
    "באג": "bug",
    "פיצ'ר": "feature",
    "אפליקציה": "application",
    "דאטהבייס": "database",
}

Step 7: Phone Integration (Twilio)

Setting Up Twilio with Israeli Numbers (+972)

from twilio.rest import Client
from twilio.twiml.voice_response import VoiceResponse, Gather

TWILIO_ACCOUNT_S

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [squadcodercom](https://github.com/squadcodercom)
- **Source:** [squadcodercom/squadcoder](https://github.com/squadcodercom/squadcoder)
- **License:** MIT
- **Homepage:** https://squadcoder.com

Install and usage instructions live in the source repository linked above.