Vmprotect

VMProtect Deobfuscation & Unpacking

Comprehensive reference for identifying, analyzing, and deobfuscating VMProtect-protected binaries. VMProtect (VMP) is a commercial software protection tool that uses code virtualization, mutation, packing, and anti-debug to hinder reverse engineering.

> Versions covered: VMProtect 2.x and 3.x (including 3.5+). Techniques differ between versions — always identify the version first. > Parent skill: [../SKILL.md](../SKILL.md) — generic deobfuscation framework and reusable techniques (mprotect monitoring, API-level string capture, section dumping, deduplication).

---

1. Detection Heuristics

Before attempting deobfuscation, confirm the target is VMProtect-protected. These heuristics range from trivial to behavioral.

1.1 Section Names

VMProtect adds characteristic sections to the binary. This is the fastest indicator.

.vmp0    — packed/virtualized code (primary VMP section)
.vmp1    — additional VMP section (second layer or data)
.vmp2    — rare, seen in heavily protected binaries
.VMP     — older VMProtect 2.x naming
UPX0/UPX1 — VMProtect sometimes mimics UPX section names as a decoy

Detection with radare2:

iS~vmp          # list sections matching "vmp"
iS~.vmp         # stricter match
iS~VMP

Detection with rabin2:

rabin2 -S binary | grep -i vmp

1.2 Entry Point Analysis

VMProtect-packed binaries have a distinctive entry point pattern:

• VMP 3.x: Entry point jumps into a .vmp0 section, not .text
• VMP 2.x: Entry point contains a push of all registers followed by a jump to the VM dispatcher
• The original entry point (OEP) is virtualized or redirected

# Check if EP lands in a VMP section
ie              # show entry point
iS              # compare EP address against section ranges
s entry0; pd 20 # disassemble first 20 instructions at EP

Red flags at entry point:

• pushfd / pushad or push of many registers as very first instructions
• Immediate jump to a high/unusual address outside .text
• call to a location that computes the next address dynamically

1.3 VM Handler Dispatch Pattern

The core of VMProtect is a bytecode interpreter (virtual machine). Look for this dispatch loop pattern:

; Typical VMP3 dispatch loop (x86-64)
mov     reg1, [rsi]         ; fetch VM opcode from bytecode stream
add     rsi, 4              ; advance VM instruction pointer
lea     reg2, [rip + table] ; handler table base
mov     reg3, [reg2 + reg1*8] ; lookup handler address
jmp     reg3                ; dispatch to handler

; Alternative pattern (computed jump)
movzx   eax, byte [rbx]
inc     rbx
jmp     qword [rdi + rax*8]

Heuristic signals:

• A single function with hundreds or thousands of basic blocks
• Very high cyclomatic complexity (>100) in a single function
• Repeated jmp reg or jmp [reg + reg*8] patterns (indirect jumps)
• A large jump table (>50 entries) used as a handler dispatch

# Find functions with abnormally many basic blocks
aflj | jq '.[] | select(.nbbs > 200) | {name, addr: .offset, blocks: .nbbs}'

# Search for indirect jump patterns (handler dispatch)
/c jmp rax
/c jmp qword [rdi
/c jmp qword [rbx

1.4 Import Table Anomalies

VMProtect wraps or virtualizes imports:

• No imports or very few imports visible in the IAT — VMProtect resolves them at runtime via GetProcAddress/dlsym
• Import thunks jump through VMP stubs instead of directly to the API
• GetProcAddress, LoadLibraryA/W are among the few visible imports (used by VMP's own resolver)

ii           # list imports — suspiciously few?
ii~GetProc   # VMP always needs these
ii~LoadLib

1.5 dlsym Mass-Resolution Pattern (Android/Linux)

On Android, VMP-protected .so files resolve all their imports dynamically via dlsym at startup. This produces a distinctive burst of 100-300+ dlsym calls immediately after the library loads — covering everything from malloc and memcpy to OpenGL functions and JNI helpers.

This pattern is a strong behavioral indicator: normal libraries use the ELF dynamic linker for most imports and only dlsym a handful of optional symbols.

Hook dlsym and count calls originating from the target module — if you see 100+ distinct symbol resolutions at load time, VMP is almost certainly present:

var dlsymCount = 0;
Interceptor.attach(Module.getGlobalExportByName('dlsym'), {
    onEnter(args) {
        if (!isFromTarget(this.returnAddress)) return;
        var name = args[1].readCString();
        if (name) {
            dlsymCount++;
            console.log('[dlsym #' + dlsymCount + '] ' + name);
        }
    }
});

1.6 String Obfuscation Indicators

VMProtect encrypts strings referenced by virtualized code:

• iz (data section strings) returns very few meaningful strings
• izz (all strings) may reveal encrypted blobs or base64-like data
• API name strings are absent (resolved dynamically)
• String references (axt) point into VMP sections, not .text

iz                     # meaningful strings are missing
izz~flag               # try to find something
izz | wc -l            # compare count — VMP binaries have far fewer readable strings

1.7 Code Mutation Indicators

Mutated (but not virtualized) functions show:

• Junk instructions: meaningless arithmetic that cancels out (e.g., add rax, 5; sub rax, 5)
• Opaque predicates: conditional jumps where the condition is always true/false
• Register shuffling: excessive mov reg, reg chains
• Constant unfolding: simple constants split into multi-step computations
• Dead stores: writes to registers/memory immediately overwritten

# Look for mutation patterns — high instruction count, low semantic density
pdf @ fcn.addr | grep -c "nop\|xchg\|stc\|clc\|cmc"

1.8 Anti-Debug and Anti-Tamper Signatures

VMProtect includes runtime checks:

• Calls to IsDebuggerPresent, NtQueryInformationProcess, CheckRemoteDebuggerPresent
• RDTSC timing checks (measure time between two points)
• INT 2D / INT 3 exceptions used as anti-debug
• CRC/hash checks over code sections (anti-tamper)
• TLS callbacks that run before main()
• On Android/Linux: reads /proc/self/status and checks TracerPid field

ii~Debugger
ii~NtQuery
/c rdtsc
/c int 0x2d
/c int 3
iS~.tls          # TLS section present?

1.9 mprotect Storm at Startup (Android/Linux)

VMP-protected .so files call mprotect repeatedly during initialization to:

1. Make packed sections writable (r-x → rwx)
2. Unpack/decrypt code into those sections
3. Re-protect them (rwx → r-x)

Hook mprotect and filter for calls targeting the library's address range. Seeing 5-20+ mprotect calls on a single library at load time is a strong VMP indicator (normal libraries make 0-1 calls). See the [parent skill](../SKILL.md#33-mprotect-monitoring) for the hook template.

1.10 uncompress / zlib Usage During Unpacking

VMProtect uses zlib's uncompress function to decompress packed code sections at runtime. If you see uncompress calls with output landing inside the target library's memory range right after load, this confirms VMP packing.

Interceptor.attach(Module.getGlobalExportByName('uncompress'), {
    onEnter(args) {
        this.dest   = args[0];
        this.srcLen = args[3].toUInt32();
    },
    onLeave(retval) {
        if (retval.toInt32() !== 0) return;
        if (this.dest.compare(targetBase) >= 0 && this.dest.compare(targetEnd)  100)] | length'

Scoring:

• .vmp sections found → confirmed VMProtect
• 3+ other heuristics match → highly likely VMProtect
• 1–2 heuristics match → possibly VMProtect or other virtualizer (check for Themida/Code Virtualizer patterns)

Behavioral confirmation (requires running the binary):

• dlsym mass-resolution (100+ calls at startup) → confirmed VMP import obfuscation
• mprotect storm (5+ calls on the library at load) → confirmed VMP packing
• uncompress writing into the library → confirmed VMP compression
• /proc/self/status TracerPid check → confirmed VMP anti-debug

---

2. VMProtect Architecture

Understanding the VM is essential for deobfuscation.

2.1 VM Components

+------------------+     +----------------+     +------------------+
| Bytecode Stream  | --> | VM Dispatcher  | --> | Handler Table    |
| (encrypted ops)  |     | (fetch-decode) |     | (native stubs)  |
+------------------+     +----------------+     +------------------+
                                |
                          +-----+-----+
                          |           |
                    +---------+ +---------+
                    | VM Stack| | VM Regs  |
                    | (RSP-   | | (mapped  |
                    |  based) | |  to mem) |
                    +---------+ +---------+

• Bytecode stream: Encrypted VM opcodes stored in .vmp0/.vmp1
• Dispatcher: Fetch-decode-execute loop; fetches next opcode, decrypts, jumps to handler
• Handler table: Array of native code snippets, each implementing one VM opcode
• VM stack: Separate stack area, often pointed to by a repurposed general register
• VM context/registers: Virtual registers stored in memory, not mapped 1:1 to CPU registers

2.2 VMP Opcode Classes

VMProtect VMs implement these broad opcode categories:

| Category | Examples | Purpose | |----------|----------|---------| | Stack ops | vPush, vPop | Move data on/off VM stack | | Arithmetic | vAdd, vSub, vMul, vDiv, vNor | ALU operations (NOR is used to build AND/OR/XOR/NOT) | | Memory | vLoad, vStore | Read/write native memory | | Control flow | vJmp, vCall, vRet | Branch, call native, return | | Flags | vPushFlags, vPopFlags | Manage EFLAGS/RFLAGS | | Context | vReadReg, vWriteReg | Access native CPU registers | | Crypto | vDecrypt, vRotate | Decrypt next bytecode chunk |

2.3 Version Differences

| Feature | VMP 2.x | VMP 3.x | |---------|---------|---------| | Section names | .VMP0, .VMP1 | .vmp0, .vmp1 | | Handler encoding | Direct handler addresses | Delta-encoded / encrypted handler offsets | | Bytecode encryption | Simple XOR/ADD rolling key | Multi-layer, key-dependent decryption per opcode | | VM entry | pushad; pushfd; call vm_entry | More varied, often via call [rip+disp] | | Handler count | ~30-50 | ~40-80 (more granular) | | Mutation | Light | Heavy mutation + MBA (mixed boolean arithmetic) | | Anti-debug | Basic (IsDebuggerPresent) | Layered (timing, exception, TLS, CRC, TracerPid) |

---

3. String Deobfuscation

VMProtect encrypts strings referenced by protected functions. The most effective approach is to hook the consumers of decrypted strings rather than reversing the decryption algorithm.

3.1 Strategy: Hook API Consumers (Proven Approach)

Decrypted strings must eventually flow through standard APIs — JNI calls, libc string functions, dlsym, logging. By hooking these APIs and filtering for calls originating from the target module, you capture all decrypted strings without needing to understand the decryption algorithm.

This approach was proven effective on VMProtect-protected Android .so files, recovering 2000+ unique strings including class names, method signatures, dynamically resolved API names, and file paths.

3.2 JNI Vtable Hooking (Android)

The most valuable source of strings on Android. Hook JNI functions by their vtable index to capture all strings crossing the Java-native boundary:

Java.perform(function () {
    var env = Java.vm.getEnv();
    var vtable = env.handle.readPointer();

    // NewStringUTF — native code creating Java strings
    Interceptor.attach(vtable.add(167 * Process.pointerSize).readPointer(), {
        onEnter: function (args) {
            var s = args[1].readUtf8String();
            if (s) logString('JNI:NewStringUTF', s);
        }
    });

    // GetStringUTFChars — native code reading Java strings
    Interceptor.attach(vtable.add(169 * Process.pointerSize).readPointer(), {
        onEnter: function (args) { this.jstr = args[1]; },
        onLeave: function (retval) {
            var s = retval.readUtf8String();
            if (s) logString('JNI:GetStringUTFChars', s);
        }
    });

    // FindClass — class names being loaded
    Interceptor.attach(vtable.add(6 * Process.pointerSize).readPointer(), {
        onEnter: function (args) {
            var s = args[1].readCString();
            if (s) logString('JNI:FindClass', s);
        }
    });

    // GetMethodID — method lookups (name + signature)
    Interceptor.attach(vtable.add(33 * Process.pointerSize).readPointer(), {
        onEnter: function (args) {
            var name = args[2].readCString();
            var sig = args[3].readCString();
            if (name) logString('JNI:GetMethodID', name, sig);
        }
    });

    // GetFieldID — field lookups
    Interceptor.attach(vtable.add(36 * Process.pointerSize).readPointer(), {
        onEnter: function (args) {
            var name = args[2].readCString();
            var sig = args[3].readCString();
            if (name) logString('JNI:GetFieldID', name, sig);
        }
    });

    // GetStaticMethodID
    Interceptor.attach(vtable.add(113 * Process.pointerSize).readPointer(), {
        onEnter: function (args) {
            var name = args[2].readCString();
            var sig = args[3].readCString();
            if (name) logString('JNI:GetStaticMethodID', name, sig);
        }
    });

    // GetStaticFieldID
    Interceptor.attach(vtable.add(114 * Process.pointerSize).readPointer(), {
        onEnter: function (args) {
            var name = args[2].readCString();
            var sig = args[3].readCString();
            if (name) logString('JNI:GetStaticFieldID', name, sig);
        }
    });

    // RegisterNatives — JNI method registration (name + sig + native function pointer)
    Interceptor.attach(vtable.add(215 * Process.pointerSize).readPointer(), {
        onEnter: function (args) {
            var count = args[3].toInt32();
            for (var i = 0; i  ' + fn);
            }
        }
    });
});

JNI vtable index reference:

| Index | Function | Captures | |-------|----------|----------| | 6 | FindClass | Class names (e.g., com/example/GameManager) | | 33 | GetMethodID | Instance method names + signatures | | 36 | GetFieldID | Instance field names + signatures | | 113 | GetStaticMethodID | Static method names + signatures | | 114 | GetStaticFieldID | Static field names + signatures | | 167 | NewStringUTF | Strings created by native code for Java | | 169 | GetStringUTFChars | Java strings read by native code | | 215 | RegisterNatives | JNI method bindings (name, sig, native addr) |

3.3 libc String Function Hooking

Hook libc string functions, filtering to calls originating from the target module:

var hooks = [
    { name: 'strcmp',  read: [0, 1] },
    { name: 'strncmp', read: [0, 1] },
    { name: 'strstr', read: [0, 1] },
    { name: 'strlen', read: [0] }
];

hooks.forEach(function (h) {
    var addr = Module.findGlobalExportByName(h.name);
    if (!addr) return;

    Interceptor.attach(addr, {
        onEnter: function (args) {
            if (!isFromTarget(this.returnAddress)) return;
            this.log = true;
            this.strs = [];
            for (var i = 0; i 
mprotect(section, size, PROT_READ|PROT_EXEC)              — re-protect

Dump after the section transitions from rwx back to r-x:

var mprotectLog = [];
Interceptor.attach(Module.getGlobalExportByName('mprotect'), {
    onEnter(args) {
        this.addr = args[0];
        this.len  = args[1].toUInt32();
        this.prot = args[2].toInt32();
    },
    onLeave(retval) {
        if (retval.toInt32() !== 0) return;
        if (!targetModule) return;

        var start = this.addr;
        var end   = start.add(this.

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [as0ler](https://github.com/as0ler)
- **Source:** [as0ler/skills](https://github.com/as0ler/skills)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.