Exp Simd Vectorization

SIMD Vectorization

Decision Gate

1. Check Span and MemoryExtensions first. If the operation can be expressed using built-in Span methods (e.g., Contains, IndexOf, CopyTo, SequenceEqual) or MemoryExtensions, use them — no additional dependency is needed and the runtime already vectorizes many of these internally.
2. Check for TensorPrimitives next. If one or more TensorPrimitives methods cover the operation → use them. If the .csproj does NOT already reference System.Numerics.Tensors, add the package, for example: ` (or use the versioning approach already used by your solution). Then replace the scalar loop with TP calls and stop. See the full API table below. Compose multiple TP calls when needed (e.g., finding both min and max → TensorPrimitives.Min(span) + TensorPrimitives.Max(span)` as two calls). Do NOT write manual Vector128 code for operations TP already handles.
3. Scalar loop over contiguous array/span of byte, sbyte, short, ushort, int, uint, long, ulong, nint, nuint, float, double (and char via reinterpretation as ushort)? → Implement with explicit Vector128 / Vector256 / Vector512 intrinsics using the patterns below.
4. No contiguous numeric arrays to process (dictionary lookups, tree traversals, linked lists, state machines, string formatting, small collections, enum comparisons, recursive algorithms, decimal arithmetic)? → Report [NO SIMD OPPORTUNITY] and write a full paragraph explaining WHY, referencing the specific code characteristics that prevent vectorization (e.g., "State machines require sequential branching on enum values — there are no contiguous numeric arrays to process in parallel, and each transition depends on the previous state"). This explanation is graded.

TensorPrimitives API Reference

TensorPrimitives APIs are generic and work for any primitive type that satisfies the method's generic constraints — not just float/double. For example, Sum requires IAdditionOperators + IAdditiveIdentity and works for all primitive numeric types, while CosineSimilarity requires IRootFunctions and only works for float/double. If the project doesn't already reference System.Numerics.Tensors, add it to the .csproj. Replace the entire manual loop with one or more TensorPrimitives calls as needed (prefer a single call when possible):

Reductions (span → scalar)

| Operation | API | |-----------|-----| | Sum | TensorPrimitives.Sum(span) | | Sum of squares | TensorPrimitives.SumOfSquares(span) | | Sum of magnitudes (L1 norm) | TensorPrimitives.SumOfMagnitudes(span) | | L2 norm | TensorPrimitives.Norm(span) | | Product of all elements | TensorPrimitives.Product(span) | | Min value | TensorPrimitives.Min(span) | | Max value | TensorPrimitives.Max(span) | | Index of max | TensorPrimitives.IndexOfMax(span) | | Index of min | TensorPrimitives.IndexOfMin(span) | | Dot product | TensorPrimitives.Dot(a, b) | | Cosine similarity | TensorPrimitives.CosineSimilarity(a, b) | | Euclidean distance | TensorPrimitives.Distance(a, b) |

Element-wise transforms (span → span)

| Operation | API | |-----------|-----| | Negate | TensorPrimitives.Negate(src, dst) | | Abs | TensorPrimitives.Abs(src, dst) | | Sqrt | TensorPrimitives.Sqrt(src, dst) | | Exp | TensorPrimitives.Exp(src, dst) | | Log | TensorPrimitives.Log(src, dst) | | Log2 | TensorPrimitives.Log2(src, dst) | | Tanh | TensorPrimitives.Tanh(src, dst) | | Sigmoid | TensorPrimitives.Sigmoid(src, dst) | | SoftMax | TensorPrimitives.SoftMax(src, dst) | | Sinh | TensorPrimitives.Sinh(src, dst) | | Cosh | TensorPrimitives.Cosh(src, dst) | | Round | TensorPrimitives.Round(src, dst) | | Floor | TensorPrimitives.Floor(src, dst) | | Ceiling | TensorPrimitives.Ceiling(src, dst) | | CopySign | TensorPrimitives.CopySign(src, sign, dst) | | Pow | TensorPrimitives.Pow(bases, exponents, dst) |

Two-span operations (a, b → dst)

| Operation | API | |-----------|-----| | Add | TensorPrimitives.Add(a, b, dst) | | Subtract | TensorPrimitives.Subtract(a, b, dst) | | Multiply | TensorPrimitives.Multiply(a, b, dst) | | Divide | TensorPrimitives.Divide(a, b, dst) | | Element-wise Min | TensorPrimitives.Min(a, b, dst) | | Element-wise Max | TensorPrimitives.Max(a, b, dst) |

Three-span fused operations

| Operation | API | |-----------|-----| | (x+y)z | TensorPrimitives.AddMultiply(x, y, z, dst) | | xy+z | TensorPrimitives.MultiplyAdd(x, y, z, dst) | | fma(x,y,z) | TensorPrimitives.FusedMultiplyAdd(x, y, z, dst) |

> AddMultiply and MultiplyAdd are distinct — they optimize differently depending on whether the dependency chain flows from the addend or the multiplier. FusedMultiplyAdd is the IEEE 754 fused form of (x*y)+z with a single rounding step.

Manual SIMD with Vector128/Vector256/Vector512

Use this when TensorPrimitives doesn't have a single API for the operation. This is required for byte-level operations, character class counting, range validation, bitwise bulk ops, cross-type conversions, and custom patterns.

Required imports

using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Runtime.Intrinsics;

Prefer cross-platform APIs (System.Runtime.Intrinsics). Only use platform-specific intrinsics (System.Runtime.Intrinsics.X86, .Arm) when there is a significant performance advantage that justifies the increased code complexity of maintaining separate code paths.

Three-tier dispatch pattern

Always include all three tiers. Use if/else if so that small inputs hit only one branch before reaching the scalar fallback — a fallthrough pattern (sequential ifs) pessimizes the scalar case by requiring up to three not-taken branches that may mispredict. The IsHardwareAccelerated checks are JIT-time constants, so dead paths are eliminated at compile time:

ref var src = ref MemoryMarshal.GetReference(span);
uint i = 0;
uint length = (uint)span.Length;

if (Vector512.IsHardwareAccelerated && Vector512.IsSupported)
{
    uint vec512Count = (uint)Vector512.Count;
    while (i + vec512Count .IsSupported)
{
    uint vec256Count = (uint)Vector256.Count;
    while (i + vec256Count .IsSupported)
{
    uint vec128Count = (uint)Vector128.Count;
    while (i + vec128Count  range means out-of-range (unsigned wraparound catches b .Zero));

This same technique works for popcount (LUT = {0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4}). For simpler cases (single byte value, adjacent range), use Equals + Count or range check instead.

Pattern: Cross-type conversion (widening chains)

When the source and destination types differ (e.g., byte→float for dequantization, short→byte for narrowing):

// Widen: byte → short → int → float
var bytes = Vector128.LoadUnsafe(ref src, offset);
var (lo16, hi16) = Vector128.Widen(bytes);
var (lo32a, lo32b) = Vector128.Widen(lo16);
var f0 = Vector128.ConvertToSingle(lo32a.AsInt32());

// Narrow: int → short → byte (with saturation via Min/Max clamping)
var clamped = Vector128.Min(Vector128.Max(vec, Vector128.Zero), Vector128.Create((short)255));
var narrowed = Vector128.Narrow(clamped.AsUInt16(), nextVec.AsUInt16());

Trailing elements

• Idempotent ops (validation, search): overlap last vector — re-processing is safe
• Aggregations (sum, count, min/max): scalar loop for remainder to avoid double-counting
• Store ops (transform in-place): use ConditionalSelect to merge with last stored vector

Key Rules

• Preserve original method signature — drop-in replacement
• Keep scalar code as fallback — never delete it
• Use Vector128 / Vector256 / Vector512 explicitly — never Vector
• Prefer portable Vector128/Vector256/Vector512 APIs over platform-specific intrinsics (Avx2, Sse42, AdvSimd, Fma) unless there is a significant performance advantage
• Testing: use dotnet run (NOT dotnet test) — xunit.v3 is an in-process runner

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: dotnet
• Source: dotnet/skills
• License: MIT

Install and usage instructions live in the source repository linked above.