# Why I Rewrote Our Hot Path in RustA Go service was burning 40% CPU on JSON serialization. A targeted Rust rewrite via FFI cut p99 latency by 3x.

## The Problem

We had a video fingerprinting pipeline processing ~12k requests/sec. The core matching logic was fast — but the service was spending **40% of CPU time** serializing response payloads. `pprof` made this obvious:

```
(pprof) top10
Showing nodes accounting for 4.2s, 42% of 10s total
      flat  flat%   sum%        cum   cum%
     2.1s 21.00% 21.00%      2.8s 28.00%  encoding/json.(*encodeState).marshal
     0.9s  9.00% 30.00%      0.9s  9.00%  encoding/json.(*encodeState).string
     0.6s  6.00% 36.00%      0.6s  6.00%  runtime.mallocgc
```

The response struct had nested slices of fingerprint matches — each response was ~15KB of JSON with hundreds of float64 arrays. Go's `encoding/json` uses reflection, allocates per-field, and can't be easily vectorized.

## Why Not Just Use a Faster Go Serializer?

I tried the obvious fixes first:

| Approach | Result |
|----------|--------|
| `json-iterator/go` | 15% faster — not enough |
| `easyjson` (codegen) | 25% faster — better, but still reflection-free only at top level |
| `sonic` (JIT) | 35% faster — close, but requires amd64 and CGo |
| Protocol Buffers | Would require client changes across 4 teams |

None of these solved the fundamental issue: we were allocating thousands of small objects per response, and the GC was paying for it.

## The Rust Approach

Instead of rewriting the entire service, I wrote a shared library (`.so`) that handles *only* serialization:

```rust
use serde::Serialize;
use serde_json::to_vec;

#[repr(C)]
pub struct SerializeResult {
    ptr: *mut u8,
    len: usize,
    cap: usize,
}

#[no_mangle]
pub extern "C" fn serialize_matches(
    data_ptr: *const u8,
    data_len: usize,
) -> SerializeResult {
    let input = unsafe {
        std::slice::from_raw_parts(data_ptr, data_len)
    };

    // Deserialize from a compact binary format (MessagePack)
    let matches: Vec<MatchResult> = rmp_serde::from_slice(input)
        .expect("invalid msgpack input");

    // Serialize to JSON using simd-json when available
    let mut output = to_vec(&matches).unwrap();
    let result = SerializeResult {
        ptr: output.as_mut_ptr(),
        len: output.len(),
        cap: output.capacity(),
    };
    std::mem::forget(output); // caller frees
    result
}
```

The Go side calls this via CGo:

```go
// #cgo LDFLAGS: -L./lib -lfingerprint_serial
// #include "serial.h"
import "C"

func serializeMatches(packed []byte) []byte {
    result := C.serialize_matches(
        (*C.uchar)(unsafe.Pointer(&packed[0])),
        C.size_t(len(packed)),
    )
    defer C.free_buffer(result.ptr, result.len, result.cap)
    return C.GoBytes(unsafe.Pointer(result.ptr), C.int(result.len))
}
```

## Results

After deploying to one pod and comparing:

| Metric | Before | After | Change |
|--------|--------|-------|--------|
| p50 latency | 12ms | 4ms | **-67%** |
| p99 latency | 45ms | 15ms | **-67%** |
| CPU usage | 78% | 41% | **-47%** |
| Allocs/op | 847 | 12 | **-98.5%** |
| GC pause p99 | 3.2ms | 0.4ms | **-87%** |

The key win wasn't just "Rust is faster" — it's that we moved ~800 allocations per request out of Go's GC entirely. The Rust code allocates once (the output buffer) and the Go side copies it out immediately.

## When Is This Worth It?

A partial FFI rewrite makes sense when:

1. **The hot path is isolated** — serialization is a pure function with clear input/output boundaries
2. **The bottleneck is allocation, not logic** — if your code is slow because of algorithmic complexity, Rust won't magically fix it
3. **You can't change the protocol** — if we could switch to Protobuf everywhere, that would've been simpler
4. **The team can maintain it** — a 200-line Rust library with CI is manageable; rewriting the whole service isn't

## What I'd Do Differently

- **Start with flamegraphs, not intuition** — I almost rewrote the matching logic first, which was already fast
- **Use `cbindgen` from day one** — I hand-wrote the C header initially and got bitten by alignment issues
- **Benchmark the FFI overhead itself** — CGo has ~50ns overhead per call. For us that's negligible at 12ms baseline, but for microsecond-level work it matters

The full serialization library is ~200 lines of Rust. The CGo wrapper is ~40 lines of Go. Total effort: 3 days including benchmarking and rollout. The service has been running this in production for 6 weeks with zero issues.

---

*This is a pattern I'll write more about — using Rust as a "performance escape hatch" from Go/Python/Node without committing to a full rewrite.*