I Allocated 1.5 GB Just to Add Vectors Together
I had a loop that did something embarrassingly simple: fold a stream of feature vectors into one running total. Add a vector, add the next, keep going. A few hundred thousand of them. The math is a for-loop over some dou
I had a loop that did something embarrassingly simple: fold a stream of feature vectors into one running total. Add a vector, add the next, keep going. A few hundred thousand of them. The math is a for-loop over some doubles, nothing that should trouble the garbage collector.
The allocation profile said otherwise. Gigabytes. For adding numbers.
I knew why the second I looked at the operator. I'd written + the tidy, immutable way years ago and never thought about it again:
public static Vec operator +(Vec a, Vec b)
{
var result = new double[a.Data.Length];
for (int i = 0; i < result.Length; i++)
result[i] = a.Data[i] + b.Data[i];
return new Vec(result);
}
That's a perfectly reasonable operator. Every call hands back a brand new Vec with its own fresh array, nobody's state gets stomped, and you can reason about it without fear. It's also a fresh 8 KB array on the heap every single time you add two 1024-dimension vectors. In a one-off expression that's free. In a fold over a stream, it's a steady firehose into gen 0.
My accumulation loop was the firehose:
var acc = new Vec(Dims);
for (int i = 0; i < input.Length; i++)
acc = acc + input[i];
Read that acc = acc + input[i] again. I'm allocating a whole new accumulator on every iteration and throwing the old one away. I never needed a new object. I wanted to pour the next vector into the one I already had.
The operator I couldn't write before
Until C# 14 there was no clean way to say that. += was always just sugar for a = a + b: it called your operator +, took the result, and assigned it back. You couldn't hook the "in place" part because there was no in-place part to hook.
C# 14 adds user-defined compound assignment operators. You write operator += as its own thing. It's an instance method, it returns void, and it mutates this.
public void operator +=(Vec other)
{
var mine = Data;
var theirs = other.Data;
for (int i = 0; i < mine.Length; i++)
mine[i] += theirs[i];
}
No new. No array. It reaches into the buffer this instance already owns and adds the other vector straight into it. The call site doesn't change at all: acc += input[i] now finds this operator instead of falling back to +. If I hadn't written it, the compiler would've quietly used operator + like always. You only get the new behavior when you opt in.
I counted it
I don't trust "should be faster," so I measured. I built 200,000 vectors of 1024 doubles each, folded them both ways, and tracked wall-clock time (best of five) alongside bytes allocated with GC.GetAllocatedBytesForCurrentThread(). This is a container with workstation GC, not a lab bench, so I care about the ratio and the allocation column, not the exact milliseconds.
dims=1024, vectors=200,000, rounds=5, GC=workstation
acc = acc + v best 740.0 ms allocated 1571.7 MB checksum 102400896.09
acc += v best 301.9 ms allocated 0.0 MB checksum 102400896.09
same result: True
time ratio: 2.45x
1.5 GB versus zero. Same checksum, so the two loops genuinely compute the same answer. The in-place version came out 2.45x faster, and most of that gap isn't the addition. It's the collector cleaning up a gigabyte and a half of short-lived arrays that never had any reason to exist. Zero megabytes still makes me grin a little. The whole fold runs without handing the GC a single byte.
When I'd leave the old operator alone
Here's the part the benchmark won't tell you: mutating in place throws away the one thing immutable + was buying you. With operator +, nobody's vector changes under them. With operator +=, if two references point at the same Vec and one of them does +=, the other just silently got mutated too. That's a classic aliasing bug, and it's exactly the kind of thing that's fine in a tight local loop and nasty in a shared domain model.
So I'm not about to replace every + with an in-place +=. For value-ish types you pass around, compare, and cache, immutability is worth more than the allocations. My honest rule: reach for the in-place operator when you've got a genuine hot accumulation loop and the accumulator is local and yours. Everywhere else, let + keep making new things.
One more sharp edge. The compiler doesn't enforce consistency, so you can define += without +, or make them disagree, and it won't say a word. I'd keep the two in sync regardless, because a += that quietly diverges from + is a debugging session waiting to happen.
I normally drop a link to the runnable sample here, but my code repo push got blocked this time, so the whole thing is right below. Target net10.0 with <LangVersion>14</LangVersion> and it runs as-is:
using System.Diagnostics;
sealed class Vec
{
public readonly double[] Data;
public Vec(int dims) => Data = new double[dims];
public Vec(double[] data) => Data = data;
// Immutable style: a fresh Vec + array on every add.
public static Vec operator +(Vec a, Vec b)
{
var result = new double[a.Data.Length];
for (int i = 0; i < result.Length; i++)
result[i] = a.Data[i] + b.Data[i];
return new Vec(result);
}
// C# 14 in-place compound assignment: mutates this buffer, allocates nothing.
public void operator +=(Vec other)
{
var mine = Data;
var theirs = other.Data;
for (int i = 0; i < mine.Length; i++)
mine[i] += theirs[i];
}
public double Checksum()
{
double s = 0;
foreach (var d in Data) s += d;
return s;
}
}
static class Program
{
const int Dims = 1024;
const int Count = 200_000;
const int Rounds = 5;
static Vec[] BuildInputs()
{
var rng = new Random(42);
var vecs = new Vec[Count];
for (int i = 0; i < Count; i++)
{
var d = new double[Dims];
for (int j = 0; j < Dims; j++) d[j] = rng.NextDouble();
vecs[i] = new Vec(d);
}
return vecs;
}
static (double sum, long bytes, double ms) RunPlus(Vec[] input)
{
var sw = Stopwatch.StartNew();
long before = GC.GetAllocatedBytesForCurrentThread();
var acc = new Vec(Dims);
for (int i = 0; i < input.Length; i++)
acc = acc + input[i];
long after = GC.GetAllocatedBytesForCurrentThread();
sw.Stop();
return (acc.Checksum(), after - before, sw.Elapsed.TotalMilliseconds);
}
static (double sum, long bytes, double ms) RunCompound(Vec[] input)
{
var sw = Stopwatch.StartNew();
long before = GC.GetAllocatedBytesForCurrentThread();
var acc = new Vec(Dims);
for (int i = 0; i < input.Length; i++)
acc += input[i];
long after = GC.GetAllocatedBytesForCurrentThread();
sw.Stop();
return (acc.Checksum(), after - before, sw.Elapsed.TotalMilliseconds);
}
static void Main()
{
Console.WriteLine($"dims={Dims}, vectors={Count:N0}, rounds={Rounds}, GC=workstation");
var input = BuildInputs();
_ = RunPlus(input); // warm up the JIT
_ = RunCompound(input);
double plusChk = 0, compChk = 0, plusBest = double.MaxValue, compBest = double.MaxValue;
long plusBytes = 0, compBytes = 0;
for (int r = 0; r < Rounds; r++)
{
var p = RunPlus(input);
var c = RunCompound(input);
plusChk = p.sum; compChk = c.sum;
plusBytes = p.bytes; compBytes = c.bytes;
if (p.ms < plusBest) plusBest = p.ms;
if (c.ms < compBest) compBest = c.ms;
}
Console.WriteLine($"acc = acc + v best {plusBest,8:F1} ms allocated {plusBytes / (1024.0 * 1024.0),8:F1} MB checksum {plusChk:F2}");
Console.WriteLine($"acc += v best {compBest,8:F1} ms allocated {compBytes / (1024.0 * 1024.0),8:F1} MB checksum {compChk:F2}");
Console.WriteLine($"same result: {Math.Abs(plusChk - compChk) < 1e-6}");
Console.WriteLine($"time ratio: {plusBest / compBest:F2}x");
}
}
Have you hit a spot where += was quietly allocating behind your back? I'd like to hear where it bit you.
β still benchmarking things nobody asked me to benchmark
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.