175 MB of Substrings to Read One Column

I had a boring little job: a pile of CSV-shaped lines in memory, and I needed one number out of each — the amount column, third field, that's it. The rest of the row was along for the ride. I wrote the obvious loop, it gave the right total, and I moved on. Then the allocation profiler lit up on that exact loop and I went back to look. A million rows in, and I'd minted somewhere north of a hundred megabytes of garbage to read a single integer per line. Every other field I split out, I threw straight away.

Here's the loop that did it:

long total = 0;
foreach (string line in lines)
{
    string[] fields = line.Split(',');
    total += long.Parse(fields[2], CultureInfo.InvariantCulture);
}

line.Split(',') is doing precisely what I asked and nothing I wanted. For each row it allocates a string[] to hold the pieces, then allocates a fresh string for every field it finds, four of them on my data, copying the characters out of the original line into each one. I read fields[2] and let the array and the other three strings fall on the floor. Four columns, five allocations a row, one of them actually used.

The split that doesn't cut anything out

The span-based Split has been around for a couple of versions now and it works on a different idea. Instead of handing you new strings, it tells you where the pieces are. You give it a destination buffer of Range values, it fills in the start and end of each field, and you slice the original line yourself when you want to look at one. Nothing gets copied. Nothing gets allocated.

long total = 0;
Span<Range> ranges = stackalloc Range[8];
foreach (string line in lines)
{
    ReadOnlySpan<char> span = line;
    int count = span.Split(ranges, ',');
    ReadOnlySpan<char> amount = span[ranges[2]];
    total += long.Parse(amount, CultureInfo.InvariantCulture);
}

The stackalloc Range[8] buffer sits on the stack and gets reused on every iteration — one buffer for the whole loop, not one per row. span.Split writes the field boundaries into it and returns how many it found. span[ranges[2]] is a view over the amount field, still pointing into the original line's characters, and long.Parse has had a ReadOnlySpan<char> overload for ages, so it parses that slice straight without a substring in between. The row never becomes an array, the fields never become strings, and the only thing that leaves the loop is the running total.

The numbers

A million rows, built in memory, parsed best-of-7 in a container. Not a lab — I care about the ratio between the two, not the absolute milliseconds on whatever box this ran on.

rows                 : 1,000,000
string.Split sum     : 4,998,543,546
Span<Range> sum      : 4,998,543,546
sums match           : True

string.Split  best   :     75.7 ms   alloc   174.7 MB
Span<Range>   best   :     32.1 ms   alloc     0.0 MB

The sums matching is the part I care about most. Same answer, so this is a fair fight and not a trick where I quietly dropped work. On that footing the span path ran in a bit under half the time and allocated essentially nothing. The 174.7 MB the naive loop burned is all throwaway: arrays and substrings created and collected inside a single pass. Dropping it doesn't just save memory, it saves the GC the job of walking all that back out later, which is a chunk of where the time goes too.

I'll admit I expected the allocation win and only hoped for the speed one. Getting both was the nice surprise.

When I'd leave Split alone

I would not reach for this everywhere, and I'd push back on anyone who tried. If you actually need the fields as strings — you're stashing them in a list, returning them, putting them in a dictionary — then the span version just means you call .ToString() on each slice and you've bought back every allocation you saved, plus more awkward code. The ranges approach pays off exactly when you read the fields, use them, and let them go without keeping them around. Parsing to a number, comparing, summing: yes. Collecting rows into objects you hold onto: no, Split into strings is honest about what you need.

The other caveat is size. At a few hundred rows none of this registers; the naive loop allocates a rounding error and you'd never see it in a profile. This only became worth touching because the row count had three commas and a half-decent chance of growing. Reaching for stackalloc Range[] on a cold path nobody runs in a loop is effort spent making code harder to read for zero measured gain, and I think that trade is a bad one often enough to say it out loud.

One real gotcha: size the Range buffer for the most fields you expect, because Split stops once the destination is full and folds the rest of the line into the last range. Eight slots for four columns gives me headroom; a buffer that's too small turns into a silent correctness bug, not an exception.

Here's the whole thing if you want to run it yourself. It's a plain net10.0 console app — one Program.cs, nothing else, build and run with dotnet run -c Release.

using System.Diagnostics;
using System.Globalization;

const int Rows = 1_000_000;
var lines = new string[Rows];
var rng = new Random(42);
string[] regions = ["emea", "apac", "amer", "latam"];
for (int i = 0; i < Rows; i++)
{
    int amount = rng.Next(1, 10_000);
    lines[i] = $"{i},{regions[i & 3]},{amount},{(i % 2 == 0 ? 1 : 0)}";
}

_ = SumWithSplit(lines);   // warm up both paths before timing
_ = SumWithRanges(lines);

const int Trials = 7;
var (splitSum, splitMs, splitBytes) = Measure(() => SumWithSplit(lines), Trials);
var (rangeSum, rangeMs, rangeBytes) = Measure(() => SumWithRanges(lines), Trials);

Console.WriteLine($"sums match        : {splitSum == rangeSum}");
Console.WriteLine($"string.Split  best: {splitMs,8:F1} ms   alloc {splitBytes / (1024.0 * 1024.0),7:F1} MB");
Console.WriteLine($"Span<Range>   best: {rangeMs,8:F1} ms   alloc {rangeBytes / (1024.0 * 1024.0),7:F1} MB");

static long SumWithSplit(string[] lines)
{
    long total = 0;
    foreach (string line in lines)
    {
        string[] fields = line.Split(',');
        total += long.Parse(fields[2], CultureInfo.InvariantCulture);
    }
    return total;
}

static long SumWithRanges(string[] lines)
{
    long total = 0;
    Span<Range> ranges = stackalloc Range[8];
    foreach (string line in lines)
    {
        ReadOnlySpan<char> span = line;
        span.Split(ranges, ',');
        total += long.Parse(span[ranges[2]], CultureInfo.InvariantCulture);
    }
    return total;
}

static (long sum, double bestMs, long bytes) Measure(Func<long> work, int trials)
{
    long sum = 0;
    double best = double.MaxValue;
    long bytesForBest = 0;
    for (int t = 0; t < trials; t++)
    {
        GC.Collect();
        GC.WaitForPendingFinalizers();
        GC.Collect();
        long before = GC.GetAllocatedBytesForCurrentThread();
        var sw = Stopwatch.StartNew();
        sum = work();
        sw.Stop();
        long after = GC.GetAllocatedBytesForCurrentThread();
        if (sw.Elapsed.TotalMilliseconds < best)
        {
            best = sw.Elapsed.TotalMilliseconds;
            bytesForBest = after - before;
        }
    }
    return (sum, best, bytesForBest);
}

Have you swapped a hot Split over to ranges, or found a spot where it wasn't worth the trouble? I'd like to hear where the line fell for you.

— still benchmarking the boring loops nobody asked me to touch