Lab / Open methodology
.NET bulk import: faster catalog lookups
50,000 catalog import rows: 708.22 ms → 1.51 ms. Synthetic lab; median of 10 runs. Open code and raw data.
Scan the list for every rowtoDictionary lookup by product code
.NET 10 LAB MEASUREMENT
Stop searching the full catalog for every row.
Match product codes in an order import against a 10,000-product catalog. Same .NET version and output; keyed access instead of repeated list scans.
Reduce waiting and repeated CPU work in large product and order imports. Each incoming row no longer triggers another search through the entire catalog.
Scan the list for every row
List.Find with a capturing predicate per row. Identical catalog ordering and business rule.
- CPU time
- 705.682 ms
- Managed allocation (bytes)
- 4,801,064
Dictionary lookup by product code
Includes index construction and the output array. Dictionary is also available in older .NET versions.
- CPU time
- 1.528 ms
- Managed allocation (bytes)
- 684,080
FrozenDictionary control
Modern read-only API with construction included. It is not assumed to be faster for every file.
- CPU time
- 1.991 ms
- Managed allocation (bytes)
- 1,610,160
Bars start at zero and share the same time scale across implementations.
Replace repeated List.Find searches with a Dictionary keyed by product code. Rebuild the index for every file and include that construction time in the total. Measure FrozenDictionary as an additional alternative.
All load points and measurements
| Implementation | Median duration | CPU time | Managed allocation (bytes) |
|---|---|---|---|
| Scan the list for every row | 14.216 ms | 14.237 ms | 97,064 |
| Dictionary lookup by product code | 0.21 ms | 0.229 ms | 292,080 |
| FrozenDictionary control | 0.897 ms | 0.915 ms | 1,218,160 |
| Implementation | Median duration | CPU time | Managed allocation (bytes) |
|---|---|---|---|
| Scan the list for every row | 143.355 ms | 142.927 ms | 961,064 |
| Dictionary lookup by product code | 0.466 ms | 0.488 ms | 364,080 |
| FrozenDictionary control | 1.059 ms | 1.076 ms | 1,290,160 |
| Implementation | Median duration | CPU time | Managed allocation (bytes) |
|---|---|---|---|
| Scan the list for every row | 708.219 ms | 705.682 ms | 4,801,064 |
| Dictionary lookup by product code | 1.506 ms | 1.528 ms | 684,080 |
| FrozenDictionary control | 1.961 ms | 1.991 ms | 1,610,160 |
CPU cost at your workload
Compare the recorded CPU time at your own monthly volume.
One batch run = 50,000 Imported file rows.
CPU time reduced / month1.956 hours
Processing cost difference / month—Calculated after you enter a unit price.
Lab CPU time × your volume and unit price. This is not a hosting bill forecast; idle capacity, networking, storage, licences and migration are excluded.
Try it yourself
Replay the recorded lab results side by side.
As the workload grows
Duration by workload
- Before
- After
Duration by workload (ms)
Show as a table
| Imported file rows | Before | After |
|---|---|---|
| 1,000 | 14.22 ms | 0.21 ms |
| 10,000 | 143.36 ms | 0.47 ms |
| 50,000 | 708.22 ms | 1.51 ms |
How we measured
Environment, protocol and technical analysis
Environment and data
- Environment
- Apple M3; Docker Linux arm64; 1 CPU, 768 MiB; .NET 10.0.13. Shared developer workstation.
- Dataset
- 10,000 products; 1,000/10,000/50,000 import rows. Every 17th row has an unknown SKU; fixed order, prices and quantities.
- Tools
- .NET 10.0.13 · Stopwatch · Linux CLOCK_PROCESS_CPUTIME_ID · GC.GetTotalAllocatedBytes(true) · Independent Python SHA-256 output oracle · DOTNET_TieredCompilation=0
- Repetitions
- 3 sizes × 3 implementations × 10 measurements, plus 18 warmups. Same process, fixed shuffled order; every output verified.
Stop searching the full catalog for every row.
Match product codes in an order import against a 10,000-product catalog. Same .NET version and output; keyed access instead of repeated list scans.
Identical catalog and import rows on the same .NET 10 runtime. Two warmups and ten measurements per arm and size; 108 observations in total. End-to-end duration includes index construction and output-array creation. Input generation, pre-run GC and output SHA-256 are outside every arm. An independent Python business-rule oracle checks the little-endian int64 output hash, count, missing SKUs and cent total. CPU uses the Linux process CPU clock; memory is managed allocation delta. The primary point is the preselected 50,000 rows.
Inspect the measurement script ↗What changes for the business?
Reduce waiting and repeated CPU work in large product and order imports. Each incoming row no longer triggers another search through the entire catalog.
Change and scope
Replace repeated List.Find searches with a Dictionary keyed by product code. Rebuild the index for every file and include that construction time in the total. Measure FrozenDictionary as an additional alternative.
.NET 10.0.13; Linux arm64 Docker, 1-CPU quota, 768 MiB. Match rows already prepared in memory; CSV reading and database writes are excluded. An algorithm change on the same runtime, not a .NET Framework → 10 speed comparison. Managed allocation is not peak memory or RSS. Index construction and allocation costs remain visible at small file sizes.
Building the index is part of the job
Dictionary and FrozenDictionary are rebuilt for every file. The construction times below are included in the main end-to-end chart, not subtracted.
| File rows | Dictionary construction | FrozenDictionary construction |
|---|---|---|
| 1,000 | 0.183 ms | 0.871 ms |
| 10,000 | 0.187 ms | 0.873 ms |
| 50,000 | 0.183 ms | 0.923 ms |
FrozenDictionary is a candidate for long-lived catalogs with many reads. If it costs more in this single-file workflow, that result remains visible. At small file sizes, index allocation may exceed the old list implementation; every size and allocation counter is in the interactive table.
Separate the runtime from the algorithm
Repeated list scans do not disappear automatically when code moves to a modern runtime. A Dictionary can also improve an older .NET application. This experiment does not measure .NET Framework or old Windows Server support/security; runtime and dependency migration needs its own compatibility, patching and deployment checks.
Where to start in your system
Identify the slow operation, data volume and peak workload. Establish its baseline, change the code and access pattern, then compare the same result.
What we did
Business rule and data
AIPrepared the synthetic workflow, tests and measurement automation.
VerificationPublished code and raw outputs are available for review; this is not a client project.
Implementation change
AIReplace repeated List.Find searches with a Dictionary keyed by product code. Rebuild the index for every file and include that construction time in the total. Measure FrozenDictionary as an additional alternative.
VerificationOutput equivalence was independently checked; every repetition is retained.