OCR and .NET: Architectures for Intelligent Document Processing at Scale

Introduction

If you have ever worked at a company that deals with a lot of paperwork, you know the scene: invoices, contracts, medical records, and receipts arrive as images or scanned PDFs, without a single searchable line of text. As long as the content is stuck in the image, it’s impossible to automate, search, or analyze anything. That’s where optical character recognition (OCR) comes in, transforming these images into text and serving as the foundation for any intelligent document processing project.

Today, extracting text from an image is the easy part. The hard part is doing this with millions of documents without losing any along the way, without blowing the budget, and without exposing personal data. In this article, we’ll build a distributed OCR architecture in .NET 10, covering asynchronous ingestion, parallel processing, image treatment, machine learning, observability, and GDPR.

A Bit of Context

From Classical OCR to Document Understanding

OCR has evolved a lot. Classic engines like Tesseract follow a well-defined script: they analyze the layout, separate the lines and classify character by character. Then came deep learning models, such as recurrent convolutional networks (CRNN), which read the entire sequence at once, without needing to crop each letter. More recently, models like LayoutLM have also started to consider where the text is on the page. With that, the question is no longer just “what is written?” but also “what does this document mean?”, paving the way to extract fields, tables, and forms.

Why .NET?

Because .NET 10 already has almost everything this type of system needs: ASP.NET Core for APIs, System.Threading.Channels and Parallel.ForEachAsync to control concurrency, ML.NET and ONNX Runtime to run models locally, as well as good integration with containers and OpenTelemetry. ML.NET, in fact, was created precisely to bring machine learning to production .NET applications without depending on external runtimes (AHMED et al., 2019).

The Challenges of Processing at Scale

Scalability and fault tolerance

OCR is heavy: it consumes a lot of CPU (or GPU), and the processing time can vary greatly from one document to another. If you process everything within the HTTP request itself, throughput drops and the client ends up waiting until getting a timeout. The natural solution is to put a queue in the middle, separating whoever receives from whoever processes. However, queues bring their own problems: duplicate messages, lost messages, and reprocessing. The golden rule is to assume that every message may arrive more than once (at-least-once) and to make each step idempotent (KLEPPMANN, 2017).

Extraction Quality

In real life, documents don't arrive in perfect condition. There are crooked photos taken by mobile phones, low-resolution scans, stamps over text, and each provider with a different layout. For this reason, the pipeline needs to adapt: apply the right treatment for each type of document and combine OCR with layout models to recover the structure, not just the text.

Security and Privacy

Corporate documents are full of personal data: name, national ID, address, and, in some sectors, even health information. The Brazilian General Data Protection Law (LGPD) requires legal basis, collection of only what is necessary, security, and the ability to prove that all this is being upheld (BRASIL, 2018). In other words, privacy needs to be in the design from the beginning, not patched on at the end.

Hands-On: Reference Architecture in .NET

One service for each step

The idea is to break the pipeline into small services, each with a single responsibility: ingestion, pre-processing, OCR, semantic extraction, anonymization, indexing, and delivery. The big advantage is that each step scales in its own way, which makes all the difference, since OCR consumes much more resources than receiving a file or indexing text (NEWMAN, 2021). The services communicate via messaging (Azure Service Bus, RabbitMQ, or Kafka), so one doesn't have to wait for the other to be available to keep working (RICHARDSON, 2018).

Asynchronous ingestion with 202 Accepted

The entry point doesn't do any OCR. It just saves the file in an object storage, registers the job, publishes a message to the queue, and responds immediately with 202 Accepted, informing the client where they can check the status:

public sealed record DocumentMessage(Guid JobId, Uri BlobUri);

[ApiController]
[Route("api/ocr/documents")]
public sealed class OcrDocumentsController(
    IBlobStorage storage,
    IDocumentQueue queue,
    IJobStore jobs) : ControllerBase
{
    [HttpPost]
    [RequestSizeLimit(20_000_000)]
    public async Task<IActionResult> Submit(IFormFile document, CancellationToken ct)
    {
        if (document.Length == 0)
        {
            return BadRequest("Empty document.");
        }

        var jobId = Guid.CreateVersion7();

        await using var content = document.OpenReadStream();
        var blobUri = await storage.UploadAsync(jobId, content, ct);

        await jobs.CreateAsync(jobId, ct);
        await queue.EnqueueAsync(new DocumentMessage(jobId, blobUri), ct);

        return AcceptedAtAction(nameof(GetStatus), new { jobId }, new { jobId });
    }

    [HttpGet("{jobId:guid}")]
    public async Task<IActionResult> GetStatus(Guid jobId, CancellationToken ct)
        => await jobs.FindAsync(jobId, ct) is { } job ? Ok(job) : NotFound();
}

A detail worth highlighting: Guid.CreateVersion7() generates time-ordered identifiers. This makes database indexes more efficient and makes it much easier to track a document through pipeline logs.

Parallel workers with backpressure

Inside each worker, concurrency needs a limit. A channel with fixed capacity solves this with backpressure: if the workers can't keep up, the message producer waits a bit, instead of accumulating everything in memory until the container crashes.

var channel = Channel.CreateBounded<DocumentMessage>(new BoundedChannelOptions(capacity: 500)
{
    FullMode = BoundedChannelFullMode.Wait
});

public static Task RunOcrWorkersAsync(
    ChannelReader<DocumentMessage> reader,
    IOcrProcessor processor,
    int degreeOfParallelism,
    CancellationToken ct)
{
    var options = new ParallelOptions
    {
        MaxDegreeOfParallelism = degreeOfParallelism,
        CancellationToken = ct
    };

    return Parallel.ForEachAsync(reader.ReadAllAsync(ct), options, async (message, token) =>
    {
        // Idempotence: duplicate deliveries do not cause reprocessing
        if (await processor.IsAlreadyProcessedAsync(message.JobId, token))
        {
            return;
        }

        await processor.ProcessAsync(message, token);
    });
}

Since OCR is CPU-intensive, a good starting point for the degree of parallelism is the number of container cores. From there, adjust by looking at real latency and resource usage metrics.

Image pre-processing

Bad image, bad text, it's that simple. Before OCR, it's worth converting to grayscale, correcting orientation, boosting contrast, binarizing, removing noise, and straightening skewed pages (deskew). In .NET, ImageSharp handles the basics with no native dependencies; for more advanced operations, like deskew, you can use OpenCV via OpenCvSharp.

using SixLabors.ImageSharp;
using SixLabors.ImageSharp.PixelFormats;
using SixLabors.ImageSharp.Processing;

public static async Task<Stream> PreprocessAsync(Stream input, CancellationToken ct)
{
    // L8 loads the image directly in grayscale (8 bits)
    using var image = await Image.LoadAsync<L8>(input, ct);

    image.Mutate(ctx => ctx
        .AutoOrient()
        .Contrast(1.2f)
        .BinaryThreshold(0.5f));

    var output = new MemoryStream();
    await image.SaveAsPngAsync(output, ct);
    output.Position = 0;
    return output;
}

Adding Machine Learning to the Pipeline

What document is this?

With the text in hand, the next step is to find out what kind of document it is: an invoice, a contract, a receipt? This answer defines which field extractor will come into play. With ML.NET, you can train and run a multiclass classifier within the .NET process itself (AHMED et al., 2019):

public sealed class OcrDocument
{
    public string OcrText { get; set; } = string.Empty;
    public string DocumentType { get; set; } = string.Empty;
}

public sealed class DocumentTypePrediction
{
    [ColumnName("PredictedLabel")]
    public string DocumentType { get; set; } = string.Empty;
}

var mlContext = new MLContext(seed: 42);
IDataView trainingData = mlContext.Data.LoadFromEnumerable(labeledDocuments);

var pipeline = mlContext.Transforms.Conversion
        .MapValueToKey("Label", nameof(OcrDocument.DocumentType))
    .Append(mlContext.Transforms.Text.FeaturizeText("Features", nameof(OcrDocument.OcrText)))
    .Append(mlContext.MulticlassClassification.Trainers.SdcaMaximumEntropy())
    .Append(mlContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));

var model = pipeline.Fit(trainingData);

var engine = mlContext.Model.CreatePredictionEngine<OcrDocument, DocumentTypePrediction>(model);
var prediction = engine.Predict(new OcrDocument { OcrText = extractedText });

Heads up: the PredictionEngine is not thread-safe. In APIs and web services, use the PredictionEnginePool from the Microsoft.Extensions.ML package.

Tables, layouts, and external models

To understand tables and forms, it’s not enough to just read the words in sequence: the model needs to know where each text is on the page. Usually, these models are trained in Python and exported to ONNX, running on .NET via ONNX Runtime. Another option is to publish them as dedicated GPU microservices, accessed via gRPC. The right choice depends on how much you can spend on hardware and how much latency each step can tolerate.

Observability and Autoscaling

Metrics with OpenTelemetry

Without per-step metrics, you don’t know where the bottleneck is, and scaling becomes a guessing game. At a minimum, track latency by document type, failure rate per step, and queue length. These are the signals that will back your SLOs and alerts (BEYER et al., 2016). In .NET 10, the System.Diagnostics.Metrics API communicates directly with OpenTelemetry, and you can export the data to Prometheus, Azure Monitor, or any backend compatible with OTLP:

using System.Diagnostics.Metrics;

public sealed class OcrMetrics
{
    public const string MeterName = "Leaders.Ocr";

    private readonly Histogram<double> _duration;
    private readonly Counter<long> _failures;

    public OcrMetrics(IMeterFactory meterFactory)
    {
        var meter = meterFactory.Create(MeterName);
        _duration = meter.CreateHistogram<double>("ocr.document.duration", unit: "ms");
        _failures = meter.CreateCounter<long>("ocr.document.failures");
    }

    public void RecordSuccess(TimeSpan duration, string documentType)
        => _duration.Record(duration.TotalMilliseconds, new KeyValuePair<string, object?>("document.type", documentType));

    public void RecordFailure(string stage)
        => _failures.Add(1, new KeyValuePair<string, object?>("ocr.stage", stage));
}

// Program.cs
builder.Services.AddSingleton<OcrMetrics>();
builder.Services.AddOpenTelemetry()
    .WithMetrics(metrics => metrics
        .AddMeter(OcrMetrics.MeterName)
        .AddOtlpExporter());

Scaling by Queue

For OCR workers, CPU usage isn’t always the best indicator of when to scale. Queue length gives a much clearer view of how much work is waiting. In Kubernetes, you can combine the Horizontal Pod Autoscaler with KEDA to scale pods according to the queue in Service Bus, RabbitMQ, or Kafka, even reaching zero replicas when there’s nothing to process (BURNS; BEDA; HIGHTOWER, 2019).

Security, Compliance and Governance

Encryption and anonymization

Data must be protected both at rest, with keys managed in services like Azure Key Vault or HashiCorp Vault, and in transit, with TLS 1.2 or higher. And before indexing anything, mask personal identifiers: this is the LGPD minimization principle in practice (BRASIL, 2018). For a first layer of protection, a compile-time generated regular expression already helps a lot:

using System.Text.RegularExpressions;

public static partial class PiiMasker
{
    // Accepts formatted CPF (000.000.000-00) or just digits
    [GeneratedRegex(@"\b\d{3}\.?\d{3}\.?\d{3}-?\d{2}\b")]
    private static partial Regex CpfRegex();

    public static string MaskCpf(string text)
        => CpfRegex().Replace(text, "***.***.***-**");
}

Just don’t rely only on regex: it generates false positives and doesn’t catch names or addresses. In production, combine the verification digit validation with entity recognition models (NER) trained for your domain.

Audit and traceability

Each step must log who processed the document, with which model version and what the result was. This audit trail is what enables accountability, as required by the LGPD (BRASIL, 2018). And there’s even a bonus: when a model is updated, you know exactly which documents need to be reprocessed.

Storage, Search and Delivery

Indexing for search

The original files remain in object storage (Azure Blob Storage or Amazon S3), while the already anonymized text and metadata go to a search engine. If you still use NEST, here's a tip: it was discontinued as of Elasticsearch 8, and the official client now is Elastic.Clients.Elasticsearch:

using Elastic.Clients.Elasticsearch;

public sealed record OcrIndexedDocument(
    Guid Id,
    string FileName,
    string DocumentType,
    string MaskedText,
    DateTimeOffset ProcessedAt);

var settings = new ElasticsearchClientSettings(new Uri("https://localhost:9200"))
    .DefaultIndex("ocr-documents");
var client = new ElasticsearchClient(settings);

var document = new OcrIndexedDocument(jobId, fileName, documentType, PiiMasker.MaskCpf(ocrText), DateTimeOffset.UtcNow);
var response = await client.IndexAsync(document, ct);

if (!response.IsValidResponse)
{
    throw new InvalidOperationException($"Failed to index document {jobId}.");
}

Integration with legacy systems

The result can reach the client via status query (polling), webhooks, or events published on the bus. When the destination is an ERP or another legacy system, an anti-corruption layer translates the pipeline model to the already existing contracts. This way, internal changes do not affect those who consume the data (RICHARDSON, 2018).

In Practice: A High-Volume Multitenant Pipeline

The scenario

To put all of this into action (no pun intended), imagine a financial group that needs to process contracts and receipts from various companies. The objectives are: 1 million documents per day, peaks of 10,000 documents per minute, average latency of 2 seconds per document, table extraction, anonymization, and a complete audit trail. The scenario is illustrative, but the numbers help clarify the decisions.

How many workers are needed?

Little’s Law provides the answer: the required concurrency is the arrival rate multiplied by the latency. At peak, 10,000 documents per minute is about 167 per second. With 2 seconds per document, the system needs to handle about 334 processing tasks at the same time. With workers with 8 parallelism slots each, that means around 42 OCR pods at peak, versus just a few replicas during regular operation. Keeping all of that running all the time would be wasteful, which is why scaling by the queue length makes so much sense.

Architectural decisions

• Tenant isolation: each client has its own queues and storage prefixes, so a tenant with huge volume doesn’t affect the others.
• SLA-based priority: urgent documents and batch processing go to different queues.
• GPU only where it matters: layout and NER models run in dedicated pools, accessed via gRPC; classic OCR stays on CPU.
• Dead-letter queues: documents that repeatedly fail are separated for analysis, without blocking the rest of the flow.

Risks and how to mitigate them

The main risks here are the enormous variety of layouts and the silent degradation of model quality over time. To mitigate these, classify the document before extracting fields, maintain validation sets for each document type, and monitor the distribution of input data to detect when it starts to change (SCULLEY et al., 2015).

Best Practices and Lessons Learned

1. Treat models as code

Models, pipelines, and infrastructure deserve versioning and deployment via CI/CD, with Infrastructure as Code (Terraform, Bicep, or Helm). Machine learning systems quickly accumulate technical debt when data dependencies and configurations are left implicit (SCULLEY et al., 2015).

2. Test with real documents

Unit and integration tests are important, but not enough. Assemble a set of reference documents with the expected text and measure the error rate per character with each new version. That way, you catch accuracy drops that no functional test would show.

3. Plan for failures from day one

Timeouts, retries with exponential backoff, idempotency, and dead-letter queues need to be part of the design from the beginning—not something rushed in after the first incident (KLEPPMANN, 2017).

Conclusion

In the end, OCR at scale is much more a distributed architecture problem than a text recognition one. And .NET 10 provides the right pieces to solve it: asynchronous, parallel, and observable pipelines, bringing together classic OCR, layout models, and machine learning in the same ecosystem.

Asynchronous ingestion, backpressure, idempotency, scaling through the queue, and anonymization before indexing: with these ingredients, document processing stops being a headache and becomes a predictable and auditable capability. And the future is promising: multimodal models, which understand text, image, and layout at the same time, should increasingly reduce the need for specific rules for each document type. If you are starting out, do not try to build everything at once: start with the queue and idempotency, measure everything, and evolve from there.