rustedf.cloud
Menu

Generate thousands of PDFs in C#: pacing, 429 and safe retries

Published 3 min read

Month-end invoices, a certificate for every attendee, a statement for every customer: sooner or later you need thousands of PDFs, not one. The obvious code, a list of tasks and Task.WhenAll, sends all the requests at once, and the service answers with 429 for most of them. This guide builds a batch runner that keeps to the rate of your plan, retries what is safe to retry, and can be stopped and started again without doing the same work twice. It uses the typed client style of the C# and ASP.NET Core guide.

Know the limits first

  • The request rate belongs to the account. Your plan sets how many requests per minute the account may make, and every key shares it. A second key does not raise the limit; it only splits the same allowance. The numbers are on the plans page.
  • A render costs one credit, and a failed render costs none. A credit is taken when a render starts and returned if it fails, so retrying a failure is safe.
  • No credits means a 402. Nothing was rendered, and every other request would fail the same way, so the whole batch should stop.

Check the balance before you begin. GET /v1/me returns your plan and your credits:

public sealed record Me(string Plan, long Credits);
var me = await http.GetFromJsonAsync<Me>("v1/me", cancellationToken);
if (me!.Credits < jobs.Count)
throw new InvalidOperationException($"{me.Credits} credits for {jobs.Count} documents.");

Pace the calls

A token bucket lets one request through every so often, whatever the number of workers. .NET has one in System.Threading.RateLimiting:

using System.Threading.RateLimiting;
var limiter = new TokenBucketRateLimiter(new TokenBucketRateLimiterOptions
{
TokenLimit = 1,
TokensPerPeriod = 1,
ReplenishmentPeriod = TimeSpan.FromSeconds(60.0 / requestsPerMinute),
QueueLimit = int.MaxValue,
QueueProcessingOrder = QueueProcessingOrder.OldestFirst,
AutoReplenishment = true,
});

Set requestsPerMinute a little below the rate of your plan, so that other calls from the same account still fit. Each worker takes a token before it sends a request.

The runner

using System.Collections.Concurrent;
using System.Net;
using System.Net.Http.Json;
public sealed record Job(string Number, string TemplateId, string Locale, Dictionary<string, object?> Data);
public sealed class OutOfCreditsException : Exception;
public sealed class BatchRenderer(HttpClient http, TokenBucketRateLimiter limiter, int workers)
{
public async Task<IReadOnlyList<string>> RunAsync(
IEnumerable<Job> jobs, string folder, CancellationToken cancellationToken)
{
var failed = new ConcurrentBag<string>();
var options = new ParallelOptions
{
MaxDegreeOfParallelism = workers,
CancellationToken = cancellationToken,
};
await Parallel.ForEachAsync(jobs, options, async (job, token) =>
{
var path = Path.Combine(folder, $"{job.Number}.pdf");
if (File.Exists(path)) return; // done in an earlier run
try
{
var pdf = await RenderAsync(job, token);
await File.WriteAllBytesAsync(path + ".part", pdf, token);
File.Move(path + ".part", path, overwrite: true);
}
catch (OutOfCreditsException) { throw; } // stops the whole batch
catch (Exception) when (!token.IsCancellationRequested)
{
failed.Add(job.Number); // one bad document does not stop the rest
}
});
return failed.ToArray();
}
}

Three habits are in this code:

  • Write to a temporary name, then move it. A run that is killed halfway never leaves a half file that looks finished.
  • Skip what exists. Start the same batch again and it only does the documents that are missing, so a stopped run costs nothing to resume.
  • Keep the failures. The method returns the numbers that failed, so you can look at them, fix the data and run again.

Retry only what is safe

This method belongs inside BatchRenderer:

private async Task<byte[]> RenderAsync(Job job, CancellationToken cancellationToken)
{
for (var attempt = 1; ; attempt++)
{
using var lease = await limiter.AcquireAsync(1, cancellationToken);
using var response = await http.PostAsJsonAsync(
"v1/render",
new { template_id = job.TemplateId, data = job.Data, locale = job.Locale },
cancellationToken);
if (response.IsSuccessStatusCode)
return await response.Content.ReadAsByteArrayAsync(cancellationToken);
if (response.StatusCode == HttpStatusCode.PaymentRequired)
throw new OutOfCreditsException();
var retry = response.StatusCode is HttpStatusCode.TooManyRequests
or HttpStatusCode.ServiceUnavailable
|| (int)response.StatusCode >= 500;
if (!retry || attempt == 5)
throw new InvalidOperationException($"{job.Number}: {(int)response.StatusCode}");
var wait = response.Headers.RetryAfter?.Delta ?? TimeSpan.FromSeconds(Math.Pow(2, attempt));
await Task.Delay(wait + TimeSpan.FromMilliseconds(Random.Shared.Next(0, 500)), cancellationToken);
}
}
Status What the runner does
200 Returns the PDF.
402 Stops the whole batch: no credits, nothing was rendered.
429, 503, 5xx Waits for Retry-After, or backs off, and tries again, up to five times. A failed render costs no credit.
Other 4xx Does not retry. The request is wrong; the document is recorded as failed.

A little random delay is added to each wait, so workers that were slowed down together do not come back in the same instant.

One case to avoid

If the service finishes a render but your side never receives the answer, for example because of a network timeout, and you send the request again, it is a new render and costs another credit. Keep the HttpClient timeout above the time limit of your plan so a slow document is not given up too early, and read X-Render-Id from each answer into your log, so you can tell the documents that were made from the ones that were not.

Keep each request small

Send a saved template and only the data, as in this guide, and not the whole HTML each time. The size of an input is limited by your plan; an oversized request answers 413. See Render with a saved template.

Read Errors and limits and Rendering a PDF. For a queue-based version of the same idea, see Generate a PDF in Azure Functions.

← All posts