Managing reference/lookup data in a microservices architecture
The problem - reference data needed in many services
Reference data (e.g. a list of countries, order statuses, product categories, currency exchange rates) is needed in many microservices at once - OrderService needs order statuses, PaymentService needs exchange rates, ShippingService needs the list of countries. The question: where should this data "live" and how does each service get access to it?
Approach #1 - a dedicated reference data service + synchronous calls
public interface IReferenceDataClient
{
Task<List<CountryDto>> GetCountriesAsync();
Task<string> GetCurrencyRateAsync(string code);
}
public class ReferenceDataClient : IReferenceDataClient
{
private readonly HttpClient _httpClient;
public async Task<List<CountryDto>> GetCountriesAsync()
{
return await _httpClient.GetFromJsonAsync<List<CountryDto>>("api/reference/countries");
}
}
The advantage: a single source of truth - the data is always up to date, with no risk of inconsistency between services.
The disadvantage: every call requires a synchronous request to another service - if ReferenceDataService is slow/unavailable, all the services that depend on it are affected (the classic problem of cascading failure in microservices).
Approach #2 - local replication (caching) of reference data in each service
This is the much more common approach in practice - each service keeps its own local copy of the reference data, synchronised asynchronously:
public class OrderService
{
private readonly IMemoryCache _cache;
private readonly IReferenceDataClient _referenceDataClient;
public async Task<List<CountryDto>> GetCountriesAsync()
{
return await _cache.GetOrCreateAsync("countries", async entry =>
{
entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(1); // cache for an hour
return await _referenceDataClient.GetCountriesAsync();
});
}
}
The advantage: no synchronous dependency on another service on every request - reading from the local cache is instantaneous.
The disadvantage: the data may be briefly stale (eventual consistency) - if the "Pending" status gets renamed to "Awaiting", services with a cache will show the old name until the cache expires.
Approach #3 - propagation via events (Event-Driven Reference Data)
This approach combines the previous two, using exactly the same event-publishing mechanism we discussed with the Saga and Outbox:
// ReferenceDataService publishes an event on EVERY change to the reference data
public async Task UpdateCountryAsync(int id, string newName)
{
var country = await _context.Countries.FindAsync(id);
country.Name = newName;
await _context.SaveChangesAsync();
await _eventBus.PublishAsync(new CountryUpdatedEvent(id, newName)); // + the Outbox Pattern for atomicity!
}
// EVERY interested service listens and UPDATES its LOCAL copy
public class CountryUpdatedHandler
{
public async Task Handle(CountryUpdatedEvent evt)
{
var localCountry = await _context.LocalCountryReplicas.FindAsync(evt.CountryId);
localCountry.Name = evt.NewName;
await _context.SaveChangesAsync(); // LOKALNA replika, w bazie TEGO serwisu
}
}
The advantage: no synchronous dependency between services (reads always come from the local database), and at the same time the data is propagated relatively quickly after a change (not instantly, but in seconds, not hours as with a plain cache TTL), without waiting for the cache to expire.
The disadvantage: more infrastructure (a message broker, event handling in every service), and every service duplicates the reference data in its own database.
Which approach when
| Approach | When to use |
|---|---|
| Synchronous calls (no cache) | Data queried rarely, a requirement for absolute freshness, tolerance for a dependency between services |
| A local cache with a TTL | Data changes rarely, momentary staleness is acceptable (minutes/hours) |
| Propagation via events + a local replica | Data needed very often (every request), independence from another service's availability required, but also relative freshness |
An important rule - reference data is "owned" by one service
Regardless of the synchronisation approach, one specific service should be the owner (source of truth) of a given set of reference data - other services read (and possibly cache/replicate) it, but do not modify this data directly in their own database. This is analogous to the "one Bounded Context = one data owner" rule from Domain-Driven Design.
Summary
In a microservices architecture, reference data requires a conscious decision about the trade-off between data freshness and service independence. Synchronous calls give always-current data, but create a temporal coupling between services. A local cache with a TTL eliminates that dependency, at the cost of momentary staleness. Propagation via events (using the same Outbox Pattern we discussed) gives the best compromise for frequently queried data - a local replica with no synchronous dependency, with relatively fast propagation of changes.