When you tap your credit card at a coffee shop, a sequence of messages starts flowing between multiple financial institutions. Within seconds, these messages traverse a global network, validating your card, checking your balance, evaluating risk rules, and ultimately deciding whether to approve your purchase of that morning latte.
Card payments involve four key players: the acquirer (who enables merchants to accept card payments), the card network (like Visa or Mastercard, who operate the payment scheme), the processor (who handles the technical aspects of card management), and the issuer (who provides cards to consumers). These parties communicate using ISO 8583 messages - a standardized format that's been the backbone of card payments for decades. However, each card network has its own flavor of ISO 8583, adding custom fields and rules to support network-specific features.
In this article, we'll explore:
- How ISO 8583 messages form the foundation of card payments
- Why and how we normalize messages across different card networks
- The intricacies of authorization processing and rule evaluation
- Architectural patterns for building a highly available authorization platform
Let's dive into how modern card authorization platforms handle millions of transactions while maintaining reliability and speed.
ISO 8583 Messages
In the early days of card payments, there was no standard way for financial institutions to exchange transaction data. ISO 8583 emerged as the universal language of the payments world, establishing a common format that all networks could adopt. While networks have since added their own dialects, the core structure remains the same.
An ISO 8583 message consists of three key components: the Message Type Indicator (MTI) that signals the message's purpose, one or more bitmaps that act as a table of contents, defining which Data Elements (fields) are present, and Data Elements containing the actual transaction details.
The Message Type Indicator encodes multiple aspects of the message: its function (like authorization or clearing), its source (whether it's coming from an acquirer or issuer), and its message class (request, response, or advice). For example, when you tap your card at a coffee shop, it triggers an authorization request message asking "Can this cardholder spend $5 at Coffee Shop X?". The issuer must respond within a fairly strict timeout to ensure a smooth checkout experience. In contrast, advice messages inform recipients about events that have already occurred. Common examples include clearing messages (notifying the issuer that a previously authorized transaction has settled) or notifications that a transaction was approved by the network's stand-in service when the issuer's systems were temporarily unavailable.
Each card network maintains its own version of ISO 8583 with custom fields and extensions. Some formatting and contents even depend on processor-level configuration. For modern processors like Lithic that connect to multiple networks, this creates an interesting challenge: how do we integrate with multiple networks and support their unique capabilities, without requiring significant modifications to our entire stack for each new network integration?
Message Normalization
A naive solution to this challenge would be propagating the raw ISO 8583 format throughout our service architecture. However, this approach quickly becomes unwieldy. Each service in our processing pipeline would need to understand multiple vendor-specific formats, lacking schema validation or versioning capabilities. This becomes particularly challenging when networks update their specifications - which happens quarterly - as every service would need to maintain compatibility with both old and new message formats.
Instead, we’ve taken a more elegant approach: We normalize card network messages at our system's edge, converting various ISO 8583 formats into a single, well-defined internal format we call CardNetworkEvent. This translation happens as soon as messages enter our system and just before they leave, allowing our internal services to process transactions without needing to understand the intricacies of each network's ISO 8583 implementation.
Our internal CardNetworkEvent format uses an explicit, machine-readable schema and includes all the necessary information:
- Common fields that exist across all networks, such as transaction amount, country code, merchant details, and track data
- Network-specific fields that capture unique features of each network (like Visa's Security Level Indicator or Mastercard's Electronic Commerce Indicator)
- Lithic-internal fields that track operational details like message receipt time and ingress switch identification
The machine readable schemas for our CardNetworkEvent messages are tracked in an internal schema registry. The schema registry helps to ensure that schemas are both forwards, and backwards compatible. This enables Lithic to gracefully evolve schemas (and the applications reading and writing the messages) at their own pace.
This approach to schema evolution is particularly useful when handling network specification changes in our service-oriented architecture. When a card network announces updates to their ISO 8583 format, we can implement these changes gradually, starting with the edge services that handle the translation between ISO 8583 and CardNetworkEvent formats. Because we enforce bidirectional schema compatibility, downstream services continue processing messages without interruption - they can be updated independently and at their own pace. This decoupling between network-facing changes and internal processing gives us significant operational flexibility - we can roll out changes safely and incrementally, without requiring coordinated deployments across our entire platform.
Authorization Processing
One of the tasks Lithic performs on behalf of its customers is processing authorization requests when cardholders attempt to transact. When such a request arrives through the card network, Lithic must respond within a strict timeout. Our authorization service, operating on our normalized message format, processes these requests through multiple evaluation layers.
At the foundation, we perform universal validation checks that apply to all transactions and are not configurable. These checks cover dozens of basic security measures, such as validating security codes like CVC2 for card-not-present transactions and performing cryptographic verification of EMV chip data for card-present transactions. These basic validations ensure fundamental transaction security regardless of the card program.
The next layer consists of Lithic's internal risk and compliance rules. These include controls like spend velocity monitoring and enforcing constraints required by our partner banks. For card programs where Lithic acts as the program manager, our Risk & Compliance team may implement additional controls specific to the program requirements. We also evaluate any Authorization Rules set by our customers, enabling them to implement their own transaction controls and business logic. Both these internal and external rules are executed on a platform which we refer to as Lithic's Rules Engine.
Besides evaluating rules, Lithic’s authorization processing service is responsible for checking account balance in Lithic’s ledger, and, if enabled for a customer, forwarding transactions to the customer’s server for real-time decision-making through a feature called Auth Stream Access (ASA).
The results from all these evaluation layers - universal validations, internal controls, customer-defined rules, ASA - are combined to reach a final approve or decline decision, which is then relayed back to the card network.
Designed for availability
Our authorization processing infrastructure is built with high availability and horizontal scalability as core design principles. This focus has enabled us to achieve over 99.99% availability in the past year.
When measuring availability in a card authorization system, it's crucial to understand different types of failure modes:
- Internal system failures, such as a service in the critical path becoming unresponsive or a database being unavailable
- Timeout scenarios where we fail to respond within the network's deadline, causing the network to stand in for authorization decisions
- Complete unavailability due to infrastructure issues like network partitions between Lithic and card networks
By combining our internal logging with network-provided advice messages (which are delivered upon restoration of connectivity, if that’s the issue), we can accurately classify and measure these different failure modes. This granular understanding helps us maintain and improve our availability targets.
The two pillars of our availability strategy are redundancy and resiliency. For redundancy, we eliminate single points of failure throughout our authorization processing infrastructure. For example, we maintain multiple active connections to card networks, distributed across separate geographical locations. Similarly, every service in the critical authorization path is deployed across multiple regions.
Our infrastructure implements an "active-active" strategy, rather than an "active-passive" which is also commonly used in scenarios where geographic redundancy is required. This means that every deployed instance of our authorization processing service and its dependencies actively handle a portion of production traffic. We can re-route traffic at various layers in our stack, and we continuously exercise this capability by shifting a small fraction of our authorization traffic between geographical regions. This continuous testing gives us confidence in our ability to perform full failovers when needed.
On the resiliency front, we make extensive use of automated retry mechanisms, circuit breakers, and fail-over mechanisms. Automated retries prevent transient failures from becoming critical issues, allowing applications to recover gracefully from brief interruptions. Circuit breakers protect our system from cascading failures - when we detect that a dependency is struggling, we temporarily stop routing requests to it and fail over to instances in other regions. For latency-critical applications, we employ request hedging - if a dependency in the local region hasn't responded within a specified timeout, we automatically dispatch a backup request to another region and use whichever response arrives first.
It is the combination of the redundancy and resiliency strategies that allows us to reach high uptime while maintaining a scalable system.
Conclusion
In this post we discussed the intricacies of building a modern card authorization platform - how card networks each adapt the ISO 8583 specification, how Lithic avoids building vendor-specific interfaces by normalizing incoming messages at our system's edge, how we perform different types of authorization checks, and how we keep the whole system reliable. The key is balancing competing demands: respecting network-specific capabilities while providing a clean, modern API; maintaining high availability while processing complex authorization rules; scaling horizontally while ensuring consistency. Through our multi-layered authorization processing and focus on redundancy and resiliency, we've built an authorization platform that serves as a solid foundation for modern card programs. We’ll dive deeper into Lithic’s Rules Engine and its capabilities in one of the next posts.