From an architectural standpoint, there are a few things to think about when scaling your system across hundreds or thousands of connections. Most commonly, we've seen roadblocks come up for:
- mirroring data;
- multi-tenancy and dynamic onboarding; and
- volume management.
Explore our recommendations for handling each one below.
In almost any software architecture, mirroring data—or trying to maintain an in-sync duplicated representation of another system’s data—is an anti-pattern. It’s costly and difficult to maintain a duplicated dataset of another system.
Unfortunately, this anti-pattern is relatively common in healthcare integrations. The reason being that your system might need to consume HL7 feeds from your connection's system due to a lack of alternative options. In these cases, your system must save data if you need to refer to it later, or on an ongoing basis.
If it can be avoided, we recommend not persisting a connection's data. It’s costly from both a technology spend perspective, but also from a maintenance and operational standpoint (especially at scale).
Consider using data on demand and designing your solution to only access the data you need when you need it. This option reduces your maintenance and operating costs while increasing your ability to scale rapidly.
To scale from a few integrations to upwards of 100, it’s important to make onboarding new connections as dynamically and painlessly as possible.
Let's say you want to exchange data with 100 connections. It takes one engineering effort-day to provision new infrastructure for each integration. That means you'd lose 100 engineering effort days to onboard. That’s a huge opportunity cost when compared to what value could be added to your product itself with 100 days' worth of engineering effort.
Consider implementing multi-tenancy on shared infrastructure and automated infrastructure partitioning. It’s tempting to provision segregated infrastructure for each integration, but this strategy is costly and quickly becomes unmaintainable at scale. Instead, choose a partitioning strategy that guarantees protection against data cross-contamination while also allowing for shared infrastructure and dynamic onboarding.
As your integrations scale up, your message volume naturally increases, especially if you have event-driven workflows. There's also a wide variance in event volume rates across different healthcare organizations.
Embrace event streaming architectural paradigms, which enables your architecture to scale resources as you scale integrations. But keep in mind these following tips:
- Make sure the layers of your architecture are horizontally and dynamically scalable, as well as tenant agnostic.
- Segregate tenant event streams, while keeping processing code as tenant agnostic as possible. The impact of high-volume event streams on other streams must be minimal.
- Control tenant-specific event streams; problems in one tenant’s event stream can’t impact another tenant’s streams.
- Automate load balancing.