From an architectural standpoint, there are a few key things to think about when you want to scale your system across hundreds or thousands of connections. Check out some common roadblocks and our recommendations.
In almost any software architecture, mirroring data—or trying to maintain an in-sync duplicated representation of another system’s data—is an anti-pattern. It’s costly and difficult to maintain a duplicated dataset of another system.
Unfortunately, this anti-pattern is relatively common in healthcare integrations. The reason being that your system might need to consume HL7 feeds from your connection's system due to a lack of alternative options. In these cases, your system must save data if you need to refer to it later, or on an ongoing basis.
If it can be avoided, we recommend not persisting a connection's data. It’s costly from both a technology spend perspective, but also from a maintenance and operational standpoint (especially at scale).
Our recommendation: Consider using data on demand and designing your solution to only access the data you need when you need. This option reduces your maintenance and operating costs while increasing your ability to scale rapidly.
To scale from a few integrations to upwards of 100, it’s important to make onboarding new connections as dynamically and painlessly as possible.
Think of it like this: if you want to integrate with even 100 connections, but it takes one engineering effort-day to provision new infrastructure each time, you're losing 100 engineering effort days to onboarding. That’s a huge opportunity cost when compared to what value could be added to your product itself with 100 days' worth of engineering effort.
Our recommendation: Consider implementing multi-tenancy on shared infrastructure and automated infrastructure partitioning. It’s tempting to want to provision segregated infrastructure for each integration, but this strategy is costly and quickly becomes unmaintainable at scale. Instead, choose a partitioning strategy that guarantees protection against data cross-contamination while also allowing for shared infrastructure and dynamic onboarding.
As your integrations scale up, your message volume naturally increases, especially if you have event-driven workflows. There's also a wide variance in event volume rates across different healthcare organizations.
Our recommendation: Embrace event streaming architectural paradigms, which enables your architecture to scale resources as needed as you scale integrations. But keep in mind these following tips:
- Ensure the layers of your architecture are horizontally and dynamically scalable, as well as tenant agnostic.
- Segregate tenant event streams, while keeping processing code as tenant agnostic as possible. Impact of high volume event streams on other streams must be minimal.
- Control tenant-specific event streams; problems in one tenant’s event stream can’t impact another tenant’s streams.
- Automate load balancing.