Enable javascript in your browser for better experience. Need to know to enable it? Go here.

Surviving integration with third-party services (part 1)

Fear of the unknown

 

Repetitive negative feedback about third-party integration has made many teams hesitate to even see it as an option. Wariness is natural, but sometimes third-party integration is inevitable, and the process might turn out to be a lot easier than it originally seemed.

 

 

Unavoidable integration

 

Most modern systems are made with services from other systems because programmers can’t reinvent the wheel every time. Inevitably, almost every system will need to be integrated with another service, for example a payment gateway. Another common reason for integration is that large systems need to be developed by multiple different departments and teams; this means integration with services made by other teams is usually unavoidable.

 

 

How to ensure smooth integration

 

So, how can we make the integration work less painful? Let’s look at the two types of third-party systems that might be integrated with: mature live services and services under development.

 

Integration with live services

 

Integrating with services that are already live is relatively easy since their functionality is complete and relatively stable. Typical examples of live third-party systems are auth systems and payment systems.

 

Design Phase: Understand the responsibilities, architecture and infra of the third party

 

As a consumer of a service or set of services, we need to clearly know how the responsibilities of each third party to be integrated are defined. This will help us determine what functionality we need to cover during the design stage.

 

This can be identified from two perspectives: behavior and data.

 

  • Behaviorally, what functionality does the third-party system provide? (For example, an auth system provides registration, login and authority management.)

     

  • In terms of data, it’s necessary to identify the key data that is hosted by third parties. That data should be treated as the single source of truth in the connections between those systems. For example, when an auth system is used as a third party, the user name, password, profile information, and authority management data obtained during registration should be controlled by the auth system. When the consumer system needs to read the user’s profile information, it should get it from the auth system, (whether in real-time or through batch or cache) instead of storing another version. If that doesn’t happen, data inconsistency will cause many problems.

     

Usually, it’s almost impossible to get comprehensive details of a third-party service quickly. That can be challenging if you only have a short time to design the integration solution. To overcome this, trying to understand the architecture of the third party can be helpful. For a mature live system, there’s usually architecture documentation that includes information about  the internal structure that can give us a relatively clear view of it. If we consider an auth system, for example, it’s divided into three services, with each service having its own responsibility: these are issue token, service to service trust connection, and authentication status management. These detailed functionalities allow us to understand the whole scope of the system.

 

Implementation Phase: Prepare the test environment for integration in advance

 

The Agile Manifesto asserts the importance of “working software over comprehensive documentation.” A working testing environment provided by the third party is essential if we are to properly verify and debug its functionality. 

 

Ideally, each testing environment in our system requires a corresponding environment in the third-party system — dev, QA, UAT and prod. In reality, third-party systems can only provide one testing environment. Our testing environments have to be shared, which leads to data mess. For example, if the third party is a login system, the account created in the high environment will be affected by the account configuration of the lower environment. This could cause accidents at critical moments like PO signoff or showcase.

 

What if the third party has no way to provide at least one test environment? In this instance, a fake third party will be necessary. Some good examples of this include Mockpass, which corresponds to Singpass, or API services (moco) generated by the contract file.

 

What if faking the third party isn’t feasible? What we can do is clarify the way in which the third-party environment might behave and what the possible consequences might be for the team. For example, perhaps it should be treated as normal if the team receives multiple notification emails when the email related functionality is triggered once, if notification services for different testing environments are listening to one message queue.

 

The impact of third-party maintenance

 

Even after going live, the third party service might still continually change (hopefully with version control), so possible downtime needs to be planned early. It will be important to ensure important dates like showcase or go-live don’t conflict with those downtime periods. We can also suggest third parties use a no-downtime deployment solution anyway.

In the next part, we’ll explain how to handle third-party integration when dealing with services still under development.

Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.

Keep up to date with our latest insights