How to manage versions and upgrades of APIs with multirepo approach

user_7804876 0
Viewed 298

I am working in a team that develops different APIs (Hadoop echosystem, scala programming language, gradle for dependency management and delivery to maven repo).
Some APIs have code dependency between each other, since we are working in different repositories, the APIs are depending on each other using the versions in maven.

This approach leads to some serious version synchronization problems:

  1. API A is used by APIs B and C (both using A v1.0); API D is using B,C so indirectly depending on A (from both B,C).
    enter image description here
    If we upgrade to A v2.0, we can find ourselves updating B to use A v2.0 and then updating D to use the newer version of B, which causes dependency error because now D has two different versions of A- A v2.0 from B and A v1.0 from C.
    enter image description here
    This happens because we have no way to synchronize between our API versions and don't have a proper systematically way to manually update all the projects that use A directly or indirectly, or even be aware of them (hard task when you have a lot of API projects and dependencies).

  2. By developing on A, we can't check whether it integrates well with B,C,D without manually checking, because A should not be familiar with B,C,D.
    Sometimes we find out for example that C is not well compatible with the new version of A, so now we have to release a hotfix for A.

  3. We have no one true source of control over versions between our APIs, It would help if something could automatically say which versions works with each other in an organized manner, like:

    'A': 2.0
    'B': 1.4
    'C': 1.2
    'D': 3.0

I know this problem is yelling "use MONOREPO!" (with direct code dependencies), but we would like to manage older versions of some APIs and control when to update them, if we used monorepo in that manner, upgrading A will automatically update B,C,D since now they importing A directly, without any proper independent version management.
Using monorepo with different branches for different versions is also an option, but it will be very hard to manage different versions of different APIs in different branch names since there's no single product with a version we can align to.
We could pack all the different versions in one general API version like example 3, but it's still hard to manage different API versions in a single repo (actually harder to logically pack them together while working in different repos, which is part of my question about how to do it well).

I understand that I have to accept some pros and cons that are given by choosing monorepo/multirepo, so assumine we're staying with multirepo and not complaining about manually upgrade versions for each project and dealing with incompatibility issues, only asking about better approaches to manage different versions across multiple repositories in order to ease some pains which are automatically given by choosing multirepo.

To sum it all, what do you think is the best approach (technically and culturally for a team work) to manage versions across multirepo APIs depending on each other, developed by a single team?

dependency-management version dependencies version-control api
1 Answers

I can think of three ways to limit the chaos.

  1. Try hard to make your APIs backwards compatible, and plan for future extension. In some ways it is easier to add public facing new methods (while keeping the old), and slowly deprecate the old, than it is to do a hard switch. When you want to change sth. about your API, it might be better to mark some functionality as deprecated while keeping it working.

  2. Do slower release cycles the further up the dependency tree you are. API-A should never have a faster release cycle than the downstream repos B,C,D. Developers of API-A should also be aware of the downstream impact they are causing and clearly communicate future changes.

  3. Use identical versioning conventions like Major.Minor.Patch etc. Require the individual projects to state the versions they depend on in a machine-readable format.

user_73219540
Related