Thinking about multi-project

In a project involving multiple smaller projects that (are):

• dependent with each other (common, tools, etc.)

• developed in parallel (meaning frequent change needs to be reflected to dependent projects)

https://blog.bitsrc.io/11-tools-to-build-a-monorepo-in-2021-7ce904821cc2

Automatically links projects, manage large number of small projects from a central point. Looks like the most sophisticated solution.

https://rushjs.io/pages/intro/why_mono/

Above link listed reasons for one big repo.

Tunnel vison - people ignore what happens to the other part of the whole picture... I'm not sure this is really a problem, isn't that what code isolation wants to achieve? clearly defined interface.....

Cascading publishing - complicated dependency triggers cascaded building, and multi-repo slows down the speed (people needs to update, rebuild, publish, in right order), and when the chain becomes long, the speed is really slow. Ok, this sounds valid to me. and then

Downstream victims: when a downstream victim finds problem week after... makes sense

Linking madness: npm link can be messy.....

With one big repo:

• fast and easy to do cross-repo changes

• update becomes more frequent

• also more frequent merging

'npm install' will symbol link workspace folders to node_modules, allowing absolute reference

• Can be used together with git submodule / git subtree

• So that the sub-modules and subtree do not need to be put under source folder and referenced in relative path

Manages submodule as independent module, maintains a link to a specific commit (SHA).

• Requires separate steps to manage sub module

• Linking - sub module checked out in source tree (with everything else, package.json tsconfig.json etc.)

Subtree files are stored in main repository, changes merged in parent repository, or changes in parent repository can be merged back to sub repository

• Requires separate steps to manage

• Linking - checkout in source tree (also with everything else including package.json, tsconfig.json etc.)

Developer Experience

Mono (better experience when developing multiple related projects in parallel):

• View everything, easy to check into dependency code

• Able to contribute to dependency code

• Able to change multiple module in one atomic commit (refactor)

• Able to apply common process (lint) over all modules

Multi-repo:

• Focus on own code

• Able to control access / life cycle / branching strategy / hooks of individual modules

Dependency Linking and Upgrades (mono wins)

Mono:

• Google "force teams to share code at source level instead of linking in previously built binaries" (source: https://trunkbaseddevelopment.com/monorepos/)

• "have no version numbers for their own dependencies, just an implicit ‘HEAD’" - everyone uses the latest dependency. Indeed, because of lock step upgrades for all, releases from their monorepos mean ‘HEAD’ is their effective version number

• Since no upgrade is "deferred", problems are easier to spot (who breaks it who fixs it)

• Third-party libraries (like JUnit) will be checked into the repo with a specific version number (like 4.11), and all teams will use that version if they use it at all.

Multi:

• each user upgrade their dependencies at their own paces

Branching Strategy (multi can use a more flexible strategy)

Multi-repo:

Since one repo one project, repo's branching / tags can be useful for managing the one project

• https://nvie.com/posts/a-successful-git-branching-model/

• GitHub Flow

• GitHub Workflow

• Combine with CI / CD (release cycle)

• Branches for:

  • develop - main develop branch

  • features

  • fixes

  • release

  • main (lineal releases)

Monorepo:

Challenges to repo. Repo history, repo branching, repo tags are less useful. Developer is also affected by changes in unrelated parts (has to pull, merge, rebase).

• branching:

  • if the monorepo is still for a single macro project, branching & CI/CD can still be used https://medium.com/monsoon-engineering/monorepo-ci-cd-using-codebuild-and-git-branching-workflow-2f1025850139 but additional tools is needed to check which is needed to be built, which is not (however rush can do this....)

  • 
    

• "trunk based" - everything commits to the trunk

• 
  

Versioning, change log

Multi-repo (individual repo):

• 'standard-version' etc tools support automatic versioning and change log through commit messages

Mono:

• non-public packages: no need for change log and versioning - change is immediately integrated (rush)

• rush: provide own commands to manage changes & versioning, as traditional tools has wrong assumption that whole repo is for one project

Publishing

Multi-repo (individual):

• manual publish or in pipeline

Mono:

• (rush) provides own command to publish changed packages

Deploying

Multi-repo (individual):

• manual or in pipeline

Mono:

• (rush) provides command to prepare for deployment

Releasing

Multi-repo:

• Release can be triggered by pushing to certain branch

• Change file can be compiled from commit messages

• Versioning can be done from commit messages

• Tags (git tag) used to tag a version

• tool ('standard-version') can automatically update version number from commits, update version in package.json, shrinkwrap files, and create git tags, create change log...

Mono:

• Release - (Google) "Release branches are cut from a specific revision of the repository. Bug fixes and enhancements that must be added to a release are typically developed on mainline, then cherry-picked into the release branch""a release is typically a snapshot of head, with an optional small number of cherry-picks pulled in from head as needed" https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext

• Not possible / not compatible:

  • Auto-versioning from commit logs: commits are not just for the project but for the repo with many projects inside

  •