英文:
Is it possible to form a multi-level monorepo (Q about tooling & structure)?
问题
我正在研究将我的代码库转换为单一代码库并在 2023 年初进一步划分代码库的研究阶段。
- 我使用 TypeScript
- 我有多个服务器,我想将它们划分为微服务以进行进一步的开发
- 我有一些 React 前端,我将对它们进行重构
我想要的最终结构如下:
/project-root
/desktop # 电子应用程序
/documentation
/examples
/lib # 非常常见的库
/mobile # ionic 应用程序
/server # node/express
/lib # 服务器代码的库
/validation
/...
/gateway
/api # 公共 API
/ms-xxx # 微服务(Docker 容器)
/ms-yyy
/...
/web # react 应用程序
/lib # 前端代码的库
/uilib
/...
/webapp-1
/webapp-2
/...
/website # 产品网站的代码,可能是 CMS
我研究了最先进的工具,这些标准似乎是可行的:
- 包管理器:pnpm 与工作区
- 构建工具:Vite
- 单一代码库/打包:我研究了 Nx 和 Turborepo/Turbopack,但我觉得它们有一些限制,但我对这些不是专家,所以我可能是错误的。
我希望有多层次的 tsconfig
和 package.json
文件(现在正在研究 Vite),位于根目录下(lint/Prettier 等),紧挨着 server 和 web,当然也在每个应用程序下。我希望服务器和 web 应用程序使用相同的依赖项(例如相同的 node/express/react 版本)。
- 我尝试过 Nx,但 @nrwl 的工具版本落后,导致依赖问题
- Turbo 脚手架是面向 Next.js 的,而 Nx 创建的脚手架需要大量的修改。但除非你使用它们,否则你需要硬编码你的包
- pnpm 存在与对等依赖项有关的问题,我目前无法解决。
除了一些简单的示例外,我没有看到使用这些工具的大型代码库,因此我的问题是:这是否可能,或者我弄错了?
英文:
I'm in the research phase for converting my repos into a monorepo and further divide the codebase as of early 2023.
- I use TypeScript
- I have several servers and I want to divide them into microservices for further development
- I had several React frontends, I'll refactor them to another set
The final structure I want is something like this:
/project-root
/desktop # electron apps
/documentation
/examples
/lib # very common libs
/mobile # ionic apps
/server # node/express
/lib # libraries for server code
/validation
/...
/gateway
/api # public api
/ms-xxx # microservices (Docker containers)
/ms-yyy
/...
/web # react apps
/lib # libraries for frontend code
/uilib
/...
/webapp-1
/webapp-2
/...
/website # code for product website, possibly a CMS
I examined state-of-the-art tooling and these criteria seem to be plausible:
- Package manager: pnpm with workspaces
- Build tool: Vite
- Monorepo/packaging: I examined Nx and Turborepo/Turbopack but I find them restrictive, but I'm no expert on these so I might be wrong.
I would like to have multi-level tsconfig
and package.json
files (researching about Vite now), at the root (lint/Prettier etc), right under server and web, and of course under each app. I want servers and webapps use the same dependencies (e.g. same node/express/react etc versions).
- I tried Nx, but the @nrwl tooling versions were coming behind, resulting in dependency problems
- Turbo boilerplate is Next oriented, while Nx creates boilerplate that requires a lot of hacking. But unless you use them you need to hardcode your packages
- pnpm has issues with peer dependencies which I couldn't resolve now.
Except some simple examples, I don't see such larger code bases using these tools, hence the question:
Is this possible or am I getting this wrong?
答案1
得分: 1
扩展 tsconfig 文件
您可以在您的配置文件中使用 extends
来实现多层 tsconfig(例如,创建一个基本的 tsconfig 文件,所有项目都可以扩展并自定义,参见示例文档)。
避免重复依赖项
虽然没有办法使一个 package.json
继承另一个,就像你可以在 tsconfig
文件中那样,但如果你在不同项目的 package.json
文件中指定相同的版本号,包管理器如 npm 将确保每个依赖项只在顶级 node_modules
文件夹中安装一次(参见文档中的npm install --install-strategy 标志)。
TypeScript 项目的增量构建
您还可以查看TypeScript 项目引用,它可以方便地指定项目之间的依赖关系并实现增量构建。
英文:
Extending tsconfig files
> I would like to have multi-level tsconfig...
You can achieve this with extends
in you config files (e.g. having a base tsconfig file that all your projects extend and customize, see docs for an example).
Avoiding duplicate dependencies
> I want servers and webapps use the same dependencies (e.g. same node/express/react etc versions).
While there's no way for one package.json
to inherit from another like you can with tsconfig
files, if you specify the same version numbers in all the package.json
files for your different projects, package managers like npm will ensure that each dependency is installed once in a top-level node_modules
folder (see the npm install --install-strategy flag in the docs).
Incremental builds for TypeScript projects
You might also be interested in TypeScript Project References which make it easy to specify the dependencies between the projects and get incremental builds.
答案2
得分: 1
创建一个新的monorepo时,我使用了Lerna和Yarn(classic)Workspace,速度可能不是最快的,但是能正常工作。如今,我通常选择pnpm,并使用workspaces:
协议,与Wesley之前的回答类似,但有一个小区别,我建议使用Lerna-Lite代替Lerna。 Lerna和Lerna-Lite的主要优点在于它们都很容易设置,并提供了出色的version
和publish
命令,还可以选择使用Conventional Commits。因此,通过很少的pnpm工作区和Lerna/Lerna-Lite的设置,您可以轻松发布,并自动更新整个项目的changelogs。
何时以及为什么选择Lerna-Lite:
- 没有什么可以与Lerna/Lerna-Lite的
version
和publish
以及可选的Conventional Commits相媲美。一些项目,比如pnpm,使用Changesets,但似乎需要更多的设置,而且我不太喜欢它们的changelogs(我更喜欢常规的changelogs)。 - Lerna-Lite比Lerna更模块化。Lerna是一个一体化工具,包含15个内置命令和大量的开发依赖项,而Lerna-Lite只有7个命令,几乎都是可选的(只安装您使用的部分)。
- Lerna-Lite不需要Nx(新的Lerna >=6.x在后台安装了Nx,即使您不使用它,他们选择安装Nx的唯一原因是为了提高
lerna run
的速度,从而在Lerna >=6.x中成为了必需,这并不令人意外,因为Nrwl接管了Lerna的管理)。 - Lerna-Lite还可以使用pnpm的
workspace:
和pnpm的工作区上的run
,项目一开始使用npm工作区,但pnpm更好,所以我切换到了pnpm(我是维护Lerna-Lite的人)。使用pnpm工作区也非常适合确保库与pnpm和yarn的workspace:
协议正常工作。 - Jest也切换到了Lerna-Lite,他们使用Yarn Berry的
workspace:
协议,只关注version
和publish
命令,所以他们发现Lerna-Lite是最合适的选择。 - 您可以轻松地添加TurboRepo(或其他工具)或简单地使用
pnpm run
(或只是npm run --workspaces
)。 - Lerna-Lite还可以自动更新对等依赖项(尽管这是一个可选项,参见--allow-peer-dependencies-update),而Lerna还不提供此选项。
何时使用原始的Lerna:
- 如果您已经使用Nx,那么最好使用Lerna而不是Lerna-Lite,因为Lerna在内部利用并使用Nx,而Lerna-Lite不使用。由于它由Nrwl维护,您可以确保他们的产品Nx和Lerna能够很好地协同工作。
- 使用Lerna,您还可以安装其他工具,比如TurboRepo,但然后您可能会不知不觉地安装TurboRepo和Nx(也许并不知道),那么...为什么要这样做,以及为什么要这样做呢?
- Lerna是最初的工具/库,拥有数千名用户和每月数百万次的下载量,是Lerna-Lite的数百倍。
总结
所以总之,对于monorepo,我通常会选择pnpm workspaces和Lerna-Lite,这两个工具的设置过程都非常简单,使用这两个工具时不会安装大量的开发依赖项。通常,我也会选择使用pnpm run
,即使Lerna-Lite也可以选择使用lerna run
,如果您想提高run
命令的速度,那么可以考虑使用TurboRepo、Nx或其他类似的工具。最后的选择可能是直接使用pnpm来发布版本,这是许多流行项目如Vite、Vue等项目采用的方法,如果选择这条路线,可能会需要更多的设置和复杂性,与简单的pnpm + Lerna-Lite方法相比,但即使在这种情况下,也可以使用一些外部依赖项。
英文:
When creating a new monorepo, I started with Lerna and Yarn (classic) Workspace and it wasn't the fastest but it was working... Nowadays, I always go with pnpm and the use of workspaces:
protocol and similarly to previous answer by Wesley with a small difference in which I would suggest to use Lerna-Lite instead of Lerna. The main advantages with Lerna & Lerna-Lite is that they are both still easy to setup and offer great version
and publish
commands with optional use of Conventional Commits, so with very few setup of pnpm workspace and Lerna/Lerna-Lite you get easy publishing and automatically updated changelogs across the project.
When and why would you choose Lerna-Lite:
- nothing really compares to Lerna/Lerna-Lite
version
andpublish
with optional Conventional Commits. Some projects, like pnpm, use Changesets but it seems to require a bit more setup and I don't really like their changelogs (I prefer conventional changelogs). - Lerna-Lite is much more modular than Lerna. Lerna is an all-in-one tools with 15 built-in commands with lots of dev dependencies, on the other hand Lerna-Lite has only 7 commands and they are nearly all optional (install only what you use).
- Lerna-Lite doesn't require Nx (the new Lerna >=6.x installs Nx behind the scene even if you don't use it, the only reason they chose to install Nx was to improve
lerna run
speed and with that Nx became a requirement in Lerna >=6.x, it's not surprising since Nrwl took over stewardship of Lerna) - Lerna-Lite also works and actually uses pnpm
workspace:
and pnpmrun
on the workspace, the project started with npm workspaces but pnpm is way better so I switched it to pnpm (I'm the person who maintain Lerna-Lite). Using pnpm workspace is also great for making sure the lib works with pnpm & yarnworkspace:
protocol. - Jest also switched to Lerna-Lite, they are using Yarn Berry
workspace:
protocol and are only interested inversion
andpublish
commands, so they found that Lerna-Lite was their best fit. - you could easily add TurboRepo (or other tools) or simply use
pnpm run
(or justnpm run --workspaces
) - Lerna-Lite can also auto update peer dependencies (though it's an opt-in option, see --allow-peer-dependencies-update), Lerna doesn't yet offer this option.
When would you use the original Lerna
- if you are already using Nx then it's probably better to use Lerna instead of Lerna-Lite since Lerna takes advantage, and uses, Nx internally while Lerna-Lite does not. Since it's maintained by Nrwl, you can be sure that their product Nx and Lerna are working well together.
- with Lerna, you could also install other tools like TurboRepo with Lerna but then you would end up installing (perhaps not knowingly) TurboRepo and Nx behind the scene, so... why would you do, and want, that to happen?
- Lerna is the original tool/lib, they have thousands of users and millions of downloads per month which is hundred times the size of Lerna-Lite
Summary
So in summary, for a monorepo I most often go with pnpm workspaces and Lerna-Lite, the setup process of both tools is quite simple and you don't end up installing a ton of dev dependencies when using both tools. The use of pnpm run
is also what I usually go for, even if Lerna-Lite does offer lerna run
optionally, and if you ever want to improve the speed of your run
commands, then you can start looking at TurboRepo, Nx or other similar tools.
A last option might be to use pnpm directly to also publish versions, which is the approach used by many popular projects like Vite, Vue, ... these projects created their own tooling and when going with that route, you would probably end up with a bit more setup and complexity compare to a simpler pnpm + Lerna-Lite approach, but even then the idea is that it's also doable with a few external dependencies as well.
答案3
得分: 0
你可能会发现Lerna对于这种结构很有用。
尽管我更喜欢保持包分开且更自包含。使用Yarn V1 Workspaces
来实现单一仓库感觉比它值得的麻烦多,特别是考虑到可能发生的嵌套包冲突(例如 eslint、jest等)。
当看你的结构时,我建议尝试将事情分解并分别处理每个包。对于共享库,尽量将它们通用化,以便更多的包可以将它们安装为依赖项。
英文:
You may find Lerna useful for this structure.
Although I do prefer keeping packages separate and more self-contained. Working with Yarn V1 Workspaces
to achieve a monorepo felt like more hassle than it was worth, particularly considering nested packaged conflicts that can occur (like with eslint, jest etc).
When looking at your structure, I would try to break things down and work with each package separately. For shared libraries, make them as generalized as possible so more packages can install them as dependencies.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论