Road to native ES modules

[mrtnzlml]

There are some dependencies that started pushing native ES modules in a very invasive way, see:

• strip-ansi 6.0.0 -> 7.0.0 (Bump strip-ansi from 6.0.0 to 7.0.0 #2249)
• hast-util-to-html 7.1.3 -> 8.0.0 (Bump hast-util-to-html from 7.1.3 to 8.0.0 #2323)
• refractor 3.3.1 -> 4.0.0 (Bump refractor from 3.3.1 to 4.0.0 #2366)
• chalk 4.1.2 -> 5.0.0 (Bump chalk from 4.1.2 to 5.0.0 #3484)
• remove all dependencies restrictions from .github/dependabot.yml

We need to replace or upgrade these. However, we are not ready for it. So here is the plan to make it happen:

• cleanup our legacy quirky ES modules system Remove support for non-standard module field in package.json #2339
• remove useESModules from @babel/runtime and maybe others, see: https://babeljs.io/blog/2021/02/22/7.13.0#automatic-es-modules-in-babelruntime-12632httpsgithubcombabelbabelpull12632
• implement conditional packages exports: https://nodejs.org/api/packages.html#packages_conditional_exports and https://nodejs.org/api/packages.html#packages_dual_commonjs_es_module_packages (including support in NPM publisher)
• read through https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c

Goals:

• be able to use ESM packages within Universe
• be able to export package to NPM with both CJS and ESM exports (in a non-invasive way if possible)

Eventually, we can drop support for CJS altogether.

Resources:

• https://nodejs.org/api/packages.html
• https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c
• https://babeljs.io/blog/2021/02/22/7.13.0#automatic-es-modules-in-babelruntime-12632httpsgithubcombabelbabelpull12632
• Babel 8 might be pure ESM as well (https://babeljs.io/docs/en/roadmap#babel-8)

Known problems:

• Jest support is currently experimental (https://jestjs.io/docs/ecmascript-modules)
• Next.js support is currently experimental (ERR_ESM_REQUIRE - Webpack 5 config should support ESM packages vercel/next.js#23725, add support for esm externals vercel/next.js#27069)

[michalsanger]

So what is our goal? I see two different ones:

1. To be able to use ESM packages within Universe
2. Export libs from Universe to NPM as to support both ESM and CJS

[mrtnzlml]

@michalsanger I see both of the points to be goals of this issue. I updated the issue description to be more clear about the goals. Thanks!