Meanwhile anyone using an intersection of TypeScript with jest and any of sindresorhus' libraries when he flipped to ESM for his bajillion libraries immediately felt the downside and moved hard away from ESM.
Imagine the mind-boggling hours lost just to get these export/import formats to glue.
However, the intersection of typescript, nodejs, and ES modules is consistently the most frustrating experience I ever have. Trying to figure out which magic incantation of tsconfig/esbuild/tsc/node options will let me just write code and run it is a fools errand. You might figure something out, and then you try to use Jest and then you descend into madness again.
The biggest tip I can give people is to ditch ts-node and just use (the awkwardly named) tsx https://github.com/privatenumber/tsx, which pretty much just "mostly works" for running Typescript during dev for node.
The problem mostly seems to stem for all the stakeholders being pretty dogmatic to whatever their goals are, rather than the pragmatic option of just meeting people where they are. I really wish the Node, Typescript, Deno/Bun, and maybe some bundler people would come together and figure out how to make this easier for people.
> I really wish the Node, Typescript, Deno/Bun, and maybe some bundler people would come together and figure out how to make this easier for people.
Bun has solved this. Bun is straight-up magic; they've implemented tons of hacks and heuristics so everything just works. Bun can even handle ridiculous or otherwise invalid code, like having import and require in the same file.
It's doubly frustrating because a standard for authoring modules across browser and server platforms such as ESM is a good thing. But it's a bit arrogant to expect module authors across TS and JS ecosystems to ship overnight. Beginners may just turn to Deno or Bun simply because hundreds of coding tutorials and snippets no longer work.
Or, when you finally get a TS config that works but then you import @aws-sdk/* or prisma seeds and then you really rip your hair out.
I’d also argue that outsiders looking into all the complexity are ignoring the complexity within their own specialization: https://bower.sh/front-end-complexity
It’s hypocritical thinking.
To some degree, I think the typescript-team itself also has to take some blame here. I understand their point that they do not want to do any rewrites, and to some degree it makes sense, but if the ecosystem as a whole really wants to move forward to a common understanding of how it should work, someone needs to do the heavy lifting for dev-experience, and right now they're best-equipped to actually solve the problem, or at the very least help us a lot in doing so.
Their dogmatic approach makes sense for the scope they set out with when starting with typescript but in my eyes refuses a bit the reality the ecosystem currently finds itself in. And I'm saying this as an absolute ts-fanboy; it's one of the very few things about typescript that I take an issue with.
The biggest tip I would add is to bin jest and start using vitest.
Node.js released initial ESM support [1] in Node.js 12.17 in May 2020, 2 years later (!), TypeScript finally added support for ESM [2].
Here's a straight forward guide on how to use TypeScript with ESM: https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3...
[1]: https://nodejs.org/en/blog/release/v12.17.0
[2]: https://devblogs.microsoft.com/typescript/announcing-typescr...
This point stings for me, personally, since _I_ was the TypeScript language dev _in_ this wg trying to make our concerns noted, because we certainly did participate. However the group largely deadlocked on shipping with ecosystem compatibility measures, and what you see in node today is the "minimal core" the group could "agree" on (or be bullied into by group politic - this was getting shipped weather we liked it or not, as the last holdouts). The group was dissolved shortly after shipping it, making said "minimal core" the whole thing (which was the stated goal of some engineers who have since ascended to node maintainer status and are now the primary module system maintainers), with the concerns about existing ecosystem interoperability brought up left almost completely unaddressed. It's been a massive "I told yo so" moment (since a concern with shipping the "minimal core" was that they would never be addressed), but it's not like that helps anyone.
Like this shipped, because _in theory_, it'd be a non-breaking from a library author perspective to get node's CJS to behave reasonably with ESM (...like it does in `bun`, or any one of the bundler-like environments available like `tsx` or `webpack` or `esbuild`), and _in theory_ they're open to a PR for a fix... I wish anyone who tries good luck in getting such a change merged.
How did they not? This is confusing to me, as I’ve been authoring and emitting ESM in TS for quite a few years now.
- module authors increasingly adopt ESM
- TypeScript experience with Deno is butter smooth
- npm packages work with Deno
- performance characteristics are similar
- DX for beginners just works (compared to thousands of stale articles with `require('pkg')` and `npm install pkg@latest` for nodejs)
They have only last year actually rolled out a release that uses ESM modules by default, when the bulk of the community has adopted TypeScript and moved on to using ESM imports (even if they are importing CJS modules under the hood)
They were really late to async/await, and even more so to promises. The chaotic situation where each dependency bundled its own promise library remained in Node.js years after browsers shipped built-in promises. And it still hasn't trickled down to their standard library, most of which is still callback-based and needs to be "promisified" manually. Even their support for fetch, which is eight years old at this point, is still marked as experimental.
This makes sense but honestly means a lot of QoL fixes are kinda impossible to do. At this point I, personally, would almost want tsc (the compiler component) to stop being almost a good bundler and remove some features. Every project I've worked on that uses tsc directly ends up needing a "come to Jesus" moment where a "real" bundler gets introduced and suddenly a lot of stuff becomes easier. Doubly frustrating for me to feel this because tsc does a lot of stuff and it is hard to imagine how Typescript the language moves forward without Typescipt the compiler. I just really don't want to type `.js` for files that are, pre-build, `.ts`.
[0]: "blame" here meant in the weakest sense of "there are people who could make decisions in another way to make this better". Not so much in assigning moral blame
TS would be a lot better if it were just a type checker and nothing else (which is more or less what you get when you switch to a proper bundler).
Indeed, Node's test support can handle dynamic test creation, so one can do crazy things like https://github.com/andrewaylett/prepackage-checks/blob/main/... -- that dynamically asynchronously executes NPM builds from subdirectories, loading and running per-build expectations.
But that was just a short migration period. It's only been three years and we almost fixed all these issues. This one is probably the last one, this month.
I remember spending a weekend simply inlining all his module code directly into a project to sidestep the type module change. Fun.
For me typescript syntax and jest is a no go.
Typescript is useful to check types, but i write just JavaScript with JSDoc, and check it with eslint in typescript modus.
https://gist.github.com/joepie91/bca2fda868c1e8b2c2caf76af7d...
It's not clear to me how you get top-level await with CJS with significant drawbacks OR breaking changes that would just end up back up in ESM land.
But this problem has been solved by modulepreload[0], also natively supported in the browser, which lets you specify the modules upfront in the HTML, avoiding the need for additional roundtrips. Tooling could help with generating the list of preloads but is not necessary.
[0] https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes...
It makes me wonder why Deno [1] still seems to be such a niche choice. Most of these headaches just go away.
Does anyone here prefer Deno for new projects? And if not, why?
[1] And Bun, although that's much newer.
I wouldn't necessarily suggest that a newbie programmer go right to Deno yet because today you still have to be prepared to deal with some differences and quirks.
But here is why I like Deno:
- Typescript pretty much Just Works (TM) out of the box without having to think about it
- More web/browser APIs right out of the box with less Node-specific weirdness. Anything that is Deno-specific (such as anything that lives on the Deno global) is, in my opinion, better designed than Node's APIs.
- Requiring file extensions in import specifiers makes total sense to me. Some people think this is stupid, but I see no reason not to make them actual file paths and not this sort of pseudo file path that omits the extension.
- I like how Deno can bundle your application into a single executable.
- Testing framework and linter are built right in. Which is great because, let's face it, most of us have been using such things for a very long time now.
- Being able to pull in modules from the web without NPM is fantastic.
- Node.js and NPM compatibility has gotten way better recently.
- Binary data is handled as Uint8Array rather than using a non-standard Buffer like in Node. Makes total sense. Skip the middle-man and just give me the bytes in a widely compatible form.
- Granular security with strict permissions by default is really nice. Sometimes I do want to import a module but then have a guarantee that third-party code won't phone home. It's also flexible enough that you can allow your own code to reach your own domains but then some module you import from elsewhere doesn't send data to some other domain. I'd still choose to not use such modules, but it's good that you can mitigate the risk of malicious code without having to do anything.
And sure, Node has improved a lot since the inception of Deno and will continue to improve. It can probably do some of these things and may end up having more parity with Deno. I don't think that Node is bad. I still use Node, primarily at work, and in cases where a package has maintainers that refuse to support Deno (ex. Playwright).
I would highly advise any startup considering using them to reconsider. You don't want your business to being running into rough edges all day. You don't want to be reinventing things all day.
They're fun, and especially deno is really fun. It feels more like go, where you control everything.
But they're still not nearly as productive as node
That said, most of my Deno projects have been fairly basic, so I'm curious if there's a point at which the early wins give way to hidden pain points.
First, my personal choice is not to use TypeScript. For me it just adds a layer of complexity that adds very little. That I am a sole developer certainly is a factor in that decision. And history. I worked with Java for a long time, with all the safety it provides, only to find that PHP and JavaScript programmers could ship in 1/4 the time with the same level of functionality. I find that passing named parameters: `foo({animal: "duck"})` makes my plain old javascript much more robust, with very little cost.
So strike #1 against Deno is that it puts typescript much more in the foreground.
Strike #2 is that Node works. Not because it is right, but because services make sure they work with node. They don't put the same effort into making things work with Deno.
I needed to write code using Deno that talked to a Digital Ocean managed Postgres database. I spent three days. Then I decided not to use Deno. Was this Deno's fault? No. But if you need to get things done and one in 1000 of those things works in Node but not in Deno, then the choice is straight forward.
I'm not saying these are good reasons, but I did try Deno and wanted it to work.
If you have specific issues with ARM, I'm certainly interested in fixing those. I don't have any timeline for when we'll add Linux ARM as a binary release target but I'm happy to ensure any ARM-only bugs get the proper attention.
deno: Mach-O 64-bit executable arm64
heh. It really is https://github.com/esbuild-kit/esno/blob/master/esno.js
Of course, Deno offers a better experience for scripts or programs.
tsx should be able to handle Playwright as of v4+, and hopefully the test coverage you're referring to.
Before, it was compiling ESM syntax to CJS as an effort to ease the ecosystem's CJS -> ESM migration, and hiccuping whenever it encountered `eval()`. Now it includes smarter checks to determine if a file needs to be compiled at all and skips processing most dependencies.
Hope it works well for you!
One problem with this guide that makes that trickier is that the tsconfig.json itself imports https://github.com/sindresorhus/tsconfig/blob/5c87dc118e057d... and overrides a bunch of values.
So if you really want to understand what it's doing, you need to get rid of that dependency and inline / merge that config file. That would also add stability to your project as it's one less upstream dependency out of your control (and a big one at that as it controls how you entire project is built!).
For example the imported tsconfig.json changes these:
"compilerOptions": {
// ...
"module": "node16",
"moduleResolution": "node16",
"moduleDetection": "force",
// ...
"allowSyntheticDefaultImports": true, // To provide backwards compatibility, Node.js allows you to import most CommonJS packages with a default import. This flag tells TypeScript that it's okay to use import on CommonJS modules.
"jsx": "react",
// ...
}
One of my major gripes with the JS/TS ecosystem is that "explanations" are sorely lacking. See https://www.typescriptlang.org/tsconfig for the relevant documentation for tsconfig files. Tutorials are on the page, how-to guides abound on the wider internet (like the OP), and the linked TSConfig Reference and JSON Schema (used in code completion in IDEs) are together absolutely massive.
But an explanation is missing! There is no official documentation about how different options interact to say: as I'm walking a file tree as the Typescript compiler, this is how I will interpret a certain file I encounter, what will be outputted, and how that will be interpreted by bundlers and browsers, especially in an ESM world.
https://medium.com/extra-credit-by-guild/tsconfig-json-demys... is in the right direction, but outdated as ESM has become much more popular in the past 3 years, and still organized by option (so it's already somewhat in the "reference" world).
IMO even independent of documentation, the industry's move to ESM is problematic: https://gist.github.com/joepie91/bca2fda868c1e8b2c2caf76af7d... describes many of the issues. But they're certainly exacerbated by good explanation-style documentation that helps people understand how ESM works under the hood!
Working with TS has been somewhat frustrating.
(don't remember if there's a decent video out there but I figure you probably don't need to listen to me waffle to see how the analogies might work ;)
> But an explanation is missing! There is no official documentation about how different options interact to say: as I'm walking a file tree as the Typescript compiler, this is how I will interpret a certain file I encounter, what will be outputted, and how that will be interpreted by bundlers and browsers, especially in an ESM world.
Perhaps you missed it, but Andrew (from the TS team) recently finished a massive overhaul of our module docs: https://www.typescriptlang.org/docs/handbook/modules/introdu...
The "theory" page describes TypeScript's perspective on modules. The "reference" page documents things from the "as I'm walking a file tree" perspective (among many other details). The "guides" page also provides recommendations for certain kinds of projects.
Since TS 4.5 and their support for `.mts`, I finally settled on just chaining `tsc --build && node ./build/main.mjs`.
When I actually need to build and test, it's very fast (<2 seconds). I use Yarn workspaces/subprojects and TypeScript references with incremental builds so it's not really an issue.
Here is the `package.json` [0] for a serialization lib I did, you can check the `scripts` section: it's very minimal and I'm quite happy with it. I'm an old Gulp maintainer, so for a long time I had heavy Gulp config with a lot of processing. Over the years I could get rid of it. ESM was the last thing holding me back; and now I'm so happy that I can just use a few simple commands.
[0]: https://github.com/demurgos/kryo/blob/master/packages/kryo/p...
This eventually turns into an incantation and magic. It's how crap gets built up and poor decisions get propagated.
Some of them have internal dependencies that just _no longer exist_, or aren't compatible with Node past 14.
Meanwhile, all I had to do to update old Python Lambdas was just update the runtime itself.
As of November 2023, what is the canonical way to set up a Node project with Typescript and hot reload?
Minimal setup with least amount of configs and tooling. I am not after any other tools like Bun.
Do you need all that other stuff for a simple node app?
What a frustrating experience it has been.
Using unbuild "works", but
- For the life of me, I can't get unbuild to generate `.d.mts` files when my source files don't have `.mts` extensions. Luckily, when the library is used in a downstream project and a `.mjs` file is imported, TypeScript properly loads the `.d.ts` file anyways
- When it comes to a downstream project, TypeScript doesn't seem to work with export maps. People say it does, but maybe because I don't have `.d.mts` files TypeScript is saying it can't find type information? It _does_ work with simple export maps, but if I say `'./': './dist/esm/'` so the downstream project doesn't have to manually import from `dist/esm` I see the issue
- Using the "esm" module / moduleResolutions + converting my files to `.mts` + changing imports in the source to import the non-existant `.mjs` files results in a CJS bundle that tries to require a `.mjs` file, which unbuild has built with a `.js` extension, so it throws an error because it can't find the file.
- For some reason unbuild mucks with the hashbang line in my _CJS_ `bin` script, the ESM one it doesn't touch
Ugh.
Then do a check with attw cli tool and it’ll report your compat mistakes with links to explainer docs.
Like I said, these are libraries whose "build" before was just `tsc`, there's nothing funny going on.
I had to tsc then node --test the built files.
tsx also is missing the test runner https://github.com/privatenumber/tsx/issues/257
node --import=tsx --test __tests__/\*/\*.test.tsSo true, I really wandering if there was a better alternative, they essentially broke a lot of packages and I don't know how many dev hours will be put now to fix all of this, because so many tools are broken now. Additionally to that, they did some other not backwards compatible changes, like removing __direname in ESM, which is IMHO not the best decisions, why not for example just keep it as it is, but deprecate it and have a warning message.
Deno?
I completely disagree. This has always felt immediately straight forward to me. I suspect this struggle, a struggle I completely don’t understand, explains the complexity of hiring for these kinds of jobs. I really felt that to get hired for these jobs you had to be willing to play stupid games and abandon all reason to worship at the pulpit of giant frameworks and third party solutions. Fortunately, I have moved on to something else.
In my tsconfig.json I use
{
"compilerOptions": {
"alwaysStrict": true,
"module": "ES2020",
"moduleResolution": "node",
"outDir": "./js/lib",
"noEmit": true,
"noImplicitAny": true,
"pretty": false,
"strictFunctionTypes": true,
"target": "ES2020",
"types": ["node"],
"typeRoots": ["./node_modules/@types"]
},
"exclude": [
"js",
"lib/terminal/test/storageTest/temp",
"**/node_modules",
"**/.*/"
],
"include": [
"**/*.ts"
]
}
For an ecosystem that purports to be ready for professional use it absolutely boggles my mind that the tooling is stuck at this very amateurish level.
Why isn’t there a ready preset for me to use ESM syntax with Next.js / Typescript?
What’s wrong with the culture around Node/TS/JS tooling in general that it’s so consistently broken all the time? What the hell.
Qwik feels so right to use, if you have 5 minutes you should read about it: https://qwik.builder.io/docs/concepts/think-qwik/
I'm maintaining a few pristine example projects to keep track of these ways and test them periodically. This iteration probably had the shortest lifetime. I didn't manage to develop even a single toy project before something got obsoleted again. Love this community. Looking forward to solve a handful of brand new issues with tsx.
I wish there was some IDEish starter pack which you could install and start writing code anytime, without investigating issues with running a damn interpreter.
All we can do is adapt and keep building. Using Node v20 (LTS) currently with the config linked in the parent post.
- ts-node is much slower than something like esbuild/tsx right? As long as you rely on your ide type checking or run type checking before deploying your app.
Also, this project can be compiled with `tsc` or a bundler, of course.
In terms of speed, this should consistently start up an app or CLI in <3 seconds (depending on size of course).
Part of the problem is the fact that web browsers, Node.js, and other JS runtimes have slightly different needs and expectations. Part of the problem arises when you're trying to mix and match web code, Node.js code, and other JS runtime code in the same monorepo. There's no single magic-bullet configuration.
This attitude just doesn't work for js ecosystem. I have some shell scripts, vim config files, etc. that I don't remember what they do, but I just copy over and they work.
Coding projects are different; they break all the time. Especially with how fast the js/ts ecosystem works. One day, a crucial library just decides that it's changed something that doesn't work with the build process because there's so many variations of them that they can't possibly all be tested against.
You might ask why without bundling?
Sometimes you just want to start something simple on the browser and compile to JavaScript on the fly.
I tried the dev server from Modern Web [0], and I liked it. I program in TypeScript and the browser reloads whenever I save a file. Of course I could set up a bundler and for a small program waiting times are negligible. But I hate bundlers. I know it's irrational, but nowadays I program for fun so I think I should have the choice to reject bundlers.
This fails for many Node dependencies. There is a conflict between CommonJS and ESM. I am not 100% sure that what I want to achieve is impossible without forking dependencies and making a small change.
I even found a way to have a CommonJs and ESM polyglot, but this hack is extremely ugly. I named the hack modglot [1]. I don't think this is a good idea and I don't understand enough to propose something. I am somewhat dejected about the current state of TypeScript development for the browser and paused development.
Now I am programming in Rust again just for fun, but if I return to TypeScript, probably I will try out Deno.
[0]: https://modern-web.dev/guides/dev-server/getting-started/
If you Google the error you will get stackoverflow posts with hundreds of upvotes.
And that's the main problem with using ESM - 3rd party library support. It is trivial to do hello world in ESM with Node.js, but try that with 10 dependencies and quickly you will be hit errors.
Fellow scrounging raccoons, raise your chipped mugs and cracked cups! Let us toast to our fortune, we truly are the blessed ones.
- `importsNotUsedAsValues` is deprecated [0] since TypeScript 5.2, in favor of `verbatimModuleSyntax` [1].
- I could set `module` to `Node16`. This automatically set `esModuleInterop` to true.
- Also, to catch more issues, set `allowUnreachableCode` to false and set `strict`, `noImplicitReturns`, `noImplicitOverride`, `noFallthroughCasesInSwitch`, `exactOptionalPropertyTypes` to true.
- Set `types` to the empty array `[]` to avoid loading unwanted types.
- Enable `skipLibCheck` to avoid checking imported module types.
- Not sure that `declarationMap` is still useful nowadays. TypeScript is now able to match directly against source files.
- Enable `composite` that in turns enables `incremental` and `declaration` (declaration file emit). `composite` enables project references which is useful in a monorepo setting or to separate source and test files into two projects. See [2]
- Enable `checkJs` to type-check JavaScript files
To summarize:
{
"$schema": "https://json.schemastore.org/tsconfig",
"compilerOptions": {
"lib": ["ES2022"],
"module": "Node16",
"target": "ES2022",
"outDir": "./dist",
"composite": true,
"sourceMap": true,
"types": [],
"isolatedModules": true,
"resolveJsonModule": true,
"skipLibCheck": true,
"verbatimModuleSyntax": true,
"allowUnreachableCode": false,
"checkJs": true,
"exactOptionalPropertyTypes": true,
"noFallthroughCasesInSwitch": true,
"noImplicitOverride": true,
"noImplicitReturns": true,
"strict": true
}
"include": ["./src/**/*.ts"],
"exclude": ["./src/specific-file.ts"]
}
[1] https://www.typescriptlang.org/tsconfig#verbatimModuleSyntax
[2] https://www.typescriptlang.org/docs/handbook/project-referen...
I will say that it's kind of sad that Node.js has devolved into this.
https://github.com/bhouston/template-typescript-monorepo
This has a lot of features that work well together: - TypeScript - ESM - Node.JS Server + React + Cli - Monorepository - Fast Incremental Updates
Feedback welcome. I use this for about a dozen different projects now.
I'm currently using this setup to run multiple services on AWS.
Ideally things are great — do ur frontend dev in js and backend in js. With TS added for better dx. Except 90% of the time ur fighting the configs of why its not importing/requiring
U change from .js, .ts, .cjs, .mjs and nothing works
I’m not very familiar with the ecosystem and trying to understand what pain ports people are describing here in the comments.
Until this github repo dries up the same way similar attempts at this have.
For type checking on development, you can do it with ESLint and JSDoc in Typescript modus. You have the same type checking like you have in ts files. You can even import types from typescript files, like .d.ts
Best of both worlds, no transformation of the code, and on development you have some help from Typescript.
ts-node --esm --swc
Haven't really had any issues with this in a long time.