ken/crashtestdev
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

yo blog

Browse Source
This commit is contained in:
Ken
2019-08-16 20:35:55 -07:00
parent 1d7c9df8e4
commit b4f5e90447
8 changed files with 236 additions and 1991 deletions
Download Patch File Download Diff File Expand all files Collapse all files
content/assets/localize-bg.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.1 MiB

content/posts/localize-react-without-bloating-the-bundle.md
+28 -25
View File
@@ -4,9 +4,11 @@ date: 2019-08-16T22:56:11.257Z
title: Localize React without Bloating the Bundle
heroImage: /assets/localize-bg.png
---
There are so many possibilities when you want to localize your application with your React application. I believe that localization is difficult because it requires excellence in several pieces of a stack at the same time. You have to have a working pipeline that can take string resources that would get translated. Then, you have to have a way to load these strings onto your page or application. Finally, you have to have a way to take these strings and inject them into the components inside your application. As you can see, you can choose just about any tech to help you accomplish these goals. I am documenting a particular set of stack that I believe helps you achieve this in the most performant way with tools that you're already probably using.
## Translation Pipeline
At work, we have a translation-as-a-service API that we rely on to refresh localized strings for us every night. There's a growing team that has built a simple Azure DevOps task we add as a step in one of our pipelines which runs nightly.
Not everyone is as fortunate that has something they can use from their own company. Given that, I'll suggest a pattern here as the first step. Do a search for "localization as a service" and look for a vendor that can help add a step in your CI pipeline of choice. Set up a nightly job to refresh your application's localized strings as JSON like this:
@@ -19,6 +21,7 @@ Not everyone is as fortunate that has something they can use from their own comp
```
## Rendering Localized Strings
React is a large ecosystem. So then the paradox of choice is real when finding supplemental libraries for React. Conventional wisdom is to find the most popular packages from npmjs.org. So given this, I first looked at react-intl to help me inject localized strings into my application. The issue here is that react-intl uses higher order components all over the place. One of the explicit goals (as I heard it from ReactConf 2018) of React hooks is to do away with depth of the component tree caused by the higher order components. Higher order component, or HOC, is a neat idea until the consumer needed to access the ref to the original wrapped component. When all your components that use localized strings are wrapped in HOCs, your application start to look like a sideways mountain. (aside: go look at your component tree in React DevTool to see if you're suffering from HOC-itis)
Enter react-intl-universal. The Alibaba Group created this library to get around the HOC issues of react-intl. On top of this, there are times where strings are needed from outside of the component's render() method. It takes 2 steps to place strings inside your components.
@@ -30,37 +33,36 @@ Let's pretend that ./locales/en-US.json has the same content as the example abov
```js
// locale data
const locales = {
"en-US": require('./locales/en-US.json'),
"zh-CN": require('./locales/zh-CN.json'),
};
"en-US": require("./locales/en-US.json"),
"zh-CN": require("./locales/zh-CN.json"),
}
```
Then, we initialize the react-intl-universal library inside a componentDidMount() call. And we'll use the localized string inside the render() method with the .get() function:
```js
import intl from 'react-intl-universal';
import intl from "react-intl-universal"
class App extends Component {
state = { isLoading: false }
componentDidMount() {
this.loadLocales();
this.loadLocales()
}
async loadLocales() {
await intl.init({
currentLocale: 'en-US',
currentLocale: "en-US",
locales,
});
this.setState({ isLoading: true });
})
this.setState({ isLoading: true })
}
render() {
return (
!this.state.isLoading &&
<div>
{intl.get('HELLO_NAME', {name: 'world'})}
</div>
);
!this.state.isLoading && (
<div>{intl.get("HELLO_NAME", { name: "world" })}</div>
)
)
}
}
```
@@ -68,16 +70,17 @@ class App extends Component {
Note that the init() call returns a Promise. This means that we can use the async / await syntax to write our string load code. Once this is added, we look at the way we retrieve the strings by key. For that, we use the get(). Get takes in two parameters: the key and some object. Sometimes the strings have slots that can be replaced by the object values.
## Loading Localized Strings
This is where it gets interesting. So far, we've assumed that we had the locale data all upfront. This means that all the localized strings would had been loaded inside a bundle or onto the page somehow. Loading all the language strings in one go can only be feasible if the app barely contain any text. If we're using Webpack, we should take advantage of a feature that I recently came to know. We all have seen the dynamic import() syntax:
```js
const SomeModule = import('some-module');
const SomeModule = import("some-module")
```
But, have you seen what Webpack can do with something like this?
```js
const getLocale = (locale) => import(`./locales/${locale}.json`);
const getLocale = locale => import(`./locales/${locale}.json`)
```
Based on the .json files it finds inside ./locales, Webpack is smart enough to generate chunks for dynamic loading! That means your main bundle will not incur the weight of the entire library of localized strings. Putting all these concepts together, I've created a repo to demonstrate concepts from this post:
@@ -87,20 +90,20 @@ https://github.com/kenotron/react-intl-example
I'll go over some of the points from that repo. First, I created a HOC that you place at the ROOT of the application. Don't worry! It is only one HOC for the entire app. It is called LocaleComponent - I'm keeping this strange little name until React.createResource() becomes a thing maybe in the future.
```js
const getLocale = locale => import(`./locale/${locale}.json`);
const getLocale = locale => import(`./locale/${locale}.json`)
class LocaleComponent extends React.Component {
state = { isLoading: true };
async loadLocales() {
const locales = await getLocale('en');
const currentLocale = 'en';
await intl.init({ currentLocale, locales });
this.setState({ isLoading: false });
state = { isLoading: true }
async loadLocales() {
const locales = await getLocale("en")
const currentLocale = "en"
await intl.init({ currentLocale, locales })
this.setState({ isLoading: false })
}
render() {
return !this.state.isLoading ? <>this.props.children</> : null;
return !this.state.isLoading ? <>this.props.children</> : null
}
}
```
content/posts/speeding-up-webpack-typescript-incremental-builds-by-7x.mdx
+146
View File
@@ -0,0 +1,146 @@
---
path: /speeding-up-webpack-typescript-incremental-builds-by-7x/
date: 2019-08-16T22:56:11.257Z
title: Speeding Up Webpack Typescript Incremental Builds by 7x
heroImage: /assets/localize-bg.png
---
What if I told you your Webpack is doing too much work all this time? Webpack 4 brought a lot of goodies for developers but to use it at scale, the Outlook team at Microsoft had to take a hard look at the incremental build numbers to find out. Here’s how we made our incremental builds go from 35s to a consistent 5s.
I guess it goes without saying that you MUST measure in order for you to know you have made progress!
Let’s name some enemies of incremental build speed:
1. stats.toJson()
2. Competing resolution logic between Webpack and its loaders (ts-loader)
3. Garbage Collection
4. Subtle v8 ES6 perfomance issues
5. The Base Line
To begin, there are already a few things beyond setting the mode in webpack.config.js we already apply so we’re not doing too much optimization during incremental builds:
```json
{
"optimization": {
"removeAvailableModules": false,
"removeEmptyChunks": false,
"splitChunks": false
}
}
```
Alright, so let’s establish the baseline by looking at a typical inner loop flame graph:
As you can see, we’re clocking in at around 40s here per incremental build. This is not exactly true because we lose about 5s of it due to profiling. In measuring with our internal telemetry, we noticed that our devs are hitting around 30–35s on avg (and sometimes over a minute at the 75th percentile) incremental builds.
So, as soon as you look at those colors, you would recognize three separate phases of the incremental build process. With this in mind, let’s tackle the first enemy.
## Enemy #1: stats.toJson is VERY heavy in WP4
If you were looking at the the CPU profile flame graph, you would notice that the last phase of the process is dominated by a bunch of stats.toJson calls. Where does it come from? It’s right inside webpack-dev-server’s Server.js:
```js
const clientStats = { errorDetails: false };
...
comp.hooks.done.tap('webpack-dev-server', (stats) => {
this.\_sendStats(this.sockets, stats.toJson(clientStats));
this.\_stats = stats;
});
```
The issue here is that Webpack 4 gave `toJson` a lot more information, but it also regressed the performance tremendously as a result. The fix is in a pull request:
https://github.com/webpack/webpack-dev-server/pull/1362
This is the big one — it brought our incremental speeds from 30s to around 15s.
> Update: the webpack-dev-server maintainers had accepted my patch! So, go ahead and use webpack-dev-server@3.1.2. Personally, I have observed a slight 0.5s regression between 2.x release and the 3.x release, so we’re keeping the 2.x for now until we can move to using webpack-serve.
Since we’re waiting for the authors to merge this for v2, I’ve published a temporary fork for it for the v2 branch:
https://www.npmjs.com/package/webpack-dev-server-speedy
For the fix on v2, you’ll have to use the node API to take advantage of it like this package in your build process:
```js
const Webpack = require("webpack")
const WebpackDevServer = require("webpack-dev-server-speedy")
const webpackConfig = require("./webpack.config")
const compiler = Webpack(webpackConfig)
const devServerOptions = Object.assign({}, webpackConfig.devServer, {
stats: {
colors: true,
},
})
const server = new WebpackDevServer(compiler, devServerOptions)
server.listen(8080, "127.0.0.1", () => {
console.log("Starting server on http://localhost:8080")
})
```
## Enemy #2: Competing resolution logic between Webpack and ts-loader
If I were to ask you to build an incremental compiler based on Typescript, you would likely first reach into the Typescript API for something that it is using for its watch mode. For the longest time, Typescript safe guarded this API from external modules. During this time, ts-loader was born. The author of the loader tracked the progress of another Typescript-centric loader called awesome-typescript-loader and brought back the idea of doing type checking on a separate thread. This transpileOnly flag worked remarkably well (with a rather glaring caveat that const enums are not supported out of the box — here’s a workaround from the ts-jest repo) until the codebase reaches a certain size.
In OWA, we have nearly 9000 modules that we shove across this loader. We have found that the first phase of that incremental build is linearly growing as our repo grows.
Things looked pretty grim until the Typescript team decided to take on this mammoth work of expose the watch API to external modules. Specifically, after this was merged, ts-loader is super charged with the ability to limit the amount of modules to transpile at a time per iteration!
We just add this to our webpack config module.rules:
```json
{
"test": /\.tsx?\$/,
"use": [
{
"loader": "ts-loader",
"options": {
"transpileOnly": true,
"experimentalWatchApi": true
}
}
]
}
```
Don’t forget to the typechecker when appropriate: https://www.npmjs.com/package/fork-ts-checker-webpack-plugin (we have a mode to turn type checker OFF for even faster rebuilds)
The incremental builds now only rebuilds around 30–40 modules rather than 50% of our modules! I also have a way to CAP the growth of the incremental builds in the first phase.
This optimization cuts our 15s to around 8s.
## Enemy #3: Garbage Collection
So Garbage Collection is a great invention. But not in a tight loop. Perhaps there’s a perf bug inside node or v8, but I’ve discovered that a global string.replace(/…/g, ‘…..’) can cause a lot of GC when placed inside a loop. Webpack 4 introduced the path info in the generated dev mode replacing module ids with more useful path info. This is done with, you guessed it, global string replace with regex. It then created a LOT of unnecessary GCs along the way. (as an aside, perhaps I should file a bug against either Webpack, node, or v8…)
Okay, let’s turn that sucker off in webpack.config.js in the output.pathinfo:
```json
{
"output": {
"pathinfo": false
}
}
```
Just ask yourself if you REALLY need that pathinfo or that build speed. For us, we chose speed. This made our 8s builds to around 6s
## Enemy #4: Subtle v8 ES6 perfomance issues
Most everyone would be pleased with that 6s figure, but why should we humans not demand MOAR? Yes, MOAR speed!!!
In chatting with a colleague of mine, John-David Dalton, about his project, esm, he told me about node.js performance issues with ES6 data structures like Map and Set. Having dug into Webpack source code previously and by looking at the remaining profile slowdowns (looking at the “heavy” or “bottom-up”), I noticed that Webpack’s internal algorithm is dominated by calling their SortableSet methods. Since SortableSet extends Set, it would follow that Webpack is actually greatly affected by the speed of the Map/Set implementation of V8. Here’s the bug:
https://github.com/nodejs/node/issues/19769
So, I advise everyone doing heavy Webpack development to switch to the LTS (v10+ or stick with v8.9.4)
Using that version, the incremental build is down to 4.5s
## Why? Inventing on Principle!
Finally, I want to leave you with the best motivation on why we should reduce this incremental build speeds down to almost nothing:
Hey! follow me on twitter @kenneth_chau to get more articles like these :)