Jason Miller | Scaling Vite at Shopify | ViteConf 2024

Scaling Vite at Shopify

Introduction to the Challenge

• Jason Miller introduces himself and discusses his role at Shopify, focusing on the challenges faced with a large React codebase bundled with Webpack.

• The codebase consists of approximately 5 million lines of TypeScript and 800 routes, which made local development slow and cumbersome.

Migration to Vite

• Last year, the team migrated from Webpack to Vite, transforming their application into a Remix single-page app.

• The migration resulted in faster local development compared to previous cloud-based IDEs and enabled production builds in CI with smoke testing.

Custom Plugins and Transformations

• The migration involved hard work, codemods, and custom Vite plugins while eliminating custom AST transformations to avoid Babel parse costs.

• Examples of custom plugins include:

• CSS modules for all imports by default.

• Bespoke translation loading.

• GraphQL TypeScript type generation.

• Module graph preloading as a new feature in Vite.

Benefits Realized Post-Migration

• The CSS modules plugin was successfully deployed in production; translation loading facilitated hot module replacement for translations.

• A GraphQL plugin optimized JavaScript output size by eliminating unnecessary TypeScript transpiled enums, reducing bundle sizes by up to 80%.

Initial Surprises After Switching

• Upon switching from Webpack to Vite, the homepage initially sent 18,000 modules to the browser, leading to long load times (30–60 seconds).

• Despite this challenge, one-second hot reload times were achieved.

Issues with Barrel Files

• Barrel files—modules that re-export other modules—were extensively used but caused significant issues during bundling.

Dead Code Elimination Challenges

• Dead code elimination struggles arose when using react.memo, preventing certain unused exports from being removed due to unknown dependencies.

Clarity Reduction Due to Barrel Files

Understanding Module Re-exports and Their Implications

The Nature of Re-exports

• The value of a re-exported module is determined by the last re-export that exports it, not the first. This means all dependencies must be loaded and executed to resolve the final value.

Accidental Import Cycles

• Import cycles can occur when a module references itself indirectly through barrel files, leading to undefined values at runtime due to incomplete execution.

Infrastructure Strain in Vite

• Increased file loading leads to higher memory usage and longer serving times in Vite, as more modules need to be read from disk.

Issues with Test Code in Production

• Test helpers inadvertently included in production code due to improper re-exporting practices can lead to bloated builds, highlighting flaws in dead code elimination strategies.

Challenges with Barrel Files

Definition and Misuse of Barrel Files

• "Bin files," or misused barrel files, may start as aggregators but evolve into standard JavaScript modules, complicating imports and lazy loading.

Linting Limitations

• Linters struggle with whole program analysis for import statements; custom lint rules can significantly slow down CI processes.

Optimizing Module Imports

Debarrel Plugin Introduction

• A debarrel plugin was developed for Vite that performs on-the-fly tree-shaking by processing import statements into their final locations before serving them.

Impact on Performance

• Implementing the debarrel plugin reduced served modules by over 50%, significantly improving initial page load times—this was noted as the largest optimization achieved within a year.

Refactoring Application Structure

Cold Page Load Improvements

• Initial cold page load times were reduced from 15 seconds by restructuring how critical data fetching occurs without unnecessary upfront JavaScript loading.

Separation of Concerns

How App Context and Data Loading Work in React Router 6

Overview of App Context

• The app context is an object outside the React tree, initialized before React and UI toolkit components load. It serves as a centralized location for global concerns.

• This context operates independently while data is being fetched from the network, ensuring that critical tasks are prioritized during loading.

Data Loading Mechanism

• Data loading occurs immediately after minimal JavaScript is shipped, functioning like a bootloader to fetch necessary data in parallel with heavier UI code.

• The module graph structure includes dynamic imports for rendering and routing chunks, which can lead to complications with bundling.

Challenges with Bundling

• A bundler issue arises when common dependencies are pushed into the entry chunk due to its guaranteed loading order, potentially leading to bloated bundles.

• Import cycles occur when changes in route loaders affect multiple chunks, causing cascading hash invalidations across the entire module graph.

Solutions Implemented

• To resolve import cycles, it was determined that the entry chunk should not be imported by any other chunk. Manual chunks were created for broadly shared modules.

• This approach prevents cascading hash invalidation; if a route loader changes, only affected chunks receive new hashes without impacting others.

Engineering Culture Impact

• A Vite plugin was developed to enforce bundle constraints and prevent nonsensical dependencies from appearing in critical chunks.

Hot Module Replacement: A Year in Comparison

Transition from Webpack to Vite

• The speaker discusses the significant improvements in development speed when using Vite for hot module replacement (HMR) compared to their previous Webpack setup, noting it feels "extremely weird" at first but is ultimately "10 times faster."

• A live demo illustrates the performance difference: after saving changes, Vite's HMR responds in just five milliseconds, while Webpack takes approximately 12 seconds for the same action. This stark contrast highlights the efficiency of Vite.