Clever Geek Handbook

How Chrome DevTools moved from bike to standard



A quick note on how the Chrome DevTools migrated from the internal module loader to standard JavaScript modules. We will tell you how and why the migration was delayed, about the hidden costs of migration and about the conclusions of the DevTools team after the migration was completed. But let's start with the history of web developer tools.



Introduction



As you probably know, Chrome DevTools is an HTML, CSS and JavaScript web application. Over the years DevTools has become feature rich, smart, and knowledgeable about the modern web platform. Although DevTools has expanded, its architecture is largely reminiscent of the original when the tool was part of WebKit .



We will tell the story of DevTools, describe the benefits and limitations of the solutions, and what we have done to mitigate those limitations. So let's dive deeper into modular systems, how to load code, and how we ultimately used JavaScript modules.



In the beginning there was nothing



The frontend now has many modular systems and their tools, as well as a standardized JavaScript module format . None of this was when DevTools started. The tool is built on top of WebKit code written over 12 years ago.



The first mention of the modular system in DevTools dates back to 2012: it was the introduction of a list of modules with a corresponding list of sources . Part of the Python infrastructure used at the time to compile and build DevTools. In 2013, the modules were checked out into a file by frontend_modules.json this commit , and then, in 2014, into separate ones module.json( here ). Example module.json: 



{
  "dependencies": [
    "common"
  ],
  "scripts": [
    "StylePane.js",
    "ElementsPanel.js"
  ]
}


Since 2014, it has been module.jsonused in developer tools to point to modules and source files. In the meantime, the web ecosystem grew rapidly and many module formats were created: UMD, CommonJS, and eventually standardized JavaScript modules. However DevTools is stuck on module.json. The non-standardized, unique modular system had several disadvantages:



  1. module.json required its own build tools.
  2. There was no IDE integration. Of course, she required special tools to create files that she understood: ( jsconfig.jsonfor VS Code ).
  3. Functions, classes and objects have been placed in the global scope to allow sharing between modules.
  4. The order in which the files were listed was important. There was no guarantee that the code you rely on was uploaded other than human verification.


In general, evaluating the current state of the DevTools modular system and other more widely used module formats, we came to the conclusion that it module.jsoncreated more problems than it solved.



Advantages of the standard



We've chosen JavaScript modules. When this decision was made, modules in the language were still flagged in Node.js and a large number of NPM packages did not support them. Regardless, we concluded that JavaScript modules were the best option.



The main advantage of modules is that they are a language-standardized format . When we listed the consmodule.json, we realized that almost all of them were associated with the use of a non-standardized, unique module format. Choosing a non-standardized module format means that we have to invest time building integrations using build tools and our colleagues' tools. These integrations were often fragile and lacking support for features, requiring additional maintenance time and sometimes tricky bugs. The bugs ended up hitting users.



Since JavaScript modules were standard, this meant that IDEs like VS Code, type checking tools like the Closure / TypeScript compiler, and build tools like Rollup and minifiers would be able to understand the written source code. What's more, when a new person joins the DevTools team, they don't have to waste time learning the proprietary module.json.



Of course, when DevTools first started out, none of the above benefits existed. It took years of work in standards groups to implement the runtime. It took time for feedback from developers - users of the modules. But when modules appeared in the language, we had a choice: either to continue supporting our own format, or to invest in the transition to a new format.



How much does the shine of novelty cost?



Even though JavaScript modules had many benefits that we wanted to use, we stayed in the world module.json. Taking advantage of the modules of the language meant that we had to invest considerable effort in technical debt. In the meantime, the migration could break functions and introduce regression bugs.



It was not about whether we are using JavaScript modules. The question was how expensive is the ability to use JavaScript modules . We had to balance the risk of annoying users with regressions, the time it took for engineers to migrate, and a period of deterioration in the state of the system in which we would be operating.



The last point turned out to be very important. Even though we could theoretically get to JavaScript modules, during the migration we would end up with code that would take into account both types of modules . Not only is this technically challenging, but it also means that all engineers working on DevTools need to know how to work in such an environment. They would have to constantly ask themselves, "What's going on in this code, is it module.jsonJS, and how can I make the change?"

The latent cost of migration in terms of training colleagues was higher than we expected.
After analyzing the costs, we came to the conclusion that it is still worth switching to modules in the language. Therefore, our main goals were:



  1. Make sure the standard modules are as useful as possible.
  2. Make sure that the integration with existing modules on the base is module.jsonsafe and does not lead to negative impacts on the user (regression errors, user frustration).
  3. Provide DevTools migration guides. Primarily through checks and balances built into the process to prevent accidental mistakes.


Spreadsheet, conversions and technical debt



The goal was clear. But the limitations were module.jsonhard to get around. It took several iterations, prototypes, and architectural changes before we came up with a usable solution. We ended up writing a project document with a migration strategy. This document gave an initial estimate of the time: 2-4 weeks.

The most intensive part of the migration took 4 months, and 7 months passed from start to finish!
The original plan, however, has stood the test of time: we wanted to teach the DevTools runtime to load all files in the old way to use those listed in the array scripts module.json, while all files listed in the array moduleshad to be loaded by dynamic language import . Any file that will be in the array modulescan work with importand exportfrom ES6.



In addition, we wanted to migrate in 2 phases. Ultimately, we split the last phase into 2 sub-phases: export and import. Modules and phases were tracked in a large spreadsheet:





A snippet of the migration table here.



Export phase



The first step was to add export statements for all entities that need to be shared between modules / files. The transformation was automated by running a script for each folder . Let's say module.jsonthere is such an entity: 



Module.File1.exported = function() {
  console.log('exported');
  Module.File1.localFunctionInFile();
};
Module.File1.localFunctionInFile = function() {
  console.log('Local');
};


Here Moduleis the name of the module. File1- file name. In the code tree, it looks like this: front_end/module/file1.JS.



The code above translates to this: 



export function exported() {
  console.log('exported');
  Module.File1.localFunctionInFile();
}
export function localFunctionInFile() {
  console.log('Local');
}

/** Legacy export object */
Module.File1 = {
  exported,
  localFunctionInFile,
};


We originally planned to rewrite the import into one file at this stage. For example, in the example above, we would rewrite Module.File1.localFunctionInFileto localFunctionInFile. However, we realized that it would be easier to automate and safer to separate the two transformations. Thus, "transferring all entities into one file" will become the second import sub-phase.



Since adding a keyword exportturns the file from a "script" into a "module", much of the DevTools infrastructure had to be updated accordingly. The framework included a dynamic import runtime as well as tools such as ESLint for running in module mode.



One nuisance was that our tests were run in a "non-strict" mode. JavaScript modules imply that the files run in strict mode. This affected the tests. As it turned out, a non-trivial number of tests relied on a non-strict mode, including a test in which the operator was present with.



In the end, updating the very first folder (adding export) took about a week and several attempts at reloading .



Import phase



After all the entities were exported using the export statements, while remaining in the global scope due to legacy, we had to update all entity references, if they are in multiple files, to use ES imports. The ultimate goal is to remove all expired exports by clearing the global scope. The transformation was automated by running a script for each folder .



For example, the following entities module.json: 



Module.File1.exported();
AnotherModule.AnotherFile.alsoExported();
SameModule.AnotherFile.moduleScoped();


Converted to: 



import * as Module from '../module/Module.js';
import * as AnotherModule from '../another_module/AnotherModule.js';

import {moduleScoped} from './AnotherFile.js';

Module.File1.exported();
AnotherModule.AnotherFile.alsoExported();
moduleScoped();


However, this approach has caveats:



  1. Not every entity was named by principle Module.File.symbolName. Some entities have been named Modele.Fileor even Module.CompletelyDifferentName. The mismatch meant that we had to create an internal mapping from the old global object to the new imported object.
  2. moduleScoped. , Events, , Events. , , , import Events .
  3. , . , . , , . , , ( DevTools). , , .


JavaScript



In February 2020, 6 months after the start in September 2019, the last cleanups were performed in the ui / folder. So the migration ended unofficially. When the dust settled, we officially marked the migration complete on March 5, 2020 .



Now DevTools only work with JavaScript modules. We are still putting some entities in the global scope (in legacy files module.js) for legacy tests or integrations with other parts of the architect's tools. They will be removed over time, but we do not consider them as blocking development. We also have a style guide for JavaScript modules .



Statistics



Conservative estimates of the number of CL (changelist - a term used in Gerrit, similar to the GitHub pull request) involved in this migration is around 250 CL, mostly done by 2 engineers . We don't have final statistics on the size of the changes made, but a conservative estimate of the changed rows (the sum of the absolute difference between inserts and deletions for each CL) is roughly 30,000 rows, which is about 20% of all DevTools front-end code .



The first file to be exported is supported in Chrome 79, which was released in the stable release in December 2019. The last change to switch to import comes in Chrome 83, which was released in the stable release in May 2020.



We are aware of one regression due to migration in stable Chrome. Autocompletion of code snippets in command bar broke due to extraneous default export . There have been several other regressions, but our automated test cases and Chrome Canary users have reported them. We fixed bugs before they could make it into stable Chrome releases.



You can see the whole story by registering here . Not all, but most CL are tied to this error.



What have we learned?



  1. . , JavaScript ( ) , DevTools . , , , .
  2. — , . , , . , , , .
  3. (, ) . -. , Python Rollup.
  4. (~20% ), . , , . , .
  5. , . . , , , . , , — . , , .


image


, Level Up , - SkillFactory:





E






More articles:


All Articles