Source Maps: quick and clear

The Source Maps mechanism is used to map the source codes of the program to scripts generated on their basis. Despite the fact that the topic is not new and a number of articles have already been written on it (for example, this , this, and this ), some aspects still need clarification. The presented article is an attempt to organize and systematize everything that is known on this topic in a concise and accessible form.



The article Source Maps is considered in relation to client development in the environment of popular browsers (for example, DevTools Google Chrome), although their scope is not tied to any particular language or environment. The main source for Source Maps is, of course, the standard , although it has not yet been adopted (status - proposal), but still widely supported by browsers.



Work on Source Maps began in the late 2000s, with the first version being created for the Firebug Closure Inspector plugin. The second version was released in 2010 and contained changes in terms of reducing the size of the map file. The third version was developed as part of a collaboration between Google and Mozilla and proposed in 2011 (last revised in 2013).



Currently, there is a situation in the client development environment when the source code is almost never integrated directly into the web page, but it goes through various stages of processing: minification, optimization, concatenation, moreover, the source code itself can be written in languages ​​that require translation ... In this case, for debugging purposes, a mechanism is needed that allows you to observe the original, human-readable code in the debugger.



Source Maps requires the following files:

• the actual generated JavaScript file
• the set of source files used to create it
• map file mapping them to each other

Map file

All work of Source Maps is based on a map-file, which may look, for example, like this:

{
    "version":3,
    "file":"index.js",
    "sourceRoot":"",
    "sources":["../src/index.ts"],
    "names":[],
    "mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAC;AAEvC,SAAS,SAAS;IACd,MAAM,OAAO,GAAG,QAAQ,CAAC,aAAa,CAAC,KAAK,CAAC,CAAC;",
    "sourcesContent": []
}

Usually, the name of the map-file is made up of the name of the script to which it belongs, with the addition of the ".map" extension, bundle.js - bundle.js.map. This is a regular json file with the following fields:

• "Version" - Source Maps version;
• "File" - (optional) the name of the generated file to which the current map file belongs;
• "SourceRoot" - (optional) prefix for paths to source files;
• "Sources" - a list of paths to source files (resolved in the same way as src addresses of the script tag, you can use file: //.);
• "Names" - a list of names of variables and functions that have been changed in the generated file;
• “Mappings” - coordinates of the mapping of variables and functions of the source files to the generated file in Base64 VLQ format;
• "SourcesContent" - (optional) in the case of a self-contained map file, a list of lines, each of which contains the source text of the file from sources;

Download Source Maps

In order for the browser to load the map file, one of the following methods can be used:

• The JavaScript file came with an HTTP header: SourceMap: <url> (previously deprecated X-SourceMap: <url> was previously used)
• the generated JavaScript file has a special comment of the form:

//# sourceMappingURL=<url> ( CSS /*# sourceMappingURL=<url> */)

Thus, having loaded the map-file, the browser will pull in the sources from the "sources" field and, using the data in the "mappings" field, will display them on the generated script. In the Sources DevTools tab, you can find both options.



The pseudo-protocol file: // can be used to indicate the path. Also, the contents of the base64 encoded map file may be included in the <url>. In Webpack terminology, similar Source Maps are called inline source maps.

//# sourceMappingURL=data:application/json;charset=utf-8;base64,<source maps Base64 code>

Self-contained map files

The source file code can be included directly in the map file in the "sourcesContent" field, if this field is available, there is no need to download them separately. In this case, the names of the files in "sources" do not reflect their real address and can be completely arbitrary. That is why, in the Sources tab of the DevTools, you can see such strange "protocols": webpack: //, ng: //, etc.

Mappings

The essence of the mapping mechanism is that the coordinates (row / column) of the names of variables and functions in the generated file are mapped to the coordinates in the corresponding source code file. For the display mechanism to work, the following information is required:



(# 1) line number in the generated file;

(# 2) column number in the generated file;

(# 3) index of the source in "sources";

(# 4) source line number;

(# 5) source column number;



All this data is in the "mappings" field, the value of which is a long string with a special structure and values ​​encoded in Base64 VLQ.



The line is separated by semicolons (;) into sections corresponding to the lines in the generated file (# 1).



Each section is separated by commas (,) into segments, each of which can contain 1,4 or 5 values:

• column number in the generated file (# 2);
• source index in "sources" (# 3);
• source line number (# 4);
• source column number (# 5);
• the index of the variable / function name from the "names" list;

The values ​​of line and column numbers are relative, indicate the offset relative to the previous coordinates and only the first from the beginning of the file or section.



Each value is a Base64 VLQ number. VLQ (Variable-length quantity) is a principle of encoding an arbitrarily large number using an arbitrary number of binary blocks of fixed length.



Source Maps uses six-bit blocks that follow in order from the lowest part of the number to the highest. The most significant 6th bit of each block (continuation bit) is reserved, if it is set, then the current one is followed by the next block related to the same number, if it is cleared, the sequence is complete.



Since the value must have a sign in Source Maps, the least significant 1-bit (sign bit) is also reserved for it, but only in the first block of the sequence. As expected, a set sign bit means a negative number.



Thus, if a number can be encoded with a single block, it cannot be modulo 15 (1111 ₂ ), since in the first six-bit block of the sequence two bits are reserved: the continuation bit will always be cleared, the sign bit will be set depending on the sign of the number.



Six-bit VLQ blocks are mapped to Base64, where each six-bit sequence corresponds to a specific ASCII character.







Decode the number mE. Invert the order, the last part is Em. We decode numbers from Base64: E - 000100, m - 100110. In the first, we discard the high continuation bit and two leading zeros - 100. In the second we discard the high continuation and the low sign bits (the sign bit is cleared - the number is positive) - 0011. As a result, we get 100 0011 ₂ , which corresponds to decimal 67.



It is possible in the opposite direction, encode 41. Its binary code is 101001 ₂, we split into two blocks: the high part - 10, the low part (always 4-bit) - 1001. To the high part we add the high-order continuation bit (cleared) and three leading zeros - 000010. To the low part we add the high-order continuation bit (set) and the least significant sign bit (cleared - the number is positive) - 110010. We encode numbers in Base64: 000010 - C, 110010 - y. We invert the order and, as a result, we get yC.



The library of the same name is very useful for working with VLQ .