Mini Program Dependency Analysis Practice

[[350074]]

Students who have used webpack must know webpack-bundle-analyzer, which can be used to analyze the dependencies of the current project's js files.

webpack-bundle-analyzer

Because I have been working on mini-programs recently, and mini-programs are particularly sensitive to package size, I wondered if I could make a similar tool to view the dependencies between the main packages and sub-packages of the current mini-program. After a few days of hard work, I finally made it, and the results are as follows:

[[350075]]

Mini Program Dependencies

Today's article will show you how to implement this tool.

Mini Program Entrance

The pages of the mini program are defined by the pages parameter of app.json, which is used to specify which pages the mini program consists of. Each item corresponds to the path (including file name) information of a page. For each page in pages, the mini program will look for the corresponding json, js, wxml, and wxss files for processing.

For example, the development directory is:

 ├── app.js
 ├── app.json
 ├── app.wxss
 ├── pages
 │ │── index  
 │ │ ├── index .wxml
 │ │ ├── index .js
 │ │ ├── index .json
 │ │ └── index .wxss
 │ └── logs
 │ ├── logs.wxml
 │ └── logs.js
 └── utils

You need to write in app.json:

 {
 "pages" : [ "pages/index/index" , "pages/logs/logs" ]
 }

For the convenience of demonstration, we first fork an official demo of the applet, and then create a new file depend.js, in which the work related to dependency analysis is implemented.

 $ git clone [email protected]:wechat-miniprogram/miniprogram-demo.git
 $ cd miniprogram-demo
 $ touch depend.js

The general directory structure is as follows:

Directory Structure

With app.json as the entry point, we can get all the pages under the main package.

 const fs = require( 'fs-extra' )
 const path = require( 'path' ) 
 
 const root = process.cwd() 
 
 class Depend {
 constructor() {
    this.context = path.join (root, 'miniprogram' )
 }
 // Get the absolute address
 getAbsolute(file) {
 return path.join (this.context, file )
 }
 run() {
    const appPath = this.getAbsolute( 'app.json' )
    const appJson = fs.readJsonSync(appPath)
 const { pages } = appJson // All pages of the main package
 }
 }

Each page will correspond to four files: json, js, wxml, wxss:

 const Extends = [ '.js' , '.json' , '.wxml' , '.wxss' ]
 class Depend {
 constructor() {
 //Store the file
 this.files = new Set ()
    this.context = path.join (root, 'miniprogram' )
 }
 // Modify the file suffix
  replaceExt(filePath, ext = '' ) {
    const dirName = path.dirname(filePath)
    const extName = path.extname(filePath)
    const fileName = path.basename(filePath, extName)
 return path.join (dirName, fileName + ext )
 }
 run() {
 //Omit the process of getting pages
 pages.forEach(page => {
 // Get the absolute address
      const absPath = this.getAbsolute(page)
      Extends.forEach(ext => {
 // Each page needs to determine whether js, json, wxml, wxss exists
        const filePath = this.replaceExt(absPath, ext)
        if (fs.existsSync(filePath)) {
          this.files.add (filePath)
 }
 })
 })
 }
 }

Now all files related to the pages in pages are stored in the files field.

Constructing a tree structure

After getting the files, we need to construct a tree-structured file tree based on each file for subsequent display of dependencies.

Suppose we have a pages directory, and there are two pages under the pages directory: detail and index. There are four corresponding files under these two page folders.

 pages
 ├── detail
 │ ├── detail.js
 │ ├── detail.json
 │ ├── detail.wxml
 │ └── detail.wxss
 └── index  
 ├── index.js
 ├── index.json
 ├── index .wxml
 └── index .wxss

Based on the directory structure above, we construct a file tree structure as follows. Size is used to indicate the size of the current file or folder. Children stores the files in the folder. If it is a file, there is no children attribute.

 pages = {
 "size" : 8,
 "children" : {
 "detail" : {
 "size" : 4,
 "children" : {
 "detail.js" : { "size" : 1 },
 "detail.json" : { "size" : 1 },
 "detail.wxml" : { "size" : 1 },
 "detail.wxss" : { "size" : 1 }
 }
 },
 "index" : {
 "size" : 4,
 "children" : {
 "index.js" : { "size" : 1 },
 "index.json" : { "size" : 1 },
 "index.wxml" : { "size" : 1 },
 "index.wxss" : { "size" : 1 }
 }
 }
 }
 }

We first construct a tree field in the constructor to store the file tree data, and then we pass each file into the addToTree method to add the file to the tree.

 class Depend {
 constructor() {
 this.tree = {
 size : 0,
 children: {}
 }
 this.files = new Set ()
    this.context = path.join (root, 'miniprogram' )
 } 
   
 run() {
 //Omit the process of getting pages
 pages.forEach(page => {
      const absPath = this.getAbsolute(page)
      Extends.forEach(ext => {
        const filePath = this.replaceExt(absPath, ext)
        if (fs.existsSync(filePath)) {
 // Call addToTree
          this.addToTree(filePath)
 }
 })
 })
 }
 }

Next, implement the addToTree method:

 class Depend {
 //Omit the previous code 
 
 // Get the relative address
 getRelative(file) {
 return path. relative (this. context, file)
 }
 // Get the file size in KB
 getSize(file) {
    const stats = fs.statSync(file)
 return stats.size / 1024
 } 
 
 // Add the file to the tree
 addToTree(filePath) {
    if (this.files.has(filePath)) {
 // If the file has been added, it will not be added to the file tree
 return  
 }
    const size = this.getSize(filePath)
    const relPath = this.getRelative(filePath)
 // Convert the file path into an array
    // 'pages/index/index.js' =>
    // [ 'pages' , 'index' , 'index.js' ]
    const names = relPath.split(path.sep)
    const lastIdx = names.length - 1 
 
 this.tree.size += size  
    let point = this.tree.children
    names.forEach(( name , idx) => {
 if (idx === lastIdx) {
        point[ name ] = { size }
 return  
 }
 if (!point[ name ]) {
 point[ name ] = {
 size , children: {}
 }
 } else {
        point[ name ]. size += size  
 }
      point = point[ name ].children
 })
 // Add files to the file
    this.files.add (filePath)
 }
 }

We can output the file to tree.json after running it.

 run() {
 // ...
 pages.forEach(page => {
 //...
 })
  fs.writeJSONSync( 'tree.json' , this.tree, { spaces: 2 })
 }

tree.json

Get Dependencies

The above steps seem to be fine, but we are missing an important part, that is, before we construct the file tree, we need to get the dependencies of each file, so that the output is a complete file tree of the mini program. The file dependency relationship needs to be divided into four parts, namely, the way to obtain dependencies for the four types of files: js, json, wxml, and wxss.

Get .js file dependencies

Mini programs support modularization in the CommonJS way. If es6 is enabled, they can also support modularization in ESM. If we want to obtain the dependency of a js file, we must first clarify the three ways of writing js files to import modules. For the following three syntaxes, we can introduce Babel to obtain dependencies.

 import a from   './a.js'  
 export b from   './b.js'  
 const c = require( './c.js' )

Convert the code to AST through @babel/parser, then traverse the AST nodes through @babel/traverse, get the values ​​of the above three import methods, and put them into an array.

 const { parse } = require( '@babel/parser' )
 const { default : traverse } = require( '@babel/traverse' ) 
 
 class Depend {
 // ...
 jsDeps(file) {
 const deps = []
    const dirName = path.dirname(file)
 // Read the js file content
    const content = fs.readFileSync(file, 'utf-8' )
 // Convert code to AST
    const ast = parse(content, {
 sourceType: 'module' ,
      plugins: [ 'exportDefaultFrom' ]
 })
 // Traverse the AST
 traverse(ast, {
      ImportDeclaration: ({ node }) => {
 // Get the import from address
        const { value } = node.source
        const jsFile = this.transformScript(dirName, value)
 if (jsFile) {
 deps.push(jsFile)
 }
 },
      ExportNamedDeclaration: ({ node }) => {
 // Get the export from address
        const { value } = node.source
        const jsFile = this.transformScript(dirName, value)
 if (jsFile) {
 deps.push(jsFile)
 }
 },
      CallExpression: ({ node }) => {
 if (
          (node.callee. name && node.callee. name === 'require' ) &&
          node.arguments.length >= 1
 ) {
 // Get the require address
          const [{ value }] = node.arguments
          const jsFile = this.transformScript(dirName, value)
 if (jsFile) {
 deps.push(jsFile)
 }
 }
 }
 })
 return deps
 }
 }

After obtaining the path of the dependent module, you cannot add the path to the dependency array immediately, because according to the module syntax, the js suffix can be omitted. In addition, when the require path is a folder, the index.js in the folder will be imported by default.

 class Depend {
 // Get the script file of a certain path
 transformScript(url) {
    const ext = path.extname(url)
 // If there is a suffix, it means that the current file is already a file
    if (ext === '.js' && fs.existsSync(url)) {
 return url
 }
 // a/b/c => a/b/c.js
    const jsFile = url + '.js'  
    if (fs.existsSync(jsFile)) {
 return jsFile
 }
    // a/b/c => a/b/c/ index .js
    const jsIndexFile = path.join (url, 'index.js' )
    if (fs.existsSync(jsIndexFile)) {
 return jsIndexFile
 }
 return   null  
 }
 jsDeps(file) {...}
 }

We can create a js and see if the output deps is correct:

 // File path: /Users/shenfq/Code/fork/miniprogram-demo/
 import a from   './a.js'  
 export b from   '../b.js'  
 const c = require( '../../c.js' )

image-20201101134549678

Get .json file dependencies

The json file itself does not support modularization, but the mini program can import custom components through the json file. You only need to declare the reference through usingComponents in the json file of the page. usingComponents is an object, the key is the tag name of the custom component, and the value is the custom component file path:

 {
 "usingComponents" : {
 "component-tag-name" : "path/to/the/custom/component"  
 }
 }

Custom components, like mini-program pages, also correspond to four files, so we need to obtain all dependencies in usingComponents in json, determine whether the four files corresponding to each component exist, and then add them to the dependencies.

 class Depend {
 // ...
 jsonDeps(file) {
 const deps = []
    const dirName = path.dirname(file)
    const { usingComponents } = fs.readJsonSync(file)
    if (usingComponents && typeof usingComponents === 'object' ) {
      Object.values ​​(usingComponents).forEach((component) => {
        component = path.resolve(dirName, component)
 // Each component needs to determine whether the js/json/wxml/wxss file exists
        Extends.forEach((ext) => {
          const file = this.replaceExt(component, ext)
          if (fs.existsSync(file)) {
 deps.push(file)
 }
 })
 })
 }
 return deps
 }
 }

Get .wxml file dependencies

wxml provides two file reference methods: import and include.

 <import src= "a.wxml" />
 <include src= "b.wxml" />

The wxml file is essentially an html file, so the wxml file can be parsed by an html parser. For the principles related to the html parser, you can read my previous article "Vue Template Compilation Principles".

 const htmlparser2 = require( 'htmlparser2' ) 
 
 class Depend {
 // ...
 wxmlDeps(file) {
 const deps = []
    const dirName = path.dirname(file)
    const content = fs.readFileSync(file, 'utf-8' )
    const htmlParser = new htmlparser2.Parser({
      onopentag( name , attribs = {}) {
        if ( name !== 'import' && name !== 'require' ) {
 return  
 }
        const { src } = attribs
 if (src) {
 return  
 }
       const wxmlFile = path.resolve(dirName, src)
        if (fs.existsSync(wxmlFile)) {
 deps.push(wxmlFile)
 }
 }
 })
 htmlParser.write(content)
 htmlParser. end ()
 return deps
 }
 }

Get .wxss file dependencies

Finally, the syntax of importing styles in wxss files is consistent with that of css. You can use the @import statement to import external style sheets.

 @import "common.wxss" ;

You can use postcss to parse the wxss file and then get the address of the imported file, but here we are lazy and do it directly through simple regular matching.

 class Depend {
 // ...
 wxssDeps(file) {
 const deps = []
    const dirName = path.dirname(file)
    const content = fs.readFileSync(file, 'utf-8' )
    const importRegExp = /@import\s*[ '"](.+)[' "];*/g
 let matched
    while ((matched = importRegExp. exec (content)) !== null ) {
 if (!matched[1]) {
 continue  
 }
      const wxssFile = path.resolve(dirName, matched[1])
      if (fs.existsSync(wxmlFile)) {
 deps.push(wxssFile)
 }
 }
 return deps
 }
 }

Get .wxss file dependencies

Finally, the syntax of importing styles in wxss files is consistent with that of css. You can use the @import statement to import external style sheets.

 class Depend {
 addToTree(filePath) {
 // If the file has been added, it will not be added to the file tree
    if (this.files.has(filePath)) {
 return  
 } 
 
    const relPath = this.getRelative(filePath)
    const names = relPath.split(path.sep)
    names.forEach(( name , idx) => {
 // ... add to the tree
 })
    this.files.add (filePath) 
 
 // ===== Get file dependencies and add them to the tree =====
    const deps = this.getDeps(filePath)
 deps.forEach(dep => {
 this.addToTree(dep)
 })
 }
 }

Get subpackage dependencies

Students who are familiar with mini programs must know that mini programs provide a sub-package mechanism. After using sub-packages, the files in the sub-packages will be packaged into a separate package and loaded only when they are used, while other files will be placed in the main package and loaded when the mini program is opened. In subpackages, the configuration of each sub-package has the following items:

So when we run, in addition to getting all the pages under pages, we also need to get all the pages in subpackages. Because we only care about the contents of the main package before, there is only one file tree under this.tree. Now we need to mount multiple file trees under this.tree. We need to create a separate file tree for the main package first, and then create a file tree for each subpackage.

 class Depend {
 constructor() {
 this.tree = {}
 this.files = new Set ()
    this.context = path.join (root, 'miniprogram' )
 }
 createTree(pkg) {
 this.tree[pkg] = {
 size : 0,
 children: {}
 }
 }
 addPage(page, pkg) {
    const absPath = this.getAbsolute(page)
 Extends.forEach(ext => {
      const filePath = this.replaceExt(absPath, ext)
      if (fs.existsSync(filePath)) {
        this.addToTree(filePath, pkg)
 }
 })
 }
 run() {
    const appPath = this.getAbsolute( 'app.json' )
    const appJson = fs.readJsonSync(appPath)
    const { pages, subPackages, subpackages } = appJson 
     
 this.createTree( 'main' ) // Create a file tree for the main package
 pages.forEach(page => {
      this.addPage(page, 'main' )
 })
 // Since subPackages and subpackages in app.json are both effective
 // So we get both properties, and use whichever one exists
    const subPkgs = subPackages || subpackages
 // Only traverse when subpackages exist
    subPkgs && subPkgs.forEach(({ root, pages }) => {
      root = root.split( '/' ) .join (path.sep)
 this.createTree(root) // Create a file tree for subpackage
 pages.forEach(page => {
        this.addPage(`${root}${path.sep}${page}`, pkg)
 })
 })
 // Output file tree
    fs.writeJSONSync( 'tree.json' , this.tree, { spaces: 2 })
 }
 }

The addToTree method also needs to be modified to determine which tree to add the current file to based on the passed in pkg.

 class Depend {
  addToTree(filePath, pkg = 'main' ) {
    if (this.files.has(filePath)) {
 // If the file has been added, it will not be added to the file tree
 return  
 }
    let relPath = this.getRelative(filePath)
    if (pkg !== 'main' && relPath.indexOf(pkg) !== 0) {
 // If the file does not start with a subpackage name, it means that the file is not in a subpackage.
 // The file needs to be added to the file tree of the main package
 pkg = 'main'  
 } 
 
 const tree = this.tree[pkg] // Get the corresponding tree based on pkg
    const size = this.getSize(filePath)
    const names = relPath.split(path.sep)
    const lastIdx = names.length - 1 
 
 tree.size += size  
 let point = tree.children
    names.forEach(( name , idx) => {
 // ... add to the tree
 })
    this.files.add (filePath) 
 
 // ===== Get file dependencies and add them to the tree =====
    const deps = this.getDeps(filePath)
 deps.forEach(dep => {
 this.addToTree(dep)
 })
 }
 }

One thing to note here is that if a file in the package/a sub-package depends on a file that is not in the package/a folder, then the file needs to be placed in the file tree of the main package.

Drawing with EChart

After the above process, we can finally get a json file as follows:

tree.json

Next, we use the drawing ability of ECharts to display this json data in the form of a chart. We can see a Disk Usage case in the example provided by ECharts, which is in line with our expectations.

ECharts

The configuration of ECharts will not be described here. Just follow the demo on the official website. We just need to convert the tree.json data into the format required by ECharts. The complete code is in codesandbod. You can see the effect by going to the online address below.

Online address: https://codesandbox.io/s/cold-dawn-kufc9

Final effect

Summarize

This article is more practical, so a lot of code is posted. In addition, this article provides an idea for obtaining the dependencies of each file, although only such a dependency graph is constructed here using the file tree.

In business development, the mini program IDE needs to be fully compiled every time it is started, and it takes a long time to wait when previewing the development version. Now that we have file dependencies, we can only select the pages currently being developed for packaging, which can greatly improve our development efficiency. If you are interested in this part, you can write another article to introduce how to achieve it.