Progressive Web Application, theme build & configuration

The post will share with you the difference between the production and development builds of PWA. If you are not familiar to PWA (Progressive Web Application) please visit that blog post.

1. PWA Webpack
   • Why we need Webpack
   • Source & destination folders
2. Webpack: Production And Development Set-Ups
   2.1. Code Minification And Splitting
   2.2. The Manifest
   2.3. Service Worker Pre-cache
   2.4. The Source Map
   2.5. The Hot Reload & the Development Server
   2.6. The HTML entry file name
   2.7. Debugging & Building
   2.8. The build speed
3. The config folder
4. Webpack commands

1. PWA Webpack

What is Webpack? Webpack is a tool that allows us to bundle a theme or and entire application. The created prioritized bundles of modules and files then will be run as application starts. Why do we need to create a bundle out of our application? Because images, styles and some other assets of an application are not included in it by default. Besides, there is number of features are not supported by all browsers by default.

Why do we need Webpack?

In order to understand why we need Webpack let’s see the challenges when building a PWA
Challenge 1. As we develop in our application and have some code imports:

import 'abc' 
import 'react'

Yet node_modules folder is not shared, so we can’t access the exact react file from there. Without it we can’t open that file in browser.

Challenge 2. Web browser does not understand scss, eg. the following:

import 'style.scss'

Neither browser knows what 'react' or the other modules are, so we need to include them into the target script as well.

Challenge 3. Arrow functions ( ) => {…}, classes and class proposal properties (eg., prop = 1;) are not included by default, nor supported by most browsers. One needs to transform these attributes into formats that older browsers go along with. The code written in ECMAScript6 has to be transformed into one of ES2015.

Suppose we have a CSS flexbox, using the shorthand flex notation:

flex: 100;

How should it be translated into the final PWA that is to be executed by most of browsers? Webpack has a postcss-loader, which will do this for us using the autoprefixer plug-in. So, the processed code will look like the following:

-webkit-flex: 101;  # Chrome and Safari
-moz-flex: 101;     # Mozilla
-o-flex: 101;       # Opera
-ms-flex: 101;      # Internet Explorer

To summarize:

• Webpack replaces the imports and handles the imports incl. dependencies – (addresses challenge 1).
• CSS loaders* (sass-loader) are to change SCSS into CSS – (addresses challenge 2).
• Job of Babel (babel-loader) is to transform JS from one ESMA script/ES to another – (addresses challenge 3).

*Loader is a part of Webpack to load a specific type of asset.

Webpack is a code transformation tool that handles all these transformings, loadings… The init codes are taken from multiple sources and the results are put into the destination folder.

Besides all the transformations Webpack performs code optimizations.

Webpack: source and destination folders

A PWA default folder structure looks like the following:

📂pwa
 ┣ 📂Magento_Theme
 ┣ 📂src
 ┃ ┣ 📂app			 
 ┃ ┣ 📂config
 ┗ ┗ 📂public  	

So, what happens when one runs webpack? It takes a resources from src/ folder and outputs them into a destination, for example Magento_Theme/.

📂pwa
 ┣ 📂Magento_Theme
 ┃ ┣ 📂templates
 ┃ ┗ 📂web
 ┣ 📂src
 ┃ ┣ 📂app    
 ┃ ┃ ┣ 📜<name>.js
 ┃ ┃ ┗ 📜<name>.scss 
 ┃ ┣ 📂config
 ┃ ┗ 📂public  
 ┗   ┗ 📜index.production.phtml

Below, the webpack or, more specifically html-webpack-plugin,  takes the index.production.phtml or index.development.phtml , renames it to root.phtml  while placing it into Magento_Theme/templates folder. See the following structure:

📂pwa
 ┣ 📂Magento_Theme
 ┃ ┣ 📂templates
 ┃ ┃ ┗ 📜root.phtml
 ┃ ┗ 📂web
 ┣ 📂src
 ┃ ┣ 📂app    
 ┃ ┃ ┣ 📜<name>.js
 ┃ ┃ ┗ 📜<name>.scss 
 ┃ ┣ 📂config
 ┃ ┗ 📂public  
 ┗   ┗ 📜index.production.phtml

But where are the styles placed?

Similar moves are done for JavaScript and style files, that after processing will appear in the Magento_Theme/web folder. This is how webpack helps Magento get PWA as a theme.

📂pwa
 ┣ 📂Magento_Theme
 ┃ ┣ 📂templates
 ┃ ┃ ┗ 📜root.phtml
 ┃ ┣ 📂web
 ┃ ┃ ┣ 📂assets
 ┃ ┃ ┣ 📂icons
 ┃ ┃ ┃ ┗📜<name>.bundle.js
 ┃ ┗ ┗ 📂<...>
 ┣ 📂src
 ┃ ┣ 📂app    
 ┃ ┃ ┣ 📜<name>.js
 ┃ ┃ ┗ 📜<name>.scss 
 ┃ ┣ 📂config
 ┃ ┗ 📂public  
 ┗   ┗ 📜index.production.phtml

Yet, the final JS code in Magento_theme/web is very different from our original files because Babel optimizes it.

2. Compare Prod and Dev setups

The root.phtml and index.production.phtml files are very similar. However it is because process of parsing the code the style files must be completely different.

Let’s take a look at the differences between the production and development set-up in a one table.

2.1. Code Minification & Splitting

The babel plug-in minifies the code removing unnecessary characters thus making it to become a single line.

Webpack splits code into multiple parts, or bundles. Code splitting allows one to load each bundle of code when needed. This improves website speed and overall performance. Moreover, webpack also prioritizes the code loading as needed by PWA.

2.2. The manifest file

Manifest does not differ in the development and production set-ups. The manifest file is a specific feature of PWA. It is automatically generated and serves to install an application correctly.

PWA manifests usually include things like app name, author, version, description, list of resources among other things.

2.3. Service Worker Pre-cache

The precache-manifest, however, is only generated in production. Based on the manifest file the Service Worker Pre-cache pre-loads code chunks and assets (without app originally needing them in browser). Thus in the off-line mode all the application (bundles, assets) is available to us**.

**This makes PWA to stand out among regular web applications.

The pre-loaded bundles will also ensure that all of the application parts will be available to you offline even though you haven’t visited it. This only applies to the application’s parts like ‘my account’, ‘card’, ‘checkout’, etc., not the actual data.

2.4. The Source references

Development setup keeps a source map. Source maps provide a references for the code for debug purposes. It maps the minified or transformed code to its original source.

2.5. The Hot Reload at Development server

In the development env. the PWA is kept running on the development server with periodic injections of new file versions that are edited at runtime.

The value process.env.NODE_ENV === ‘development’ ensures that a React application or any other dependent application will have hot reload enabled, that is – the app will reload automatically any time our code changes.

Some application logic is needed only in the development mode. If we go to the source folder and open the main index.js file we may see that some logic runs only in development, more specifically it’s the hot reload.

📂source
 ┣ 📂<…>
 ┣ 📂src
 ┃ ┣ 📂app	
 ┃ ┃ ┣ 📂<…>
 ┃ ┃ ┗ 📜index.js		 
 ┃ ┣ 📂config
 ┃ ┣ 📂public
 ┗ ┗ 📂sw 	

The main advantage of the hot reload is that the application state is not lost, which is especially useful when customizing a theme.

The development server is what makes one to use hot reload and memory cache. Only changed files are regenerated, which allows us to perform bundling quicker.

If we run npm run build it takes much time each time. While npm run watch takes much time only once initially and subsequent re-loads will be very fast.

2.6. HTML entry file name

index.production.phtml in a production environment vs index.development.html in development environment. Note the production file extension contains p in phtml.

2.7. Debugging and Building

The production set-up application performs a two-step build process:

• First the Service Worker Pre-Cache is built
• Then the actual application is built

The big difference as to the debugging is the fact that no files are generated in development mode, due to the development server’s memory cache. This can be seen by browsing the folder structures in dev. mode where the Magento-Theme folder will not exist.

2.8. The build speed

The built speed is faster in the Development mode than in the Production mode based on the following factors:

• files are supplied from cache
• hot reload option

3. The config folder

Take a look at the config folder contents:

📂source
 ┣ 📂<…>
 ┣ 📂src
 ┃ ┣ 📂app     
 ┃ ┣ 📂config
 ┃ ┃ ┣ 📜<…>
 ┃ ┃ ┣ 📜babel.config.js
 ┃ ┃ ┣ 📜meta.config.js
 ┃ ┃ ┣ 📜tests.config.js
 ┃ ┃ ┣ 📜webpack.core.config.js
 ┃ ┃ ┣ 📜webpack.development.config.js
 ┃ ┃ ┣ 📜webpack.extract-translations.config.js
 ┃ ┃ ┣ 📜webpack.production.config.js
 ┃ ┃ ┗ 📜webpack.sw.config.js
 ┃ ┣ 📂public
 ┗ ┗ 📂sw     

The config folder consists of the following:

• webpack.development.config.js and webpack.production.config.js
  • Their differences see in section 2.
• webpack.extract-translations.config.js
  • Extracts the internationalization strings from the application. We need to do this periodically in order to update the .json files with new translations.
  • One may execute this webpack config file by running the extract-translations command. Execute npm commands by typing npm run [name-of-command], eg. npm run extract-translations. Running a npm command this way makes the configuration preset in package.json file to be used. In section 4 you’ll get all the description of them.

One can read more about npm here.

• webpack.sw.config.js
  • The Service Worker configuration file is a part of the two-step build production set-up.
• webpack.core.config.js
  • Contains a configuration of core contribution.
• babel.config.js
  • Configuration for the babel-loader, read more of it in babel docs.
• meta.config.js
  • This file contains a configuration of meta tags that are to be included into HTML document.

You may get more info in webpack documentation.

4. Webpack commands

Node package manager (npm) allows us to use some nifty aliases that are defined in the package.json file. The package.json file is located in the main source folder.

npm run build is the most valuable command preset in the package.json file, because running it allows Magento to recognize a production ready theme.

Look at two of the ready made scripts commands in the package.json file:

"scripts": {

  "build": "npm run build-sw && webpack --config ./src/config/webpack.production.config.js",

  "build-sw": "webpack --config ./src/config/webpack.sw.config.js --mode=production",
}

One can find all of npm presets in the package.json file.

What to run in what case

If you’re using our Docker environment you don’t have to run anything. Docker does the job on its own.

Developing on a local machine, the following commands should be useful to you.

1) npm ci

If you’re running an existing Magento instance and you want to install a theme you need to run npm ci first. Not npm install, nor npm i. This npm ci must be the first command for preparing our project.

npm ci makes sure that you’ll get a more reliable build by firstly deleting your node_modules folder giving you a fresh start, as well as looking into your package-lock.json file and installing dependencies of a specified version, instead of modifying the package-lock.json file like npm i does.

npm ci is a security measure that allows us to not break our replication in case of one of the dependencies having a new broken minor release.

2) npm run build

The next command you run will be different depending on your needs. In order to compile a theme and test if Magento is able to recognize it, you need to run npm run build. This will build the theme and generate a Magento_Theme folder.

📂<parent-folder>
 ┣ 📂Magento_Theme
 ┃ ┣ 📂templates
 ┃ ┃ ┗ 📜root.phtml
 ┗ ┗ 📂web

Having the Magento_Theme folder with root.phtml file and web folder, it means that Magento should be able to recognize the theme.

If your application has the Magento_Theme folder with its contents, but Magento is not able to recognize the theme, you should check the Magento tables and look whether your theme’s name is added to the list.

3) npm run pm2-watch

This command will start the development server, which in turn will be restarted every time a new folder is created. We need to restart the development server upon a new folder creation cause of the memory cache renewal. If you’re creating a new folder with what you want to override a specific component, you need to recompile and the pm2 or process manager will handle this and restart the watching process.

The development server will be started in the port :3003, keep in mind that Magento might not function properly, because the request to /graphql might not be properly sent.

In this case you might need to change the request-URI constant cons REQUEST_URI in the util-request file which is in source folder.

Another approach would be by adding a NGINX config, which you can locate in our front-end configuration containers as well.

If none of these solutions work, we would recommend developing your application in our Docker environment which includes everything you need to start building applications seamlessly.

4) npm run extract-translations

This command will parse through all of your files and find every single template literal —(` `); that you’ve added, not a variable. Notice that there should be a simple string in between the back ticks, not a variable. After that, the contents of the back ticks will be put into the source/i18n internationalization folder under the specific language, eg. all English strings will be appended to the en_US.json file.

If a language file’s string translation is empty, null, this means that there is no yet translation for the specific term.

Which commands to apply in Docker and which in local machine

1. npm ci Docker Local
2. npm run build Local
3. npm run pm2-watch Local
4. npm run extract-translations Local

The use of npm run build, npm run pm2-watch and npm run extract-translations commands (commands 2-4) is not needed if you develop in a Docker environment. These commands should only be applied if you install on your local machine.

The npm ci command is however always in need so that we pull the latest changes. This way we ensure that the latest working versions of the packages are installed.