Scully and Angular Universal, comparing benchmarks

Scully - a Static Site Generator for Angular series

This article is part of a growing series about Scully - a Static Site Generator for Angular. If you get excited about this article be sure to check the others!

1. Custom plugins for Scully - Angular Static Site Generator
2. Disabling Angular when statically generating with Scully
3. Scully or Angular Universal, what is the difference?
4. Scully and Angular Universal, comparing benchmarks

Target audience

This post is a follow-up on my previous post about the differences between Angular Universal and Scully. In that post I mentioned that Universal can be much faster in the (pre)rendering process compared to Scully. This difference is mainly significant and noticeable in the order of thousands of pages and more as we will show.

When building and maintaining a large website, for example a business directory, where rerenders are often necessary, this can become very important. If it takes an hour or more to prerender all the pages the data might have changed again before the process ended. So let's see how both technologies behave on the same application and come up with the actual metrics!

Important: This post is not intended to break down or bash any of the 2 technologies. I love both of them very much and I believe they both have their use case! As a good software engineer you should always try to make the best and informed decisions about technology choices, especially if you can base those decisions on simple benchmarks!

Example application

The example application used for the benchmarks is a simple Angular application, in an Nx monorepo, that has a home route, a news overview route and news detail route. The news functionality is loaded via a lazy-loaded angular module. You can find the repository here to play with the setup and try your own scenarios.

Due to the different implementation details of Universal and Scully the application was build twice, but with the same structural setup, markup and styles. This means the code is not completely identical; the only difference is the injection of the ScullyLibModule in the Scully application. This difference should not have a significant influence on the test results.

Benchmarking details

Test cases

Scully and Universal will prerender all the pages that are available on the application. These pages include the auto-discovered pages and the list of pages that is generated based on the available news data. For Scully, the list of dynamic pages is created using a custom routerPlugin. For Universal we used a node script. The result is the same.

The HTML render-output will be minified using html-minifier. The dynamic data, that is loaded on the news overview and detail pages, is inlined in the HTML.

We will monitor the time it takes for rendering 100, 1,000 (one thousand) and 10,000 (ten thousand) news detail pages with Scully and Universal. Each testcase will be done 3 times and the average of those will be used as the final result.

System environment

The benchmarks were executed on my own device, and after a reboot. For reference, here are the system details:

• MBPro 15inch 2018, 2.9Ghz 6core Intel Core i9
• RAM: 32 GB 2400 Mhz DDR4
• Graphics: Radeon Pro 560x 4GB & Intel UHD Graphics 640 1536 MB

Monitoring tools and measuring metrics

• The execution time is monitored with gnomon
• The payload is measured with Chrome Devtools > Network

Running the tests

Universal

We have to serve the application (line 1) while the prerender process is ongoing to be able to load the data for the news overview and detail pages. In a separate terminal (line 3) we group the prerendering and minification process and pipe it to the gnomon command to measure the execution time.

npx nx serve universal
# parallel terminal
( npm run prerender:universal && npm run minify:universal ) | npx gnomon

Scully

The same is true for Scully, the data needs to be available so we serve the application in a separate terminal (line 1). In the other terminal (line 3) we first do a clean --prod build of the application as the Universal process also starts with a clean build. We pass the option --scanRoutes to start with and force a clean route discovery process. The final configuration option defines the location of the config file for Scully.

npx nx serve scully
# parallel terminal
( npx nx build scully --prod && npx scully --scanRoutes --configFile .scully/scully.scully.config.js ) | npx gnomon

To avoid setting up an API server for this test, I have used news-100.json to represent 100 entries in JSON data, news-1000.json for 1,000 entries and news-10000.json for 10,000 entries. For testing the different amount of pages we need to modify the reference to news-100.json in the scully.scully.config.js, the overview.component.ts and generate-routes.ts files.

Benchmark results

Execution time

The first table shows the results for the execution time of the prerendering process of both technologies.

Asset and HTML page sizes

The second table shows an overview of the sizes of the generated index.html files and the total of assets loaded when requesting a specific page that bootstraps the Angular application on pageload.

General findings

Before we dive into the comparison of the metrics of the different technologies and test-cases, let's first discuss some import findings and things to consider.

• Execution time will be influenced by other processes running. If you can run the process on a clean and isolated node it will give the most stable results.
• The HTML minification process for the Univeral prerendering process can be considered as sub-optimal. There is currently no easy way to directly hook into the process. So we need a separate post-process that will do a read and write operation for minifying each HTML file.
• Both technologies allow for more smart rerendering of the prerendered pages. You can build smart systems that will only prerender a part of your application, or even only one specific page, after data changes. You will always need an initial prerendering of all pages.

Comparing the metrics

• Universal is notably faster, but in the order of 100 pages the differences are neglectable. For around 1,000 pages the execution time for Scully triples compared to Universal. For 10,000 of pages Universal is a factor ~10 faster.
• Using Scully adds around 10kb to the total asset size of the Angular application. 10kb is not little, but with around 1/30 of the total asset size of a very basic application I do no consider this to be significant.

How to optimize Scully

Scully is still in alpha, nearing their first beta release and there will probably follow more internal optimizations. However, Scully, using Puppeteer, will never be able to perform as optimal and fast as Universal. Let's explore other ways and techniques to optimize Scully.

Limit the to-render list of pages

If your application has a lot of pages you definitely need strategies that will only rerender specific pages when your content changes. These algorithms can be build in your own custom router plugin. You could for example query a service that will only give back the changed identifiers since a given datetime. That way your list of to render routes will be limited to the updated pages.

Another but related technique to the strategy described above is the use of the --baseFilter option. If this option is specified only the routes that start with this prefix are taken into account. This way you can prerender only a part of your application routing-tree.

Disable specific resource types

Using the option --ignoreResourceTypes we can block specific request to resources from happening in Puppeteer when our pages get rendered. This might only slightly decrease the prerendering execution time but potentially dramatically decrease the amounts of data that is loaded during prerendering. The following configuration will block all image, other media, font and stylesheet resources from being loaded.

scully.your-project-name.config.js

exports.config = {
    projectRoot: "./apps/<your-project-name>/src",
    projectName: "<your-project-name>",
    outDir: './dist/apps/<your-project-name>-static',
    ignoreResourceTypes: [
      'image',
      'media',
      'font',
      'stylesheet'
    ]
};

Conclusion

Please don't take my writings for granted on this and don't use these metrics for your project. The example application is far from a real-world application, but the concepts and benchmarking process explained can easily be applied on your own use-case. Adding 3rd party libraries or loading a content-heavy and/or functionality-heavy page might yield completely different results. If you are willing to share your benchmarking results I would be very happy! :)

One thing is for sure though.. If you implement Universal, the prerendering process time will dramaticaly decrease (read: be faster) if you are generating pages in amounts of 1000+. If your content is highly volatile but you want to keep benefiting from prerendering, consider implementing Universal or at least apply smart rerendering strategies for your Scully powered application when data changes.

Just because these tests showed that Scully becomes slow in a higher order of amounts of pages, in this particular example application, doesn't mean it's bad at what it does. Scully solves other problems and the time/money you might invest on adding Universal support might be more then the extra build and render time Scully introduces.

Choose wisely and consider all aspects! The Scully team is working hard to keep improving. If you want more advice on the Scully vs Angular Universal debate, check my other blogpost on the differences between Scully and Universal. Thanks for reading!

Further reading

1. Scully - Github organization
2. Server-side rendering (SSR) with Angular Universal
3. Minify the HTML of your prerendered Angular application - Scully plugin
4. Guess parser

Special thanks to

• Leonel Elimpe

for reviewing this post and providing valuable and much-appreciated feedback!