Why You Shouldn't Merge JavaScript in Magento

Most people - myself included - thought that merging all of your separate Javascript files was a healthy way to speed up the front end of your site. The logic was that by merging the files, you reduce the number of requests the browser makes when loading your site. This allows the request to complete quicker and results in your page loading faster. Let's walk through an example of this process in Magento and see what happens.

Imagine you have just enabled Javascript Merging in the Magento Admin and then a user visits your site. Magento collates all of the XML layout files and determines which Javascript files should be included for the homepage request. These files are then merged into a single file and saved using an md5 hash of their filenames. This new file, named f0eb853c09ba3e46ad21fb89b8d57626.js, is then served to the user, who downloads it and stores it in their browser cache.

Next, the user clicks a link to one of your categories. Once again, Magento collates the XML layout files and determines what Javascript files are needed for the category page. At this point, Magento realises that the exact same Javascript files used on the homepage are needed on this page. Instead of remaking the single Javascript file, Magento serves the Javascript file created when the user visited the homepage. This time though, the browser knows that it already has this file and therefore doesn't download it again. Instead, the file is loaded from the user's cache, saving time, bandwidth and computer resources!

So far everything has gone well; we have made the page load faster while decreasing the load on the server, but...

Next, the user is enticed by one of your products and clicks through to a product page. Once again, Magento determines which Javascript files are needed based on the layout XML files. At this point, Magento realises that the same Javascript files on the homepage are needed as well two new files. As a merged file containing this combination of Javascript doesn't exist, Magento merges the files and creates a new Javascript file named 139f9771ba2ba0cae754978fab4a3c35.js. Roughly 80% of this file is the Javascript that was used on the homepage and has been downloaded and cached by the user already. The final 20% accounts for the new Javascript files used solely on the product page. This file is then served to the user who is forced to download it all! Although this file is made of 80% code that has already been cached, the user's browser is completely unaware of this and will download 100% of the file and cache it again!

The intended result of merging Javascript is a decreased page load time, but in the above scenario, the user has been forced to re-download a massive chunk of Javascript! This will no doubt have increased page load time (downloading ~40kb of Javascript that doesn't need to be downloaded), which goes against the idea of Javascript merging.

Let's consider what would happen if we hadn't of merged our Javascript...

When the user first requests the homepage, the merged Javascript files would be downloaded individually. Although the combined size of these Javascript files matched the size of the merged Javascript file, this request would take longer as each file would need to be requested, downloaded and cached separately. 1-0 merging!

Second, the user requested the category page, which uses the same files that were just downloaded and cached on the homepage. As a result this takes exactly the same amount of time it would if the Javascript files had been merged. In both instances, all of the required Javascript has already been downloaded and cached, meaning the browser can skip the Javascript altogether.

Lastly, the user visits the product page, which uses the Javascript files found on the home and category pages as well as two new files. The files used on both the homepage and category page have already been cached, so the browser skips these files and downloads and caches the two new files that haven't been experienced before! In the merging scenario, we mentioned that files used on the homepage in Magento make up about 80% of the total Javascript on the product page. With merging disabled, we have managed to skip downloading the 80% and just download the 20%. In the merging example, the full 100% had to be re-downloaded!

This problem is compounded for each page your user visits that has a combination of Javascript not found on another page. Each time this happens, the user will be forced to re-download Javascript that it has already downloaded and cached under a different filename.

What Can Be Done?

While the current Magento merging system is flawed, it would only take a minor adjustment to fix this critical issue.

In Magento, Javascript is added to the Head block through XML and stored in an array, which is then iterated over and displayed in the head section of your HTML code. Our suggestion is that instead of simply adding Javascript files to the head, an extra parameter be included that allows you to specify a group for the Javascript. Consider the following code:

<default>
 <reference name="head">
   <action method="addJs"><script>prototype/prototype.js</script><group>global</group></action>
   <action method="addJs"><script>scriptaculous/builder.js</script><group>global</group></action>
   <action method="addJs"><script>scriptaculous/effects.js</script><group>global</group></action>
   <action method="addJs"><script>varien/form.js</script><group>global</group></action>  </reference>
</default>

<catalog_product_view>
 <reference name="head">
   <action method="addJs"><script>varien/product.js</script><group>product</group></action>
   <action method="addJs"><script>varien/configurable.js</script><group>product</group></action>
 </reference>
</catalog_product_view>

Notice the addition of the paramter group? Instead of all of the Javascript being merged into a single file, the group parameter would be be used to decide which Javascript files get merged together. The above code would result in two separate merged files being created for the product page; one containing the global Javascript that is used on the homepage (and the whole of the site) and a second containing the Javascript used solely on the product page. Although this would create an extra page request, time would still be saved by not having to re-download the Javascript used on the homepage! If this technique were applied to the whole Magento site, each piece of Javascript would be downloaded once and once only!

Update: Since writing this article, we have decided to take this into our own hands and have added this functionality to our Magento minify extension. For truly fast speeds, use this with our Magento Full Page Cache extension!.

21 thoughts on “Why You Shouldn't Merge JavaScript in Magento”

  • Kevin Woolf

    Hey Fishpig,

    Where would you make these changes exactly? I've taken a look at all the JS add by Magento core, added extensions, and installed templates only find that identifying each and every combination of JS files that gets loaded on any given page would be a daunting task.

    Is there any short cut that would allow me to avoid just visiting every page and noting all the JS files in the head section?

    Kevin

    Reply
  • BT

    Hi Kevin, This would require changing Mage_Page_Block_Html_Head. You would then need to update all of the layout XML files to ensure a group is specified when adding the JS. We are working on an extension that should help with this but hopefully this is something Magento will build into the core of Magento 2.0

    Reply
  • beep logic

    In your proposed solution: what if <catalog_product_view/> handle includes a different list of "product" group JS files than <catalog_product_compare/>? Seems like you'd end up with multiple versions of the compiled "product" group.

    Reply
  • BT

    Hi BL, the above example was just a small bit of code to ilustrate our point. In a working example, each product type could have it's own group (eg. product_grouped, product_configurable). The rest of the JS that is on all product pages could have the JS group 'product'. This would then stop any JS being redownloaded by the browser.

    I hope this answers your question.

    Reply
  • beep logic

    Your comment system didn't escape my tags for the <handles>. Even if you "chunk" up the JS files by a group, the list could vary by handle: catalog_product_view and catalog_product_compare could potentially generate different "product_configurable" groups as some JS files are not required in one view. Hopefully I'm not misunderstanding your solution.

    Reply
  • BT

    Yes, the JS files could be different per handle. You would have to make a judgement call about how you split the groups. This would be different for each site and would depend on the JS you use, as well as the popularity of particular pages.

    Regardless of how the JS is split, as long as you group the Magento core JS (Prototype, Scriptaculous, Varien, Mage etc) into a single file, you should still see some improvements in the overall load time of your page as this JS - which accounts for a large percentage of the overall JS - would not have to be redownloaded time and time again

    Reply
  • Alexander Sibert

    Nice Blog post. I merge always in Magento all JS/CSS Files to one and use additionally the free fooman Speedster Enterprise Extension to Minify, Compress, Versioning and caching the files.

    Reply
  • steve snyder

    I also do merge Java Script , these techniques help make boost magento load time which otherwise is difficult to achieve. Any news about Magento 2, would merging also have to used in Magento 2?

    Reply
  • Peter O'Callaghan
    Peter O'Callaghan September 20, 2012 at 12:06 pm

    It would appear that if you use a themes local.xml file to call addJs, the added JS file will not even be included on the page.

    Reply
  • Mathew Porter

    Nice article and I agree that it is best to keep the js files separate and only load specific ones on pages where required.

    Also worth noting that if people are developing on Magento CE 1.4 and above these xml updates to call individual js files should be made in the themes local.xml file and not by editing core files to aid with upgrading.

    Reply
  • Gordon

    Thank you for the nice article,
    it inspired me to write a extension, which does exactly what you talk about. I uploaded it to github: https://github.com/GordonLesti/Lesti_Merge and Magento Connect: http://www.magentocommerce.com/magento-connect/catalog/product/view/id/16559/

    Best Regards
    Gordon

    Reply
  • Mark Shust

    I believe a better way of handling this would be to write a module that fixes Magento's flawed combining logic, as it allows addJs references nodes other than default. My personal opinion would be to combine all external javascript (those referenced with an addJs or simliar) into one big file. This would cause the javascript not to regenerate and load all the javascript at once. I believe this makes the most sense. Then you might allow the option for specific files (perhaps super large ones only on specific pages) to bypass the combining logic and load themselves separately.

    Reply
  • Damu

    Nice post! when i merge js not only frontend but admin broke too. so i always avoid merging js.

    Reply
  • Thiyagarajan Veluchamy
    Thiyagarajan Veluchamy June 3, 2013 at 12:01 pm

    Thanks! nice post.

    Reply
  • Max

    Is it safe to assume this merge issue applies as well to CSS in Magento?

    Reply
    • BT

      Yes, any merging of files can cause this behaviour. With CSS, I personally prefer to manually add all CSS to a single file to avoid this. I use <a href="http://fishpig.co.uk/magento/extensions/optimisation-minification/" rel="nofollow">Opti</a> to automatically minify this file so that the smallest possible file is sent to the customer.

      Reply
  • Rob

    Hi, it's definitely an interesting argument. However I just did a few website speed tests with the javascript both merged and not merged on both the homepage and product pages and the load times on both were about 50% faster with the files merged.

    Reply
    • BT

      Agreed but customers will be visiting your whole site and not just the homepage and product page. The best solution would be to merge the files, but merge them in a clever and more efficient way. The JS files that are included on every page should be merged into a single file, the files that are on the product pages should be merged and the same for every other page. Files merged in this way would mean a customer would never have to download any JS more than once.

      The new version of <a href="http://fishpig.co.uk/magento/extensions/optimisation-minification/" rel="nofollow">Opti</a>, which will be released at the start of 2014, will do this.

      Reply
  • Nicolas

    I believe that this article is missing a few point. This would be correct when there is only a few Javascript but some of the Magento store I worked on had +10 css files and at least +20 JS scripts. Moreover this sentence "This time though, the browser knows that it already has this file and therefore doesn't download it again." is not true is the server is not configured properly for instance, its needs to be an expire time when the server deliver Javascript, also you need to use etags. Because the browser will still do a request to see if the file changed and only if it didn't change it will pass, but the request has been made either way.

    You definitely bring a good point. To me the only to fix this is to use modular Javascript, YUI has a JavaScript loader pretty good it only load the Javascript you need on the page. For the css the way Magento deal with it was a mess anyway, you should have only on CSS file and reuse element as much as possible. I would just rewrite the CSS sheet entirely.

    Reply
  • Mummy Ninja

    I just turned of my magento javascript merging and use cloudflare instead. :D

    Reply
  • Jari Ullah

    I think magento is not a good option for this..

    Reply
Leave a Reply
Post your comment

FishPig Ltd