Improving web site performance with Apache .htaccess

May 23rd, 2011 by Samuel Santos Leave a reply »

Web performance is getting more and more attention from web developers and is one of the hottest topic in web development.

Fred Wilson considered it at 10 Golden Principles of Successful Web Apps as the #1 principle for successful web apps.

First and foremost, we believe that speed is more than a feature. Speed is the most important feature. If your application is slow, people won’t use it.

Faster website means more revenue and traffic
  • Amazon: 100 ms of extra load time caused a 1% drop in sales (source: Greg Linden, Amazon).
  • Google: 500 ms of extra load time caused 20% fewer searches (source: Marrissa Mayer, Google).
  • Yahoo!: 400 ms of extra load time caused a 5–9% increase in the number of people who clicked “back” before the page even loaded (source: Nicole Sullivan, Yahoo!).

Google experiments reached similar results:

Our experiments demonstrate that slowing down the search results page by 100 to 400 milliseconds has a measurable impact on the number of searches per user of -0.2% to -0.6% (averaged over four or six weeks depending on the experiment). That’s 0.2% to 0.6% fewer searches for changes under half a second!

And speed is now a factor contributing to Google Page Rank:

Google, in their ongoing effort to make the Web faster, blogged last month that “we’ve decided to take site speed into account in our search rankings.” This is yet another way in which improving web performance will have a positive impact on the bottom line.

The good news is that some of the most important speed optimizations can be easily done with simple .htaccess rules.
These rules can make any website faster by compressing content and enabling browser cache. They also follow the Best Practices for Speeding Up Your Web Site from Yahoo!’s Exceptional Performance team.

Compress content

Compression reduces response times by reducing the size of the HTTP response.
It’s worthwhile to gzip your HTML documents, scripts and stylesheets. In fact, it’s worthwhile to compress any text response including XML and JSON.
Image and PDF files should not be gzipped because they are already compressed. Trying to gzip them not only wastes CPU but can potentially increase file sizes.

To compress your content, Apache 2 comes bundled with the mod_deflate module.

The mod_deflate module provides the DEFLATE output filter that allows output from your server to be compressed before being sent to the client over the network.

Enable gzip compression for text responses:

<ifModule mod_deflate.c>
  AddOutputFilterByType DEFLATE text/html text/xml text/css text/plain
  AddOutputFilterByType DEFLATE image/svg+xml application/xhtml+xml application/xml
  AddOutputFilterByType DEFLATE application/rdf+xml application/rss+xml application/atom+xml
  AddOutputFilterByType DEFLATE text/javascript application/javascript application/x-javascript application/json
  AddOutputFilterByType DEFLATE application/x-font-ttf application/x-font-otf
  AddOutputFilterByType DEFLATE font/truetype font/opentype
</ifModule>

In previous versions of Apache, you can use mod_gzip.

Enable browser cache

Web page designs are getting richer and richer over time, which means more scripts, stylesheets and images in the page. A first-time visitor to your page will make several HTTP requests to download all your sites files, but by using the Expires and Cache-Control headers you make those files cacheable. This avoids unnecessary HTTP requests on subsequent page views.

Apache enables those headers thanks to mod_expires and mod_headers modules.

The mod_expires module controls the setting of the Expires HTTP header and the max-age directive of the Cache-Control HTTP header in server responses.
To modify Cache-Control directives other than max-age, you can use the mod_headers module.

The mod_headers module provides directives to control and modify HTTP request and response headers. Headers can be merged, replaced or removed.

Rule for setting Expires headers:

# BEGIN Expire headers
<ifModule mod_expires.c>
  ExpiresActive On
  ExpiresDefault "access plus 5 seconds"
  ExpiresByType image/x-icon "access plus 2592000 seconds"
  ExpiresByType image/jpeg "access plus 2592000 seconds"
  ExpiresByType image/png "access plus 2592000 seconds"
  ExpiresByType image/gif "access plus 2592000 seconds"
  ExpiresByType application/x-shockwave-flash "access plus 2592000 seconds"
  ExpiresByType text/css "access plus 604800 seconds"
  ExpiresByType text/javascript "access plus 216000 seconds"
  ExpiresByType application/javascript "access plus 216000 seconds"
  ExpiresByType application/x-javascript "access plus 216000 seconds"
  ExpiresByType text/html "access plus 600 seconds"
  ExpiresByType application/xhtml+xml "access plus 600 seconds"
</ifModule>
# END Expire headers

Rule for setting Cache-Control headers:

# BEGIN Cache-Control Headers
<ifModule mod_headers.c>
  <filesMatch "\.(ico|jpe?g|png|gif|swf)$">
    Header set Cache-Control "public"
  </filesMatch>
  <filesMatch "\.(css)$">
    Header set Cache-Control "public"
  </filesMatch>
  <filesMatch "\.(js)$">
    Header set Cache-Control "private"
  </filesMatch>
  <filesMatch "\.(x?html?|php)$">
    Header set Cache-Control "private, must-revalidate"
  </filesMatch>
</ifModule>
# END Cache-Control Headers

Note 1: There is no need to set max-age directive with Cache-Control header since it is already set by mod_expires module.
Note 2: must-revalidate means that once a response becomes stale it has to be revalidated; it doesn’t mean that it has to be checked every time.

Disable HTTP ETag header

ETags were added to provide a mechanism for validating entities that is more flexible than the last-modified date.

The problem with ETags is that they typically are constructed using attributes that make them unique to a specific server hosting a site.
ETags won’t match when a browser gets the original component from one server and later tries to validate that component on a different server.

In Apache, this is done by simply adding the FileETag directive to your configuration file:

# BEGIN Turn ETags Off
FileETag None
# END Turn ETags Off

Last-Modified header

In a previous post I stated that:

If you remove the Last-Modified and ETag header, you will totally eliminate If-Modified-Since and If-None-Match requests and their 304 Not Modified responses, so a file will stay cached without checking for updates until the Expires header indicates new content is available!

Well, I was wrong and this is why: we still want Last-Modified header for static files. If a user presses the refresh button, then browser will send a conditional request and server will respond a 304 Not Modified. If you disable both Last-Modified and ETag, the browser will have to download the whole content again whenever a user presses refresh.

Merge and minify your static files

Reducing the number of components in turn reduces the number of HTTP requests required to render the page. This is the key to faster pages.
Minification is the practice of removing unnecessary characters from code to reduce its size thereby improving load times.

See Combine and minimize JavaScript and CSS files for faster loading for more information on this topic.

Web performance tools

Always check your changes. Use either YSlow or Page Speed browser plugins.
They are super easy to use and the best tools currently available for the job.

Final file

Copy the following .htaccess file into the root directory of your site and enjoy the performance improvements.

# BEGIN Compress text files
<ifModule mod_deflate.c>
  AddOutputFilterByType DEFLATE text/html text/xml text/css text/plain
  AddOutputFilterByType DEFLATE image/svg+xml application/xhtml+xml application/xml
  AddOutputFilterByType DEFLATE application/rdf+xml application/rss+xml application/atom+xml
  AddOutputFilterByType DEFLATE text/javascript application/javascript application/x-javascript application/json
  AddOutputFilterByType DEFLATE application/x-font-ttf application/x-font-otf
  AddOutputFilterByType DEFLATE font/truetype font/opentype
</ifModule>
# END Compress text files

# BEGIN Expire headers
<ifModule mod_expires.c>
  ExpiresActive On
  ExpiresDefault "access plus 5 seconds"
  ExpiresByType image/x-icon "access plus 2592000 seconds"
  ExpiresByType image/jpeg "access plus 2592000 seconds"
  ExpiresByType image/png "access plus 2592000 seconds"
  ExpiresByType image/gif "access plus 2592000 seconds"
  ExpiresByType application/x-shockwave-flash "access plus 2592000 seconds"
  ExpiresByType text/css "access plus 604800 seconds"
  ExpiresByType text/javascript "access plus 216000 seconds"
  ExpiresByType application/javascript "access plus 216000 seconds"
  ExpiresByType application/x-javascript "access plus 216000 seconds"
  ExpiresByType text/html "access plus 600 seconds"
  ExpiresByType application/xhtml+xml "access plus 600 seconds"
</ifModule>
# END Expire headers

# BEGIN Cache-Control Headers
<ifModule mod_headers.c>
  <filesMatch "\.(ico|jpe?g|png|gif|swf)$">
    Header set Cache-Control "public"
  </filesMatch>
  <filesMatch "\.(css)$">
    Header set Cache-Control "public"
  </filesMatch>
  <filesMatch "\.(js)$">
    Header set Cache-Control "private"
  </filesMatch>
  <filesMatch "\.(x?html?|php)$">
    Header set Cache-Control "private, must-revalidate"
  </filesMatch>
</ifModule>
# END Cache-Control Headers

# BEGIN Turn ETags Off
FileETag None
# END Turn ETags Off
Don't be shellfish...Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedInPin on PinterestBuffer this pageEmail this to someone
Advertisement

39 comments

  1. Justin says:

    Thanks!

  2. Rod says:

    Fantastic info – thank you very much!

  3. Great article, good resource.

  4. Sharry says:

    Great article, thanks. My site seems 10x faster already :)

  5. Thank you for this excellent piece of code! :)

    I have two questions:

    should this be put before or after the wordpress code on the htaccess ?

    Can it be used with the most common cache plugins( Wp Total Cache, or Wp Super Cache?)

    Thanks again

    • Hi Nuno,
      You can place WordPress code anywhere you want, the order doesn’t really matter.
      As for the cache plugins, I assume that it’s safe to use any of them, but I can’t say with total confidence since I don’t use those plugins.

  6. George says:

    Great info.

    I used your settings but I have a problem when I update some content (eg js files)
    I get the cached version, not the new one. Unless, of course I refresh the page. It is easy for me to do since I know I have done an update. But visitors?

    How do I overcome this problem?

    Thanks.

  7. sk says:

    Amazing bit of info, went from a result of 49 in page speed to 79 just using the info on this page.

  8. Chris says:

    Cheers for this fantastic piece of information!

    Would you know how to prevent caching if the url starts ‘/admin/’
    If admin/ was a real folder I would just store a new .htaccess in there, but in my case all the paths go through a redirect rule to my php framework main page handler:
    RewriteRule ^(.*)$ app.php [QSA,L]

    I tried using directoryMatch, Location in the above configuration…. but it all failed miserably..

    Any idea to add path exceptions to that?

  9. Ramon says:

    Thanks for the helpful and detailed explanation. Very useful.

    You mention at ‘Compress content’: “The following rule will gzip all your *.css, *.js, *.html, *.html, *.xhtml, and *.php files”. But I don’t see any ‘link/mention’ for the *.php-files in the provided code.

    So my question is: Are the provided ‘Compress content’-code lines complete? Or did you forget to mention *.php in de line of code.

    Thanks in advance for you answer.

  10. Ray says:

    Nice work Samuel.

    Would the .htaccess you have developed be added to the root directory only or to each sub directory?

  11. Matt says:

    Thanks a lot, Samuel! It works amazingly!

  12. Tanveer says:

    These snippets comes along with w3total cache. isn’t?

  13. Kaidul says:

    Very helpful really.I was in a fix on how to gzip components and cache-control and now I get my solution.Thanks the author.

  14. Erik says:

    This is a great article!
    Pingdom score from 52 > 92

    Thanks!

  15. Ofrou says:

    Hello!! Thanks for your time and for help all of us.

    I have 1 Q. I have a hosting that allow me to have more than 1 domain/web.

    What do I need to do, 1 .htaccess for each domain or only 1 in the root folder of server?

    All my domains use wordpress.

    Thanks in advance!

    • You need one .htaccess file for each domain.
      You should already have one in both of them since WordPress uses it in conjunction with the mod_rewrite Apache module to produce permalinks.

      • ofrou says:

        Hi Samuel and thaks a lot for your reply. My pages now very fast, but I still have a big issue for me.

        When I click on a google link of any of my pages, It allways takes a lot of seconds (sometimes 15 or 17 seconds) to respond and start charging. Do you know what I should do?

        Muito obrigado, Saudações da Galiza.

  16. Maxi says:

    Great post!!!
    Thanks for sharing!

  17. Erick says:

    Thank you! Very useful!

  18. Erick says:

    It is important to specify one of Expires or Cache-Control max-age, and one of Last-Modified or ETag, for all cacheable resources. It is redundant to specify both Expires and Cache-Control: max-age, or to specify both Last-Modified and ETag.

    • Hi Erick,
      The Expires HTTP header is still useful for supporting older HTTP/1.0 proxies.
      For conditional requests, my suggestion referred in the post is to disable the ETag HTTP header and only enable Last-Modified.

  19. Does it matter were the entire code is placed in the HTAcess file? I’m seeing no difference. My site is in the details. WPTotal cache doesn’t work on my site either.

  20. Jerry says:

    Thanks for the detailed explanation.

    I just have one question though:

    Should the:

    ExpiresByType image/jpeg “access plus 2592000 seconds”

    be written as :

    ExpiresByType image/jpe?g “access plus 2592000 seconds”

    to cover both jpeg and jpg extensions, or is that not necessary.

    Thanks,

    Chuck

  21. Rob says:

    Excellent! Made a HUGE difference!

  22. thanks for the post it is very useful.

    i have check my website in pingdom. it give me below recommendation.

    The following resources are missing a cache validator. Resources that do not specify a cache validator cannot be refreshed efficiently. Specify a Last-Modified or ETag header to enable cache validation for the following resources:

    http://www.vishaldigitalprint.com/images/Hoarding-radhika.jpg
    http://www.vishaldigitalprint.com/images/arrows.png
    http://www.vishaldigitalprint.com/images/bg-slider1b.jpg

    can you help me out in resolving this recommendation.

    • Pingdom is right!
      As you can see in the section “Last-Modified header” of this post, if you disable both Last-Modified and ETag, the browser will have to download the whole content again whenever you refresh the page.
      My suggestion is to always specify the Last-Modified HTTP header for cached files.

Leave a Reply