I’m working through an SEO Performance and Pagespeed Obsession, if this were a disease it would be called SEO Performance and Pagespeed Obsessive Disorder: SPPOD :-)
I’ve already picked the low hanging performance fruit and have very good results on page speed tools like Lighthouse audits tools. The Lighthouse test for this sites home page below was taken a couple of weeks ago, I did the speed test again today and it’s pretty much the same.
Very Fast Lighthouse Google Mobile Speed Test Results
I could stop with the above page speed results (they are VERY good), but pagespeed is my current SEO obsession and my next (failed) attempt at squeezing a few more milliseconds of speed was completely disabling .htaccess files for this domain.
The plan was (it failed) to move ALL .htaccess rules from the roots .htaccess and any other .htaccess files on the site (got to move ALL the rules not just the root .htaccess files) into either the main Apache servers httpd.conf file or the domains specific conf file: each of my virtual servers/domains have a separate conf file.
If I managed to move all the .htaccess files rules to the conf file I’d be able to completely disable .htaccess files (disable AllowOverride) on the virtual server (this domain) and this will have a performance improvement.
Why Move .htaccess Rules into the httpd.conf File?
The vast majority of domains use at least a root .htaccess file, for example it’s a prerequisite for WordPress SEO friendly permalinks to set relevant rewrite rules within the root .htaccess file.
At the very start of the Apache HTTP Server Tutorial: .htaccess files there’s a notice:
You should avoid using .htaccess files completely if you have access to httpd main server config file. Using .htaccess files slows down your Apache http server. Any directive that you can include in a .htaccess file is better set in a Directory block, as it will have the same effect with better performance.
The reason why .htaccess files have a performance hit (not an expert on this, but this is how I understood it) is because when AllowOverride is active every webpage loaded has to look through the directory structure for .htaccess files even if there aren’t any present.
For example a default WordPress site will load lots of resources from multiple directories like the WordPress themes main CSS file:
/wp-content/themes/themename/style.css
When AllowOverride is active the server has to check for a .htaccess file in these directories:
/wp-content/themes/themename/.htaccess
/wp-content/themes/.htaccess
/wp-content/.htaccess
/.htaccess
Could go even deeper back
To check for relevant server override rules inside the .htaccess files.
In the standard WordPress theme style.css file, Apache checks for at least 4 .htaccess files even though for most sites there’s only the root .htaccess file.
I don’t know how much of a performance hit this has on a server (my attempt to disable AllowOverride failed), but for me as long as I can make a server improvement without too much hassle I’ll make the change. Unfortunately my setup made it impractical to move all .htaccess file rules to either the main Apache httpd.conf or the virtual servers (the domains) conf file.
Even so I did move some rules including the W3 Total Cache Browser Cache Rules and below I’ll explain why I couldn’t move them all.
W3 Total Cache Plugin Creates a Lot of .htaccess files
As mentioned above this sites powered by WordPress, but it also uses the very popular caching plugin W3 Total Cache and unfortunately the Page Cache module creates a .htaccess file for every single webpage it caches & they are regularly deleted/recreated!
When I ran the following search command on the server (this finds all files with .htaccess in the file name):
find /home/domainname/public_html/ | grep '.htaccess'
It output a LOT of .htaccess files, 378 .htaccess files to be precise, below is a small selection:
/home/domainname/public_html/.htaccess
/home/domainname/public_html/.well-known/acme-challenge/.htaccess
/home/domainname/public_html/updates/cache/.htaccess
/home/domainname/public_html/updates/includes/.htaccess
/home/domainname/public_html/updates/logs/.htaccess
/home/domainname/public_html/updates/packages/.htaccess
/home/domainname/public_html/wp-content/cache/minify/.htaccess
/home/domainname/public_html/wp-content/hiddenfolder/.htaccess
/home/domainname/public_html/wp-content/cache/page_enhanced/.htaccess
/home/domainname/public_html/wp-content/cache/page_enhanced/seo-gold.com/.htaccess
/home/domainname/public_html/wp-content/cache/page_enhanced/seo-gold.com/2017/.htaccess
/home/domainname/public_html/wp-content/cache/page_enhanced/seo-gold.com/2018/.htaccess
/home/domainname/public_html/wp-content/cache/page_enhanced/seo-gold.com/anchor-text-of-internal-links-seo-test/.htaccess
/home/domainname/public_html/wp-content/cache/page_enhanced/seo-gold.com/anchor-text-of-internal-links-seo-test/anchor-text-seo-test-of-internal-links/.htaccess
/home/domainname/public_html/wp-content/cache/page_enhanced/seo-gold.com/anchor-text-of-internal-links-seo-test/external-link-anchor-text-seo-test-result/.htaccess
/home/domainname/public_html/wp-content/cache/page_enhanced/seo-gold.com/anchor-text-of-internal-links-seo-test/google-exact-match-search-for-hire-a-freelance-seo-expert/.htaccess
/home/domainname/public_html/wp-content/cache/page_enhanced/seo-gold.com/anchor-text-of-internal-links-seo-test/internal-link-anchor-text-seo-test-result/.htaccess
I should be able to manually move the contents of the first 10 .htaccess files, but the 300+ .htaccess files under /home/domainname/public_html/wp-content/cache/page_enhanced/seo-gold.com/ are created by the W3 Total Cache plugins Page Cache module and they are regularly rebuilt with new rules every time the WordPress theme updates.
No way I can manually move them all and regularly update them, plus every time a new WordPress Post, WordPress Page or WordPress Image Attachment is created a new .htaccess file is created under the /wp-content/cache/page_enhanced/ directory structure!
Was a nice idea to move all .htaccess rules to the httpd.conf or virtual server conf file, but it’s impractical with this setup :-(
When you hit a wall like this it’s time to decide if it’s worth the time/effort to find a work around. I could maybe switch to another caching plugin or see if there’s another solution to replicating the W3 Total Cache plugins Page Cache .htaccess files. I made the decision to stop trying to disable AllowOverride for now and come back to the issue at a later date: what I’ll do is keep an eye out for articles related to disabling AllowOverride when there’s a lot of .htaccess files and in the meantime look for lower hanging fruit.
I did move some rules from .htaccess files which should have a small performance improvement, though I assume no where near as much if I managed to disable .htaccess files completely.
How to Move WordPress .htaccess Rules to the httpd.conf File
WordPress SEO friendly permalinks are set via rewrite rules within the root .htaccess file.
Standard .htaccess file WordPress SEO friendly permalink rules.
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress
The above sort of .htaccess file rules in theory can be moved (you don’t HAVE to) to the httpd.conf or domain specific conf file via a Directory directive like this added to the domains VirtualHost directive:
<Directory /home/domainname/public_html>
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress
</Directory>
After moving the rules your conf file will look something like this:
Move WordPress .htaccess Rules to the httpd.conf File
Note: It’s important to wrap the WordPress rules inside a Directory directive, without it, it won’t work.
I added the rules to a conf files which holds the directives/rules for this domain rather than the main httpd.conf file, end result is the same.
When I tried the above it failed (resulted in 10 mins of head scratching why it didn’t work) because I use the W3 Total Cache Plugin and the plugins Page Cache .htaccess file rules are saved ABOVE the main WordPress .htaccess file rules (inside the .htaccess file) and if you only move the main WordPress rules without the plugin Page Cache rules it breaks the site: the rules are loaded in the wrong order (main WordPress rules are loaded first).
I tried moving all W3 Total Cache Plugin rules into the Apache conf file with the main WordPress rules (in the correct order) and though it works at first, as soon as I manually clear the plugins cache the .htaccess files are resaved to the .htaccess file and it breaks the site.
A fix is make the .htaccess file non writable, change it’s permissions, this way W3 Total Cache will try to resave the rules into the .htaccess file and fail. Would be nice if W3 Total Cache added an option to disable automatic .htaccess file rule saving for this scenario.
David Law
Website - SEO Gold Services
Hi David – great results. I can see how your speed has improved since I first found your website about a year ago.
I’ve been down SPPOD road a few times myself. :)
It’s always the smallest problems that give us the biggest headache. That’s how I ended up finding SEO Gold, due to W3TC unsetting my ETags. Anyway now I’m concentrating on the Lighthouse family rather than outdated Pingdom and GT Metrix.
Drill down on your Best Practices, it will most likely be simple things like contrast ratios, then you’ll be able to get the Lighthouse Google Easter Egg.
Will be sure check out your other articles – didn’t realise you had so many!
Regards,
David
Sorry, my bad, your site is returning 100 on Best Practices now.
Regarding Performance – your site is still running on http/1.1 so update it to http/2 and you’ll be golden.
Good luck!