We recently tested a new server management tool – it starts and stops servers. It works by updating the DNS record for the server so that when its off, users are redirected to a webpage that re-starts the server.
During testing, we saw something a bit strange – the server DNS entry was updated, but the local browser was still showing the old webpage!
After quite a bit of digging we discovered that browsers cache the DNS entries to make pages load faster – and this behaviour can't easily be bypassed.
How It Works: DNS Refresh
Before we proceed, let's remind ourselves how DNS works:
1. Your computer needs to find the IP address for “abc.com”.
2. It asks a local DNS server, which then:
- Contacts multiple intermediate DNS servers
- Finally reaches the authoritative DNS server for “abc.com”
3. The DNS server responds with the IP address and a Time-To-Live (TTL) value.
4. The TTL tells systems how long your computer should remember this information.
Traditionally, DNS caching is handled at the operating system level, respecting the TTL values. This system has worked well for decades – which makes the browser manufacturers' decision to implement their own caching layer a bit surprising.
The DNS Caching Problem
Browser manufacturers – specifically Microsoft and Google – want to maximise browser performance. Modern webpages often have hundreds of images or scripts each of which may be located at a new URL. Calling the Operating System DNS lookup API can take quite a few milliseconds and over the whole page it adds-up. So browser manufacturers added an internal DNS cache to their products.
If a user loads a page which references ‘acme.com’, the browser will do a normal DNS lookup for acme.com, and then store this in its cache. If the browser loads something else from acme.com, it reads the IP from the cache and doesn't even ask the OS.
The cache delay – how long it takes before acme.com is removed from the cache – appears to be about a minute. We’ve not found any setting for this in Chrome (we didn’t check Edge).
The downside of this behaviour is that if some user updates a DNS entry to point to a new IP, but the domain is already in the cache, the browser won't see the change.
Moreover, the browser doesn't use the TTL value – which specifies how long things should stay in cache - because operating systems don't have an API to read this value! Even if Google or Microsoft wanted to, they can’t read the TTL.
The final kicker: if a user presses refresh, it seems the browser cache delay is reset. So if a user is sitting on their browser, pressing refresh waiting for a site to load – it never will! They need to stop pressing refresh, wait a minute and then refresh again!
Practical Solutions to Browser DNS Caching
If you find yourself in a situation where browser DNS caching is affecting your work, here are some strategies:
- Use incognito/private browsing mode (sometimes works as it's like a new browser instance).
- Restart the browser.
- Use command-line tools like dig or nslookup to verify DNS changes.
- Close all tabs for the domain in question and wait for a short period before reopening (this may resolve the issue, as background processes might automatically terminate after a period of inactivity).
- Manually clear the browser DNS cache:
- Firefox: go to about:networking#dns
- Chrome/Edge/Chromium based: go to chrome://net-internals/#dns
- Then click the clear cache button.
The best fix we've found is to create variant domains/URLs – but this only works in our specific case. If the site in question is “acme.blueberrytest.com”, we also register “acme#.blueberrytest.com” and point this to the target IP. We change “acme.blueberrytest.com” to point to the target IP – but then load the other version to force the browser to re-read the DNS.
Conclusion
This is obviously quite a specialised problem – clearly the browser developers decided that the performance gain was worth annoying a few crazy people who like to change DNS!
So far, we've not found any very good solutions. We've found very little documentation on this "feature" of modern browsers. We do find it a bit annoying that Chrome and Edge don't have any internal diagnostic screen or a settings screen to control how this works. Ideally there would a Chrome JS API to allow us to clear the cache – that would solve it completely.
While the performance benefits of browser DNS caching are clear, it can create unexpected challenges for developers and system administrators who need to be aware of its existence.
Remember, for most standard web applications and typical internet usage, OS-level and DNS server caching still play the primary role in DNS resolution behaviour. Always consider the entire DNS resolution chain, including this browser-specific behaviour, when planning for and implementing DNS changes.