NEWAI content checker: see which pages read like AI text

Knowledge baseUptime

Website uptime monitoring

For an agency, uptime monitoring is about hearing that a client's site is down before the client does, and knowing which alarms to believe. The monitor asks for the site from the outside on a schedule and tells you when it stops answering. In Google, a few hours of downtime leave no mark; days of errors do.

Updated Nov 21, 20269 min read

How does an uptime monitor decide a site is down?

A monitor requests the page the way a visitor's browser would and reads the status code and how long the answer took. No answer, a failed connection, a timeout or a 5xx server error counts as down, and Google's crawlers treat timeouts and DNS errors like server errors too. One failed check isn't enough for a good monitor. It waits for a second round or asks from a second location, because one miss can be a hiccup on the checker's side. Two checks that must agree is what you want; a dozen locations add little for a client site.

Some answers look like trouble without being an outage. A 403 Forbidden, a 429 Too Many Requests or a firewall's challenge page means something stood between the checker and the site, while real visitors may get in fine. If the homepage answers 404, that's a different problem, but still one to fix.

Even a 200 doesn't prove much. Back in 2011 Google's Search Central blog wrote about "a lot of problems with sites that return 200 (OK) result codes for server errors, downtime, bandwidth-overruns or for temporary placeholder pages". Read only the status code and such a page counts as up. A keyword check, which looks for a word that should be on the page, catches it.

How often to check depends on what you want to catch. A one-minute check sees the server that drops out for a few minutes and comes back by itself, which matters for a webshop that loses orders by the minute. Google shrugs at those. Gary Illyes said in 2024, as Search Engine Journal reported it, that "10-15 minutes every now and then is not 'extended' by any means, so you should be fine." For search, a check every 30 minutes catches every outage that matters.

Why does the monitor say the site is down when it isn't?

Most of the time the site is fine and something in front of it refused the checker: the client's firewall, a CDN's bot protection, a rate limit or a security plugin. The third "site is down" mail of the week arrives, you open the site, and it loads. One site owner on a hosting forum summed it up: "When I use Cloudflare proxy, I get a lot of false downtime alerts. When I turn off proxy, they stop."

A monitor that cries wolf is worse than no monitor: after the third false alarm nobody looks anymore. Often it's the client's own firewall saying no to the checker. Let it through with a header rule, never with a list of IP addresses.

I wanted to know how common that is, so I looked at the business sites in my account, mostly small and mid-size Dutch companies and webshops, scanned in November 2026. One in five refused our scanner on part of their pages with a 403 or a 429, on most of them a fifth to three quarters of the site. And one in four refuses at least one AI bot outright. A scan sends hundreds of requests in a row and a monitor one every half hour, so a rate limit bites a crawl harder. But the firewall making the call is the same one.

That firewall decides per visitor. Bot rules, rate limits, challenge pages, country blocks and security plugins can all let people in and turn an automated visitor away, and none of them knows your checker is a friend. So a monitor that counts every refusal as down keeps sending alarms about a server that is fine. One that shows a refusal as blocked tells you to look at a firewall rule before you call the host.

It matters beyond the monitor too. In its post on CDNs and crawling, Google warns that "occasionally the bots that you do want on your site may end up in your CDN's blocklist, typically in their Web Application Firewall (WAF)", which can keep a site out of search results. The reverse happens too: some CDNs keep serving the site while the server behind them is down.

The fix is a rule. Give the checker a secret value in a request header, and add a firewall rule that lets requests with that value through: Skip in Cloudflare, Allow in AWS WAF. Put that rule at the top, so no other rule fires first. Bot Fight Mode on Cloudflare's free plan is the exception, because no rule can skip it.

MarketingTracer's Uptime monitor shows a refusal as its own state, Blocked, apart from Down, in the incident list and in its alerts. A blocked check doesn't lower the uptime percentage. Its checks say who they are in the user agent and carry a secret header value meant for that one site. In the site settings, "Let MarketingTracer through your firewall" has ready-made rules for Cloudflare and AWS WAF, and our bot page explains how it works.

The Uptime page: status, uptime over 7 and 30 days, hourly bars with one bad night, and incidents marked Down and Blocked

Demo account, sample data.

Does an outage hurt your rankings, and how long is too long?

An outage hurts rankings only when the errors last more than a day or two.

Sites go down more often than people think, often at night when heavy scheduled jobs run. An hour at night costs you nothing in Google; a day of errors does, because then pages start to drop out. So when traffic dips, the first thing I check is the incidents: was it down, and for how long?

Google's documentation backs up the calm part. Server errors and 429s make its crawlers slow down, indexed pages stay for now, and once the server answers normally Google "gradually increases the crawl rate for the site".

The line sits at about two days. Google's page on reducing the crawl rate warns against errors "longer than 1-2 days", because "if Googlebot observes these status codes on the same URL for multiple days, the URL may be dropped from Google's index." How crawl rate works is in the Googlebot article.

Nor is there a penalty afterwards. During the Cloudflare outage of November 2025, John Mueller put it on Bluesky as "5xx = Google crawling slows down, but it'll ramp back up," and pages that did drop out "will pop back in fairly quickly".

One file breaks this calm. When robots.txt answers with a server error, Google stops crawling the whole site for the first 12 hours and then works from the last good version for up to 30 days, while every page may load fine. Your homepage monitor won't notice. A 404 on robots.txt is harmless: Google treats that as no file at all.

In minutes: a 30-day month has 43,200 of them, so 99.9% uptime allows about 43 minutes of downtime. A monitor that checks every 30 minutes and wants two failed checks in a row calls an outage 30 to 60 minutes after it starts, so anything shorter than half an hour never sets off an alert and anything over an hour always does. Google's "1-2 days" is 1,440 to 2,880 minutes: 48 to 96 checks, or 33 to 67 times the whole 99.9% allowance of a month. Your 30-minute monitor has told you long before Google starts dropping pages.

Planned maintenance is the outage you control. For a day or two, serve an informational page with a 503 status code and a Retry-After header. Never a 403, 404 or 410, which "will remove the website's URLs from Google Search", never a 200 page saying you'll be back, and never a 503 on robots.txt, "because this blocks all crawling."

After an outage, Search Console shows what Google ran into. Settings > Crawl stats > Host status "describes whether or not Google encountered availability issues when trying to crawl your site", with a check each for robots.txt fetching, DNS resolution and server connectivity. Next to the incident list, that tells you whether the outage cost anything.

Search Console Crawl stats, Host status: robots.txt fetching, DNS resolution and server connectivity

Search Console, names changed.

How do you watch uptime on all your client sites?

One monitor for every client homepage, the checker let through each firewall, the alert to you first, and the month in the report. With any decent monitor:

  1. Pick a monitor that confirms an outage before it alarms and shows a refusal apart from an outage. Every 30 minutes is enough for search.
  2. Add every client homepage in one place, so you don't log in to a separate tool per client.
  3. Let the checker through each client's firewall with a header rule, before the first false alarm.
  4. Send the alert to the agency first, right away. The client gets it too only if they ask.
  5. Look at the incident list once a week. An outage that fixed itself still has a cause, and a blocked check is a firewall rule to fix.
  6. Put the month in the client report: availability, the incidents and how long they lasted, and what you did. Read it before it goes out.

Step four is the one people skip, until a developer opens a forum question with "this morning I had a call from an angry client complaining that his site was down."

A client should never be the one who tells you the site is down. I want the alert first, fix it or call the host, and then tell the client what happened. In the monthly report it's one line: was it up, if not for how long, and what we did about it.

In MarketingTracer the Uptime monitor sits next to each site's SEO data: one click per site ("Turn on uptime monitor"), in every plan at no extra cost. The "Site is down" alert comes after two failed checks in a row and again when the site is back, by email or to a Slack channel, right away or every morning, per person and site. The client report gets a section "Availability and incidents", white label like the rest of the app. It checks the homepage only, every 30 minutes, with no status page and no keyword check, so a checkout needs a faster monitor next to it.

The report section Availability and incidents: availability for the month and one short incident

Demo account, sample data.

What should you watch on a client site besides "is it up"?

Whether it's blocked or down, how fast it answers, the certificate, robots.txt, and what a homepage check can't see. Per client site:

  • Status, and since when. The incident list shows when it started, how long it lasted and what happened. A 403, a 429 or a challenge page is the firewall; a 5xx or a timeout is the server, and each needs a different person to fix it. An outage at the same hour every night is a scheduled job, and it will come back until someone moves it.
  • Response time. A server that slows down under load is the next one to throw errors. About four in ten of those business sites answered a few pages with a server error or a timeout while our scanner crawled them, mostly a handful in a thousand. Watch the trend; what the number means is in time to first byte.
  • The certificate's lifetime. Since 15 March 2026 a new public certificate lasts at most 200 days, from March 2027 at most 100 and from March 2029 at most 47. You see the expiry date in the browser and in your monitor, and Search Console's HTTPS report flags an invalid certificate.
  • The certificate's renewal. Nine in ten business sites in my account already run on 90-day certificates that renew themselves, and none was within a month of expiring. What goes wrong now is a renewal that quietly stops, and Let's Encrypt ended its expiry emails in June 2025. No Google source says an expired certificate costs rankings; it costs the visitor who sees "Your connection is not private" and turns back.
  • robots.txt. It should answer with a 200, or a 404 when there's no file, and never with a server error or a timeout. Open /robots.txt yourself or look at robots.txt fetching under Host status; a site crawl checks that it's there.
  • What a homepage check can't see. The checkout, the forms, a broken product page, a hacked page that still answers 200: a homepage monitor calls them all up. A shop's checkout wants its own check in a second tool, one that runs through a purchase or looks for a word on the page.

Free 14-day trial

MarketingTracer smart A.I. tool shows you what you need to rank. Then we'll help you get it done.

  • No credit card required
  • No installation needed
  • No strings attached
Start your 14-day trial