Check your site: five tests you can run yourself

All you need is a browser, and for tests 3 and 4 a Google or Microsoft account. For each test: how to run it, what the result tells you, and what it doesn't.

1. Is the door open?

Type yoursite.com/robots.txt into your browser. You'll see your robots.txt, the file that tells crawlers which pages they may visit (Google).

What it tells you: which crawlers you've asked to stay away. Compare the names with the list on AI crawlers. No file at all? Google says "Not having a robots.txt file is fine, and means that Google can crawl all URLs on your site" (Google).

What it doesn't: whether your host blocks crawlers before they ever reach the file. If your site runs on Cloudflare, check its Security Settings too: there, "Block" stops Googlebot, Bingbot and Applebot as well (Cloudflare).

Want your assistant to make these fixes? That is what Croftweaver does.

2. How fast is it?

Open pagespeed.web.dev, enter your address, and run it three times.

What it tells you: how your page loads on a phone and a computer, with suggestions. Google explains two kinds of result: lab data from "a simulated load of a page on a single device", and field data from real visitors over "the previous 28-day collection period" (Google, Oct 2024).

What it doesn't: give you one true number. Google says the score can change "from run to run" without any change to your page, which is why we suggest three runs and reading the spread. A new page may have no real-visitor data yet (Google, Oct 2024). And "having good lab data does not necessarily mean real-user experiences will also be good" (Google, Oct 2024).

Want your assistant to make these fixes? That is what Croftweaver does.

3. Does Google know your pages?

Set up Search Console, "a free service offered by Google" (Google). Add your site and prove it's yours. If you use WordPress, Wix or Squarespace, Google says your platform "might provide a special plugin or setting to verify your property easily, or even automatically" (Google). Adding it "does not affect your website", and data "should begin to appear in your property in a few days" (Google).

Then type a page's address into the URL Inspection bar at the top.

What it tells you: whether Google has that page in its index. "URL is not on Google means that the URL can't appear in Search results" (Google). Since 31 August 2026 it also holds the switch for Google's AI answers, the Search generative AI control (Google).

What it doesn't: promise a place in results. Google warns: "URL is on Google" doesn't actually guarantee that your page will appear in Search results, and the inspection shows the last indexed version, "not the live version on the web" (Google).

Want your assistant to make these fixes? That is what Croftweaver does.

4. How does Bing, and its AI, see you?

Open a "free Bing Webmaster Tools account" at bing.com/toolbox/webmaster. You can sign in with a Microsoft, Google or Facebook account (Microsoft). If you already use Search Console, you can import your site from there and it gets "automatically verified". Know that Bing will then "periodically validate the site ownership status by accessing your Search Console account" (Microsoft).

What it tells you: besides Bing search, an AI Performance report: how often your pages are shown as sources in Copilot and Bing's AI answers (Bing, Feb 2026). Data "usually takes 48 hours" to appear (Microsoft).

What it doesn't: tell you how important a page is. Bing says the report "reflects how often pages are cited, not page importance, ranking, or placement" (Bing, Feb 2026).

Want your assistant to make these fixes? That is what Croftweaver does.

5. Ask your AI assistant, and make it show its sources

Paste this into ChatGPT, Claude or Gemini:

Please review my website [your address] for how easy it is to find on Google and in AI answers. List the five most important improvements. For each one, name and link the official source (documentation from Google, Bing, OpenAI, Anthropic or Perplexity) it is based on. If there is no official source, say so.

Then ask the same thing a second time, or in a second assistant, and compare.

What it tells you: ideas worth checking. The sources let you check each one, which is one of the two marks of good advice Google names: it "backs up their claims by citing official Google Search guidance" (Google, Jun 2026).

What it doesn't: give you a verdict you can trust without checking. OpenAI lists "Fabricated quotes, studies, citations or references to non-existent sources" among ChatGPT's possible mistakes (OpenAI), and Anthropic says "users should review Claude's cited sources" (Anthropic). Open every link it gives you. And if it gives you a score, see the section about scores on the E-E-A-T page (E-E-A-T).

Want your assistant to make these fixes? That is what Croftweaver does.

For one list that points to these tests, and to the pages behind them, see the checklist.

Checked on 3 November 2026.