.Google's John Mueller responded to a concern about why Google marks web pages that are prohibited from creeping through robots.txt and why the it's safe to neglect the relevant Browse Console reports about those creeps.Robot Website Traffic To Concern Parameter URLs.The individual talking to the inquiry documented that bots were actually making links to non-existent query parameter Links (? q= xyz) to webpages along with noindex meta tags that are actually additionally blocked in robots.txt. What cued the inquiry is that Google.com is crawling the web links to those web pages, obtaining blocked out through robots.txt (without noticing a noindex robotics meta tag) at that point obtaining turned up in Google Look Console as "Indexed, though obstructed through robots.txt.".The individual talked to the observing concern:." Yet right here is actually the major question: why would certainly Google.com index webpages when they can not also view the content? What is actually the perk during that?".Google.com's John Mueller affirmed that if they can't crawl the web page they can't see the noindex meta tag. He also creates an exciting mention of the web site: search operator, recommending to overlook the outcomes considering that the "normal" customers won't view those results.He created:." Yes, you're appropriate: if our company can not creep the web page, we can't find the noindex. That said, if our team can't crawl the web pages, after that there's certainly not a great deal for us to index. Thus while you may find a few of those pages along with a targeted internet site:- query, the common customer won't see them, so I would not bother it. Noindex is also great (without robots.txt disallow), it just indicates the Links are going to find yourself being actually crept (as well as wind up in the Look Console document for crawled/not recorded-- neither of these standings result in concerns to the remainder of the website). The vital part is that you don't make all of them crawlable + indexable.".Takeaways:.1. Mueller's solution validates the limits in operation the Internet site: hunt evolved search driver for diagnostic explanations. One of those reasons is since it is actually not attached to the normal search index, it is actually a distinct thing completely.Google's John Mueller commented on the site hunt operator in 2021:." The brief response is that a web site: inquiry is actually certainly not indicated to be total, nor made use of for diagnostics reasons.An internet site question is a particular kind of search that limits the outcomes to a specific web site. It's essentially just the word website, a bowel, and then the web site's domain.This inquiry limits the outcomes to a certain web site. It is actually certainly not implied to be a thorough collection of all the webpages coming from that website.".2. Noindex tag without utilizing a robots.txt is great for these kinds of scenarios where a crawler is actually linking to non-existent pages that are actually acquiring found by Googlebot.3. URLs along with the noindex tag are going to produce a "crawled/not listed" item in Look Console and that those won't possess an adverse impact on the rest of the website.Read through the question as well as address on LinkedIn:.Why would certainly Google mark pages when they can not also observe the content?Included Photo by Shutterstock/Krakenimages. com.