I get the basic working of a web crawler. They start at a page and index it, then follow all its links and index those, and so on. Eventually they have a good enough representation of the entire internet.
Recently I made a new web page on a new domain, and soon enough it showed up in a Google search. How did Google know the address of my site (or any other one) to begin with? Is there an actual directory of every site on the internet? Or does it try random combinations of domain names?
In: 2
if your domain is www.IMovedYourCheese.com, then the fact of its registration is public (it is part of DNS database), so google got it from there.
if your domain is www.IMovedYourCheese.provider.com, then provider.com probably tells google about new domains that it created, or actually has a page listing all www.*.provider.com domains.
Latest Answers