Google Dorking

Task 1 Ye Ol' Search Engine

Roger dodger!

No answer needed

Task 2 Let's Learn About Crawlers

Name the key term of what a "Crawler" is used to do

index

What is the name of the technique that "Search Engines" use to retrieve this information about websites?

crawling

What is an example of the type of contents that could be gathered from a website?

keywords

Task 3 Enter: Search Engine Optimisation

Use the same SEO checkup tool and other online alternatives to see how their results compare for https://tryhackme.com and http://googledorking.cmnatic.co.uk

No answer needed

Task 4 Beepboop - Robots.txt

Where would "robots.txt" be located on the domain "ablog.com"

ablog.com/robots.txt

If a website was to have a sitemap, where would that be located?

/sitemap.xml

How would we only allow "Bingbot" to index the website?

User-Agent: Bingbot

How would we prevent a "Crawler" from indexing the directory "/dont-index-me/"?

Disallow:/dont-index-me/

What is the extension of a Unix/Linux system configuration file that we might want to hide from "Crawlers"?

.conf

Task 5 Sitemaps

The easier a website is to "Crawl", the more optimised it is for the "Search Engine"

What is the typical file structure of a "Sitemap"?

xml

What real life example can "Sitemaps" be compared to?

map

Name the keyword for the path taken for content on a website

route

Task 6 What is Google Dorking?

What would be the format used to query the site bbc.co.uk about flood defences

site:bbc.co.uk flood defences

What term would you use to search by file type?

filetype:

What term can we use to look for login pages?

intitle: login