Task 1 Ye Ol' Search Engine
Roger dodger!
Task 2 Let's Learn About Crawlers
Name the key term of what a "Crawler" is used to do
What is an example of the type of contents that could be gathered from a website?
Task 3 Enter: Search Engine Optimisation
Use the same SEO checkup tool and other online alternatives to see how their results compare for and
Task 4 Beepboop - Robots.txt
Where would "robots.txt" be located on the domain "ablog.com"
If a website was to have a sitemap, where would that be located?
How would we only allow "Bingbot" to index the website?
How would we prevent a "Crawler" from indexing the directory "/dont-index-me/"?
What is the extension of a Unix/Linux system configuration file that we might want to hide from "Crawlers"?
Task 5 Sitemaps
The easier a website is to "Crawl", the more optimised it is for the "Search Engine"
What is the typical file structure of a "Sitemap"?
What real life example can "Sitemaps" be compared to?
Name the keyword for the path taken for content on a website
Task 6 What is Google Dorking?
site:bbc.co.uk flood defences
What term would you use to search by file type?
What term can we use to look for login pages?