Accessibility / Findability Issues
| Index in Google | Search google with site:sitename.com |
| Crawl vs. Index Status | Need google webmaster |
| Server Uptime | Check status on https://www.uptrends.com/tools/uptime |
| Robots.txt | Check status on http://sitename.com/robots.txt |
| Use of Meta Robots Tag | Go to screaming frog tool and run a scan. In the ‘internal all’ report, scroll to the right to get the column named ‘Meta Robots 1’. Scroll down and check if any pages are marked as ‘no index’. Check if the marked pages are good and if so, why they have been ‘no-indexed’. Mark has fail if there’s a high proportion of pages no-indexed. |
| Meta Refresh | Use Screaming frog internal_all report to check the meta refresh column. It should be blank for all pages. |
| Sitemap html | http://www.gatherby.org/sitemap.html & without html |
| Blocked By Robots.txt, Blocked Resources | Screaming frog > Response Code
|
| No response | Should ideally be 0. |
| Success 2xx |
|
| Redirections 3xx , 4xx , 5xx |
|
Page XML Sitemaps
| Found Page XML Sitemaps? | Check if www.website.com/sitemap.xml exists. Yes – pass, no – fail Then check https://www.xml-sitemaps.com/validate-xml-site and http://freetools.webmasterworld.com/tools/site-validator/map.html by entering website.com. If valid, pass, else fail. |
| Page XML Sitemap(s) Listed in Robots.txt | Open the robots.txt file and check if the sitemap has been listed in it. |
| Page XML Sitemap with Broken Links | Check your XML Sitemap for errors with Screaming Frog Download and save your xml sitemap Sort on Status code. If there are 4xx, 5xx, it’s a fail, else pass. |
Mobile XML Sitemaps
| Found Mobile Pages? | Check if the site has a separate mobile version like http://m.website.com. (Most websites now are responsive and don’t have a separate mobile site, in which case, mark as NA and the below checks can be skipped) |
| Found Mobile XML Sitemaps | If there is a mobile version of the site, there should be a separate xml site map of the form http://m.website.com/sitemap.xml. Check if it exists manually and validate using https://www.xml-sitemaps.com/validate-xml-site and http://freetools.webmasterworld.com/tools/site-validator/map.html |
| Mobile XML Sitemap(s) Listed in Robots.txt | Open the robots.txt file and check if the mobile sitemap has been listed in it |
| Mobile XML Sitemap(s) with Broken Links | Check the mobile Sitemap for errors with Screaming Frog Download and save your xml sitemap Sort on Status code. If there are 4xx, 5xx, it’s a fail, else pass. |
Video XML Sitemaps
| Found Videos? | If the site has videos, mark as pass, else NA and skip the following rows in this section |
| Found Video XML Sitemaps | A video sitemap helps search engines to quickly identify video content and index it for video searches. |
| Video XML Sitemap(s) Listed in Robots.txt | Visit yoursite.com/sitemap_video.xml – check if one exists. |
| Video XML Sitemap(s) with Broken Links | Check the video sitemap using the screaming frog tool as mentioned in the page XML sitemap section. |
Image XML Sitemaps
| Found Images? | Check if the site has images. (Most websites do) |
| Found Images XML Sitemaps | Check if https://gatherby.org/sitemap_image.xml is found. If yes, pass, else fail. |
| Image XML Sitemap(s) Listed in Robots.txt | If the image sitemap exists, check if it is listed in the robots.txt. |
| Image XML Sitemap(s) with Broken Links | Run screaming frog check for the image sitemap and see if there are broken links. |
Pagination
| Pagination Present? | Check if the site has prev and next buttons on each page. Many blog sites have this, when a large article is broken into multiple pages.This also applies to eCommerce sites with multiple product pages and long guides that is broken down into multiple pages. |
| View-All page Available | Make sure view-all page is quick to load. User can easily navigate and find relevant information. Whenever a view all page is present for paginated content google makes effort to index the view-all page and google will also consolidate indexing properties to the view-all page. |
| Use of session IDs or other unique identifiers/parameters | Check what happens when you more across paginated content. <link rel=”next” href=”page2&sid=9876” />. The sid should be the same. Keeping session id consistent you can tell google what paginated content belongs together |
| Use of splash or gateway pages | Check if there is a splash page which loads before the home page is displayed. |
Subdomains
| Found subdomains?: | Check if the website has subdomains like http://subdomain.website.com. Go to https://pentest-tools.com/information-gathering/find-subdomains-of-domain and enter the website URL into the box. If there are subdomains, they will be listed. Ignore the email.website.com, smtp.website.com etc which are used for emails. |
| Sub-domains have robots.txt | If there is a separate sub-domain, it should have its own robots.txt – http://subdomain.website.com/robots.txt |
| Sub-domains have sitemaps | If there is a separate sub-domain, it should have its own robots.txt – http://subdomain.website.com/sitemap.xml |
| Sub-domain indexing | Google for site:subdomain.website.com and see if there are any pages returned. If the subdomain is intentionally created, like a blog which you want to be indexed and you can’t see any results, mark as fail, else pass. However, if it is an old version of the website which you don’t want Google to index, then mark pass if it doesn’t return any results. |
Breadcrumbs
| Breadcrumbs Found and Available in All The Pages | A trail, or secondary navigation, clearly visible to website users to help them navigate your website. Check for items in the submenu. Is it showing something like Home> Blog> How to check for Breadcrumbs for sub topics? |
| Breadcrumb Visual Representation (last trail path NOT clickable) | If the breadcrumb path is visible, check if the individual items are clickable and if you are able to go back to the previous step in the path. |
Architecture Issues
| Site accessible without Javascript (disable JS) | Disable javascript on the browser and check if the site is loading normally. Navigate through the site. If everything functions as usual, mark as pass, else fail. |
| Breadcrumbs NOT in JS | The breadcrumb path should work normally even if javascript is disabled. Check a few links after disabling javascript on browser. |
| Links coded in HTML (NOT JavaScript) | Turn off javascript and try clicking links on the site. They should work as usual. |
| TLM Reflects Order in Terms of Relevance of Content | Whether every page has a link from the home page via Top level menu. Check some random URLs obtained after doing a Screaming Frog crawl. |
| Footer optimization | Check if the page footer has items like FAQ, locations page, privacy policy, careers, HTML sitemap and other pages important pages that search engines look for to determine the quality of your website. |
| Site Directory Structure | Do a screaming frog scan, choose a few random URLs and see how far they are from the home page. Ideally, they should be within 4 clicks from the home page. |
URL Issues
| Hyphens used as default delimiter in URLs | Run screaming frog crawl and check the URLs in the list for delimiters other than -. |
| Overall URL friendliness (short and easy to share) | Check the same URL list on screaming frog and see if the URLs are short and easy to share. |
| Overall URL optimization (usage of target keywords) | Check the screaming frog URL list to see if all URLs have meaningful slugs and if you are able to make out what the page is about, just by looking at the URL. Do a quick scan of the list. |
| Absolute vs. Relative URL References in Source Code | On any given page on the website, right click and select “view source code”. Find an internal link. If it’s coded as <a href = “http://yoursite.com/link/” > anchor</a>, it’s an absolute link.. Absolute URLs are better from SEO point of view but relative are better from page load time point of view. |
On-Page Tagging/Markup Issues
| Page Source META tags (ie.: <title>, meta description in head) | Check the overview box on the right side of screaming frog. Under page title, the ‘missing’ should be 0% and under meta description, the missing should be 0%. |
<title> tag issues
| Missing <title> | Check Screaming frog results box on the right side. Missing percentage should be 0 for it to Pass. 1-10% – Ok. Above 10% – fail. |
| Duplicate <title> | Duplicate under title should be 0 for pass.1-10% – Ok. Above 10% – fail. |
| Over 65 characters | 0 for pass.1-10% – Ok. Above 10% – fail. |
| Below 30 characters | 0 for pass.1-10% – Ok. Above 10% – fail. |
| Over 571 pixels | 0 for pass.1-10% – Ok. Above 10% – fail. |
| Below 200 pixels | 0 for pass.1-10% – Ok. Above 10% – fail. |
| Same as H1 | The H1 and Title should ideally be the same but it cannot be strictly enforced. One can make use of 2 variations of a keyword for better SEO (for eg. ‘Digital Marketing’ and ‘Online Marketing’. However, you need to make sure these are not completely different as it can lead to poor user experience and SEO penalty. Screaming frog tool will only check exact matches. If this score is above 70% it can be marked as a pass. Between 50-70% is ok and below 50% needs to be investigated by checking the title and H1 of each non-matching page individually and if the variation is only a little (keyword synonyms used for SEO benefit) they can be manually marked and the percentage of matches can be calculated manually. |
| Multiple | Pages which have more than one <title> tag. This has to be 0% for pass; >0% but < 15% – OK; > 15% – Fail |
<meta description> issues
| Missing Meta Description | 0% – pass; >0% but < 15% – OK; > 15% – Fail |
| Duplicate Meta Descriptions | 0% – pass; >0% but < 15% – OK; > 15% – Fail |
| Over 156 characters | 0% – pass; >0% but < 15% – OK; > 15% – Fail |
| Below 70 characters | 0% – pass; >0% but < 15% – OK; > 15% – Fail |
| Over 928 pixels | 0% – pass; >0% but < 15% – OK; > 15% – Fail |
| Below 400 pixels | 0% – pass; >0% but < 15% – OK; > 15% – Fail |
The meta description has to be between 70-156 characters in length for best result. Below 70 characters is bad because it is an opportunity to showcase your page to search engine visitors and entice them to click the link to your page. Above 156 is bad because search engines will truncate the characters.
The pixel length is another way of looking at it because some characters are wider than the others, eg ‘l’ takes up lesser number of pixels than ‘w’.
Page Headings (H1-H6))
Check screaming frog results in the right side box.
| Missing H1 Heading | 0% – pass; >0% but < 15% – OK; > 15% – Fail |
| Duplicate H1 Headings | 0% – pass; >0% but < 15% – OK; > 15% – Fail |
| Over 70 characters | 0% – pass; >0% but < 15% – OK; > 15% – Fail |
| Multiple H1 | 0% – pass; >0% but < 15% – OK; > 15% – Fail |
| Missing H2 | 0-20%- pass; 20-50% – OK, >50% fail (It’s ok to have missing H2 tags but not for a large proportion of pages. H2 tags are used when a page is logically split into sections. If most of the pages have a missing H2 tag, it could mean that the content is not split into sections and is written as one big paragraph, which makes it less readable.) |
| Over 70 characters | 0% – pass; >0% but < 15% – OK; > 15% – Fail |
Structured Data
| Use of Structured Datamarkup | Go to https://search.google.com/structured-data/testing-tool and enter the URL of the site. Check if there is any data on the right side box. If yes, pass, else fail. |
Main Body Content
| Key Content Location | Choose a few random pages, right click and select click view source. Check the location of H1, H2 tags. They should be near the top of the pages. Mark as pass if the header tags and main content of page are near the top. |
| Rich Semantic Content (NOT thin) | Open Screaming Frog and enter the domain. Scroll over to the right to find “Word Count” and look for pages with less than 300 words (outside of trivial pages ie. contact, about). (Consider downloading the internal_all file and checking thru excel) It should ideally be less than 20%. |
| Expired Content | Manually check the pages – blog, news etc to see if there is any time-related content and if there are very old articles – dated 3-4 years back. |
SEO Video
| Found Videos? | Check if the site has any videos posted |
| Video XML Sitemap | If there are videos, there has to be a video XML sitemap |
YouTube Optimization
Applies to sites which have official YouTube channel.
| Average Optimization Score | |
| Channel Optimization: |
Domain hosted Videos
Applies when the videos are hosted on the same domain.
| Video files in one directory | |
| Keyword in the filename and URL |
SEO Images
Run screaming frog scan. Check the summary section on the right. Check the ‘Images’ section.
| Image File Size (>100 kb) | Larger image sizes slow load time on a page. Slow load time leads to bad user experience having to wait for images to render. Mark as fail if >20% are over 100kb, OK if less than 20% is over 100kb. Pass for 0% |
| Missing Alt Text | Check screaming frog result under images for the Missing Alt Text value 0 – pass; <20% is ok. > 20% – fail |
| Alt Text over 100 Characters | Check screaming frog result under images 0 – pass; <20% is ok. > 20% – fail |
International SEO
Applies to international websites
| Client’s business targets: multiple countries or languages? | |
| Use of Multiple ccTLDS | |
| Use of subfolders or subdomains (within one ccTLD) | |
| Language Markup | |
| Language set | |
| Alternate Tags (Rel=”Alternate” Hreflang=”x”) | |
| Alternate language HREFLANG directive set in head of Source Code | |
| Alternate language directive set in page XML sitemap. |
Potential Triggers of Duplicate Content
| Potential Trigger: www vs. non-www | Open the www version and non-www version of the site and a few of the pages. Either all should redirect to www or non-www. There shouldn’t be a situation where some pages load as www and some load as non-www and there are multiple versions of the same page – one on www and one on non-www. Screaming frog scan can reveal this, if the problem exists. Mark as pass if there are no two version and fail if there are two versions |
| Potential Trigger: htttp vs. https | Check if the site has both http and non http pages. If yes, there is a potential duplicate content issue and mark as fail. If there is only one version, mark as pass. |
| Potential Trigger: Redirected Index File Names | Check if the home page is being redirected to some other page. (Screaming frog will show any 301/302 redirects if they exist; ignore the redirections to www or non www versions). If there are redirections, mark as fail. Otherwise, pass. |
| Potential Trigger: Redirected Slash and Non Slash URLS (ie.: www.example.com/sample/ vs. www.example.com/sample) | A quick scan of the internal_all file downloaded from screaming frog will reveal if there are multiple versions of the same page with ‘/’ and without ‘/’. Download the file and sort on the first column. The same URLs with / and without / will appear adjacent to each other. If there are 2 versions, mark as fail, else mark as pass. |
| Potential Trigger: Uppercase/Lower Characters Automatically Fixed | Select a few random URLs on the site from screaming frog internal_all spreadsheet. Change some random characters in the URL to upper case and check what happens. If the pages load fine, pass. Else fail. |
| Other Duplicate Content Found Within the Same Domain | Check if there is any chance of duplicate content by submitting the URL to http://siteliner.com/. If there are any duplicates listed, examine them and see if they are just snippets and headers/footers or any main content has been duplicated. |
| Other Subdomains or Client’s Domains | Check if the page has subdomains where the content can be replicated, for example www.website.com/subpage and subpage.website.com Mark as pass if there are no subdomains and fail if there are subdomains and the possibility exists. |
| Duplicate Content Handling Use of the Rel=Canonical Tag: | If there is duplicate content as shown in siteliner, check if there is a redirect set up from one page to the other. Pass if it is set, fail if not set and NA if there is no duplicate content. |
| Duplicate Content Handling : Within Pages in Domain | If there is any other method of handling duplicate content, mention pass. |
Social Issues
| Google + account: | Pass if the page has a G+ page associated.Type in the name, phone and google plus in the Google search bar. |
| Implemented rel=”author” | Shows a rich snippet of the author in the SERP. Attributes content on a page or post to a particular author (i.e. digital signature). Currently Google has suspended authorship markup; however may bring it back in future. Since this feature isn’t in use now, mark as NA and move on. |
| Social Sharing: | Check if the articles have FB/Twitter/G+ sharing links which allow visitors to post the page link onto their own social media pages. Mark pass if yes and fail if no. |
| Open Graph Snippet: Facebook (custom search in SF for meta property=”og:”) | Check the source code. Type in the meta tags “og” manually. If tags like these exist, mark pass, else fail. meta property=”og:title” content=”Gather By MGO 1,069 Bioactive Honey”/> |
| Does have OG directive? | |
| Does have Facebook sharing button | Check if articles have the Facebook sharing button. If yes, mark pass, else fail. |
| Not multiple | |
| Editorial optimized | |
| Twitter Card Snippet (searched “<custom search in SF for meta name=”twitter:”): | Manually check for the meta name “twitter” in the source code. Mark pass if it’s present, and fail if not present. |
| Does have Twitter directive? | |
| Does have Twitter sharing buttons | Check if there are twitter sharing buttons, mark pass if they are present and fail if they are not. |
| Not multiple | |
| Optimized for social sharing |
Page Speed
Go to https://developers.google.com/speed/pagespeed/insights/ and enter URL. Pass if the test is in green, fail if Red and Ok if Amber
| Google Page Speed Insights (desktop) – home page, landing page | Pass if the test is in green, fail if Red and Ok if Amber |
| Google Page Speed Insights (mobile) – home page, landing page | Pass if the test is in green, fail if Red and Ok if Amber |
| Speed test using GTMetrix – home page, landing page (PageSpeed) | |
| Speed test using GTMetrix – home page, landing page (YSlow Score) |
Protocols
Check Screaming frog results. It will give the percentage of pages which are http and https.
| HTTP | |
| HTTPS |
Crawler Tools
| Xenu Crawler |
|
| Screaming Frog Crawler |
|