Screaming Frog: Crawling a SF site fully generated by JS

by Vecta   Last Updated May 16, 2018 14:04 PM

I'm trying to crawl a Salesforce-generated site. This site appears to be fully generated by Javascript (if you load the page with JS disabled only a white screen shows up). For instance, Salesforce's own KB behaves this way.

Some of the pages come back as rendered and crawled, most come back as just blank. I'm unsure of if there's a way to get Screaming Frog to wait until page content loads on each article.

I currently have rendering set to "Javascript" which allows me to view some pages but not all.

Thanks for your help.

Tags : web-crawlers


Related Questions


Crawl Errors - ?2016-jun-764.html

Updated June 14, 2015 23:01 PM


How to prevent page counts by search indexers

Updated May 26, 2016 08:01 AM

How to keep unfriendly bots from accessing site?

Updated June 23, 2016 08:01 AM