What is the purpose of user agent node-fetch/1.0 (+https://github.com/bitinn/node-fetch) fetching base URL and no other resources loaded

by zod   Last Updated August 14, 2019 00:04 AM - source

The user agent node-fetch/1.0 (+https://github.com/bitinn/node-fetch) has been fetching the base URL from IP's belonging to

It is fairly agressive and coming in from quite a number of IP addresses and I'm wondering why the user agent only fetches the base URL and no other resources are loaded.

user agent

I'd like to block it but am not sure it would be wise.

Related Questions

Google's definition of Cloacking

Updated May 18, 2019 18:04 PM

Invalid traffic coming from Sogou

Updated May 24, 2019 07:04 AM

User agent string proper syntax

Updated February 17, 2016 01:01 AM