java - crawler4j always returns fatal transport error -
this seed add crawler4j.
error [crawler 1] fatal transport error: connection http://example.com refused while fetching http://example.com/page.html (link found in doc #0)
this weird me. don't know causes it.
try increase politeness setting of crawler. happens me when server want crawler not good.
Comments
Post a Comment