artofx.org link crawling
A long time ago, jd2 could crawl artofx.org's links to the filehosters, but atm you need to manually open thier links in browser to get to the hosts and add that to jd2. I think the website is throwing jd2 off because of referrer url.
example links:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Could the crawling be restored? I tried setting up a link crawler rule but without success.
|