Thread
: [Solved]
Can't get this URL, Linkcrawler runs forever
View Single Post
#
3
13.04.2020, 06:20
raztoki
English Supporter
Join Date: Apr 2010
Location: Australia
Posts: 17,611
utilise deepPattern like tony mentioned,
fyi: maxdecryptdepth is to prevent crawling for ever if you for example listen to base url, and then find (sub)pages within. say you only want to check 2 sub pages deep instead of unlimited.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170
Don't fight the system, use it to your advantage. :]
raztoki
View Public Profile
Send a private message to raztoki
Find all posts by raztoki