#1
|
|||
|
|||
Crawler not finding link
The link looks like this **External links are only visible to Support Staff****External links are only visible to Support Staff**
It's inside some javascript, but the crawler should still find it, right? The only thing slightly unusual is that it's a https instead of http. Last edited by raztoki; 13.01.2017 at 23:40. |
#2
|
||||
|
||||
if you're deep analysing, typically this is done on url containing html which may or may not have javascript within. If the url is within this base page it should find it. If the url is within another <script src it wont be caught as deep analyse only checks the original url (doesn't follow).
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#3
|
|||
|
|||
The link is in the main page, yes. At least it is when I load the page with Firefox. It's possible that the site is serving jdownloader a different page. Can I tell jdownloader what browser to emulate?
|
#4
|
||||
|
||||
@Luke_M
simple answer is no not so simple answer, yes. (via proxy, or editing our source code). can you provide the page you're trying to deep analyse? raztoki
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#5
|
|||
|
|||
What's the easiest way to test the crawler? Can I point it to a local file somehow, or do I have to install a web server?
|
#6
|
||||
|
||||
setup eclipse & jd workspace ? scan the page/url that you have issues with?
That's what I was going todo for you, once I was given the page/url to test with. raztoki
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#7
|
||||
|
||||
Please provide some example link, then we can check/fix or add support/help.
__________________
JD-Dev & Server-Admin |
#8
|
|||
|
|||
hello, also having problem like this....
JD use to be able to crawl links from this site: **External links are only visible to Support Staff****External links are only visible to Support Staff** but now it wont. Last edited by raztoki; 19.01.2017 at 10:42. |
#9
|
||||
|
||||
Do you use "Add new links"?
Please provide example links, links with "__" should contain external link that can be found with "Add new links".
__________________
FAQ: How to upload a Log |
#10
|
||||
|
||||
You can easily create support for this with LinkCrawler Rules (use board search)
You have to create rules for the '__' links
__________________
JD-Dev & Server-Admin |
Thread Tools | |
Display Modes | |
|
|