JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 24.04.2020, 20:43
emilio530 emilio530 is offline
Storm
 
Join Date: Oct 2011
Location: In the paradise
Posts: 210
Default link crawler giving false positive on iframe pages

Hi! This is to report an issue with the link crawler. When it makes deep scanning it's giving false positives on offline files. I mean, embedded videos on iframes that are online are marked as offline.

Consider this examples

**External links are only visible to Support Staff****External links are only visible to Support Staff** I copy all the content of this page and ask for a deep analysis to the crawler. It gets correctly the 3 pages containing embedded videos.

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

This pages have <iframes> to show the video, but it seems that in some part of the crawl, its setted that the payload of url must be 12 character long, and and when the iframe loads a url that have the payload 12 character long it loads correcly and shows it online. But newer posts are coming 14 chars long and are being detected incompletely, so they are marked offline as the file doesn't exist

**External links are only visible to Support Staff****External links are only visible to Support Staff** is crawled for **External links are only visible to Support Staff****External links are only visible to Support Staff**

**External links are only visible to Support Staff****External links are only visible to Support Staff** is crawled for **External links are only visible to Support Staff****External links are only visible to Support Staff**

and

**External links are only visible to Support Staff****External links are only visible to Support Staff** is crawled for **External links are only visible to Support Staff****External links are only visible to Support Staff**

Hope this helps to fix issue.

Thanks!!

Edit:

After a analysis with old links from the same host, i'm getting the same issue with the last 2 letters from the payload being ignored.

**External links are only visible to Support Staff****External links are only visible to Support Staff** gets
**External links are only visible to Support Staff****External links are only visible to Support Staff** and the iframe have **External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by emilio530; 24.04.2020 at 20:49.
Reply With Quote
  #2  
Old 24.04.2020, 21:17
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,524
Default

Thanks, found the cause and will fix it next week
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 30.04.2020, 18:30
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,524
Default

should be working as expected with next update
__________________
JD-Dev & Server-Admin
Reply With Quote
  #4  
Old 30.04.2020, 21:13
emilio530 emilio530 is offline
Storm
 
Join Date: Oct 2011
Location: In the paradise
Posts: 210
Default

Thank you! Will test as soon i can go again to my machine.

:)
Reply With Quote
  #5  
Old 02.05.2020, 09:43
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,524
Default

You're welcome
__________________
JD-Dev & Server-Admin
Reply With Quote
  #6  
Old 06.05.2020, 20:43
emilio530 emilio530 is offline
Storm
 
Join Date: Oct 2011
Location: In the paradise
Posts: 210
Default

Quote:
Originally Posted by Jiaz View Post
You're welcome
Hi, just tested it and works like a charm again. Many thanks!!
Reply With Quote
  #7  
Old 07.05.2020, 09:29
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,088
Default

Thanks for your feedback

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 01:04.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.