JDownloader Community - Appwork GmbH
 

 
 
Thread Tools Display Modes
Prev Previous Post   Next Post Next
  #1  
Old 24.04.2020, 20:43
emilio530 emilio530 is offline
Storm
 
Join Date: Oct 2011
Location: In the paradise
Posts: 210
Default link crawler giving false positive on iframe pages

Hi! This is to report an issue with the link crawler. When it makes deep scanning it's giving false positives on offline files. I mean, embedded videos on iframes that are online are marked as offline.

Consider this examples

**External links are only visible to Support Staff****External links are only visible to Support Staff** I copy all the content of this page and ask for a deep analysis to the crawler. It gets correctly the 3 pages containing embedded videos.

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

This pages have <iframes> to show the video, but it seems that in some part of the crawl, its setted that the payload of url must be 12 character long, and and when the iframe loads a url that have the payload 12 character long it loads correcly and shows it online. But newer posts are coming 14 chars long and are being detected incompletely, so they are marked offline as the file doesn't exist

**External links are only visible to Support Staff****External links are only visible to Support Staff** is crawled for **External links are only visible to Support Staff****External links are only visible to Support Staff**

**External links are only visible to Support Staff****External links are only visible to Support Staff** is crawled for **External links are only visible to Support Staff****External links are only visible to Support Staff**

and

**External links are only visible to Support Staff****External links are only visible to Support Staff** is crawled for **External links are only visible to Support Staff****External links are only visible to Support Staff**

Hope this helps to fix issue.

Thanks!!

Edit:

After a analysis with old links from the same host, i'm getting the same issue with the last 2 letters from the payload being ignored.

**External links are only visible to Support Staff****External links are only visible to Support Staff** gets
**External links are only visible to Support Staff****External links are only visible to Support Staff** and the iframe have **External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by emilio530; 24.04.2020 at 20:49.
Reply With Quote
 

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 19:06.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.