JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 03.04.2016, 01:04
BWEL BWEL is offline
JD Alpha
 
Join Date: Feb 2016
Location: Toronto, ON
Posts: 24
Exclamation LinkCrawler will not decrypting PHP encoded links

Hello,

I did actually search the forum for hours, but still scratching my head as to why this code in "LinkCrawler Rules" in "Advanced Settings" is not working.

Code:
{
"enabled" : true,
"maxDecryptDepth" : 1,
"name" : "gfxxtra",
"pattern" : "http?://(www.)?gfxxtra\\.com/engine/go\\.php\\?url\\=(.+)",
"rule" : "DEEPDECRYPT"
}
I am trying to grab only the resulting link that comes upon crawling all links that have the following common pattern:

Code:
**External links are only visible to Support Staff**
The above link is usually posted in the download SAMPLE URL
**External links are only visible to Support Staff****External links are only visible to Support Staff**

under the following anti-leech protection code embedded into an hoster logo image:
Code:
<p style="text-align: center;"><!--dle_leech_begin-->
<a rel="nofollow" href="**External links are only visible to Support Staff** 
target="_blank" rel="nofollow"><!--dle_image_begin:/uploads/2.fileboom.gif|-->
<img src="/uploads/2.fileboom.gif" alt="Animating Charts and Graphs in CINEMA 4D" title="Animating Charts and Graphs in CINEMA 4D"  />
<!--dle_image_end--></a><!--dle_leech_end--></p>
The odd thing is that JDownloader will crawl any other link present in the page with the exception of the "gfxxtra.com/engine/go.php" one that I am trying to extract and I am still baffled as to why, because whenever I paste the link manually in JDownloader in Analyze and Add Links, it does automatically crawl to the correct fileboom.me link.

I know this is much to ask but the problem I have with Analyze and Add Links is that it will only work one link at a time, if several links are present there, it won't crawl anything.

I would really appreciate some input from any of you who might have an idea as to what is actually going on with this.

Thanks,
BWEL

Last edited by raztoki; 03.04.2016 at 04:52.
Reply With Quote
  #2  
Old 03.04.2016, 04:57
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default

problem as I see it
without plugin support it will be problematic
we have code to detect the use of base64 (url wrapping), and it returns a *.rar which then the directhttp plugin kicks in. Yet its another redirect and not hosting the file so the directhttp plugin will fail.

we will need to add plugin support in order to address this.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #3  
Old 04.04.2016, 07:42
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default

added support via decrypter plugins, please delete you rules that you have made.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #4  
Old 06.04.2016, 09:39
BWEL BWEL is offline
JD Alpha
 
Join Date: Feb 2016
Location: Toronto, ON
Posts: 24
Default

thank you so much. It does work now. Amazing You're the greatest.
Reply With Quote
  #5  
Old 08.04.2016, 01:54
BWEL BWEL is offline
JD Alpha
 
Join Date: Feb 2016
Location: Toronto, ON
Posts: 24
Exclamation Some Encoded Links work and some others don't

Raztoki I have been using JDownloader with the latest update I notice the following weird behaviour when it comes to base 64 encoded php links.

The following formatting is automatically redirected to the correct URL with a simple drag and drop from Firefox to JDownloader Link Grabber:

Code:
div id="news-id-764093" style="display:inline;"><p style="text-align: center;">&nbsp;</p>
<p style="text-align: center;"><!--dle_leech_begin--><a rel="nofollow" href="**External links are only visible to Support Staff** target="_blank"><!--dle_image_begin:/uploads/2.fileboom.gif|--><img src="/uploads/2.fileboom.gif" alt="InDesign Secrets (2016)" title="InDesign Secrets (2016)"><!--dle_image_end--></a><!--dle_leech_end--><br>------------------------------------------------------------------------------------------<br><!--dle_leech_begin--><a rel="nofollow" href="**External links are only visible to Support Staff** target="_blank"><!--dle_image_begin:/uploads/1.nitroflare.gif|--><img src="/uploads/1.nitroflare.gif" alt="InDesign Secrets (2016)" title="InDesign Secrets (2016)"><!--dle_image_end--></a><!--dle_leech_end--><br>&nbsp; <br></p></div>

Checkout this **External links are only visible to Support Staff**VIMEO Video for a better understanding.

Whereas this other formatting always fails, JDownloader cannot reveal the download link with drag and drop. The only way it works is if I was to paste one encoded link at a time manually via Analyse and Add Links

Code:
</center><p><span id="post-img"></span></p><div id="news-id-764011" style="display:inline;"><p style="text-align: center;">&nbsp;</p>
<p style="text-align: center;"><!--dle_leech_begin--><a rel="nofollow" href="**External links are only visible to Support Staff** target="_blank"><!--dle_image_begin:/uploads/2.fileboom.gif|--><img src="/uploads/2.fileboom.gif" alt="Learn to Texture for 3D" title="Learn to Texture for 3D"><!--dle_image_end--></a><!--dle_leech_end--></p>
<div style="text-align: center;"><br><br>RARPASS:gfxtra</div></div>
<center>

Checkout this **External links are only visible to Support Staff**VIMEO Video for a better understanding.

I am looking at the encoded HREF and I cannot find any difference between the first working links from the second non-working one.

Can you give it a try on your end and figure what what might be failing?

Thanks a lot for your valuable and precious contribution to the community.

Regards,
BWEL
Attached Files
File Type: rar WorkingLinkBehaviour.rar (554.9 KB, 2 views)

Last edited by BWEL; 08.04.2016 at 02:35. Reason: adding screen recording videos
Reply With Quote
  #6  
Old 08.04.2016, 02:31
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default

You actually don't have the latest update, we have had core build block which has prevented plugin updates for over a week now. You will need to wait until that clears before the plugin changes I made becomes live.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #7  
Old 08.04.2016, 02:37
BWEL BWEL is offline
JD Alpha
 
Join Date: Feb 2016
Location: Toronto, ON
Posts: 24
Default

Thank you Raztoki for the prompt assistance. Could you kindly have a look at the screen recordings I took, just to create an idea. It's odd, but I've noticed a weird bug in Analyse and Add Links, anytime there are many links there, it will stop crawling them. Perhaps you've fixed it, perhaps not.

Your contribution is always valued and I thank you for taking time to assist the community.
Reply With Quote
  #8  
Old 08.04.2016, 02:52
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default

add links dialog has options to:
a) add and analyse links
or
b) add and deep analyse links.

breakdown
a) only matches against known regex patterns in our plugins or core.
b) opens each url and analyses website source code and add series of findings, core then matches them against supported plugins regex patterns.

Jiaz has core feature that if you copy a single link and it doesn't match a) it will also so b). But if you copy numerous urls to clipboard and add links dailog and do a) it wont do b).

core has feature to analyse base64 within url structure, but if that returned url is say **External links are only visible to Support Staff****External links are only visible to Support Staff** it will be picked up by directhttp plugin, and it will try and download it as hotlinkable. Since its not hotlinkable and redirects to other hosters, it failed.

I'll check your videos shortly.

hope that helps.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #9  
Old 08.04.2016, 04:03
BWEL BWEL is offline
JD Alpha
 
Join Date: Feb 2016
Location: Toronto, ON
Posts: 24
Default

thanks for the breakdown raztoki, it makes sense. The bug I am trying to report is the fact that upon adding multiple links divided by lines at the same time in the "add and analyse" window, even the links that match plugins don't get recognized anymore. Known patterns will be simply ignored. If I just enter one link, then it will crawl and extract the known plugin/pattern links just fine. One of my videos shows exactly this behaviour.
Reply With Quote
  #10  
Old 08.04.2016, 04:39
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default

click and drag, must use provide htmlflavor features, as its analysing the image and the links behind it. fboom is actually not double wrapped which I find strange that link didn't add.

when plugin comes out just use right click copy url of the go.php link and it will be processed aok

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 19:28.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.