JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 21.04.2014, 07:32
mwl mwl is offline
JD Fan
 
Join Date: Mar 2013
Posts: 79
Default 4chan HTML update (patch included)

4chan has recently been updated with a new internal layout for the HTML. Threads have been moved from /res to /thread and now have a slug at the end describing the thread. This broke the URL detection in the plugin, as well as the thread number detection. Images have had /src removed from their image path.

Strip off .txt extension, apply to JDownloader project via Team -> Apply patch ... (or whatever your localised equivalent is)

URLs to test against:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

The part caught in the first if block would appear to grab all the threads on a page, but it doesn't look like it works at the moment, and I'm not sure if it worked before the change since I've never tried it. It shouldn't be too much trouble to fix at a less ungodly hour, so if that feature is needed again, let me know and I'll see if I can make it work.
Attached Files
File Type: txt 4chan-update201404.patch.txt (3.5 KB, 107 views)
Reply With Quote
  #2  
Old 21.04.2014, 17:16
Lram32's Avatar
Lram32 Lram32 is offline
Spanish Supporter
 
Join Date: Jul 2012
Location: Dark side of the moon
Posts: 9,905
Default

Please wait for developer's reply. I linked this thread with one ticket that we have.

Ticket:
__________________
» Setup JD2 / Instalador de JD2 «
Spoiler:

Installer for Windows XP/Vista/Seven/Eight
|| JD2 x86 - x64 (Beta) ||

Installer for Mac
|| JD2 (Beta) ||

Installers for Linux
|| JD2 (Beta) x86 || <---> || JD2 (Beta) x64 ||

How to Create a Log
-» Click Here «-


¿Cómo crear un registro?
-» Click Aquí «-


Support Chat / Chat de Soporte
-» Click Here / Click Aquí «-


Reply With Quote
  #3  
Old 21.04.2014, 22:08
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,918
Default

Fixed, thx for the patch!
---
[Please read]=>How/when do I get updates?
Read it carefully BEFORE complaining that something would still be not working!
---
[Bitte lesen]=>Wie/wann bekomme ich Updates?
Bitte vollständig lesen bevor ihr euch beschwert, dass etwas immernoch nicht funktionieren würde!

GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #4  
Old 22.04.2014, 22:51
mwl mwl is offline
JD Fan
 
Join Date: Mar 2013
Posts: 79
Default

Doesn't seem to have done the trick. It's missing images with long names, these have <a title="..." href="...">. Patch in the works ...

I'm thinking something along these lines, which I can get to match in a regex tester, but I can't get to work on the actual page:
Code:
br.getRegex("(?i)File: <a( title=\".*\\.(gif|jpg|png|webm)\")? href=\"(//" + IMAGERDOMAINS + "/[0-9a-z]{1,4}/(src/)?\\d+\\.(gif|jpg|png|webm))\"")


Right, got it. Instead of grabbing onto the file link, this patch grabs onto the thumbnail, and accounts for "spoiler images" on boards that support them. I think by removing the "old" check for images I've fixed something I ran into a while ago: where someone pasted in the literal URL of a 4chan image, the board turned it into a link, which the plugin picked up on - images[].length not being zero, it would then miss out on the images posted in that thread. Finally, I've corrected the board name length in the plugin definition from {1,3} to {1,4}, since a length of 3 excluded amongst other things the rather handy /test/ board that becomes public when moot makes changes to the HTML.
Attached Files
File Type: txt 4chan-longnames.patch.txt (1.6 KB, 57 views)

Last edited by mwl; 22.04.2014 at 23:36.
Reply With Quote
  #5  
Old 23.04.2014, 23:22
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,918
Default

I improved it, please check that out first.

GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #6  
Old 24.04.2014, 23:31
mwl mwl is offline
JD Fan
 
Join Date: Mar 2013
Posts: 79
Default

Looks good. You'll still need to change the {1,3} to {1,4}, and you'll need to remove this:
Code:
br.getRegex("(?i)(https?://[\\w\\.]*?" + IMAGERDOMAINS + "/[0-9a-z]{1,4}/(src/)?\\d+\\.(gif|jpg|png|webm))").getColumn(0)
otherwise if someone posts an image URL in the thread the plugin will try and download that instead.
Attached Files
File Type: txt 4chan-roguelinks.patch.txt (1.6 KB, 56 views)

Last edited by mwl; 24.04.2014 at 23:34.
Reply With Quote
  #7  
Old 25.04.2014, 21:18
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,918
Default

I changed the regex though your patch file somehow doesn't work.
Please post the full plain code.

GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #8  
Old 25.04.2014, 22:37
mwl mwl is offline
JD Fan
 
Join Date: Mar 2013
Posts: 79
Default

Changes on lines 32 and 59.
Attached Files
File Type: txt Brds4Chnrg.java.txt (4.7 KB, 107 views)
Reply With Quote
  #9  
Old 31.05.2014, 07:26
mwl mwl is offline
JD Fan
 
Join Date: Mar 2013
Posts: 79
Default

This is still broken. Threads with image URLs linked in the text are still grabbing those instead of the images posted to the thread.

Fails on this post (very NSFW): **External links are only visible to Support Staff****External links are only visible to Support Staff**
Instead of grabbing the thread images, the plugin grabs these two URLs which it should not be grabbing, and nothing else.

Fails also here (again, very NSFW): **External links are only visible to Support Staff****External links are only visible to Support Staff**
Grabs the URL in the post but no images from the thread.

Last edited by mwl; 31.05.2014 at 07:31.
Reply With Quote
  #10  
Old 31.05.2014, 19:02
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,918
Default

Simply post your fix here and I'll apply and committ it

GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #11  
Old 31.05.2014, 19:55
mwl mwl is offline
JD Fan
 
Join Date: Mar 2013
Posts: 79
Default

Quote:
Originally Posted by mwl View Post
and you'll need to remove this:
Code:
br.getRegex("(?i)(https?://[\\w\\.]*?" + IMAGERDOMAINS + "/[0-9a-z]{1,4}/(src/)?\\d+\\.(gif|jpg|png|webm))").getColumn(0)
otherwise if someone posts an image URL in the thread the plugin will try and download that instead.
This is also in the file I attached about two posts up.
Reply With Quote
  #12  
Old 31.05.2014, 20:30
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,918
Default

Line is missing.
2 nearly identical regexes are used twice in this decrypter.

GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #13  
Old 31.05.2014, 21:00
mwl mwl is offline
JD Fan
 
Join Date: Mar 2013
Posts: 79
Default

Around line 59 in the version I have in my IDE.

This:
Code:
            String[] images = br.getRegex("(?i)(https?://[\\w\\.]*?" + IMAGERDOMAINS + "/[0-9a-z]{1,4}/(src/)?\\d+\\.(gif|jpg|png|webm))").getColumn(0);
            if (images == null || images.length == 0) images = br.getRegex("(?i)File: <a (title=\"[^<>\"/]+\" )?href=\"(//" + IMAGERDOMAINS + "/[0-9a-z]{1,4}/(src/)?\\d+\\.(gif|jpg|png|webm))\"").getColumn(1);
should become this:
Code:
            String[] images = br.getRegex("(?i)File: <a (title=\"[^<>\"/]+\" )?href=\"(//" + IMAGERDOMAINS + "/[0-9a-z]{1,4}/(src/)?\\d+\\.(gif|jpg|png|webm))\"").getColumn(1);
Reply With Quote
  #14  
Old 31.05.2014, 21:04
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,918
Default

Done.
---
[Please read]=>How/when do I get updates?
Read it carefully BEFORE complaining that something would still be not working!
---
[Bitte lesen]=>Wie/wann bekomme ich Updates?
Bitte vollständig lesen bevor ihr euch beschwert, dass etwas immernoch nicht funktionieren würde!

GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 14:33.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.