JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 09.03.2013, 22:29
mwl mwl is offline
JD Beta
 
Join Date: Mar 2013
Posts: 59
Default 4chan decrypter - use thread ID

Currently, the format for these package names is "4chan.org - [board name] - [date]". The date isn't particularly useful beyond ensuring JD puts each thread in a separate folder. To me, it would seem more useful to put the thread number in there. On the slower boards, this would mean you could re-add a thread at a later date and only get the newer images, rather than re-downloading them all. Benefits are a saving in bandwidth and a saving in time. Where any threads in the queue are likely to expire, the time saved may mean the difference between getting all the images and the thread and images 404ing before you get to them.

I couldn't find an option to change this in the options, so I took to the source. It looks like it comes from this bit of code in Brds4Chnrg.java:

Code:
                String domain = "4chan.org";
                String cat = br.getRegex("<div class=\"boardTitle\">/.{1,3}/ - (
.*?)</div>").getMatch(0);
                if (cat == null) cat = br.getRegex("<title>/b/ - (.*?)</title>")
.getMatch(0);
                if (cat != null) {
                    cat = cat.replace("&amp;", "&");
                } else {
                    cat = "Unknown Cat";
                }
                String date = new Date().toString();
                fp.setName(domain + " - " + cat + " - " + date);
The thread number for the image boards is after /res/ in the URL, and appears in the HTML like this:
Code:
<div class="thread" id="t12345678">
It also appears in the post form at the top, though this isn't shown if the thread is locked, so I wouldn't rely on it being there.

I'm happy to whip up a patch, though the prospect of making the change locally and getting it wiped out at each update is worrying. I suspect it's as simple as changing the last two lines above to:
Code:
                String threadno = br.getRegex("<div class=\"thread\" id=\"t([0-9]+)\">").getMatch(0);
                fp.setName(domain + " - " + cat + " - " + threadno);
What do you guys think?
Reply With Quote
  #2  
Old 10.03.2013, 03:05
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,268
Default

you will need a if and else statement with threadno isn't available otherwise presents a npe
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #3  
Old 10.03.2013, 04:00
mwl mwl is offline
JD Beta
 
Join Date: Mar 2013
Posts: 59
Default

It occurs to me that checking the URL may be faster than checking the entire HTML. This seems safe to me, since we can't get here unless the URL that matches the plugin conditions, the page doesn't 404, and the result contains at least one content image.

Code:
				// String suffix = br.getRegex("<div class=\"thread\" id=\"t([0-9]+)\">").getMatch(0);
				String suffix = new Regex(parameter, "/res/([0-9]+)").getMatch(0);
				if (suffix == null) {
					// Fall back to date if we can't resolve
					suffix = new Date().toString();
				}
				fp.setName(domain + " - " + cat + " - " + suffix);
For obvious reasons, it's difficult to put together a viable test case, so you'd have to actually visit the place to get URLs to test. It works for me for valid cases, though I've not really stress-tested any possible invalid ones, since I can't really see anything which could pass the checks to get to that block but not match the that URL regex.
Reply With Quote
  #4  
Old 15.03.2013, 21:13
mwl mwl is offline
JD Beta
 
Join Date: Mar 2013
Posts: 59
Default

After a bit of wrestling with the formatter, here's the change in patch form. Let me know if you want to do something with it - replacing the plug-in after JD over-writes it every time is getting tedious. As far as the IP goes, this fix is in the public domain and I assert no rights over it.

Code:
Index: src/jd/plugins/decrypter/Brds4Chnrg.java
===================================================================
--- src/jd/plugins/decrypter/Brds4Chnrg.java	(revision 20212)
+++ src/jd/plugins/decrypter/Brds4Chnrg.java	(working copy)
@@ -71,8 +71,13 @@
                 } else {
                     cat = "Unknown Cat";
                 }
-                String date = new Date().toString();
-                fp.setName(domain + " - " + cat + " - " + date);
+                // extract thread number from URL
+                String suffix = new Regex(parameter, "/res/([0-9]+)").getMatch(0);
+                if (suffix == null) {
+                    // Fall back to date if we can't resolve
+                    suffix = new Date().toString();
+                }
+                fp.setName(domain + " - " + cat + " - " + suffix);
                 for (String image : images) {
                     if (image.startsWith("/") && !image.startsWith("h")) image = image.replace("//", prot + "://");
                     DownloadLink dl = createDownloadlink(image);
Attached Files
File Type: txt 4chan-threadno.patch.txt (1.1 KB, 194 views)
Reply With Quote
  #5  
Old 07.04.2013, 03:00
mwl mwl is offline
JD Beta
 
Join Date: Mar 2013
Posts: 59
Default

No more word on this yet?
Reply With Quote
  #6  
Old 07.04.2013, 21:55
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 56,557
Default

No...

GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #7  
Old 11.04.2013, 21:12
mwl mwl is offline
JD Beta
 
Join Date: Mar 2013
Posts: 59
Default

That's a little unhelpful, especially given I've effectively done all the work for you (and even provided a patch ready to apply against the trunk). Is there anything else I can do to help you get this in?
Reply With Quote
  #8  
Old 12.04.2013, 17:48
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 56,557
Default

Please just post the complete code and I'll apply it...

GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #9  
Old 12.04.2013, 20:44
mwl mwl is offline
JD Beta
 
Join Date: Mar 2013
Posts: 59
Default

Quote:
Originally Posted by pspzockerscene View Post
Please just post the complete code and I'll apply it...
There's a svn diff attached to post #4. Team > Apply patch in Eclipse and you're done.
Reply With Quote
  #10  
Old 14.04.2013, 03:04
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 56,557
Default

Please just give me the complete code as text, thx.

GreeZ psp
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #11  
Old 21.04.2013, 00:36
mwl mwl is offline
JD Beta
 
Join Date: Mar 2013
Posts: 59
Default

It's attached to post #4 ...
Reply With Quote
  #12  
Old 21.04.2013, 05:23
tony2long's Avatar
tony2long tony2long is online now
English Supporter
 
Join Date: Jun 2009
Posts: 6,406
Default

Please provide some example links so we can make sure it works before commit.
__________________
FAQ: How to upload a Log
Reply With Quote
  #13  
Old 21.04.2013, 14:27
mwl mwl is offline
JD Beta
 
Join Date: Mar 2013
Posts: 59
Default

These should be around for a few days at least, and shouldn't contain anything too objectionable (no guarantees though).
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Shouldn't be any difference between SSL and non-SSL.
Reply With Quote
  #14  
Old 21.04.2013, 16:08
tony2long's Avatar
tony2long tony2long is online now
English Supporter
 
Join Date: Jun 2009
Posts: 6,406
Default

It's done.
__________________
FAQ: How to upload a Log
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 04:31.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.