[RickyPickles]

When media files are posted in 4chan.org threads, the file names get changed to a random series of numbers. But the original title is what's shown as the URL text linking to the file.

Would it be at all possible to have LinkGrabber automatically use the URL text as the file name...?

The URL text also includes the file extension, so the switch would be seamless.

I have a Chrome extension called '4chan X' that does this via a little download icon next to the URL text. Unfortunately though there's no option to scrape the whole thread, so you have to go one by one and that gets pretty tiresome. Would be awesome to just use JDownloader instead!

[pspzockerscene]

Hi,
please post example URLs.

-psp-

[RickyPickles]

SFW examples
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

NSFW example
**External links are only visible to Support Staff****External links are only visible to Support Staff**

[pspzockerscene]

Hm seems like I don't understand the issue you are having.
If I take this as an example:
bla.4chan.org/wsg/1601234567416.webm
--> Filename would be:
1601234567416.webm

Where is the problem / how/where are other filenames supposed to come from?

-psp-
EDIT

Fo you mean the "file:blabla.mp4" nems in their html code?
We could try to crawl- and set those filenames but at this moment, serverside filenames are chosen which is not per-se considerable as a bug.

[Jiaz]

@pspzockerscene: yes, that is what he means

[pspzockerscene]

Done.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[Derp]

Please make this feature optional.

[pspzockerscene]

Originally Posted by Derp

Please make this feature optional.

I will create a Packagizer rule for you tomorrow to get the old names back.

If more users want the same, I may add a plugin setting for that.

-psp-

[Derp]

Originally Posted by pspzockerscene

I will create a Packagizer rule for you tomorrow to get the old names back.

If more users want the same, I may add a plugin setting for that.

-psp-

Tried that, but didn't work, see attached pic of the rule. JD2 was still using those new filenames.

Originally Posted by Jiaz

I've updated ticket, we will add plugin setting for this

Great!

[Jiaz]

I've updated ticket, we will add plugin setting for this

[Jiaz]

@Derp: that rule won't work because orgfilename is the one set by plugin. pspzocker was talking about a rule that uses the old names but I (and you) prefer to have plugin setting for this He will work on this

[pspzockerscene]

@Derp
It won't work this way - you'd have to get the names out of the direct-URLs crawled by the crawler.
Anyways - I've added a setting for the next update.
See Settings -> Plugins -> boards.4chan.org

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[pspzockerscene]

I've just prepared a huge update for our 4chan plugin.
This includes:
- Faster crawling
- Can now crawl up to 20 pages of a category (see plugin settings - default = limited to 1)

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[Derp]

Build Thu Nov 19 18:44:33 CET 2020

The plugin is completely broken now.
Regardless of the server-filename-setting, JD2 is adding every file as a .webm and checks it as such, which of course results in all the files which aren't webm-files to not be found.

Also, the default packagename just uses the 4chan-board-acronym and not the board-name, i.e.
"4chan org - sp - 103946012" instead of
"4chan org - Sports - 103946012".

**External links are only visible to Support Staff****External links are only visible to Support Staff**

**External links are only visible to Support Staff****External links are only visible to Support Staff** <-- this thread happens to have at least one file of every type (jpg, png, gif, webm) and two webms for the auto-packaging.


------------------------------

Originally Posted by Jiaz

@Derp: that rule won't work because orgfilename is the one set by plugin.

Originally Posted by pspzockerscene

@Derp
It won't work this way - you'd have to get the names out of the direct-URLs crawled by the crawler.

No idea how that would work, was the first time I used it.
In the "Dynamic Variables" there's no entry for that, but "packagename" is listed twice.

Maybe I also found a bug:
When sourceurl is checked, and the user checks "rename", the checkbox is cleared and disabled.
After unchecking "rename", the sourceurl-checkbox is checked again, but not re-enabled.

[pspzockerscene]

Originally Posted by Derp

No idea how that would work, was the first time I used it.
In the "Dynamic Variables" there's no entry for that, but "packagename" is listed twice.

It works totally different - see board search for more examples.

Originally Posted by Derp

Maybe I also found a bug:
When sourceurl is checked, and the user checks "rename", the checkbox is cleared and disabled.
After unchecking "rename", the sourceurl-checkbox is checked again, but not re-enabled.

Jiaz will check this tomorrow.

I've fixed this issue - I made a simple but silly mistake with that wrong file-extension and I didn't know that the long/full board title was so important.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[Derp]

Originally Posted by pspzockerscene

It works totally different - see board search for more examples.

I'll pass, thanks.
Re: "Packagename" being available twice: It's because of line 284 or 301 in PackagizerFilterRuleDialog.java

Originally Posted by pspzockerscene

I've fixed this issue - I made a simple but silly mistake with that wrong file-extension and I didn't know that the long/full board title was so important.

Bug seems to be fixed and the filename-feature seems to work in both states, too.

Thank you very much.

EDIT2:
Just had an interesting case:
Two different files in a thread used the same upload-filename, and JD2 treats them as mirrors of each other.
Because of that only one of them is downloaded
With server-filenames, both files are downloaded.
**External links are only visible to Support Staff****External links are only visible to Support Staff**
filename is "file.png"

EDIT3:
Considering how slow the image-server already is, a fast linkcheck option would make sense, too.

[pspzockerscene]

Originally Posted by Derp

Considering how slow the image-server already is, a fast linkcheck option would make sense, too.

"Fast linkcheck" is already always used.

You can still see the filesizes right away because this information is right away for 4chan content.
Where/how do you think that a linkcheck process is active- and slow?
I don't see how we could speed anything up here.

-psp-

[Derp]

Originally Posted by pspzockerscene

"Fast linkcheck" is already always used.

You can still see the filesizes right away because this information is right away for 4chan content.

Oh right, forgot about that.

Originally Posted by Jiaz

Limiting the request interval to api only will be available with next core update next week. I'm not finished yet

Several years ago the authors of the 4chanX-userscript were asked by 4chan-admins to set the minimum interval of hitting the API to 30 seconds or so, the same as the native inline-extension. But that was before it was cloudflare'd, IIRC.
Considering the JSON already contains the filesizes and JD2 doesn't need to hit the image-server to get that info, it shouldn't be a problem at all.

-----
Another question: Is JD2 somehow limiting the concurrent downloads from cloudflare-protected links or something?
When I start to download a package with ~30 images (~11MiB total) from 4chan with 5 concurrent downloads, it takes about 3 minutes for it to finish, with most of them being stuck at "Starting..." for 10-15 seconds.
Or are you seeing the same behavior and it's being limited server-side? I think it changed to this about a week ago.
EDIT:
Another try yesterday: 140 files, 75 MiB, 3 concurrent downloads, took 8 minutes.

[Jiaz]

@Derp: Thanks for the info, those two Packagename are different ones. I will change description

[Jiaz]

@pspzockerscene. maybe he's taking about the request limit