[Tock]

Hello,

I'm trying to figure out how to download images from Kemono.su while organizing them by post, such that the folder structure is something like:

JDownloads\KemonoArtist\Post1\Image1.jpg ... Image5.jpg
JDownloads\KemonoArtist\Post2\Image1.jpg ... Image5.jpg
JDownloads\KemonoArtist\Post3\Image1.jpg ... Image5.jpg

Unfortunately the 'Adopt Folder Structure' rule doesn't seem to apply to Kemono, since they're posts and not folders I suppose, though that's essentially what I'm trying to do.

This is about as far as I've gotten from hours of googling: **External links are only visible to Support Staff****External links are only visible to Support Staff**

URL in question (NSFW): **External links are only visible to Support Staff****External links are only visible to Support Staff**

Thanks

edit: Ideally I'd prefer to use the Page title rather than the Post ID, but couldn't figure out how to get there.

[pspzockerscene]

Using Packagizer rules with some dynamic plugin properties, you can do something like:
If Sourceurl -> Contains -> kemono.su
...then set
Download Directory -> Downloads\00TestDLs\<jd:prop:userid>\<jd:prop:title>

Important notice:
Our ticket system & knowledgebase are currently under maintenance.
If this is still the case while you are reading my posts and you can't access help articles linked by our staff, use the Internet archive/wayback machine to be able to view those articles:
archive.org/web/
Enter the URL -> Click on "Browse history" -> Select one of the latest dates available
If the date you selected does not lead you to the support article try the next oldest one.

[Tock]

Ah hah! Perfect! Thank you!

[VanTuz]

When a link to some file sharing site like mega appears in a post, then such links are not sent to the post folder, but are dumped into one pile. How do I make such links fall into the post folder?

[pspzockerscene]

Originally Posted by VanTuz

When a link to some file sharing site like mega appears in a post, then such links are not sent to the post folder, but are dumped into one pile. How do I make such links fall into the post folder?

That behavior is not a bug since MEGA links will typically need to go into the file/folder structure they provide, see:
https://support.jdownloader.org/de/k...lder-structure

If you really wanted them to go into the profile- or post-folder, you could maybe do this via Packagizer rule based on the "source URL".

[Jiaz]

@VanTuz: Please provide example links

[VanTuz]

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

[Jiaz]

@VanTuz: Would be nice to get a post with included link? You provided user overview and checked a few but did not find those in question. Please help us to save time and provide direct example links and then we can check and provide better help/solutions.

[VanTuz]

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

[VanTuz]

I tried, but I can't make the files downloaded from mega or other file sharing sites go to the post folder, they end up in a separate folder.

[pspzockerscene]

Please provide all Packagizer rules and/or EventScripter scripts you've tried so far.

[VanTuz]

Download url: kemono* (or did not use this item)
source url: mega*
download folder kemono\<jd:prop:postid>-<jd:prop:title>

I didn't use scripts because I don't know how to use them.

[Jiaz]

@VanTuz: Any customization? for me it worked fine for your first example link. any custom packagizer rule or does this randomly work/not work? most likely examples with single links work but those with folder links do not work
issue comes up when the linked folder has subfolders and then placing everything in same/one package will mess up the files structure

[VanTuz]

Originally Posted by Jiaz

@VanTuz: Any customization? for me it worked fine for your first example link. any custom packagizer rule or does this randomly work/not work? most likely examples with single links work but those with folder links do not work
issue comes up when the linked folder has subfolders and then placing everything in same/one package will mess up the files structure

If you download a specific post, there are no problems, but if you start scanning the author's page, then the posts are sorted into folders, and files from external file sharing sites are added separately. That's why I gave links to the authors' pages for the first time.

Originally Posted by Jiaz

so for example you could create packagizer rule that parses postID from URL and then places everything into custom package

How?

[Jiaz]

@VanTuz: source and download url are mixed, source = kemono and download is mega(as you download from mega).
but this will not work that way as the properties are not part of mega/dropbox folder
so for example you could create packagizer rule that parses postID from URL and then places everything into custom package
only kemono links do have access to those properties but cannot access them from other links (eg mega or dropbox)

[pspzockerscene]

I've tested Jiaz' idea because it sounded good and it makes sense but it doesn't work.

Explanation:
When a kemono.su user profile is crawled, JDownloader does never "see" or process the single "/post/..." URLs.
The posts are crawled as part of the user crawler because this is how their API works (reference: kemono.su/api/schema).
This means that from non-kemono URLs crawled as part of a user crawl process, there is no "source URL" reference on single post kemono URLs.
This ultimately means that for all items resulting from a profile crawl process, all you golt as a source URL is the kemono profile URL ("kemono.su/portal/user/username").

@VanTuz what you want is either _impossible_ or only using the following tricks:

• An EventScripter script (though to be honest, I still don't know how you want to match the results to find their origin)
• or: Modify the JDownloader source code of the kemono crawler so that the kemono plugin puts all "plugin properties" on all results [also on non-kemono results] so that you can access them also on e.g. mega.nz items.
• or: Modify the JDownloader soure code of the kemono crawler so that the user profile crawler will first return all single post URLs and then process them. This will make it waay slower and also puts an unnecessary load on the kemono API but in the end you will have the single post URLs as "sourcer URL" reference which is needed to accomplish your goal here.

[VanTuz]

It's clear. It's a pity that nothing can be done about it, because I'm not a programmer. It's just that when yiff. party existed, the program put posts in folders, plus it did this with pixiv, but for some reason it doesn't want to work with this site. And I hoped that the program sees that a link to a mega was found on such and such a post, so you can put it in a folder with the name of the post in which the link was found, but since the API of the site does not allow this, I will download as is.

[Jiaz]

@VanTuz: we have one more idea and currently testing it
Update:
Good news, I have found a way! Bad news, it will take some time before the required change will be available in update.will further discuss this with pspzockerscene

[VanTuz]

There is a new problem - the program does not pick up links from the dropbox if you insert a link to a post that contains a link to the dropbox. If you just add a link to the dropbox, everything is fine.
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

[Jiaz]

@VanTuz: works fine here, you have to Enable Settings->Plugins->kemono.su
"Crawl http links in post text"

[VanTuz]

And indeed, it was turned off. I just remember that this setting was turned on and I was afraid that something was broken.

[Jiaz]

@VanTuz: Thanks for the feedback and you're welcome!

[VanTuz]

I'm trying to download this author - **External links are only visible to Support Staff****External links are only visible to Support Staff** - but out of more than 700 files, the program finds 593.
If I give direct links to posts, it finds 160 files. In addition, the program really does not interfere with picking up links from a text file.
UPD. When I enable the Packagizer rule, the program finds fewer files.

[Jiaz]

Originally Posted by VanTuz

In addition, the program really does not interfere with picking up links from a text file.

What do you mean by that? Adding an user url requires less requests and more likely to fetch/process more links than 800 inidividual links with minimum 800 requests resulting nearly guaranteed in rate limit or protection

[Jiaz]

@VanTuz: sounds like you're running in sort of rate limit and/or crawl-/bot protection. Please provide a debug log for when this happens.

Happens for me too, first try I got around 880 links, next time only 300, next time only 450. the requests are blocking.

[VanTuz]

08.04.24 00.55.19 <--> 08.04.24 00.57.55 jdlog://7202411370661/

[Jiaz]

@VanTuz: As I thought/expected, it's caused by rate limit. Please try again with next plugin update

[VanTuz]

Thanks a lot, everything is working. And this is very pleasing, because gallery-dl ran into that very limit, and jdownloader downloaded everything.

[Jiaz]

@VanTuz: Thanks for the feedback. There was just some little thing missing by work of pspzockerscene but looks like the rate limit handling works good

[VanTuz]

Previously, when I downloaded individual posts by some author, the files that the author posted on mega fell into the same folder with the files from kemono, but now they fall into a separate folder, which is inconvenient. Is there any way to turn it off?
**External links are only visible to Support Staff****External links are only visible to Support Staff**

[pspzockerscene]

Originally Posted by VanTuz

Previously, when I downloaded individual posts by some author, the files that the author posted on mega fell into the same folder

No. Mega files always went into separate packages. That is not a bug.

Originally Posted by VanTuz

Is there any way to turn it off?

No.

MEGA links mostly lead to file/folder structures which JD will try to keep.
In order to do this, they need to go into separate packages, see:
https://support.jdownloader.org/de/k...lder-structure
This is similar to kemono plugin-properties not being set on resulting MEGA files/links, see:
https://board.jdownloader.org/showth...579#post528579

[VanTuz]

But I remember absolutely clearly that all files from all file sharing sites in the post ended up in the same folder, not in different ones.

[pspzockerscene]

Then this must have been a bug.

I just came back from holiday so I got no good idea what was changed in the meatime though then again as said, the current behavior is technically the correct one.

[VanTuz]

Got it. I will try to ask for help in the topic about event scripter programming, maybe they will help.