#1
|
|||
|
|||
kemono.party kemono.su Organize downloaded files by post
Hello,
I'm trying to figure out how to download images from Kemono.su while organizing them by post, such that the folder structure is something like: JDownloads\KemonoArtist\Post1\Image1.jpg ... Image5.jpg JDownloads\KemonoArtist\Post2\Image1.jpg ... Image5.jpg JDownloads\KemonoArtist\Post3\Image1.jpg ... Image5.jpg Unfortunately the 'Adopt Folder Structure' rule doesn't seem to apply to Kemono, since they're posts and not folders I suppose, though that's essentially what I'm trying to do. This is about as far as I've gotten from hours of googling: **External links are only visible to Support Staff****External links are only visible to Support Staff** URL in question (NSFW): **External links are only visible to Support Staff****External links are only visible to Support Staff** Thanks edit: Ideally I'd prefer to use the Page title rather than the Post ID, but couldn't figure out how to get there. Last edited by Tock; 08.03.2024 at 00:45. Reason: added info |
#2
|
||||
|
||||
Using Packagizer rules with some dynamic plugin properties, you can do something like:
If Sourceurl -> Contains -> kemono.su ...then set Download Directory -> Downloads\00TestDLs\<jd:prop:userid>\<jd:prop:title> Important notice: Our ticket system & knowledgebase are currently under maintenance. If this is still the case while you are reading my posts and you can't access help articles linked by our staff, use the Internet archive/wayback machine to be able to view those articles: archive.org/web/ Enter the URL -> Click on "Browse history" -> Select one of the latest dates available If the date you selected does not lead you to the support article try the next oldest one.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#3
|
|||
|
|||
Ah hah! Perfect! Thank you!
|
#4
|
|||
|
|||
When a link to some file sharing site like mega appears in a post, then such links are not sent to the post folder, but are dumped into one pile. How do I make such links fall into the post folder?
|
#5
|
||||
|
||||
@VanTuz: Please provide example links
__________________
JD-Dev & Server-Admin |
#6
|
|||
|
|||
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#7
|
||||
|
||||
@VanTuz: Would be nice to get a post with included link? You provided user overview and checked a few but did not find those in question. Please help us to save time and provide direct example links and then we can check and provide better help/solutions.
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 23.03.2024 at 23:25. |
#8
|
|||
|
|||
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#9
|
||||
|
||||
Quote:
https://support.jdownloader.org/de/k...lder-structure If you really wanted them to go into the profile- or post-folder, you could maybe do this via Packagizer rule based on the "source URL".
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#10
|
|||
|
|||
I tried, but I can't make the files downloaded from mega or other file sharing sites go to the post folder, they end up in a separate folder.
|
#11
|
||||
|
||||
Please provide all Packagizer rules and/or EventScripter scripts you've tried so far.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#12
|
|||
|
|||
Download url: kemono* (or did not use this item)
source url: mega* download folder kemono\<jd:prop:postid>-<jd:prop:title> I didn't use scripts because I don't know how to use them. |
#13
|
||||
|
||||
@VanTuz: Any customization? for me it worked fine for your first example link. any custom packagizer rule or does this randomly work/not work? most likely examples with single links work but those with folder links do not work
issue comes up when the linked folder has subfolders and then placing everything in same/one package will mess up the files structure
__________________
JD-Dev & Server-Admin |
#14
|
||||
|
||||
@VanTuz: source and download url are mixed, source = kemono and download is mega(as you download from mega).
but this will not work that way as the properties are not part of mega/dropbox folder so for example you could create packagizer rule that parses postID from URL and then places everything into custom package only kemono links do have access to those properties but cannot access them from other links (eg mega or dropbox)
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 25.03.2024 at 18:16. |
#15
|
|||
|
|||
Quote:
How? |
#16
|
||||
|
||||
I've tested Jiaz' idea because it sounded good and it makes sense but it doesn't work.
Explanation: When a kemono.su user profile is crawled, JDownloader does never "see" or process the single "/post/..." URLs. The posts are crawled as part of the user crawler because this is how their API works (reference: kemono.su/api/schema). This means that from non-kemono URLs crawled as part of a user crawl process, there is no "source URL" reference on single post kemono URLs. This ultimately means that for all items resulting from a profile crawl process, all you golt as a source URL is the kemono profile URL ("kemono.su/portal/user/username"). @VanTuz what you want is either _impossible_ or only using the following tricks:
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#17
|
|||
|
|||
It's clear. It's a pity that nothing can be done about it, because I'm not a programmer. It's just that when yiff. party existed, the program put posts in folders, plus it did this with pixiv, but for some reason it doesn't want to work with this site. And I hoped that the program sees that a link to a mega was found on such and such a post, so you can put it in a folder with the name of the post in which the link was found, but since the API of the site does not allow this, I will download as is.
|
#18
|
||||
|
||||
@VanTuz: we have one more idea and currently testing it
Update: Good news, I have found a way! Bad news, it will take some time before the required change will be available in update.will further discuss this with pspzockerscene
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 25.03.2024 at 23:40. |
#19
|
|||
|
|||
There is a new problem - the program does not pick up links from the dropbox if you insert a link to a post that contains a link to the dropbox. If you just add a link to the dropbox, everything is fine.
**External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#20
|
||||
|
||||
@VanTuz: works fine here, you have to Enable Settings->Plugins->kemono.su
"Crawl http links in post text"
__________________
JD-Dev & Server-Admin |
#21
|
|||
|
|||
And indeed, it was turned off. I just remember that this setting was turned on and I was afraid that something was broken.
|
#22
|
||||
|
||||
@VanTuz: Thanks for the feedback and you're welcome!
__________________
JD-Dev & Server-Admin |
#23
|
|||
|
|||
I'm trying to download this author - **External links are only visible to Support Staff****External links are only visible to Support Staff** - but out of more than 700 files, the program finds 593.
If I give direct links to posts, it finds 160 files. In addition, the program really does not interfere with picking up links from a text file. UPD. When I enable the Packagizer rule, the program finds fewer files. Last edited by VanTuz; 07.04.2024 at 17:57. |
#24
|
||||
|
||||
@VanTuz: sounds like you're running in sort of rate limit and/or crawl-/bot protection. Please provide a debug log for when this happens.
Happens for me too, first try I got around 880 links, next time only 300, next time only 450. the requests are blocking.
__________________
JD-Dev & Server-Admin |
#25
|
||||
|
||||
What do you mean by that? Adding an user url requires less requests and more likely to fetch/process more links than 800 inidividual links with minimum 800 requests resulting nearly guaranteed in rate limit or protection
__________________
JD-Dev & Server-Admin |
#26
|
|||
|
|||
08.04.24 00.55.19 <--> 08.04.24 00.57.55 jdlog://7202411370661/
Last edited by VanTuz; 08.04.2024 at 01:02. |
#27
|
||||
|
||||
@VanTuz: As I thought/expected, it's caused by rate limit. Please try again with next plugin update
__________________
JD-Dev & Server-Admin |
#28
|
|||
|
|||
Thanks a lot, everything is working. And this is very pleasing, because gallery-dl ran into that very limit, and jdownloader downloaded everything.
|
#29
|
||||
|
||||
@VanTuz: Thanks for the feedback. There was just some little thing missing by work of pspzockerscene but looks like the rate limit handling works good
__________________
JD-Dev & Server-Admin |
#30
|
|||
|
|||
Previously, when I downloaded individual posts by some author, the files that the author posted on mega fell into the same folder with the files from kemono, but now they fall into a separate folder, which is inconvenient. Is there any way to turn it off?
**External links are only visible to Support Staff****External links are only visible to Support Staff** |
#31
|
||||
|
||||
Quote:
No. MEGA links mostly lead to file/folder structures which JD will try to keep. In order to do this, they need to go into separate packages, see: https://support.jdownloader.org/de/k...lder-structure This is similar to kemono plugin-properties not being set on resulting MEGA files/links, see: https://board.jdownloader.org/showth...579#post528579
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#32
|
|||
|
|||
But I remember absolutely clearly that all files from all file sharing sites in the post ended up in the same folder, not in different ones.
|
#33
|
||||
|
||||
Then this must have been a bug.
I just came back from holiday so I got no good idea what was changed in the meatime though then again as said, the current behavior is technically the correct one.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#34
|
|||
|
|||
Got it. I will try to ask for help in the topic about event scripter programming, maybe they will help.
|
Thread Tools | |
Display Modes | |
|
|