JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 05.09.2019, 21:03
taychive taychive is offline
I will play nice!
 
Join Date: Feb 2019
Posts: 4
Default CrawlJob settings for downloading link with multiple files to custom path and keeping

Hi, I am trying to figure out what options i need to define in my crawljob file to make jdownloader have the desired output.
I need to input a Mega.nz url via crawljob then have Jdownloader save it to a custom folder path and retain the correct subfolder structure.
So far i have only been able to do one or the other but not both.


Sample input Mega.nz URL:**External links are only visible to Support Staff****External links are only visible to Support Staff**
Same as sample Mega.nz URL above but in plain text for easy reference:
/TopFolder/TopFolder.txt
/TopFolder/SubFolder1/SubFolder1.txt
/TopFolder/SubFolder1/SubSubFolder1/SubSubFolder1.txt
/TopFolder/SubFolder2/SubFolder2.txt
/TopFolder/SubFolder2/SubSubFolder2/SubSubFolder2.txt

Desired Output:
/config/Downloads/Output/unsorted/TopFolder/TopFolder.txt
/config/Downloads/Output/unsorted/TopFolder/SubFolder1/SubFolder1.txt
/config/Downloads/Output/unsorted/TopFolder/SubFolder1/SubSubFolder1/SubSubFolder1.txt
/config/Downloads/Output/unsorted/TopFolder/SubFolder2/SubFolder2.txt
/config/Downloads/Output/unsorted/TopFolder/SubFolder2/SubSubFolder2/SubSubFolder2.txt


The following is my current crawljob file and what i have tried.

My current crawljob file format:
text=**External links are only visible to Support Staff** downloadFolder=/config/Downloads/Output/unsorted/<jd:packagename>
enabled=true
autoStart=TRUE
forcedStart=TRUE
autoConfirm=TRUE

This works except it does not keep the subfolder structure:
/config/Downloads/Output/unsorted/TopFolder/TopFolder.txt
/config/Downloads/Output/unsorted/SubFolder1/SubFolder1.txt
/config/Downloads/Output/unsorted/SubSubFolder1/SubSubFolder1.txt
/config/Downloads/Output/unsorted/SubFolder2/SubFolder2.txt
/config/Downloads/Output/unsorted/SubSubFolder2/SubSubFolder2.txt

I did some searching and i think that this is because i need the packagizer rules enabled. So i update my crawljob to:
text=**External links are only visible to Support Staff** downloadFolder=/config/Downloads/Output/unsorted/<jd:packagename>
enabled=true
autoStart=TRUE
forcedStart=TRUE
autoConfirm=TRUE
overwritePackagizerEnabled=FALSE

Subfolders now have the correct structure but now my location or downloadFolder option is ignored and also a "/" is missing:
/Output/unsortedTopFolder/TopFolder.txt
/Output/unsortedTopFolder/SubFolder1/SubFolder1.txt
/Output/unsortedTopFolder/SubFolder1/SubSubFolder1/SubSubFolder1.txt
/Output/unsortedTopFolder/SubFolder2/SubFolder2.txt
/Output/unsortedTopFolder/SubFolder2/SubSubFolder2/SubSubFolder2.txt

I seen the note in the Folder Watch plugin that says "use only if text contains one single link" for the option "overwritePackagizerEnabled" but i am not sure if this is the problem or something else.

At this point i realized i am just guessing at what crawljob options/Packagizer rules i need so i decided to make this post asking for help.
I hope someone can point me in the right direction. Thanks in advance.
Reply With Quote
  #2  
Old 06.09.2019, 11:58
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 64,613
Default

JDownloader doesn't create a folder/subfolder for the entry folder/link
You're adding a folder link and it's content is added (subfolders+files) but JDownloader doesn't recursively check where the link itself is.
This is unsupported/undesired *default* behaviour.

For example: user adds your subfolder2-subsubfolder2 link.
User does expect to have a SubSubfolder2.txt file in /downloadfolder/subsubfolder2/subsubfolder2.txt (subsubfolder2 because of <jd:packagename> rule)
and not
/downloadfolder/topfolder/subfolder2/subsubfolder2/subsubfolder2.txt (because of subfolder rule)

We must add optional special handling in plugins to make this possible. Also it's not clear if you just want to keep the TopFolder in place
or you ALWAYS want the complete subfolder structure like explained in example above.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 06.09.2019, 20:28
taychive taychive is offline
I will play nice!
 
Join Date: Feb 2019
Posts: 4
Default

I see this is not default or expected behavior but it is what i am trying to accomplish.
Yes, I do always want the complete folder path including the TopFolder and all subfolders.

I don't mean for you to write special handling in plugins because i was thinking with the right options set this could be accomplished within the crawljob file since this is how i will be adding the links to JDownloader. I was just unsure what combination of crawljob or packagizer rules i need set.

Mega structure:
MegaStructure.PNG

Desired downloaded output:
DesiredOutput.PNG
Reply With Quote
  #4  
Old 06.09.2019, 20:49
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 64,613
Default

will continue here on monday
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 07:01.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.