#1
|
|||
|
|||
Folderwatch, Folder output, and Preserving Online Folder structure
I have a crawljob like this:
autoStart=FALSE extractAfterDownload=FALSE downloadFolder=C:\DOWNLOADFOLDER\ text=DOWNLOADURL It works fine, but it doesn't preserve the online folder structure. If I add overwritePackagizerEnabled=FALSE, then it downloads to the default download folder listed in general settings, even though packagizer has no rules for download directories (apart from "Create subfolders on package name" and "Adopt folder structure", the latter of which I want). So basically if the download link leads to a folder with: folder1\folder1a\file.ext folder1\file2.ext folder2\file3.ext file4.ext I want it to output to: C:\DOWNLOADFOLDER\folder1\folder1a\file.ext C:\DOWNLOADFOLDER\folder1\file2.ext C:\DOWNLOADFOLDER\folder2\file3.ext C:\DOWNLOADFOLDER\file4.ext with C:\DOWNLOADFOLDER\ being specified in the crawljob file. I tried appending <jd:subfolderbyplugin> to downloadFolder=C:\DOWNLOADFOLDER\ but jdownloader didn't parse it correctly, thinking it wanted a subfolder called <jd:subfolderbyplugin>, which is of course an invalid file name. Also writing a documentation might be helpful. It's a little annoying trying to learn this program by googling and searching through threads. Just doing a google search for "setBeforePackagizerEnabled" comes up with 0 results and same with a forum search, and I don't know what this option does (it doesn't seem to do anything). |
#2
|
||||
|
||||
Do you have example links? Because only very few plugins make use of the subfolderbyplugin feature, for example mega folders.
Please provide example links, so we can better help you. I just added 'setBeforePackagizerEnabled' recently to help a customer with his crawljobs and packagizer rules. That's why you do not find anything about it. I'm still working on a good solution to this
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 03.11.2017 at 15:36. |
#3
|
||||
|
||||
Added support with next core update
You've got to create a PackagizerRule like this Code:
match always or whatever you need set downloadDirectory to <jd:append>/<jd:subfolderbyplugin> overwritePackagizerEnabled=false the new variable <jd:append> from helps to append values and not replace/set them I've just tested with mega folder links and it is working fine for me
__________________
JD-Dev & Server-Admin |
#4
|
||||
|
||||
Quote:
That's why the new variable <jd:append> got added, so you can either set or append values
__________________
JD-Dev & Server-Admin |
#5
|
|||
|
|||
Quote:
I'm dealing with a variety of hosts (mediafire, mega, dropbox, off the top of my head) so I didn't give a link. It seems that the implementation works but perhaps the plugins do not. For example when using this link: **External links are only visible to Support Staff****External links are only visible to Support Staff** It does not output the directory recursively, but will instead take: subfolder\subfolder2\file1 subfolder\file2 to: C:\download\subfolder2\file1 C:\download\subfolder\file2 Instead of the desirable: C:\download\subfolder\subfolder2\file1 C:\download\subfolder\file2 As well this link: **External links are only visible to Support Staff****External links are only visible to Support Staff** It treats each subfolder as a separate package, rather than a subfolder. Although not related I also found an error with the mediafire plugin (at least that's where I've noticed this issue most) that if the file has an "[" (ignore quotations) in the name (off the top of my head the "[" not being closed but I'm not sure if that affects it), it will break as jdownloader will append a bunch of data relating to the download into the filename, requiring the user to manually rename the file before it can properly download. Unfortunately I don't quite have time to pin down a link for the [ error but I'll try to get it to you later if you need it. Also thanks for all your work on this program. |
#6
|
||||
|
||||
I will check those links on Monday. FolderStructure support must be implemented in Plugins. Most plugins do only provide package naming support.
__________________
JD-Dev & Server-Admin |
#7
|
||||
|
||||
In case you've got more example links, please send them to support@jdownloader.org
__________________
JD-Dev & Server-Admin |
#8
|
||||
|
||||
Quote:
__________________
JD-Dev & Server-Admin |
#9
|
||||
|
||||
I've added folder structure support to mediafire and dropbox plugins, wait for next plugin update
__________________
JD-Dev & Server-Admin |
#10
|
|||
|
|||
Quote:
**External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#11
|
||||
|
||||
Thanks! Will be fixed with next update
__________________
JD-Dev & Server-Admin |
#12
|
|||
|
|||
Hi thanks for everything, one more request, keeping the package name to the first folder name.
For example here: **External links are only visible to Support Staff****External links are only visible to Support Staff** JDownloader does not keep the package name the same for the subfolders, it instead uses a different package name for the subfolders when specified with <jd:packagename>. I believe this has to do with how Jdownloader discovers subfolders, and I'm not sure how easy it would be to fix, but it would be appreciated. In short I would like it to be: <jd:append>\HYRULE WARRIORS LINK PACK\<jd:subfolderbyplugin> Last edited by blueguy6; 17.11.2017 at 00:51. |
#13
|
|||
|
|||
bump
|
#14
|
||||
|
||||
packagename and subfolders are different things.
Each Dropbox folder has a packagename and the subfolder just specifies the relative path of that package. The root does have correct packagename but no subfolder because it is root (start of the adding process) I've added special support in DropBox Plugin for this. After next update , Settings-Plugins, search for dropbox. There will be two entries in list. One for host plugin, one for crawler. In Crawling plugin you can enable *Include Root Subfolder*.
__________________
JD-Dev & Server-Admin |
#15
|
|||
|
|||
Quote:
For mediafire it seems to work unless if the main folder has no files in it, for example compare this link which works in that matter: **External links are only visible to Support Staff****External links are only visible to Support Staff** and this one which does not: **External links are only visible to Support Staff****External links are only visible to Support Staff** From my testing with mega it seems to implement the root folder as desired regardless of whether or not there is a file in it, so I do not think any change needs to be made. (example links that work regardless, with no files in root folder: **External links are only visible to Support Staff****External links are only visible to Support Staff**, with files in root folder: **External links are only visible to Support Staff****External links are only visible to Support Staff**) Looking at Google drive, it seems to not generate the root folder, here are links: **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** Onedrive seems to have a more extreme problem in which absolutely no folder structure is preserved, see: **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** All of the links provided were added into link grabber with the only packagizer rule affecting directories being an a rule affecting all links that sets the download folder to <jd:append>/<jd:subfolderbyplugin>. Thanks again for your efforts in making this invaluable tool. |
#16
|
||||
|
||||
So mega plugin with custom setting working fine as expected now?
Or any change wishes? I've created a ticket to implement same setting for provided example links as well
__________________
JD-Dev & Server-Admin |
#17
|
|||
|
|||
Quote:
There are no changes I can think of except the already stated to onedrive, mediafire, and google drive. |
#18
|
||||
|
||||
Thanks for the feedback. I will update the other plugins as soon as I find time
__________________
JD-Dev & Server-Admin |
#19
|
|||
|
|||
Resubmitting this feature request with the issues noted in post 15 (the links are still valid)
https://board.jdownloader.org/showpo...3&postcount=15 1. Mediafire does not generate folder structure if there are no files in the root folder (ie the root has only subfolders), otherwise it does 2. Onedrive does not generate folder structure at all 3. Google drive does not generate the root folder name #1 and 2 I would really like to see fixed especially. |
#20
|
||||
|
||||
2.) already fixed with latest update, please test
__________________
JD-Dev & Server-Admin |
#21
|
||||
|
||||
Just to be sure what you exactly want to happen
You want JDownloader to preserve folder name/structure even when manually adding a single folder. currently it was meant to only work for subfolders and not a folder. adding a folder link -> adding the links within a package /test/test.txt -> /test.txt will result adding a folder with subfolders -> preserve the subfolder structure /test/test.txt -> /test.txt /test/test2/test2.txt -> /test2/text2.txt you want /test/test.txt -> /test/test.txt /test/test2/test2.txt -> /test/test2/text2.txt ?
__________________
JD-Dev & Server-Admin |
#22
|
|||
|
|||
onedrive does indeed appear to be fixed assuming it's not like mediafire (I can't seem to find a link with only subfolders in root to test).
Quote:
Edit: Looks like dropbox has a bug, see here: **External links are only visible to Support Staff****External links are only visible to Support Staff** It seems to only grab the first subfolder and not the other subfolders (which can be seen at the very bottom in the browser window). It also appears that dropbox has a specific prompt for adding subfolders that comes up after adding link, but this only sometimes comes up. An option to make grabbing subfolders default behavior (since it is default for other plugins) would be nice. Last edited by blueguy6; 25.01.2019 at 18:13. |
#23
|
||||
|
||||
Can you provide more example links please, then I will look at them one by one.
Would it be okay if we add this as optional in plugin settings as changing default behaviour would be no good. so default: /test/test.txt -> /test.txt optional: /test/test.txt -> /test/test.txt okay?
__________________
JD-Dev & Server-Admin |
#24
|
|||
|
|||
Quote:
Switching google drive and onedrive to function like the rest of the plugins seems easier and 'better' to me, but if you want to change all of them to work something like <jd:append>/<jd:root>/<jd:subfolderbyplugin> I'm fine with that. For the dropbox plugin only getting some of the subfolders, you can check any of these links: **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** For mediafire not generating the folder structure if there is no file in root: **External links are only visible to Support Staff****External links are only visible to Support Staff** vs when it works perfectly (including generating the root folder) when there is a file in root: **External links are only visible to Support Staff****External links are only visible to Support Staff** For google drive not adding the root folder, any of these: **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** For onedrive not adding root folder: **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#25
|
||||
|
||||
I know it's over 2.5 years later but I've just gone through old tickets and closed the one related to this thread.
The mentioned extra setting for dropbox.com has been removed so now our default handling for subfolder-structures of cloud services should be just fine. You can find more information about this HERE. All of our crawlers are designed in a way t try to find- and use/set the current/root folder correctly so that the structure you get afterwards resembles 100% the one from that cloud service. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
Thread Tools | |
Display Modes | |
|
|