JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 26.11.2022, 22:13
bdrifter bdrifter is offline
Junior Loader
 
Join Date: Jun 2020
Posts: 10
Default Folder Watch crawljob load order

Hello,

Is there a way for Folder Watch to load a folder full of crawljobs based on the order they were created, or the filename?

I have a folder where a stream of crawljobs would be created throughout the day. The order they are created in is important to me. Most of the time, my JDownloader is running, and is set to automatically download the links of any incoming crawljobs as they appear, one by one. But sometimes if my JDownloader isn't running, the crawljobs would accumulate, and when I launch JDownloader, it would load all the crawljobs, but not in the order they were added, which I want to avoid.

I'd be interested in any combination of settings that would fix that!

Thanks
Reply With Quote
  #2  
Old 26.11.2022, 22:39
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,044
Default

@bdrifter: right now the order of the files loaded/processed is the order they are provided by your OS/filesystem. but more important please know that JDownloader does not guarantee any adding/crawling/processing order of links that you add. Link A B C can result in processing of B C A and order in lost as C A B.

any specific reason why the order is important? and what order is important? crawling? adding? downloading? and why? maybe you can explain your use case and we can find a solution to your problem then
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 27.11.2022, 10:03
bdrifter bdrifter is offline
Junior Loader
 
Join Date: Jun 2020
Posts: 10
Default

@Jiaz: Hey thanks for the response,

I use the crawljobs to download image sequences in the order they are presented in, so their creation/modification dates naturally order them in my folders.

I use a python script to do this, and I've tried adding a sequence of links in a single crawljob, based on their order, but JDownloader doesn't seem to follow this order, so I separated each link into individual crawljob files instead, and generate them in 2 second intervals, that way at least when JDownloader is running, the links are added in order

I'd love to know some other way to do this
Reply With Quote
  #4  
Old 30.11.2022, 18:02
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,044
Default

@bdrifter: As you use a python script to create your crawljob files I would recommend to use this approach, see
https://support.jdownloader.org/Know...ame-via-anchor
Your script can add additional information (eg numbering) or set the final filename in URL and with help of packagizer rule, JDownloader will then add your additional information to filename and/or use your preset final filename.

In case you need further help with this or got questions, just ask
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 30.11.2022, 18:02
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,044
Default

Quote:
Originally Posted by bdrifter View Post
but JDownloader doesn't seem to follow this order, so I separated each link into individual crawljob files instead
As explained, there are no such guarantees.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #6  
Old 04.12.2022, 18:13
bdrifter bdrifter is offline
Junior Loader
 
Join Date: Jun 2020
Posts: 10
Default

Thanks for the explanation. I'll see if I can figure out a workaround then.
Reply With Quote
  #7  
Old 04.12.2022, 20:57
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,044
Default

Quote:
Originally Posted by bdrifter View Post
Thanks for the explanation. I'll see if I can figure out a workaround then.
In case you need help/got questions, just ask
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 21:04.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2023, Jelsoft Enterprises Ltd.