JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 16.03.2020, 21:27
D-clone D-clone is offline
Junior Loader
 
Join Date: Mar 2020
Posts: 11
Cool Login problem

Hello to everyone !
My intention is to download and automatically sort files from a single site, requiring login, into different folders in my PC. The site is (4trader.net/data/).
Most of files are *.txt, some *.exe and some *.zip.
I enabled the Packagiser.
I created appropriate rules: a separate rule for any destination folder.
I tested filter rules successfully (4trader.net/data/)! During this test I saved the login info for the source site. The username and password info were saved successfully in Basic Authentication. (When trying to manually add login info in Basic Authentication tab, the filter rules test failed)
When pressing the download button, nothing is downloaded!
Any ideas? You can find attached two screenshots!
Thank you in advance for your help!
Dimi
Attached Thumbnails
Clipboard01.jpg   Clipboard05.jpg  

Last edited by D-clone; 16.03.2020 at 21:30.
Reply With Quote
  #2  
Old 16.03.2020, 21:52
D-clone D-clone is offline
Junior Loader
 
Join Date: Mar 2020
Posts: 11
Default

2nd screenshot

Quote:
Originally Posted by D-clone View Post
Hello to everyone !
My intention is to download and automatically sort files from a single site, requiring login, into different folders in my PC. The site is (4trader.net/data/).
Most of files are *.txt, some *.exe and some *.zip.
I enabled the Packagiser.
I created appropriate rules: a separate rule for any destination folder.
I tested filter rules successfully (4trader.net/data/)! During this test I saved the login info for the source site. The username and password info were saved successfully in Basic Authentication. (When trying to manually add login info in Basic Authentication tab, the filter rules test failed)
When pressing the download button, nothing is downloaded!
Any ideas? You can find attached two screenshots!
Thank you in advance for your help!
Dimi
Attached Thumbnails
Clipboard05.jpg  
Reply With Quote
  #3  
Old 17.03.2020, 01:43
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,947
Default

please provide the log from said session http://jdownloader.org/knowledge/wik...d-log?s=lng_en
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #4  
Old 17.03.2020, 13:57
D-clone D-clone is offline
Junior Loader
 
Join Date: Mar 2020
Posts: 11
Default

Thank you for your support! My log ID is
( 17.03.20 14.51.12 <--> 17.03.20 14.52.52 jdlog://8126715302851/ )
Reply With Quote
  #5  
Old 17.03.2020, 18:01
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 52,373
Default

Your log does not contain any usable information.

Please enable: Settings --> Advanced Settings --> log debug mode --> Restart JD, make sure your error happens and upload another log.

You can also send me your logindata including example URLs via PN so I can check this.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #6  
Old 17.03.2020, 21:36
D-clone D-clone is offline
Junior Loader
 
Join Date: Mar 2020
Posts: 11
Default

Quote:
Originally Posted by pspzockerscene View Post
Your log does not contain any usable information.

Please enable: Settings --> Advanced Settings --> log debug mode --> Restart JD, make sure your error happens and upload another log.

You can also send me your logindata including example URLs via PN so I can check this.

-psp-
Thank you for your help!
Unfortunately the login problem was not solved
My updated log ID is:
17.03.20 22.13.02 <--> 17.03.20 22.20.16 jdlog://2636715302851/ .
I sent you a PM with URL and login data !
Reply With Quote
  #7  
Old 18.03.2020, 12:27
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 52,373
Default

Hi,
I'am unable to reproduce your issue here.
Your log shows several requests which di not even contain the authorization header --> Did you maybe disable your added authorization by mistake?

I've manually added your logindata to the Basic Auth logins:
**External links are only visible to Support Staff**
Works just fine here.

Your packagizer rules will always be testable like this because your URLs contain filenames and you did not enable online check in your rule.
If you really have your described login issue, your rule would not work anymore if you enable the condition: File --> Is --> Online


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #8  
Old 18.03.2020, 21:51
D-clone D-clone is offline
Junior Loader
 
Join Date: Mar 2020
Posts: 11
Default

Quote:
Originally Posted by pspzockerscene View Post
Hi,
I'am unable to reproduce your issue here.
Your log shows several requests which di not even contain the authorization header --> Did you maybe disable your added authorization by mistake?
-psp-
Thank you once more for your support !
No, the basic authentication credential is enabled.

Quote:
Originally Posted by pspzockerscene View Post
Your packagizer rules will always be testable like this because your URLs contain filenames and you did not enable online check in your rule.
If you really have your described login issue, your rule would not work anymore if you enable the condition: File --> Is --> Online
-psp-
Thank you for yor explanation!
So, I enabled in every Packagiser rule
a) "the file is online" option
b) the "enable download" option

After these modifications and, also, after installing the latest update the download finally started !

But, while downloading normally, i tried to increase the max chunks per download and max simultaneous downloads. After that, the download process was terminated, and, even when I manually start the download, nothing is really downloaded !

My updated log ID is:
18.03.20 22.36.55 <--> 18.03.20 22.24.04 jdlog://7866715302851/
Reply With Quote
  #9  
Old 19.03.2020, 17:03
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 52,373
Default

So nothing happens?
Not even an error?
Does it work better with only 1 connection per file?
According to your log, all of your downloads went through successfully.

I tested it with about 100 URLs and it worked fine as well.

-psp-
EDIT

Is your problem related to your packagizer rule?
Please keep in mind that these rules are applied only once on newly added URLs!
... or I just didn't understand what your problem is yet ...
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist

Last edited by pspzockerscene; 19.03.2020 at 17:05.
Reply With Quote
  #10  
Old 19.03.2020, 21:21
D-clone D-clone is offline
Junior Loader
 
Join Date: Mar 2020
Posts: 11
Default

Quote:
Originally Posted by pspzockerscene View Post
... or I just didn't understand what your problem is yet ...
My intention is to download and automatically sort files from a single site, requiring login, into different folders in my PC. The site is (4trader.net/data/).
I enabled the Packagiser.
I created appropriate rules: a separate rule for any destination folder.

According to your instructions, I enabled in every Packagiser rule
a) "the file is online" option
b) the "enable download" option

After these modifications and, also, after installing the latest update the download finally started to folders specified in my PC!

Quote:
Originally Posted by pspzockerscene View Post
So nothing happens?
Not even an error?
Does it work better with only 1 connection per file?
But, while downloading normally, i tried to increase the max chunks per download and max simultaneous downloads. After that, the download process was terminated, and, even when I manually start the download, nothing is really downloaded !
So, I tried to minimize both the max chunks per download and max simultaneous downloads to 1.
But again nothing is downloaded

Quote:
Originally Posted by pspzockerscene View Post
Is your problem related to your packagizer rule?
Please keep in mind that these rules are applied only once on newly added URLs!
Sorry I didn't get that!
Thank you for yor patience

--- Edit ---

The only way to start the downloads was to select:
File --> Analyze text with links
but the problem was that all files were downloaded to a single folder (meaning that packagiser rules were not applied, although I unchecked the option "information overwrites packagiser rules") !

Last edited by D-clone; 19.03.2020 at 22:16.
Reply With Quote
  #11  
Old 20.03.2020, 11:04
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 52,373
Default

Quote:
Originally Posted by D-clone View Post
My intention is to download and automatically sort files from a single site, requiring login, into different folders in my PC. The site is (4trader.net/data/).
I enabled the Packagiser.
I created appropriate rules: a separate rule for any destination folder.

According to your instructions, I enabled in every Packagiser rule
a) "the file is online" option
b) the "enable download" option
You do not have to tick the "the file is online" checkbox - this was just to tell you that with this enabled and the rule not working, it would indicate that your login does not work --> This is not the case, your login is working fine.

Quote:
Originally Posted by D-clone View Post
But, while downloading normally, i tried to increase the max chunks per download and max simultaneous downloads. After that, the download process was terminated, and, even when I manually start the download, nothing is really downloaded !
So, I tried to minimize both the max chunks per download and max simultaneous downloads to 1.
But again nothing is downloaded
Better use only 1 max. chunk - you can try to reset existing entries (context menu) and see if it works then.
As said your log contains ZERO errors.

Quote:
Originally Posted by D-clone View Post
Sorry I didn't get that!
Thank you for yor patience

--- Edit ---

The only way to start the downloads was to select:
File --> Analyze text with links
but the problem was that all files were downloaded to a single folder (meaning that packagiser rules were not applied, although I unchecked the option "information overwrites packagiser rules") !
So just to make sure I understand what you mean:
You've created a packagizer rule to change the download path based on the given filename.
You're adding the URL "***.net/data/" to JDownloader and let it parse all contents of that --> Then you start to download.
Your packagizer rule is sometimes now applied so your files get downloaded into the wrong folder?

Please export all relevant packagizer rules and post them here - you can zip them and upload them to our forum.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #12  
Old 20.03.2020, 12:05
D-clone D-clone is offline
Junior Loader
 
Join Date: Mar 2020
Posts: 11
Default

Thank you for your support!

Quote:
Originally Posted by pspzockerscene View Post
You've created a packagizer rule to change the download path based on the given filename.
Exactly! I have created appropriate rules: a separate rule for a separate destination folder, based on the given filename.

Quote:
Originally Posted by pspzockerscene View Post
You're adding the URL "***.net/data/" to JDownloader and let it parse all contents of that --> Then you start to download.
I select
"File --> Analyze text with links" the URL "***.net/data/
but the problem is that all files are downloaded to a single folder (meaning that packagiser rules are not applied, although I unchecked the option "information overwrites packagiser rules") !

Last edited by D-clone; 20.03.2020 at 12:09.
Reply With Quote
  #13  
Old 20.03.2020, 12:07
D-clone D-clone is offline
Junior Loader
 
Join Date: Mar 2020
Posts: 11
Default

These are the relevant packagizer rules!

___D-clone.zip
Reply With Quote
  #14  
Old 20.03.2020, 12:51
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 52,373
Default

That's a lot of rules!
Now I only need to understand which one should match which filenames.
First of all, I think you simply made mistakes while making the rules as some of them are working fine here e.g. the one that goes to path:
Code:
E:\Data\Analysis Daily Data\Current
Now as an example for an entry which will not be affected by any rule but probably should:
Code:
010319d.txt
Now I see you have a rule for files ending with "d.txt":
Code:
^[0-3][0-9]1019d\.txt$
If this is supposed to match for my picked example filename, you would have to change it to:
Code:
^[0-3][0-9]+d\.txt$
--> "010319d.txt" would then go to:
Code:
E:\Data\Analysis Daily Data\Current
Sure there is a lot of information I do not know e.g.:
Do you want to "sort" all of these files or are there ones which will not be affected by any rule?

Please post some example filenames and tell me which rule they're supposed to trigger this might be the best way to help.

Also please keep in mind that the way you've created the rules, they would be applied on ALL filenames you add to JDownloader no matter where they come from.
If you plan to use JDownloader to download from other sources in the future and you want to leave the rules enabled, you should consider also enabling Downloadurl --> contains --> "4trader.net" (without "")

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #15  
Old 21.03.2020, 16:27
D-clone D-clone is offline
Junior Loader
 
Join Date: Mar 2020
Posts: 11
Default

Quote:
Originally Posted by pspzockerscene View Post
Please post some example filenames and tell me which rule they're supposed to trigger this might be the best way to help.
file name (example) --> rule ---> destination folder
HTML Code:
171019d.txt  --> ^[0-3][0-9]1019d\.txt$   --> E:\Data\Analysis Daily Data\Current .
 171019i.zip   --> ^[0-3][0-9]1019i\.zip$  --> E:\Data\Analysis Intra\Current .
 171019sme.txt -->  ^[0-3][0-9]1019sme\.txt$ --->  E:\Data\Analysis Intra SME\Current . 
 x171019.txt   -->  ^x[0-3][0-9]1019\.txt$  ---> E:\Data\Analysis Xenoi Deiktes\Current . 
 ix171019.txt --->  ^ix[0-3][0-9]1019\.txt$  --->  E:\Data\Analysis Intra Xena\Current.
 usa171019.txt --->  ^usa[0-3][0-9]1019\.txt$  --->  E:\Data\Analysis USA Metoxes\Current.
 dax171019.txt --->  ^dax[0-3][0-9]1019\.txt$  --->  E:\Data\Analysis DAX30 Metoxes\Current.
 stoxx1019.txt  --->  ^stoxx[0-3][0-9]1019\.txt$  ---> E:\Data\Analysis STOXX50 Metoxes\Current .
 f171019d.txt ---> ^f[0-3][0-9]1019d\.txt$  --->  E:\Data\Analysis Forex Daily\Current  .
 f171019i.zip  --->  ^f[0-3][0-9]1019i\.zip$  --->  E:\Data\Analysis Intra Forex\Current  .
 1710xak.txt ---> ^[0-3][0-9]1019xak\.txt$  -->  E:\Data\Analysis Cyprus\Current  .
file name (always the same) --> rule ---> destination folder (same for all)
HTML Code:
Metoxes_ASE.exe   ---> ^Metoxes_ASE\.exe$  --->   E:\Data\Analysis current historic .
ASE_intraday.exe  --->  ^ASE_intraday\.exe$  --->  E:\Data\Analysis current historic .
SME_Continuous.exe ---> ^SME_Continuous\.exe$  -->  E:\Data\Analysis current historic  .
Xenoi_Deiktes.exe  --->  ^Xenoi_Deiktes\.exe$  --->  E:\Data\Analysis current historic  .
Xena_Intraday.exe  --->  ^Xena_Intraday\.exe$  --->  E:\Data\Analysis current historic  .
Metoxes_USA.exe    ---> ^Metoxes_USA\.exe$  --->  E:\Data\Analysis current historic  .
Metoxes_DAX.exe  --->  ^Metoxes_DAX\.exe$  --->  E:\Data\Analysis current historic  .
Metoxes_STOXX.exe --->  ^Metoxes_STOXX\.exe$ --->  E:\Data\Analysis current historic  .
Forex_Daily.exe   --->  ^Forex_Daily\.exe$  --->  E:\Data\Analysis current historic  .
Forex_Intraday.exe   --->  ^Forex_Intraday\.exe$  --->  E:\Data\Analysis current historic  .
Metoxes_XAK.exe  --->  ^Metoxes_XAK\.exe$  --->  E:\Data\Analysis current historic  .
The corresponding Packagiser rules are included in the file attached.

So, my problem is that none of these rules is triggered, although the positive "test filename expression" when right-clicking on rule filename expression !


Quote:
Originally Posted by pspzockerscene View Post
Now I see you have a rule for files ending with "d.txt":
^[0-3][0-9]1019d\.txt$
If this is supposed to match for my picked example filename, you would have to change it to:
^[0-3][0-9]+d\.txt$
I'll give it a chance !

Quote:
Originally Posted by pspzockerscene View Post
Do you want to "sort" all of these files or are there ones which will not be affected by any rule?
No, it is not my intention to make any kind of sorting. I just want to download to separate olders, based on the filenames!

Quote:
Originally Posted by pspzockerscene View Post
Also please keep in mind that the way you've created the rules, they would be applied on ALL filenames you add to JDownloader no matter where they come from.
If you plan to use JDownloader to download from other sources in the future and you want to leave the rules enabled, you should consider also enabling Downloadurl --> contains --> "4trader.net" (without "")
Thank you for the tip and your help !
Attached Files
File Type: zip D-clone_2020_03_21_v1.zip (1.8 KB, 0 views)

Last edited by D-clone; 21.03.2020 at 16:33.
Reply With Quote
  #16  
Old 21.03.2020, 17:50
D-clone D-clone is offline
Junior Loader
 
Join Date: Mar 2020
Posts: 11
Default

Edit:

Finally, all my Packagiser rules are now triggered!

The following tip was proved to be unnecessary!

Quote:
Originally Posted by pspzockerscene View Post
Now as an example for an entry which will not be affected by any rule but probably should:
Code:
010319d.txt
Now I see you have a rule for files ending with "d.txt":
Code:
^[0-3][0-9]1019d\.txt$
If this is supposed to match for my picked example filename, you would have to change it to:
Code:
^[0-3][0-9]+d\.txt$
I have two last questions:

My first question is the following:
The first group of rules are about files hosted in 4trader.net/data/.
The second group of rules are about files hosted in 4trader.net/data/historic/.

Is there some option, while analysing 4trader.net/data, to include "subfolders" eg 4trader.net/data/historic/? The point here is to avoid double analysing process, meaning once for the "parent folder" and once for the "subfolder". Of course, this could be bypassed by entering both links!

My second question is about the "parent folder", containg hundreds of files. Every time JDownloads analyzes it, is there the possibility to download just the files triggered by the Packagiser rules and nothing more?

HTML Code:
The only way I have found so far is to manually delete from the Linkgrabber the "various files" contents
Thank you once more for your help!

Last edited by D-clone; 21.03.2020 at 18:25.
Reply With Quote
  #17  
Old 22.03.2020, 01:30
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,947
Default

rules are processed as this time top to bottom, generic rules first and specific rules last.

in respects to /data/ vs /data/historic/ you can place source contains = /data/historic/ and prevent it from been triggered multiple times. You can tighten up further and use equals and or use patterns.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #18  
Old 22.03.2020, 22:10
D-clone D-clone is offline
Junior Loader
 
Join Date: Mar 2020
Posts: 11
Default

Thank you for the tips!
Reply With Quote
  #19  
Old 23.03.2020, 12:00
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 52,373
Default

Thanks for your feedback.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
How to create a log || Wie man einen Log erstellt
Captcha FAQ EN || Captcha FAQ DE || Erste Schritte & Tutorials
JDownloader 2 Setup Download
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 16:08.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.