[Solved] Login problem

[D-clone]

Hello to everyone !
My intention is to download and automatically sort files from a single site, requiring login, into different folders in my PC. The site is (4trader.net/data/).
Most of files are *.txt, some *.exe and some *.zip.
I enabled the Packagiser.
I created appropriate rules: a separate rule for any destination folder.
I tested filter rules successfully (4trader.net/data/)! During this test I saved the login info for the source site. The username and password info were saved successfully in Basic Authentication. (When trying to manually add login info in Basic Authentication tab, the filter rules test failed)
When pressing the download button, nothing is downloaded!
Any ideas? You can find attached two screenshots!

Thank you in advance for your help!
Dimi

[D-clone]

2nd screenshot

Originally Posted by D-clone

Hello to everyone !
My intention is to download and automatically sort files from a single site, requiring login, into different folders in my PC. The site is (4trader.net/data/).
Most of files are *.txt, some *.exe and some *.zip.
I enabled the Packagiser.
I created appropriate rules: a separate rule for any destination folder.
I tested filter rules successfully (4trader.net/data/)! During this test I saved the login info for the source site. The username and password info were saved successfully in Basic Authentication. (When trying to manually add login info in Basic Authentication tab, the filter rules test failed)
When pressing the download button, nothing is downloaded!
Any ideas? You can find attached two screenshots!

Thank you in advance for your help!
Dimi

[raztoki]

please provide the log from said session http://jdownloader.org/knowledge/wik...d-log?s=lng_en

[D-clone]

Thank you for your support! My log ID is
( 17.03.20 14.51.12 <--> 17.03.20 14.52.52 jdlog://8126715302851/ )

[pspzockerscene]

Your log does not contain any usable information.

Please enable: Settings --> Advanced Settings --> log debug mode --> Restart JD, make sure your error happens and upload another log.

You can also send me your logindata including example URLs via PN so I can check this.

-psp-

[D-clone]

Originally Posted by pspzockerscene

Your log does not contain any usable information.

Please enable: Settings --> Advanced Settings --> log debug mode --> Restart JD, make sure your error happens and upload another log.

You can also send me your logindata including example URLs via PN so I can check this.

-psp-

Thank you for your help!
Unfortunately the login problem was not solved
My updated log ID is:
17.03.20 22.13.02 <--> 17.03.20 22.20.16 jdlog://2636715302851/ .
I sent you a PM with URL and login data !

[pspzockerscene]

Hi,
I'am unable to reproduce your issue here.
Your log shows several requests which di not even contain the authorization header --> Did you maybe disable your added authorization by mistake?

I've manually added your logindata to the Basic Auth logins:
**External links are only visible to Support Staff**
Works just fine here.

Your packagizer rules will always be testable like this because your URLs contain filenames and you did not enable online check in your rule.
If you really have your described login issue, your rule would not work anymore if you enable the condition: File --> Is --> Online


-psp-

[D-clone]

Originally Posted by pspzockerscene

Hi,
I'am unable to reproduce your issue here.
Your log shows several requests which di not even contain the authorization header --> Did you maybe disable your added authorization by mistake?
-psp-

Thank you once more for your support !
No, the basic authentication credential is enabled.

Originally Posted by pspzockerscene

Your packagizer rules will always be testable like this because your URLs contain filenames and you did not enable online check in your rule.
If you really have your described login issue, your rule would not work anymore if you enable the condition: File --> Is --> Online
-psp-

Thank you for yor explanation!
So, I enabled in every Packagiser rule
a) "the file is online" option
b) the "enable download" option

After these modifications and, also, after installing the latest update the download finally started !

But, while downloading normally, i tried to increase the max chunks per download and max simultaneous downloads. After that, the download process was terminated, and, even when I manually start the download, nothing is really downloaded !

My updated log ID is:
18.03.20 22.36.55 <--> 18.03.20 22.24.04 jdlog://7866715302851/

[pspzockerscene]

So nothing happens?
Not even an error?
Does it work better with only 1 connection per file?
According to your log, all of your downloads went through successfully.

I tested it with about 100 URLs and it worked fine as well.

-psp-
EDIT

Is your problem related to your packagizer rule?
Please keep in mind that these rules are applied only once on newly added URLs!
... or I just didn't understand what your problem is yet ...

[D-clone]

Originally Posted by pspzockerscene

... or I just didn't understand what your problem is yet ...

My intention is to download and automatically sort files from a single site, requiring login, into different folders in my PC. The site is (4trader.net/data/).
I enabled the Packagiser.
I created appropriate rules: a separate rule for any destination folder.

According to your instructions, I enabled in every Packagiser rule
a) "the file is online" option
b) the "enable download" option

After these modifications and, also, after installing the latest update the download finally started to folders specified in my PC!

Originally Posted by pspzockerscene

So nothing happens?
Not even an error?
Does it work better with only 1 connection per file?

But, while downloading normally, i tried to increase the max chunks per download and max simultaneous downloads. After that, the download process was terminated, and, even when I manually start the download, nothing is really downloaded !
So, I tried to minimize both the max chunks per download and max simultaneous downloads to 1.
But again nothing is downloaded

Originally Posted by pspzockerscene

Is your problem related to your packagizer rule?
Please keep in mind that these rules are applied only once on newly added URLs!

Sorry I didn't get that!
Thank you for yor patience

--- Edit ---

The only way to start the downloads was to select:
File --> Analyze text with links
but the problem was that all files were downloaded to a single folder (meaning that packagiser rules were not applied, although I unchecked the option "information overwrites packagiser rules") !

[pspzockerscene]

Originally Posted by D-clone

My intention is to download and automatically sort files from a single site, requiring login, into different folders in my PC. The site is (4trader.net/data/).
I enabled the Packagiser.
I created appropriate rules: a separate rule for any destination folder.

According to your instructions, I enabled in every Packagiser rule
a) "the file is online" option
b) the "enable download" option

You do not have to tick the "the file is online" checkbox - this was just to tell you that with this enabled and the rule not working, it would indicate that your login does not work --> This is not the case, your login is working fine.

Originally Posted by D-clone

But, while downloading normally, i tried to increase the max chunks per download and max simultaneous downloads. After that, the download process was terminated, and, even when I manually start the download, nothing is really downloaded !
So, I tried to minimize both the max chunks per download and max simultaneous downloads to 1.
But again nothing is downloaded

Better use only 1 max. chunk - you can try to reset existing entries (context menu) and see if it works then.
As said your log contains ZERO errors.

Originally Posted by D-clone

Sorry I didn't get that!
Thank you for yor patience

--- Edit ---

The only way to start the downloads was to select:
File --> Analyze text with links
but the problem was that all files were downloaded to a single folder (meaning that packagiser rules were not applied, although I unchecked the option "information overwrites packagiser rules") !

So just to make sure I understand what you mean:
You've created a packagizer rule to change the download path based on the given filename.
You're adding the URL "***.net/data/" to JDownloader and let it parse all contents of that --> Then you start to download.
Your packagizer rule is sometimes now applied so your files get downloaded into the wrong folder?

Please export all relevant packagizer rules and post them here - you can zip them and upload them to our forum.

-psp-

[D-clone]

Thank you for your support!

Originally Posted by pspzockerscene

You've created a packagizer rule to change the download path based on the given filename.

Exactly! I have created appropriate rules: a separate rule for a separate destination folder, based on the given filename.

Originally Posted by pspzockerscene

You're adding the URL "***.net/data/" to JDownloader and let it parse all contents of that --> Then you start to download.

I select
"File --> Analyze text with links" the URL "***.net/data/
but the problem is that all files are downloaded to a single folder (meaning that packagiser rules are not applied, although I unchecked the option "information overwrites packagiser rules") !

[D-clone]

These are the relevant packagizer rules!

___D-clone.zip

[pspzockerscene]

That's a lot of rules!
Now I only need to understand which one should match which filenames.
First of all, I think you simply made mistakes while making the rules as some of them are working fine here e.g. the one that goes to path:

E:\Data\Analysis Daily Data\Current

Now as an example for an entry which will not be affected by any rule but probably should:

010319d.txt

Now I see you have a rule for files ending with "d.txt":

^[0-3][0-9]1019d\.txt$

If this is supposed to match for my picked example filename, you would have to change it to:

^[0-3][0-9]+d\.txt$

--> "010319d.txt" would then go to:

E:\Data\Analysis Daily Data\Current

Sure there is a lot of information I do not know e.g.:
Do you want to "sort" all of these files or are there ones which will not be affected by any rule?

Please post some example filenames and tell me which rule they're supposed to trigger this might be the best way to help.

Also please keep in mind that the way you've created the rules, they would be applied on ALL filenames you add to JDownloader no matter where they come from.
If you plan to use JDownloader to download from other sources in the future and you want to leave the rules enabled, you should consider also enabling Downloadurl --> contains --> "4trader.net" (without "")

-psp-

[D-clone]

Originally Posted by pspzockerscene

Please post some example filenames and tell me which rule they're supposed to trigger this might be the best way to help.

file name (example) --> rule ---> destination folder

171019d.txt  --> ^[0-3][0-9]1019d\.txt$   --> E:\Data\Analysis Daily Data\Current .
 171019i.zip   --> ^[0-3][0-9]1019i\.zip$  --> E:\Data\Analysis Intra\Current .
 171019sme.txt -->  ^[0-3][0-9]1019sme\.txt$ --->  E:\Data\Analysis Intra SME\Current . 
 x171019.txt   -->  ^x[0-3][0-9]1019\.txt$  ---> E:\Data\Analysis Xenoi Deiktes\Current . 
 ix171019.txt --->  ^ix[0-3][0-9]1019\.txt$  --->  E:\Data\Analysis Intra Xena\Current.
 usa171019.txt --->  ^usa[0-3][0-9]1019\.txt$  --->  E:\Data\Analysis USA Metoxes\Current.
 dax171019.txt --->  ^dax[0-3][0-9]1019\.txt$  --->  E:\Data\Analysis DAX30 Metoxes\Current.
 stoxx1019.txt  --->  ^stoxx[0-3][0-9]1019\.txt$  ---> E:\Data\Analysis STOXX50 Metoxes\Current .
 f171019d.txt ---> ^f[0-3][0-9]1019d\.txt$  --->  E:\Data\Analysis Forex Daily\Current  .
 f171019i.zip  --->  ^f[0-3][0-9]1019i\.zip$  --->  E:\Data\Analysis Intra Forex\Current  .
 1710xak.txt ---> ^[0-3][0-9]1019xak\.txt$  -->  E:\Data\Analysis Cyprus\Current  .

file name (always the same) --> rule ---> destination folder (same for all)

Metoxes_ASE.exe   ---> ^Metoxes_ASE\.exe$  --->   E:\Data\Analysis current historic .
ASE_intraday.exe  --->  ^ASE_intraday\.exe$  --->  E:\Data\Analysis current historic .
SME_Continuous.exe ---> ^SME_Continuous\.exe$  -->  E:\Data\Analysis current historic  .
Xenoi_Deiktes.exe  --->  ^Xenoi_Deiktes\.exe$  --->  E:\Data\Analysis current historic  .
Xena_Intraday.exe  --->  ^Xena_Intraday\.exe$  --->  E:\Data\Analysis current historic  .
Metoxes_USA.exe    ---> ^Metoxes_USA\.exe$  --->  E:\Data\Analysis current historic  .
Metoxes_DAX.exe  --->  ^Metoxes_DAX\.exe$  --->  E:\Data\Analysis current historic  .
Metoxes_STOXX.exe --->  ^Metoxes_STOXX\.exe$ --->  E:\Data\Analysis current historic  .
Forex_Daily.exe   --->  ^Forex_Daily\.exe$  --->  E:\Data\Analysis current historic  .
Forex_Intraday.exe   --->  ^Forex_Intraday\.exe$  --->  E:\Data\Analysis current historic  .
Metoxes_XAK.exe  --->  ^Metoxes_XAK\.exe$  --->  E:\Data\Analysis current historic  .

The corresponding Packagiser rules are included in the file attached.

So, my problem is that none of these rules is triggered, although the positive "test filename expression" when right-clicking on rule filename expression !

Originally Posted by pspzockerscene

Now I see you have a rule for files ending with "d.txt":
^[0-3][0-9]1019d\.txt$
If this is supposed to match for my picked example filename, you would have to change it to:
^[0-3][0-9]+d\.txt$

I'll give it a chance !

Originally Posted by pspzockerscene

Do you want to "sort" all of these files or are there ones which will not be affected by any rule?

No, it is not my intention to make any kind of sorting. I just want to download to separate olders, based on the filenames!

Originally Posted by pspzockerscene

Also please keep in mind that the way you've created the rules, they would be applied on ALL filenames you add to JDownloader no matter where they come from.
If you plan to use JDownloader to download from other sources in the future and you want to leave the rules enabled, you should consider also enabling Downloadurl --> contains --> "4trader.net" (without "")

Thank you for the tip and your help !

[D-clone]

Edit:

Finally, all my Packagiser rules are now triggered!

The following tip was proved to be unnecessary!

Originally Posted by pspzockerscene

Now as an example for an entry which will not be affected by any rule but probably should:

010319d.txt

Now I see you have a rule for files ending with "d.txt":

^[0-3][0-9]1019d\.txt$

If this is supposed to match for my picked example filename, you would have to change it to:

^[0-3][0-9]+d\.txt$

I have two last questions:

My first question is the following:
The first group of rules are about files hosted in 4trader.net/data/.
The second group of rules are about files hosted in 4trader.net/data/historic/.

Is there some option, while analysing 4trader.net/data, to include "subfolders" eg 4trader.net/data/historic/? The point here is to avoid double analysing process, meaning once for the "parent folder" and once for the "subfolder". Of course, this could be bypassed by entering both links!

My second question is about the "parent folder", containg hundreds of files. Every time JDownloads analyzes it, is there the possibility to download just the files triggered by the Packagiser rules and nothing more?

The only way I have found so far is to manually delete from the Linkgrabber the "various files" contents

Thank you once more for your help!

[raztoki]

rules are processed as this time top to bottom, generic rules first and specific rules last.

in respects to /data/ vs /data/historic/ you can place source contains = /data/historic/ and prevent it from been triggered multiple times. You can tighten up further and use equals and or use patterns.

[D-clone]

Thank you for the tips!

[pspzockerscene]

Thanks for your feedback.

-psp-