[nathan1]

Dear Staff,
I read before this thread
https://board.jdownloader.org/showth...ight=khinsider

but I don't understand how to solve for my situation.

I have a 'donator account' (similar to a premium account) and in this way I can download files as .zip and not only individual .mp3.

To download files as .zip you must have a donator account and I have one
**External links are only visible to Support Staff****External links are only visible to Support Staff**

To better explain what I would like JD to do

1.I go on this page
**External links are only visible to Support Staff****External links are only visible to Support Staff**

you see list of files



2. Then I click on this link (is just an example)
**External links are only visible to Support Staff****External links are only visible to Support Staff**

that is

Etrian Odyssey Untold (Shin Sekaiju no MeiQ) The Millennium Girl Original Soundtrack

Ok, now, iinside that link I different mp3 files but my attention is for this link




3. This page is visible only for Donator Account


I can give you my login credentials if you want to test



I see 2 links

**External links are only visible to Support Staff****External links are only visible to Support Staff**

**External links are only visible to Support Staff****External links are only visible to Support Staff**

I want retrieve both (not every album has a FLAC version, in this case there is)

My question are 2:

a) I have a list of links like

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Action Man - Search for Base X (GBC) (23 songs)
Bob the Builder - Fix it Fun! (GBC) (20 songs)
Flintstones, The - King Rock Treasure Island (GB) (12 songs)
Jimmy White's Cueball (GBC) (6 songs)

I want copy and paste these links and download them as .zip files using Jdownloader 2

b) I don't understand how to add my account on JD2. I want to add mine

Thanks for any help.

[pspzockerscene]

Hi,

because there is no JD plugin available for this website you cannot just add your account to JD.
From what I can see, I can download your direct-URLs even without your account so it should be possible to put your login-cookies into the previously posted LinkCrawler rule so JD can then (theoretically) auto-find those URLs.

If you want me to check this, you can send me your logindata via PN.
I will only use them to check/update the link crawler rule - you can change your password afterwards.

-psp-

[Jiaz]

Either send logins to pspzockerscene or to support@jdownloader.org and we can try to help with the rule

[nathan1]

@Jiaz

I sent via PM my login data to pspzockerscene

[Jiaz]

Thanks. I'm sure he will be able to help you

[pspzockerscene]

Turns out all you need is your personal "xf_session" cookie.
Here is the rule (well, two rules) - all you need to do is to replace the word CENSORED with the value of your personal "xf_session" cookie then you're good to go:

[ {
  "enabled" : true,
  "cookies" : [ ["xf_session", "CENSORED"] ],
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 1,
  "name" : "downloads.khinsider.com loggedin 1: Grab add_album URL",
  "pattern" : "https?://downloads\\.khinsider\\.com/[^/]+/album/[^/]+",
  "rule" : "DEEPDECRYPT",
  "packageNamePattern" : null,
  "passwordPattern" : null,
  "formPattern" : null,
  "deepPattern" : "\"(https?://downloads\\.khinsider\\.com/cp/add_album/\\d+)\"",
  "rewriteReplaceWith" : null
},
{
  "enabled" : true,
  "cookies" : [ ["xf_session", "CENSORED"] ],
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 1,
  "name" : "downloads.khinsider.com loggedin 2: Grab .zip files",
  "pattern" : "https?://downloads\\.khinsider\\.com/cp/add_album/\\d+",
  "rule" : "DEEPDECRYPT",
  "packageNamePattern" : null,
  "passwordPattern" : null,
  "formPattern" : null,
  "deepPattern" : "\"(https?://[^\"]+\\.zip)\"",
  "rewriteReplaceWith" : null
} ]

Rule in plaintext to get around our forum's URL censoring:
pastebin.com/M4ZQ4Thq

-psp-
EDIT

In case you're wondering why some .zip files appear twice in your linkgrabber:
This rule will simply grab ALL .zip URLs which means if there is a mirror, it will be included as well.

[nathan1]

Maybe I wrong something because don't work

1. I take cookie code using Flag Cookie extension





1. I edit your link rule crawled in this way

[ {
  "enabled" : true,
  "cookies" : [ ["xf_session", "1271103%2CODDI56dzL4KcLam9w7eCzpEs4Pg5gg3VQ#######"] ],
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 1,
  "name" : "downloads.khinsider.com loggedin 1: Grab add_album URL",
  "pattern" : "https?://downloads\\.khinsider\\.com/[^/]+/album/[^/]+",
  "rule" : "DEEPDECRYPT",
  "packageNamePattern" : null,
  "passwordPattern" : null,
  "formPattern" : null,
  "deepPattern" : "\"(https?://downloads\\.khinsider\\.com/cp/add_album/\\d+)\"",
  "rewriteReplaceWith" : null
},
{
  "enabled" : true,
  "cookies" : [ ["xf_session", "1271103%2CODDI56dzL4KcLam9w7eCzpEs4Pg5gg3VQ#######"] ],
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 1,
  "name" : "downloads.khinsider.com loggedin 2: Grab .zip files",
  "pattern" : "https?://downloads\\.khinsider\\.com/cp/add_album/\\d+",
  "rule" : "DEEPDECRYPT",
  "packageNamePattern" : null,
  "passwordPattern" : null,
  "formPattern" : null,
  "deepPattern" : "\"(https?://[^\"]+\\.zip)\"",
  "rewriteReplaceWith" : null
} ]

2. I copy and paste this code here


But JD2 doesn't crawls links

LOG

20.10.20 16.59.30 <--> 20.10.20 16.46.12 jdlog://6666325302851/

[Jiaz]

You've got to enable logging in rules. change

"logging" : false

to

"logging" : true

and try again and create log

[nathan1]

@Jiaz

No, nothing, I change false into true but these example links

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

are not crawled

LOG

20.10.20 19.23.34 <--> 20.10.20 19.23.34 jdlog://1076325302851/

[Jiaz]

Can you please send logins to support@jdownloader.org so I can check this

[pspzockerscene]

@nathan1
Works fine here!

According to your screenshot, you've taken the value of "xf_user" and put it in the crawler rule as "xf_session".
You need to get the value of your "xf_session" cookie and put it into the rule for this to work ...

-psp-

[nathan1]

I solved, thanks, I use edithiscookie of Chrome but is possible to add another rule to filter same files?

In case you're wondering why some .zip files appear twice in your linkgrabber:
This rule will simply grab ALL .zip URLs which means if there is a mirror, it will be included as well.

Is it possible to filter identical names? (I know that are same files)

[pspzockerscene]

You cannot filter same files/filenames with rules.
JD won't download the files twice anyways because they're in the same package.

... but if, for cosmetic reasons, you want JD to only find one file, the rule can be changed to nly grab the main downloadurls and not the mirrors.
For this, I've made minor changes to the 2nd rule - here is both again:

[ {
  "enabled" : true,
  "cookies" : [ ["xf_session", "CENSORED"] ],
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 1,
  "name" : "downloads.khinsider.com loggedin 1: Grab add_album URL",
  "pattern" : "https?://downloads\\.khinsider\\.com/[^/]+/album/[^/]+",
  "rule" : "DEEPDECRYPT",
  "packageNamePattern" : null,
  "passwordPattern" : null,
  "formPattern" : null,
  "deepPattern" : "\"(https?://downloads\\.khinsider\\.com/cp/add_album/\\d+)\"",
  "rewriteReplaceWith" : null
},
{
  "enabled" : true,
  "cookies" : [ ["xf_session", "CENSORED"] ],
  "updateCookies" : true,
  "logging" : false,
  "maxDecryptDepth" : 1,
  "name" : "downloads.khinsider.com loggedin 2: Grab .zip files",
  "pattern" : "https?://downloads\\.khinsider\\.com/cp/add_album/\\d+",
  "rule" : "DEEPDECRYPT",
  "packageNamePattern" : null,
  "passwordPattern" : null,
  "formPattern" : null,
  "deepPattern" : "\"(https?://[^\"]+\\.zip)\">Click here",
  "rewriteReplaceWith" : null
} ]

Rules in plaintext:
pastebin.com/158CbiPs

-psp-

[nathan1]

Your solution works great!
Thank you very much

[pspzockerscene]

Thanks for your feedback.

In case the rule ever fails:
First renew your cookies before reporting any errors here.
Session cookies do not always last forever!

-psp-

[nathan1]

Is possible to fix? This pattern seems don't work and I don't understand why

"pattern" : "https?://downloads\\.khinsider\\.com/[^/]+/album/[^/]+",

I used linkcrawl above and I set xf_session correctly

Debug LOG

30.08.22 16.40.54 <--> 30.08.22 16.40.54 jdlog://0359211370661/

[Jiaz]

@nathan1: you must update deepPattern as the URL is now relative in html, in Grab add_album URL

"deepPattern" : "\"((?:https?://downloads\\.khinsider\\.com)?/cp/add_album/\\d+)\"",

[nathan1]

Thank you Jiaz, but I don't understand why some links are still not crawled. For example these is like invisible for JD, take a look in LOG

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

LOG
30.08.22 18.03.37 <--> 30.08.22 18.03.37 jdlog://2359211370661/

[Jiaz]

@nathan1: have you updated the deepPattern? Can you send me the cookie and your current rules to support@jdownloader.org, then I will check it by tomorrow

[nathan1]

Yes, I update cookie. Ok, check email. I sent you now - problem seems when you crawl a large number of url at same time (for example 50-60 links).
I attach you also url list to control this