|
#1
|
|||
|
|||
How add a donator account and download .zip files from khinsider.com?
Dear Staff,
I read before this thread https://board.jdownloader.org/showth...ight=khinsider but I don't understand how to solve for my situation. I have a 'donator account' (similar to a premium account) and in this way I can download files as .zip and not only individual .mp3. To download files as .zip you must have a donator account and I have one **External links are only visible to Support Staff****External links are only visible to Support Staff** To better explain what I would like JD to do 1.I go on this page **External links are only visible to Support Staff****External links are only visible to Support Staff** you see list of files 2. Then I click on this link (is just an example) **External links are only visible to Support Staff****External links are only visible to Support Staff** that is Etrian Odyssey Untold (Shin Sekaiju no MeiQ) The Millennium Girl Original Soundtrack Ok, now, iinside that link I different mp3 files but my attention is for this link 3. This page is visible only for Donator Account I can give you my login credentials if you want to test I see 2 links **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** I want retrieve both (not every album has a FLAC version, in this case there is) My question are 2: a) I have a list of links like **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** Action Man - Search for Base X (GBC) (23 songs) Bob the Builder - Fix it Fun! (GBC) (20 songs) Flintstones, The - King Rock Treasure Island (GB) (12 songs) Jimmy White's Cueball (GBC) (6 songs) I want copy and paste these links and download them as .zip files using Jdownloader 2 b) I don't understand how to add my account on JD2. I want to add mine Thanks for any help. Last edited by Jiaz; 26.10.2020 at 19:46. |
#2
|
||||
|
||||
Hi,
because there is no JD plugin available for this website you cannot just add your account to JD. From what I can see, I can download your direct-URLs even without your account so it should be possible to put your login-cookies into the previously posted LinkCrawler rule so JD can then (theoretically) auto-find those URLs. If you want me to check this, you can send me your logindata via PN. I will only use them to check/update the link crawler rule - you can change your password afterwards. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#3
|
||||
|
||||
Either send logins to pspzockerscene or to support@jdownloader.org and we can try to help with the rule
__________________
JD-Dev & Server-Admin |
#4
|
|||
|
|||
@Jiaz
I sent via PM my login data to pspzockerscene |
#5
|
||||
|
||||
Thanks. I'm sure he will be able to help you
__________________
JD-Dev & Server-Admin |
#6
|
||||
|
||||
Turns out all you need is your personal "xf_session" cookie.
Here is the rule (well, two rules) - all you need to do is to replace the word CENSORED with the value of your personal "xf_session" cookie then you're good to go: Code:
[ { "enabled" : true, "cookies" : [ ["xf_session", "CENSORED"] ], "updateCookies" : true, "logging" : false, "maxDecryptDepth" : 1, "name" : "downloads.khinsider.com loggedin 1: Grab add_album URL", "pattern" : "https?://downloads\\.khinsider\\.com/[^/]+/album/[^/]+", "rule" : "DEEPDECRYPT", "packageNamePattern" : null, "passwordPattern" : null, "formPattern" : null, "deepPattern" : "\"(https?://downloads\\.khinsider\\.com/cp/add_album/\\d+)\"", "rewriteReplaceWith" : null }, { "enabled" : true, "cookies" : [ ["xf_session", "CENSORED"] ], "updateCookies" : true, "logging" : false, "maxDecryptDepth" : 1, "name" : "downloads.khinsider.com loggedin 2: Grab .zip files", "pattern" : "https?://downloads\\.khinsider\\.com/cp/add_album/\\d+", "rule" : "DEEPDECRYPT", "packageNamePattern" : null, "passwordPattern" : null, "formPattern" : null, "deepPattern" : "\"(https?://[^\"]+\\.zip)\"", "rewriteReplaceWith" : null } ] pastebin.com/M4ZQ4Thq -psp- EDIT In case you're wondering why some .zip files appear twice in your linkgrabber: This rule will simply grab ALL .zip URLs which means if there is a mirror, it will be included as well.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download Last edited by pspzockerscene; 20.10.2020 at 13:48. |
#7
|
|||
|
|||
Maybe I wrong something because don't work
1. I take cookie code using Flag Cookie extension 1. I edit your link rule crawled in this way Code:
[ { "enabled" : true, "cookies" : [ ["xf_session", "1271103%2CODDI56dzL4KcLam9w7eCzpEs4Pg5gg3VQ#######"] ], "updateCookies" : true, "logging" : false, "maxDecryptDepth" : 1, "name" : "downloads.khinsider.com loggedin 1: Grab add_album URL", "pattern" : "https?://downloads\\.khinsider\\.com/[^/]+/album/[^/]+", "rule" : "DEEPDECRYPT", "packageNamePattern" : null, "passwordPattern" : null, "formPattern" : null, "deepPattern" : "\"(https?://downloads\\.khinsider\\.com/cp/add_album/\\d+)\"", "rewriteReplaceWith" : null }, { "enabled" : true, "cookies" : [ ["xf_session", "1271103%2CODDI56dzL4KcLam9w7eCzpEs4Pg5gg3VQ#######"] ], "updateCookies" : true, "logging" : false, "maxDecryptDepth" : 1, "name" : "downloads.khinsider.com loggedin 2: Grab .zip files", "pattern" : "https?://downloads\\.khinsider\\.com/cp/add_album/\\d+", "rule" : "DEEPDECRYPT", "packageNamePattern" : null, "passwordPattern" : null, "formPattern" : null, "deepPattern" : "\"(https?://[^\"]+\\.zip)\"", "rewriteReplaceWith" : null } ] But JD2 doesn't crawls links LOG Code:
20.10.20 16.59.30 <--> 20.10.20 16.46.12 jdlog://6666325302851/ Last edited by Jiaz; 26.10.2020 at 19:46. |
#8
|
||||
|
||||
You've got to enable logging in rules. change
Quote:
Quote:
__________________
JD-Dev & Server-Admin |
#9
|
|||
|
|||
@Jiaz
No, nothing, I change false into true but these example links **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** are not crawled LOG Code:
20.10.20 19.23.34 <--> 20.10.20 19.23.34 jdlog://1076325302851/ |
#10
|
||||
|
||||
Can you please send logins to support@jdownloader.org so I can check this
__________________
JD-Dev & Server-Admin |
#11
|
||||
|
||||
@nathan1
Works fine here! According to your screenshot, you've taken the value of "xf_user" and put it in the crawler rule as "xf_session". You need to get the value of your "xf_session" cookie and put it into the rule for this to work ... -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#12
|
|||
|
|||
I solved, thanks, I use edithiscookie of Chrome but is possible to add another rule to filter same files?
Quote:
Last edited by Jiaz; 26.10.2020 at 19:46. |
#13
|
||||
|
||||
You cannot filter same files/filenames with rules.
JD won't download the files twice anyways because they're in the same package. ... but if, for cosmetic reasons, you want JD to only find one file, the rule can be changed to nly grab the main downloadurls and not the mirrors. For this, I've made minor changes to the 2nd rule - here is both again: Code:
[ { "enabled" : true, "cookies" : [ ["xf_session", "CENSORED"] ], "updateCookies" : true, "logging" : false, "maxDecryptDepth" : 1, "name" : "downloads.khinsider.com loggedin 1: Grab add_album URL", "pattern" : "https?://downloads\\.khinsider\\.com/[^/]+/album/[^/]+", "rule" : "DEEPDECRYPT", "packageNamePattern" : null, "passwordPattern" : null, "formPattern" : null, "deepPattern" : "\"(https?://downloads\\.khinsider\\.com/cp/add_album/\\d+)\"", "rewriteReplaceWith" : null }, { "enabled" : true, "cookies" : [ ["xf_session", "CENSORED"] ], "updateCookies" : true, "logging" : false, "maxDecryptDepth" : 1, "name" : "downloads.khinsider.com loggedin 2: Grab .zip files", "pattern" : "https?://downloads\\.khinsider\\.com/cp/add_album/\\d+", "rule" : "DEEPDECRYPT", "packageNamePattern" : null, "passwordPattern" : null, "formPattern" : null, "deepPattern" : "\"(https?://[^\"]+\\.zip)\">Click here", "rewriteReplaceWith" : null } ] pastebin.com/158CbiPs -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#14
|
|||
|
|||
Your solution works great!
Thank you very much |
#15
|
||||
|
||||
Thanks for your feedback.
In case the rule ever fails: First renew your cookies before reporting any errors here. Session cookies do not always last forever! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#16
|
|||
|
|||
Is possible to fix? This pattern seems don't work and I don't understand why
Code:
"pattern" : "https?://downloads\\.khinsider\\.com/[^/]+/album/[^/]+", Debug LOG Code:
30.08.22 16.40.54 <--> 30.08.22 16.40.54 jdlog://0359211370661/ Last edited by Jiaz; 30.08.2022 at 20:00. |
#17
|
||||
|
||||
@nathan1: you must update deepPattern as the URL is now relative in html, in Grab add_album URL
Code:
"deepPattern" : "\"((?:https?://downloads\\.khinsider\\.com)?/cp/add_album/\\d+)\"",
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 30.08.2022 at 20:09. |
#18
|
|||
|
|||
Thank you Jiaz, but I don't understand why some links are still not crawled. For example these is like invisible for JD, take a look in LOG
**External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** **External links are only visible to Support Staff****External links are only visible to Support Staff** LOG 30.08.22 18.03.37 <--> 30.08.22 18.03.37 jdlog://2359211370661/ |
#19
|
||||
|
||||
@nathan1: have you updated the deepPattern? Can you send me the cookie and your current rules to support@jdownloader.org, then I will check it by tomorrow
__________________
JD-Dev & Server-Admin |
#20
|
|||
|
|||
Yes, I update cookie. Ok, check email. I sent you now - problem seems when you crawl a large number of url at same time (for example 50-60 links).
I attach you also url list to control this |
#21
|
||||
|
||||
@nathan1: thanks, I will check by tomorrow. most likely some sort of rate limit causing the issue
__________________
JD-Dev & Server-Admin |
#22
|
||||
|
||||
@nathan1:
1. Your cookie doesn't seem to work here. Please keep in mind that cookies can expire and they're also often bound to a specific IP or at least a specific range/country that IP belongs to. To do further testing, please provide your mail- and password for khinsider.com. 2. I tested it using another test-account and I wasn't even able to download a single .zip file due to the following errormessage: Screenshot:
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#23
|
|||
|
|||
@psp
I sent you the credentials by replying to your last message in the email |
#24
|
||||
|
||||
I haven't tested it 100% yet but one part of the problem is definitely that khinsider.com changed the "/add_album" URLs inside their html code.
They're now represented as relative URLs vs absolute URLs and the provided LinkCrawler Rule was made for absolute URLs only. This would be the new rule: Code:
[ { "cookies" : [ [ "xf_session", "CENSORED" ] ], "deepPattern" : "(/cp/add_album/\\d+)\"", "formPattern" : null, "maxDecryptDepth" : 1, "name" : "downloads.khinsider.com loggedin 1: Grab add_album URL", "packageNamePattern" : null, "passwordPattern" : null, "pattern" : "https?://downloads\\.khinsider\\.com/[^/]+/album/[^/]+", "rule" : "DEEPDECRYPT", "enabled" : true, "logging" : true, "updateCookies" : true }, { "cookies" : [ [ "xf_session", "CENSORED" ] ], "deepPattern" : "\"(https?://[^\"]+\\.zip)\">Click here", "formPattern" : null, "maxDecryptDepth" : 1, "name" : "downloads.khinsider.com loggedin 2: Grab .zip files", "packageNamePattern" : null, "pattern" : "https?://downloads\\.khinsider\\.com/cp/add_album/\\d+", "rule" : "DEEPDECRYPT", "enabled" : true, "logging" : true, "updateCookies" : true } ] pastebin.com/raw/Si4bfhjm I've highlighted the relevant changed part.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#25
|
||||
|
||||
Quote:
__________________
JD-Dev & Server-Admin |
#26
|
||||
|
||||
Quote:
Either way nathan1 now has the complete modified rule available and also all other users who are interested in using it.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#27
|
||||
|
||||
Now I'm willing to admit that I'm also a lazy person so I'm not planning to let it run through all of your ~770 example URLs.
Let's say that the fixed rule is not enough and let's assume there is still a rate limit which is only triggered when checking all of those .zip direct-URLs. Try the following: 1. Go to Settings -> Advanced Settings -> Search for LinkCollector.dolinkcheck --> Disable this 2. Now add some- or all links. You will notice that they will get added with a blue questionmark and without onlinestatus and with unknown filesize. JDownloader will now skip the linkcheck of all added items. This has several disadvantages but it might help in this case plus they will appear in your linkgrabber faster. I recommend to turn that setting back on once you're done downloading those links.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#28
|
|||
|
|||
@psp
Thank you very much, I will follow your directions. |
#29
|
||||
|
||||
@nathan1: please test and provide feedback
__________________
JD-Dev & Server-Admin |
#30
|
|||
|
|||
@jiaz
@psp mm.. I test but this solution doesn't work well, I also disable dolinkcheck but links are not well crawled well, I copied and paste 150 links but it continues to idle detecting 5 or 6 links Code:
07.09.22 19.22.03 <--> 07.09.22 19.34.24 jdlog://8479211370661/ |
#31
|
||||
|
||||
@nathan1: hmm, log contains around 150 succesful parsed links/zip files !?
sure you don't have any filter rules active? the download domain is different in case you filter on domain and also check linkgrabber right sidebar, that you have not filtered hoster/filetype
__________________
JD-Dev & Server-Admin |
#32
|
|||
|
|||
@jiaz
No, I don't use any link filter rules. However I think to understand, I tried again on a clean server and now after about 2 minutes the links started to appear, but I usually waited at most one minute and so they didn't appear to me. Check out this video I made **External links are only visible to Support Staff****External links are only visible to Support Staff** Debug LOG Code:
07.09.22 19.21.42 <--> 07.09.22 19.21.42 jdlog://3579211370661/ |
#33
|
||||
|
||||
@nathan1:
Thanks for the video! Please check again with next core update. maybe the results will show up faster but no guarantees
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 08.09.2022 at 14:18. |
#34
|
||||
|
||||
It looks like the demand for a plugin for this website is huge so I'd add one.
In order to do so, I would need a khinsider.com donator account for testing. If any of you owns one and would agree to provide it for testing, please contact me via PN. I'd also need various example URLs for testing.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#35
|
||||
|
||||
Added plugin for khinsider.com.
More information: https://board.jdownloader.org/showth...759#post526759 I'm still looking for a donator account to do some tests.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
Thread Tools | |
Display Modes | |
|
|