**External links are only visible to Support Staff****External links are only visible to Support Staff**
|
#1
|
|||
|
|||
Access to member area by cookies
Hi
I want to download all video and photo files from a member area of the website (it is a single page area with load more button). I added following rule to the Link crawler rule but it does not work. Code:
[{ "id" : 1586007825050, "name" : "MyVideos", "enabled" : true, "logging" : true, "maxDecryptDepth" : 1, "domainPattern" : ".*fancentro\\.com", "rule" : "DEEPDECRYPT", "packageNamePattern" : null, "formPattern" : null, "deepPattern" : null, "cookies": [ ["__ax","1"], ["ConsentSettings", "2"] ,["exo_uid", "1"], "updateCookies": true }]
Spoiler:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Spoiler:
**External links are only visible to Support Staff****External links are only visible to Support Staff** |
#2
|
||||
|
||||
Hi,
1. The cookies shown in your example rule don't really seem personalized. 2. Your website might use "Basic Authentification" which would be easier than cookies. 3. Please send me the URL to the login area of that website via PN so I can check it out - I won't be able tp help you just using the ifo you've provided so far. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#3
|
|||
|
|||
Quote:
I sent you a message with all details. |
#4
|
||||
|
||||
@armindvd
I've answered your PN already but here again in public: Your PN only contains your cookies - to check your request I need your full username & password for that website. You can of course change it afterwards! -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#5
|
|||
|
|||
I thought it should be enough to login to site. credentials submitted.
|
#6
|
||||
|
||||
Hi again,
here is a simple rule that will accept "feeds" URLs and crawl video content. Of course this can be extended to grab other stuff too: pastebin.com/DZtAkgX8 Also, I've noticed that it is possible to download the "paid" content even without logging in which means even my test-rule will work fine. Also, the only cookie really needed to be logged in is the "snapcentro_auth" cookie. You can grab this from your browser and replace the "CENSORED" in my example rule with the content of said cookie. Here is the rule again so we won't use it. Our boardsoftware will censor URLs for users which is why it is so complicated and also why you should use the rule I've linked via pastebin above.
Spoiler:
Code:
[ { "enabled" : true, "cookies" : [ ["snapcentro_auth", "CENSORED"] ], "updateCookies" : true, "logging" : false, "maxDecryptDepth" : 1, "name" : "fancentro.com example rule for single videos with cookies", "pattern" : "https?://fancentro\\.com/[^/]+/feed/\\d+", "rule" : "DEEPDECRYPT", "packageNamePattern" : "<title>(.*?)</title>", "passwordPattern" : null, "formPattern" : null, "deepPattern" : "\"(https?://[^\"]+\\.mp4[^\"]+)", "rewriteReplaceWith" : null } ] -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#7
|
|||
|
|||
Quote:
After you logged in, the system will generate video and image files links and then you can download them wherever you want but the links will expire after almost 30 minutes. so you need to login again. If I enter a direct link from the feed such as **External links are only visible to Support Staff** it is working without problem. but is it possible to crawl all the link inside the root feed ? for example **External links are only visible to Support Staff****External links are only visible to Support Staff** is it possible to extend deepPattern to find multiple file type ? if yes. could you please give me an example? Last edited by armindvd; 14.07.2020 at 17:54. |
#8
|
||||
|
||||
Hi again,
1. For your "feed" URL, the contents are dynamically loaded. This means it is impossible to do this via LinkCrawler rule. A "real" JD crawler plugin would be better for that. 2. Yes sure you can extend the deepPattern to match more - its all just Regular Expressions (that's the word to google). There also exists a neat online tool for testing RegExes: regex101.com -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
Thread Tools | |
Display Modes | |
|
|