#1
|
|||
|
|||
Problem while analyse a page with jdownloader 2
Hi everyone!
I use jdownloader since years without problem and allways surprised by its power!!! Unfortunately since the last update, when I try to analyse and find links through pages, it doesn't find anymore the links hosted on these pages!! And before it worked like a charm!! What could I do? Thanks alot for your help |
#2
|
||||
|
||||
Please provide example links. Nothing has changed parser, so must be something with the website you are trying to parse
__________________
JD-Dev & Server-Admin |
#3
|
|||
|
|||
Yes thanks, but on the pages if I select and copy links, jdownloader detect it immediately:
for example: **External links are only visible to Support Staff****External links are only visible to Support Staff** It should find: **External links are only visible to Support Staff****External links are only visible to Support Staff** and **External links are only visible to Support Staff****External links are only visible to Support Staff** but it only find images from the pages and audio files from bandcamp, I don't understand what has changed. Last time I used it it was middle of july and I don't find any changement on the website. |
#4
|
|||
|
|||
**External links are only visible to Support Staff****External links are only visible to Support Staff**
|
#5
|
||||
|
||||
Have you checked page source of that link to find those 2 llinks?
I can't find it.
__________________
FAQ: How to upload a Log |
#6
|
|||
|
|||
Yes, when I see the code source (with Ctrl+U on firefox), we can find these links (I found it using Ctrl+F)
On line 496 there are the 3 links:It's a huge line ! firstly Bandcamp (which is found by JD) then zippyshare and psy-music.ru (which are not found by JD).Strange! thank you Last edited by flowlapache; 11.10.2016 at 18:33. |
#7
|
||||
|
||||
You need to add LinkCrawler Rules to tell JDownloader to auto parse this type of url.
JDownloader does not support this url and therefore does not autohandle/parse it. With a customized LinkCrawler Rule (use board search) you can tell JDownloader how to handle this url. You can also Copy/Paste the URL into JDownloader to force auto deep decrypt of the url
__________________
JD-Dev & Server-Admin |
#8
|
|||
|
|||
Ok, I will try this "link crawler rule". But it strange because it works since 4 years at least!! Maybe they change the type of url this summer. Thank you for your quick answers and help!
|
#9
|
||||
|
||||
JDownloader does not support this type of url and never did.
Copy/Paste will work fine and a Linkcrawler Rule too
__________________
JD-Dev & Server-Admin |
#10
|
|||
|
|||
Ok, so the website have changed without something changed for me.
When I try with deep decrypt, it doesn't find what I search for. I'm looking for linkcrawler Rule, JD is really surprising! |
#11
|
||||
|
||||
Simply copy/paste the url into JDownloader, it will deep decrypt and show all supported links.
__________________
JD-Dev & Server-Admin |
#12
|
|||
|
|||
Thats what I did, but it doesn't find the two links I would like. And I'm not sure what I should change in the linkcrawler rule. Can I create a new rule specific to this website? It seems to be allways on the line 96 with same type of codes, on every pages from this website (psy-music.ru) If I copy this line, JD find immediately what I look for!!
I don't find on other thread how to do..I see often "mail to support@..." to obtain help about making a rule for this website... edit: I can't find how to tell JD to auto parse pages from this website. I look some other rules for specific sites but I don't understand which code I should add to linkcrawler rules.... Last edited by flowlapache; 11.10.2016 at 22:14. |
#13
|
|||
|
|||
I have forgotten something important! (thanks to mgpai from JDchat)
links are not visible unless I login. That's why you didn't find links when you asked me. I'm still trying to make a rule for linkcrawler. For this login, it was already this since years.When I logged and had the cookie of the website, it was visible for JD. But now no more. I tried this, but it's not a valid rule... Code:
[ { "enabled" : true, "name" : "psy-music", "pattern" : " psy-music\\.ru/news/.+/[0-9-]+ ", "rule" : "DEEPDECRYPT", } ] I will sleep, maybe tomorrow regex will be clearer... Last edited by flowlapache; 12.10.2016 at 00:08. |
#14
|
||||
|
||||
pattern requires protocol prefix
Code:
https?:// raztoki
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] Last edited by raztoki; 12.10.2016 at 01:42. |
#15
|
||||
|
||||
see fixed version
__________________
JD-Dev & Server-Admin |
#16
|
|||
|
|||
Ok, thanks guys, I'm learning....
I tried with this fixed version (without and with the space after the "+" in pattern) but JD doesn't accept it. And one thing more, I have already a linkcrawler rule (I don't know where and when it comes from) for zip format: Code:
[ { "enabled" : true, "maxDecryptDepth" : 0, "id" : 1476219085998, "name" : "psy_music", "pattern" : " psy-music\\.ru/news/.+/[0-9-]+ ", "rule" : "DEEPDECRYPT", "packageNamePattern" : null, "formPattern" : null, "deepPattern" : null, "rewriteReplaceWith" : null } ] I tried the 3 solutions, but because it's not valid, the value doesn't change (only zip rule stays...) |
#17
|
||||
|
||||
your pattern is still invalid!
it must begin with https?:// an url does not start with _(space)
__________________
JD-Dev & Server-Admin |
#18
|
|||
|
|||
Sorry, it wasn't this one, I copied the (first) bad one.
I would like to show the existing one for zip format: Code:
[ { "enabled" : true, "maxDecryptDepth" : 2, "id" : 1433746432948, "name" : "Learned file extension:zip", "pattern" : "(?i).*\\.zip($|\\?.*$)", "rule" : "DIRECTHTTP", "packageNamePattern" : null, "formPattern" : null, "deepPattern" : null, "rewriteReplaceWith" : null } ] Code:
[ { "enabled" : true, "name" : "psy-music", "pattern" : "https?://psy-music\\.ru/news/.+/[0-9-]+", "rule" : "DEEPDECRYPT", } ] And must I replace the ".zip" rule or I add it wrinting something like: Code:
[ { "enabled" : true, "name" : "psy-music", "pattern" : "https?://psy-music\\.ru/news/.+/[0-9-]+", "rule" : "DEEPDECRYPT", } ] [ { "enabled" : true, "maxDecryptDepth" : 2, "id" : 1433746432948, "name" : "Learned file extension:zip", "pattern" : "(?i).*\\.zip($|\\?.*$)", "rule" : "DIRECTHTTP", "packageNamePattern" : null, "formPattern" : null, "deepPattern" : null, "rewriteReplaceWith" : null } ] |
#19
|
||||
|
||||
[{rule1},{rule2},{rule3}]
and NOT [{rule1}][{rule2}][{rule3}]
__________________
JD-Dev & Server-Admin |
#20
|
|||
|
|||
Ok thanks! I see the importance of "," and priority of "[" and "{". I will try
|
#21
|
||||
|
||||
It is json format. You can google for json beautifier. If it *eats* your input, it is valid json
__________________
JD-Dev & Server-Admin |
#22
|
|||
|
|||
Quote:
For the max decrypt depth, 0 is 0 or it's everything? Because now (with 0), when one link is parsed, it finds a lot of differents bancamp links until endless (maybe it parses all thye site). I will try with depth=1 Last thing, do you know if it's possible in a browser (firefox) to "copy the code source of the link" with a right clic? It could save me to open each page and making Ctrl+A then Ctrl+C to obtain good links in JD. Last edited by flowlapache; 13.10.2016 at 15:50. |
#23
|
||||
|
||||
Depth=0 -> only parse the input url
Depth=1 -> also parse first level of found links Ctrl+A = Select All + Ctrl+C = Copy = Copy all to Clipboard. Browser will also put html code into clipboard. It works perfectly fine for me. I don't need any logins. Browser and JDownloader do show the links just fine without any logins?!
__________________
JD-Dev & Server-Admin |
#24
|
|||
|
|||
WHAAAT!!! you are experiencing what happened since ever!! You really obtain the http link from psy-music.ru and zippishare or rusfolder and others?
it's crazy, I have only, images, bandcamp audio files, videos but no archive!! I'm lost with this new news! |
#25
|
|||
|
|||
oops, sorry (i can't edit before it's approved...)I misunderstood...
In fact yes, it works with control+A when we are on the page. But I would like to make it like before, just put the link (of the page) in JD without need to open the page... like parsing and analysing the page to find these archives! |
#26
|
|||
|
|||
I don't understand why "deepdecrypt" don't find it.
Is there a rule saying to JD to parse the code source of a page (Ctrl+A, paste by itself), because it's just what I want. I search how "copy the code source of a page without open this page in a browser" but for now I have nothing. It's boring, it's only 2 or 4 actions to do what I want ( a link>the source code>JD find the archives!), But I can't do it manually for 100 pages! I continue to search this "code source copy" feature in firefox or otherz... In all cases, thank you for your support, I'm really goodly surprised by the support by chat, or board. It's so better than a lot of applications or webshops which are not free moreover!!! |
#27
|
||||
|
||||
Do you have to log in to see those links?
__________________
FAQ: How to upload a Log |
#28
|
|||
|
|||
Thanks to come! Yes I mentionned it a bit before...
Normally I log in, then I just put links of all pages I want in the "parser" of JD and every archives are found! it works with Select All + Ctrl+C (even without be logged! like tried Jiaz) after have opened the page. But now, I can't obtain archives without open each page. Even with the deepdecrypt linkcrawler rule And I didn't find how to make a "copy the code source of this link" in Firefox. Thing which could do the job... |
#29
|
||||
|
||||
I think previously the link was shown to everybody without login, not anymore.
I am not sure, but you can try to enter your credentials in Basic Authentication and try to parse it again.
__________________
FAQ: How to upload a Log |
#30
|
|||
|
|||
Yes, I try it firstly but it changes nothing.
The website has changed but I don't know how type... Previously links were already hidden to public.Like now, it has been allways necessary to log to see the links, mostly for the http links from psy-music.ru... |
#31
|
||||
|
||||
During my tests the links were public available without login. I guess this is changed now as I no longer can see them. When you are logged in, are the links plain visible or via rightclick or do you need to do a left click?
__________________
JD-Dev & Server-Admin |
#32
|
|||
|
|||
I guess, it's the trick.
We need to left click to open it (or launch it for the http link). It's not plain visible. We see the adress only with the mouse on the link. it's directly visible in the code source. JD find it if we select it and copy. |
#33
|
||||
|
||||
In that case I would stick to *select and copy* as it seems to work
When you just copy the url then the links are no longer available as the site requires a login and your JDownloader is not logged in. Is that solution acceptable for you?
__________________
JD-Dev & Server-Admin |
#34
|
|||
|
|||
Hi, thank you.
If I found nothing else, yes I will use the "open>select all>copy". But I was used to go every one or two months on this website et download around 100 links....Which is really "hard and time eater" if I must open each page and select+copy...instead of only copy each adress like before... In the same time I'm trying with "menu wizard" on firefox to add a "copy the source code of the link" to my right click. If it works, I thing I could obtain links without open every pages. For now the "source results" are different than the source on the opened page. But they probably make it weel good to force to open each page to obtain the links... maybe they are no more JD friendly!!^^ |
#35
|
||||
|
||||
We are currently working on updated browser extensions, then it should be possible to add links from pages with logins as well
__________________
JD-Dev & Server-Admin |
#36
|
|||
|
|||
@flowlapache,
Add this script to event scripter. Change the username and password. You should now be able to grab the links which were otherwise not detected by JD. Will only detect single URLs. Create linkfilter rule to block unwanted links. @jiaz, Thanks for the browser API and code Code:
// Crawler: psy-music.ru // Trigger required: "New Crawler Job" var sourceURL = job.text; var matchURL = (/^http:\/\/psy-music.ru\/news\/.+\/[\d-]+$/).test(sourceURL); try { if (matchURL) { var mainURL = "**External links are only visible to Support Staff**; var username = "myUser"; // Change username var password = "myPasss"; // Change password var br = getBrowser(); var postURL = mainURL + "index/sub/"; var postData = "user=" + username + "&password=" + password + "&rem=1&a=2&ajax=1&rnd=691"; var mainPage = br.getPage(mainURL); var mainPage = br.postPage(postURL, postData); var links = br.getPage(sourceURL); callAPI("linkgrabberv2", "addLinks", { "links": links }); job.setText(""); } } catch (e) { alert("Error occured while adding \"" + sourceURL + "\". Please try again"); } |
#37
|
|||
|
|||
OOh yeah guys!!
Thanks so much! You made it! It works very well and fast! Thank you for this nice support! very efficient! |
#38
|
||||
|
||||
Thanks for the feedback
__________________
JD-Dev & Server-Admin |
#39
|
|||
|
|||
I try the same thing for another website but it doesn't work an dI don't understand why...
|
#40
|
||||
|
||||
@flowlapache: can you provide example links?
__________________
JD-Dev & Server-Admin |
Thread Tools | |
Display Modes | |
|
|