#41
|
||||
|
||||
Wait for next core update
__________________
JD-Dev & Server-Admin |
#42
|
|||
|
|||
Incomplete regex matching.
Jiaz could write correct(If you find free time), I will be very grateful. |
#43
|
|||
|
|||
maybe ???
Quote:
|
#44
|
|||
|
|||
Decrypt only 1 page (not work multipage)
**External links are only visible to Support Staff**... **External links are only visible to Support Staff**... Code:
[ { "enabled" : true, "maxDecryptDepth" : 1, "id" : 1454334584729, "name" : null, "pattern" : "javascript:[\\w]+\\('lbNext',''\\)", "rule" : "DEEPDECRYPT" } ] |
#45
|
||||
|
||||
crawler rules are for two things
URL patterns, for URL listening, as in trigger when you copy URL.. eg. **External links are only visible to Support Staff****External links are only visible to Support Staff** and if they are direct downloadable or deep decrypt task. for deep decrypt, our parser looks for links and returns them all. Its not designed to search for patterns that you provide it with! It wont process javascript! raztoki
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#46
|
|||
|
|||
raztoki -
Some sites dynamicalu generate web content by using JavaScript. This cause JD2 to fail to detect when loading multipages. can not force JD2 to process the JavaScript?(Eg. The simulation mouse "click") :\ |
#47
|
||||
|
||||
I can only advise on vague posts (millions of them), since you rarely give working links to sites your trying to support. I gave help based on the feature you're trying to use outside of design.
If you want complicated function, outside of the simple function that linkcrawler rules provide, _you_ will need to make your own hoster/decrypter plugins. raztoki
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#48
|
|||
|
|||
I know, in this case, the script is not enough, you probably need an additional java class code...
|
#49
|
|||
|
|||
In theory, it should work with the parser, but unfortunately it did not work.
I have no other ideas, but do not give up and still looking solutions |
#50
|
|||
|
|||
If matched, add link to download queue use backreferences as {$1}, OR {$2}, etc.?
|
#51
|
||||
|
||||
JDownloader cannot evaluate Javascript without a real plugin. Only visible links can be processed!
__________________
JD-Dev & Server-Admin |
#52
|
|||
|
|||
I used a parser with "Evaluate calculation scripts" (Enabled)
but I get an error and abort parse. href="javascript:__doPostBack('lbHome','')">Home</a><img src="../Images/spaceer" |
#53
|
|||
|
|||
Code:
import java.util.regex.Matcher; import java.util.regex.Pattern; /** <P>{@code java ParseForclosureResultsXmpl}</P> **/ public class ParseForclosureResultsXmpl { public static final void main(String[] igno_red) { String sLS = System.getProperty("line.separator", "\n"); StringBuilder sdInput = new StringBuilder(). append("<a id=\"SheetContentPlaceHolder_ctl00_gvForeclosureResutls_lbCaseNum_0\" href=\"javascript:__doPostBack('ctl00$SheetContentPlaceHolder$ctl00$gvForeclosureResutls$ctl02$lbCaseNum','')\" style=\"display:inline-block;width:100px;\">CV-13-798497</a>").append(sLS). append(" </td><td align=\"center\">488-05-029</td><td align=\"center\">I</td><td align=\"center\">01/02/2013</td>").append(sLS). append(" </tr><tr style=\"background-color:Gainsboro;\">").append(sLS). append(" <td align=\"left\">UNKNOWN HEIRS, ETC OF D.C. RUFUS, ET AL </td><td align=\"left\">10603 HAMPDEN AVENUE</td><td align=\"center\">CLEVELAND</td><td align=\"center\">44108-0000</td><td align=\"center\">").append(sLS). append(" <a id=\"SheetContentPlaceHolder_ctl00_gvForeclosureResutls_lbCaseNum_1\" href=\"javascript:__doPostBack('ctl00$SheetContentPlaceHolder$ctl00$gvForeclosureResutls$ctl03$lbCaseNum','')\" style=\"display:inline-block;width:100px;\">CV-13-798498</a>").append(sLS). append(" </td><td align=\"center\">109-16-094</td><td align=\"center\">A</td><td align=\"center\">01/02/2013</td>").append(sLS). append(" </tr><tr style=\"background-color:LightGrey;\">").append(sLS). append(" <td align=\"left\">SHARECE MILLER, ET AL </td><td align=\"left\">13514 ALVIN AVENUE</td><td align=\"center\">GARFIELD HTS</td><td align=\"center\">44105-0000</td><td align=\"center\">").append(sLS). append(" <a id=\"SheetContentPlaceHolder_ctl00_gvForeclosureResutls_lbCaseNum_2\" href=\"javascript:__doPostBack('ctl00$Shee").append(sLS); String sRqdValuePrefix = "ForeclosureResutls_lbCaseNum_"; Pattern checkerLoopPattern = Pattern.compile(sRqdValuePrefix + "\\d"); Matcher m = checkerLoopPattern.matcher(""); //Unused. so the matcher can be reused in the loop. int iLn = 0; String[] asInput = sdInput.toString().split(sLS); for(String s : asInput) { iLn++; //1st iteration: Was zero, now 1 //Resuing matcher instead of retrieving new one from Pattern each iteration m.reset(s); if(m.find()) { int iCheckerNumber = Integer.parseInt(s.substring(m.start() + sRqdValuePrefix.length(), m.end())); System.out.println("Found on line " + iLn + ", at index " + m.start() + " with checker number " + iCheckerNumber); } } } } |
#54
|
|||
|
|||
E.g. On this page JD2 works perfectly
So there is some exceptions, it works. **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#55
|
|||
|
|||
Quote:
Mini-example: 4800 links **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#56
|
|||
|
|||
Deep-Decrypt multipage - issue
What is wrong?
:confused: Code:
[ { "enabled" : true, "maxDecryptDepth" : 1, "id" : 1454334584729, "name" : null, "pattern" : "/p/(\\d+)", "rule" : "DEEPDECRYPT" } ] |
#57
|
|||
|
|||
"pattern" wrong?
Code:
/p/(\\d+) .*?/p/(\\d+) .+/p/(\\d+) |
#58
|
||||
|
||||
maxDecryptDepth=1 -> it will not deep decrypt other pages
You need to increase this value
__________________
JD-Dev & Server-Admin |
#59
|
|||
|
|||
Regex needs for full-size photos because detect very small files.
Quote:
|
#60
|
|||
|
|||
Unlimited "multipage" Value: 0 ?
|
#61
|
||||
|
||||
there is no unlimited, you can simply set to high value. I doubt that there are millions of pages :p
__________________
JD-Dev & Server-Admin |
#62
|
|||
|
|||
Deepdecrypt only detects "thumb"
**External links are only visible to Support Staff****External links are only visible to Support Staff** |
#63
|
||||
|
||||
because the full images are build via javascript.
__________________
JD-Dev & Server-Admin |
#64
|
|||
|
|||
Javascript is not a problem
If I knew the regex to full-size photos is parser all links to the full-size photos. Only needs a regex to match the full-size image. |
#65
|
|||
|
|||
Example Full-Size link:
**External links are only visible to Support Staff****External links are only visible to Support Staff** |
#66
|
|||
|
|||
yes, yes, yes !.
I tested this regex works detects full-size photos. **<bizarre-collection.biz/thumb/> |
#67
|
||||
|
||||
once again, far as I'm aware, linkcrawler rules do not look for patterns within pages, they return all urls, and JD will add all that is 'supported'. The pattern you provide is to scan specific urls added to program (either clipboard, add links dialog, etc). Pattern as I see it isn't correct, you need at least "http.+/p/\\d+". () are not required in your patterns
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#68
|
|||
|
|||
Quote:
Regex works, but not decrypts all multipages. JD2 shows 240 links Other parser ~537 links example link: **External links are only visible to Support Staff****External links are only visible to Support Staff** |
#69
|
|||
|
|||
Probably should show.
I calculated 270 + 20 + 16 = 316 links |
#70
|
|||
|
|||
1 page = 20 images
28 page (1...28) = 280 images + first page(0) (20 images) + last page (16 images) = ? |
#71
|
||||
|
||||
once again 1 page = unknown results, since it doesn't filter results for specific item, it returns all img/hrefs/etc> then returned results match given plugins will show download link in GUI. If you want isolated/filtered download links please create decrypter plugins.
raztoki
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#72
|
|||
|
|||
I know it will not work in JD2, but some tools have built such a possibility for each multi-page
|
#73
|
|||
|
|||
Example:
*bootshare\\.com/(.*)\?pageNum\\_[a-z]+=(\\d+) |
#74
|
|||
|
|||
I have no idea what is wrong.
Error Occured: [ { "enabled" : true, "maxDecryptDepth" : 1, "id" : 1454334584729, "name" : null, "pattern" : "http.+(\\&start=(\\d+)|\\&page=(\\d+)$|\\?page=(\\d+)$|/page-(\\d+)$|\\?page=(\\d+)\\&|/page/(\\d+)/$|/page(\\d+)$|/page/(\\d+)\\?|/page(\\d+)/|\\?PageNo=(\\d+)|\\?paged=(\\d+)|/page/(\\d+)$|\\?st=(\\d+)|\\&s=(\\d+)+|\\&p=(\\d+)|\\&cat=(\\d+)|\\?_pgn=(\\d+)|\\?id=(\\d+)|/p/(\\d+)|\\?go=(\\d+)", "rule" : "DEEPDECRYPT" } ] |
#75
|
|||
|
|||
without first/end patern ? ()
|
#76
|
|||
|
|||
|
#77
|
|||
|
|||
Correct...
"http.+\\&start=(\\d+)|\\&page=(\\d+)$|\\?page=(\\d+)$|/page-(\\d+)$|\\?page=(\\d+)\\&|/page/(\\d+)/$|/page(\\d+)$|/page/(\\d+)\\?|/page(\\d+)/|\\?PageNo=(\\d+)|\\?paged=(\\d+)|/page/(\\d+)$|\\?st=(\\d+)|\\&s=(\\d+)+|\\&p=(\\d+)|\\&cat=(\\d+)|\\?_pgn=(\\d+)|\\?id=(\\d+)|/p/(\\d+)|\\?go=(\\d+)" |
#78
|
||||
|
||||
Is this again your RegEx special question or is that actually JDownloader related ?
.....
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#79
|
||||
|
||||
use regex budy/or regex testing website, it would have told you that your patterns are invalid!
missing closing ) on the first , second is even worse as your not sharing the protocol.+ component so 95% pattern is invalid. no more regex based questions.. please use software or websites to test your patterns. raztoki
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
Thread Tools | |
Display Modes | |
|
|