JDownloader Community - Appwork GmbH
 

Closed Thread
 
Thread Tools Display Modes
  #41  
Old 22.02.2016, 20:00
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 67,320
Default

Wait for next core update
__________________
JD-Dev & Server-Admin
  #42  
Old 23.02.2016, 12:07
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

Incomplete regex matching.
Jiaz could write correct(If you find free time), I will be very grateful.

  #43  
Old 23.02.2016, 12:33
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

maybe ???

Quote:
javascript\:__doPostBack\('lbNext',''\)
  #44  
Old 23.02.2016, 12:50
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

Decrypt only 1 page (not work multipage)

**External links are only visible to Support Staff**...
**External links are only visible to Support Staff**...


Code:
[ {
"enabled" : true,
"maxDecryptDepth" : 1,
"id" : 1454334584729,
"name" : null,
"pattern" : "javascript:[\\w]+\\('lbNext',''\\)",
"rule" : "DEEPDECRYPT"
} ]
  #45  
Old 23.02.2016, 13:00
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,169
Default

crawler rules are for two things
URL patterns, for URL listening, as in trigger when you copy URL.. eg. **External links are only visible to Support Staff****External links are only visible to Support Staff**
and if they are direct downloadable or deep decrypt task.

for deep decrypt, our parser looks for links and returns them all.
Its not designed to search for patterns that you provide it with!
It wont process javascript!

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
  #46  
Old 23.02.2016, 14:32
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

raztoki -
Some sites dynamicalu generate web content by using JavaScript. This cause JD2 to fail to detect when loading multipages.
can not force JD2 to process the JavaScript?(Eg. The simulation mouse "click")
:\
  #47  
Old 23.02.2016, 14:36
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,169
Default

I can only advise on vague posts (millions of them), since you rarely give working links to sites your trying to support. I gave help based on the feature you're trying to use outside of design.

If you want complicated function, outside of the simple function that linkcrawler rules provide, _you_ will need to make your own hoster/decrypter plugins.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
  #48  
Old 23.02.2016, 14:42
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

I know, in this case, the script is not enough, you probably need an additional java class code...
  #49  
Old 23.02.2016, 15:24
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

In theory, it should work with the parser, but unfortunately it did not work.
I have no other ideas, but do not give up and still looking solutions

  #50  
Old 23.02.2016, 15:44
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

If matched, add link to download queue use backreferences as {$1}, OR {$2}, etc.?
  #51  
Old 23.02.2016, 15:48
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 67,320
Default

JDownloader cannot evaluate Javascript without a real plugin. Only visible links can be processed!
__________________
JD-Dev & Server-Admin
  #52  
Old 23.02.2016, 16:15
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

I used a parser with "Evaluate calculation scripts" (Enabled)
but I get an error and abort parse.

href="javascript:__doPostBack('lbHome','')">Home</a><img src="../Images/spaceer"
  #53  
Old 23.02.2016, 16:22
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

Code:
import  java.util.regex.Matcher;
import  java.util.regex.Pattern;

/**
   <P>{@code java ParseForclosureResultsXmpl}</P>
 **/
public class ParseForclosureResultsXmpl  {
   public static final void main(String[] igno_red)  {
      String sLS = System.getProperty("line.separator", "\n");

      StringBuilder sdInput = new StringBuilder().
         append("<a id=\"SheetContentPlaceHolder_ctl00_gvForeclosureResutls_lbCaseNum_0\" href=\"javascript:__doPostBack('ctl00$SheetContentPlaceHolder$ctl00$gvForeclosureResutls$ctl02$lbCaseNum','')\" style=\"display:inline-block;width:100px;\">CV-13-798497</a>").append(sLS).
         append("              </td><td align=\"center\">488-05-029</td><td align=\"center\">I</td><td align=\"center\">01/02/2013</td>").append(sLS).
         append("  </tr><tr style=\"background-color:Gainsboro;\">").append(sLS).
         append("      <td align=\"left\">UNKNOWN HEIRS, ETC OF D.C. RUFUS, ET AL  </td><td align=\"left\">10603 HAMPDEN AVENUE</td><td align=\"center\">CLEVELAND</td><td align=\"center\">44108-0000</td><td align=\"center\">").append(sLS).
         append("                  <a id=\"SheetContentPlaceHolder_ctl00_gvForeclosureResutls_lbCaseNum_1\" href=\"javascript:__doPostBack('ctl00$SheetContentPlaceHolder$ctl00$gvForeclosureResutls$ctl03$lbCaseNum','')\" style=\"display:inline-block;width:100px;\">CV-13-798498</a>").append(sLS).
         append("              </td><td align=\"center\">109-16-094</td><td align=\"center\">A</td><td align=\"center\">01/02/2013</td>").append(sLS).
         append("  </tr><tr style=\"background-color:LightGrey;\">").append(sLS).
         append("      <td align=\"left\">SHARECE MILLER, ET AL  </td><td align=\"left\">13514 ALVIN AVENUE</td><td align=\"center\">GARFIELD HTS</td><td align=\"center\">44105-0000</td><td align=\"center\">").append(sLS).
         append("                  <a id=\"SheetContentPlaceHolder_ctl00_gvForeclosureResutls_lbCaseNum_2\" href=\"javascript:__doPostBack('ctl00$Shee").append(sLS);

      String sRqdValuePrefix = "ForeclosureResutls_lbCaseNum_";
      Pattern checkerLoopPattern = Pattern.compile(sRqdValuePrefix + "\\d");
      Matcher m = checkerLoopPattern.matcher("");  //Unused. so the matcher can be reused in the loop.

      int iLn = 0;
      String[] asInput = sdInput.toString().split(sLS);
      for(String s : asInput)  {
         iLn++;    //1st iteration: Was zero, now 1

         //Resuing matcher instead of retrieving new one from Pattern each iteration
         m.reset(s);

         if(m.find())  {
            int iCheckerNumber = Integer.parseInt(s.substring(m.start() + sRqdValuePrefix.length(), m.end()));
            System.out.println("Found on line " + iLn + ", at index " + m.start() + " with checker number " + iCheckerNumber);
         }
      }
   }
}
  #54  
Old 23.02.2016, 18:03
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

E.g. On this page JD2 works perfectly
So there is some exceptions, it works.
**External links are only visible to Support Staff****External links are only visible to Support Staff**
  #55  
Old 24.02.2016, 10:50
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

Quote:
Originally Posted by Jiaz View Post
JDownloader cannot evaluate Javascript without a real plugin.
Only visible links can be processed!
I was able to process javascript parser ~880,000 links direct links, now I can easily add links to JD2

Mini-example:
4800 links
**External links are only visible to Support Staff****External links are only visible to Support Staff**
  #56  
Old 26.02.2016, 17:32
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default Deep-Decrypt multipage - issue

What is wrong?
:confused:


Code:
[ {
  "enabled" : true,
  "maxDecryptDepth" : 1,
  "id" : 1454334584729,
  "name" : null,
  "pattern" : "/p/(\\d+)",
  "rule" : "DEEPDECRYPT"
} ]
**External links are only visible to Support Staff****External links are only visible to Support Staff**
  #57  
Old 26.02.2016, 17:41
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

"pattern" wrong?

Code:
/p/(\\d+)
.*?/p/(\\d+)
.+/p/(\\d+)
  #58  
Old 26.02.2016, 17:58
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 67,320
Default

maxDecryptDepth=1 -> it will not deep decrypt other pages
You need to increase this value
__________________
JD-Dev & Server-Admin
  #59  
Old 26.02.2016, 18:02
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

Regex needs for full-size photos because detect very small files.


Quote:
bizarre-collection\\.biz/d/.+\\.jpg$
  #60  
Old 26.02.2016, 18:04
djmakinera djmakinera is offline
Banned
 
Join Date: May 2010
Location: Poland
Posts: 8,452
Default

Quote:
Originally Posted by Jiaz View Post
maxDecryptDepth=1 -> it will not deep decrypt other pages
You need to increase this value
Unlimited "multipage" Value: 0 ?
Closed Thread

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 16:25.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.