JDownloader Community - Appwork GmbH
 

Reply
Page 3 of 6 12 3 45 Last »
 
Thread Tools Display Modes
  #81  
Old 09.05.2013, 22:51
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
Interesting :
JD 2 - 30000 photos = 20 hours (if finish parsing error will crash or not)

Tested:
another application - 30000 photos = 1 hours 30 min
Reply With Quote
  #82  
Old 10.05.2013, 08:28
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
on your Original link the album is only 269 images, Took me 2.5minutes on my new code. The only way to make it faster again is to remove half decrypter code and make it into a hoster plugin. Probably the reason why the other program is so much faster, as the adding of links is only small component of the task. If online checking and dl components done its own plugin, even at the download stage you could increase the efficiency of the plugin by about 10 fold, siding scale depending on the size of the gallery. The decrypter does everything one step at a time, and this can effect over all time lapsed assuming large tasks.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #83  
Old 10.05.2013, 10:05
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
Raztoki - Is JD - first scans page 1, page2, Page3, Page4 etc. ? and then analyzes and then processes the links?
Last edited by djmakinera; 10.05.2013 at 10:58.
Reply With Quote
  #84  
Old 10.05.2013, 12:14
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
yah, not sure why we don't use the all image page (/main/pic_tape.php?ad=uid), think its always the biggest image, we would just need to goto x pages vs, x * y to get finallinks. I'll ask psp later, maybe they do not show all images/pic tape for all albums?
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #85  
Old 10.05.2013, 13:05
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
Do not use (/main/pic_tape.php?ad=uid)
because if the image is large in size (full size) - does not show the maximum size

**External links are only visible to Support Staff****External links are only visible to Support Staff** (NORMAL SIZE) (/main/pic_tape.php?ad=uid)

**External links are only visible to Support Staff****External links are only visible to Support Staff** (FULL SIZE)
Last edited by djmakinera; 10.05.2013 at 13:10.
Reply With Quote
  #86  
Old 10.05.2013, 15:37
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
I compared the link returned from the current implementation in JD and then went to pic_tape to find the same image and it was the exact same image url. So not sure what you did but I can't reproduce your findings.

If we use this method it will return all links as you only need to hit each 'x pic tape page' (* 12 a page) = total images vs 'each album page y * (48 -1) each image page = total images', in the standard album layout.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Last edited by raztoki; 10.05.2013 at 15:43.
Reply With Quote
  #87  
Old 10.05.2013, 15:42
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
Example:

_**External links are only visible to Support Staff**Red">b0.eu.IMGSRC (NORMAL)

_**External links are only visible to Support Staff**Red">o0.eu.IMGSRC (FULL)
Last edited by djmakinera; 10.05.2013 at 15:44.
Reply With Quote
  #88  
Old 10.05.2013, 16:17
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
I saw that, and that was not my findings.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #89  
Old 21.05.2013, 19:14
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default iMSRC.RU - broken
**External links are only visible to Support Staff****External links are only visible to Support Staff**

logfile:

http://jdownloader.org/pastebin/110969
Reply With Quote
  #90  
Old 26.05.2013, 17:09
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
Seems it only happens for new links, and I made support based from the html in your log. That said I was able to find some new links on the site also to test it with, and it works. In the process of rewriting the plugin to speed up efficiency. Just need to find some passworded links so I can implement/test the new code!
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #91  
Old 28.05.2013, 21:18
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
raztoki
Still loses to time hehe

Test:

JD (12 photos)
31 second
47 second

Other program (12 photos)
3 second
4 second
Reply With Quote
  #92  
Old 28.05.2013, 21:28
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
  
Join Date: Mar 2009
Location: Deutschland
Posts: 71,117
Default
JD is no site crawler/ripper, at least it's not made exactly for that.

GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #93  
Old 28.05.2013, 21:34
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
I know, but it was addressed to raztoki. We have a common theme hehe
Reply With Quote
  #94  
Old 28.05.2013, 22:39
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default "Broken parsing" (probably all links!)
"Broken parsing" (probably all links)

I do not know what happened but JD2 shows that all of the links "out of date"
Example:

Vk.com - broken
Imgsrc.ru - broken
zippyshare.com - broken
etc...



I tested different installations of JD2 (without updating!) - All the links work.
So what happened?

remove hosts and works all
Last edited by djmakinera; 28.05.2013 at 22:59.
Reply With Quote
  #95  
Old 29.05.2013, 06:36
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
not sure why I didn't post here

so answer for everyone else is..

Quote:
Originally Posted by raztoki
try closing jd
goto install folder
delete that file jdownloader\tmp\hosts.json
restart jd

could be your host file corrupted? No idea really just wild guess...

I'm going to bed now nite
raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #96  
Old 29.05.2013, 06:46
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
highly dobutful that the decrypter takes that long when everything goes right. As in time could increase/vary slightly if additional http request caused by connection timeout issues are required, or password prompting or album forwarding takes place. Also if you add links that are not in the following format, imgsrc.ru/username/a\d+.html, more page requests take place in order to make sure it's decrypting on page one of given album. The slow part of the original decrypter was processing of each of the img urls imgsrc.ru/username/(\d+).html (notice no a) to get the direct http url of the image. This is now done in the dedicated hoster plugin. Since my previous change to this plugin (a month or so ago) I placed in a time to decrypt output in the log. Though now displays in seconds and not minutes as the decrypter works fast!

the link you provided in this thread for example
62|imgsrc.ru 29/05/13 12:44:46 PM - INFO [ jd.plugins.decrypter.ImgSrcRu(decryptIt) ] -> Time to decrypt : 1 seconds. Returning 43 DownloadLinks for
a few tests of my own
84|imgsrc.ru 29/05/13 1:33:57 PM - INFO [ jd.plugins.decrypter.ImgSrcRu(decryptIt) ] -> Time to decrypt : 2 seconds. Returning 68 DownloadLinks for
86|imgsrc.ru 29/05/13 1:34:48 PM - INFO [ jd.plugins.decrypter.ImgSrcRu(decryptIt) ] -> Time to decrypt : 1 seconds. Returning 43 DownloadLinks for
114|imgsrc.ru 29/05/13 1:40:30 PM - INFO [ jd.plugins.decrypter.ImgSrcRu(decryptIt) ] -> Time to decrypt : 19 seconds. Returning 705 DownloadLinks for
151|imgsrc.ru 29/05/13 1:44:45 PM - INFO [ jd.plugins.decrypter.ImgSrcRu(decryptIt) ] -> Time to decrypt : 21 seconds. Returning 898 DownloadLinks for

considering my ping to imgsrc is half a second, performance wise that's a great outcome.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Last edited by raztoki; 29.05.2013 at 08:04.
Reply With Quote
  #97  
Old 29.05.2013, 08:13
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
That's right, now is the ultra-fast!

Although there is no original names (in some cases the useful since they contain information about the image - if detailed name image*) I know that the name comes from Vkontakte and all the pictures also;)

1. I found a problem with the name (here is not complete title)

example1:
* **External links are only visible to Support Staff****External links are only visible to Support Staff**

missing at the end of the name " .27 "
Last edited by djmakinera; 29.05.2013 at 08:36.
Reply With Quote
  #98  
Old 29.05.2013, 10:26
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
Original filenames as in 'володина даша 1.jpg' ?? I didn't wana go down that path as this way at least they will never dupe out with exact same filename. And I moved away from final link name provided by the link itself because its randomised, and they have changed it in the past. They can very quickly change it via dynamic javascript, for instance.
// document.getElementById('oripic').href='**External links are only visible to Support Staff**+o.charAt(0)+'/'+o+'/9/31970729'+n+'.jpg';
// document.getElementById('bigpic').src='**External links are only visible to Support Staff**+o.charAt(0)+'/'+o+'/8/'+'463518'+n+'.jpg';

This way, you can get the saved filename (upid) and go back to its thread if you still know the username eg. imgsrc.ru/username/upid.html.

I traced the vol.27 issue, its actually picked up fine by the plugin. There are cleanup routines somewhere else which remove it. So not a problem of this plugin.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #99  
Old 09.06.2013, 17:08
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default imgsrc.ru - not working
I do not know what's the problem. Maybe you need to skip ads or other direction links

Subject category (+18) - there are other links + advertising

**External links are only visible to Support Staff****External links are only visible to Support Staff**


http://jdownloader.org/pastebin/111659


--ID:57TS:1370789906161-13-06-09 16:58:26 - [jd.plugins.decrypter.ImgSrcRu(parsePage)] -> Possible plugin error: Please confirm in your webbrowser that this album
Last edited by djmakinera; 09.06.2013 at 17:10.
Reply With Quote
  #100  
Old 09.06.2013, 18:09
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
not a bug buddy, just it couldn't find any additional links its a warning to let you know/confirm in browser!

that album is only one photo, as albums display 48 images, the first image doesn't have the links I am looking for, I warn in case of possible plugin error. If only one pic returned within a one pic album, then theirs nothing to worry about right?
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #101  
Old 09.06.2013, 18:35
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
Hehe, One photo? I did not know I looked not to the album - the first link I gave to random testing
Reply With Quote
  #102  
Old 09.06.2013, 18:37
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
  
Join Date: Mar 2009
Location: Deutschland
Posts: 71,117
Default
Logger outputs don't mean that the plugin has to be broken.
You have to read and understand them and check it manually before reporting anything

GreeZ pspzockerscene *closed*
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #103  
Old 15.06.2013, 11:08
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default IMGSRC.RU (general problem)
IMGSRC.ru - how to turn off(disable) non-stop pop-up window to enter the password to album?
Paste a list of links but do not always know the password.
Besides, there are many such windows (for example, when trying to enter the password to one window - I can not - because next window show, etc.
Reply With Quote
  #104  
Old 15.06.2013, 11:30
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
I'll look at it when I return.. going out!
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Last edited by raztoki; 15.06.2013 at 11:33.
Reply With Quote
  #105  
Old 15.06.2013, 15:39
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
JD2 - is trying to download "delete link" (a user or a moderator has deleted) at a time when even work (when added to the download).
Is JD can "jump" and download the next line (because download still invalid links)

I hope that I have been understood?

(example invalid link, removed etc..)

**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #106  
Old 16.06.2013, 05:21
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
see how you go now.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #107  
Old 21.06.2013, 17:10
zakura6
Guest
  
Posts: n/a
Default
Thanks
Reply With Quote
  #108  
Old 04.07.2013, 12:13
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
As for me, without a complete package name completely disqualifies this program to download. That can not be.
Reply With Quote
  #109  
Old 04.07.2013, 18:55
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
djmakinera... once again non informative post == pointless. package names exist and they are adequate.

String name = Encoding.htmlDecode(username.trim()) + " @ " + Encoding.htmlDecode(fpName.trim());
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #110  
Old 04.07.2013, 20:39
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
You do not understand me. There are many similar albums and I do not want to be mixed albums. Prefer separate albums!

There should be three packages of 1000 photos. A is a single package with 3000 photos.
Last edited by djmakinera; 04.07.2013 at 20:49.
Reply With Quote
  #111  
Old 12.07.2013, 02:05
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default iMGSRC.RU - "ERROR parsePage"
--ID:8280TS:1373587405035-13-07-12 02:03:25 - [jd.plugins.decrypter.ImgSrcRu(parsePage)] -> ERROR parsePage
--ID:8280TS:1373587405035-13-07-12 02:03:25 - [jd.plugins.decrypter.ImgSrcRu(decryptIt)] -> Decrypter broken for link: **External links are only visible to Support Staff****External links are only visible to Support Staff**
--ID:8280TS:1373587405035-13-07-12 02:03:25 - [jd.plugins.decrypter.ImgSrcRu(decryptIt)] -> Time to decrypt : 0 seconds. Returning 0 DownloadLinks for **External links are only visible to Support Staff****External links are only visible to Support Staff**
--ID:8280TS:1373587405035-13-07-12 02:03:25 - [jd.plugins.PluginForDecrypt(decryptLink)] -> CrawlerPlugin out of date: jd.plugins.decrypter.ImgSrcRu@71c2db33 :21551
--ID:8280TS:1373587405035-13-07-12 02:03:25 - [jd.plugins.PluginForDecrypt(decryptLink)] -> URL was: **External links are only visible to Support Staff****External links are only visible to Support Staff**



Other (Full size galery)
Example:

**External links are only visible to Support Staff****External links are only visible to Support Staff**
Last edited by djmakinera; 12.07.2013 at 10:00.
Reply With Quote
  #112  
Old 12.07.2013, 10:37
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
Please make posts in this thread, no need to make a new thread.

Fixed

wait for updates to come.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #113  
Old 12.07.2013, 10:45
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
3 albums mashup/join.

There should be three packages and there is only one package)



Copy/paste all links

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #114  
Old 23.07.2013, 03:04
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
Update:
Not work link:


------------------------Thread: 5308: PluginErrors.log-----------------------
--ID:5308TS:1374541226609-13-07-23 03:00:26 - [jd.plugins.PluginForDecrypt(errLog)] -> CrawlerPlugin out of date: jd.plugins.decrypter.ImgSrcRu@26d8506b :21721
--ID:5308TS:1374541226609-13-07-23 03:00:26 - [jd.plugins.PluginForDecrypt(errLog)] -> URL was: **External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #115  
Old 23.07.2013, 05:44
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
" within the title named killed it. As regex didn't allow. I've allowed it now, hopefully it wont break other pages.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #116  
Old 28.07.2013, 22:08
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
Probably JDownloader is blocked(only download)... Very slow download, terminates the connection and other problems) (even if the reduction to 1 CHUNK and the speed limit my setting (MAX. 100 Kb / s)
I checked an other program, it gets/downloads much better!!... :\

JDownloader 2 - I downloads all day and not I download everything (still errors/lost connection)
I tried another program - I can download per hour, (only normal sized image)
Last edited by djmakinera; 28.07.2013 at 22:11.
Reply With Quote
  #117  
Old 29.07.2013, 07:15
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
Not a valid report djmakinera.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #118  
Old 29.07.2013, 10:04
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
Plug been killed :>
Last edited by djmakinera; 29.07.2013 at 10:06.
Reply With Quote
  #119  
Old 29.07.2013, 17:50
raztoki's Avatar
raztoki raztoki is offline
English Supporter
  
Join Date: Apr 2010
Location: Australia
Posts: 17,614
Default
still not following,

tested some of the links above, of the tested they all decrypt and download without any problems.

if you have a problem, make a proper bug report none of this circumstantial rubbish.

kthxbye
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #120  
Old 31.07.2013, 11:09
djmakinera djmakinera is offline
Banned
  
Join Date: May 2010
Location: Poland
Posts: 8,387
Default
raztoki - JDownloader is blocked.


I checked another program and nothing is blocked. (5x chunk) + multi-open program

JD2 Probably too many requests to the server... (pspzockerscene (you may know what I mean)



http://jdownloader.org/pastebin/113590

My setting 1x /3x (chunk)

and 1x / 2x still "lost connection"
Last edited by djmakinera; 31.07.2013 at 11:20.
Reply With Quote
Reply
Page 3 of 6 12 3 45 Last »

« Previous Thread | Next Thread »
Thread Tools
Display Modes
Linear Mode Linear Mode

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

All times are GMT +2. The time now is 01:31.
JDownloader Community Board - Archive - Top
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.