JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 11.01.2020, 17:55
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default MAX Simultaneous downloading not work small files???

Simultaneous downloading for small files - problem.
Pawd probably can't download more than 6-7 at the same time.
And can it be increased in any way to make downloading small files faster?


**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Last edited by raztoki; 14.01.2020 at 10:58.
Reply With Quote
  #2  
Old 12.01.2020, 02:54
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,547
Default

What you will find is that directhttp class, is quite heavy compared to other plugins as it needs to cover so many different circumstances which make it into a generic plugin. These tasks make download routine slower, thus you only ever see x dls been downloaded whilst in reality it should be 20 (assuming you have this set as max sim dl, and max sim per host is disabled) with small files as they finish before they start.

I will point out that you are approaching your goal in the wrong manner. You should use 'linkcrawler rules', or decrypter plugins to find content vs dl html > using the add links dialog to parse for you. This is a backward approach and the application is not designed for this in mind. Thus you are reaching limitations which I don't believe we will address (fixed) because its very inefficient process, we provide means with 'linkcrawler rules' and decrypter plugins which can do the task with the least overhead in the most efficient manner.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #3  
Old 12.01.2020, 03:07
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

I found the cause of this problem.

If the package is small. Maximum simultaneous download works. And faster downloads.

If a larger package. Simultaneous downloading decreases drastically and the speed decreases.

Number of links the package affects the download speed.
Reply With Quote
  #4  
Old 12.01.2020, 03:11
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

The problem exsisten even if there is a dedicated plugin. The number of links in the single package actually worsens the download speed (only tiny / small files) (if you set more simultaneous downloads).
Reply With Quote
  #5  
Old 12.01.2020, 03:31
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,547
Default

once again you have no idea what you're talking about.

size of file packages have ZERO influence on download speed of a given download.

mirror handling/file exists rules do come into play with packages but contrary to what you're stating, the smaller the package is, the faster these tasks happen and not slower.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #6  
Old 12.01.2020, 03:37
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

I tested:

80K - 100K links (one package)
Speed 30-80 KB/s

1 - 10K links (one package)
Speed up to 400KB/s

Or explain better.
The more the package decreases during download, the faster and faster the speeds will be.
Reply With Quote
  #7  
Old 12.01.2020, 03:55
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,547
Default

the only way a package could decrease during download would be cleaning up downloads whilst? GeneralSettings.cleanupafterdownloadaction

are you taking the download speed displayed in the package as proof of speed?
to me thats probably averaged and smaller packages never reach this high speed due to lack of time in which the larger ones average was built.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #8  
Old 12.01.2020, 04:05
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

The difference is that speed is noticeable when the links in the package quickly disappear.
And if it downloads slowly, then the links in the package are slowly decreasing.
This is more noticeable when the package decreases.
Maybe test it yourself, for better insight.
Reply With Quote
  #9  
Old 12.01.2020, 04:59
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,547
Default

I wont be testing. sorry

I've wasted like an hour responding to your pointless threads this morning as it is. I'm not getting that back

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #10  
Old 13.01.2020, 15:21
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

The speed is 10 times faster if the package is not more than 10k - 15k links.

Maybe Jiaz will speak and be able to optimize something?

Or how to automatically split into 10k packages?
For high speeds (shorter file download times and less memory usage)

Here it quickly downloads when the package has decreased to 10k
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
  #11  
Old 14.01.2020, 03:41
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,547
Default

believe most of these issues have been covered in dj performance thread over the years and even in here.
more link you have slower it becomes jd as a whole, and down to the package level (This is due to what I covered above (mirror handling/file checks on disk), etc.). I do not believe that there could be major gains in performance (educated guess), its more so a restriction of how the program functions. To gain significant performance you would need to drop said functions which slow things down.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #12  
Old 14.01.2020, 10:00
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

Hide my links because they were not hidden in the first post.

Code:
GeneralSettings.cleanupafterdownloadaction
Remove finished downloads: immediately
Same function?
Reply With Quote
  #13  
Old 14.01.2020, 10:07
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

Add 1000 links and see how fast it downloads. Certainly several hundred times faster than 20k
Reply With Quote
  #14  
Old 14.01.2020, 12:04
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

Here a small package, gets very quickly. (4000 links per one package)
**External links are only visible to Support Staff****External links are only visible to Support Staff**
20 simultaneous downloads work.
Larger packages block max simultaneous downloads and cannot be squeezed at high speeds.
There must be some problems.
Reply With Quote
  #15  
Old 14.01.2020, 12:10
djmakinera djmakinera is offline
JD Legend
 
Join Date: May 2010
Location: Poland
Posts: 8,445
Default

Small package = work 20 max sim. 500kb/s - 1 Mb/s

Large package 5k - 10k or 100k = 10kb/s - 30kb/s (Only work 2-4 sim.)

Test:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 08:23.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.