JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 04.02.2019, 01:12
devdaring devdaring is offline
Baby Loader
 
Join Date: Feb 2019
Posts: 7
Default Instagram accounts with up to/around 1500+ posts will stall the Linkgrabber

Most profiles with around 1500+ posts seem to freeze and bug out the linkgrabber. The best example will be any profile with 5,000+ posts.

Here is an example profile that replicates the stall and bug that will make grabbing other links glitched afterwards and only picks up 10-20 posts then freezes as well -
**External links are only visible to Support Staff****External links are only visible to Support Staff** in my case this account gave up after 2,662 files found when the account has over 5,800 posts. As another example, I've seen profiles with 2000 posts stop at sometimes 1000.

Thanks!

Last edited by devdaring; 05.02.2019 at 23:23. Reason: Via title - Above*, not "up to".
Reply With Quote
  #2  
Old 04.02.2019, 17:47
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 60,436
Default

What exactly do you mean by *freeze and bug out* ?
I did some testing and inbetween the instagram server killed the connection that stopps the crawling process

Can you provide some more profiles for testing, so we can try to add some *pauses* to avoid server killing the connection
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 04.02.2019, 18:20
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 60,436
Default

__________________
JD-Dev & Server-Admin
Reply With Quote
  #4  
Old 05.02.2019, 00:50
devdaring devdaring is offline
Baby Loader
 
Join Date: Feb 2019
Posts: 7
Default

Quote:
Originally Posted by Jiaz View Post
What exactly do you mean by *freeze and bug out* ?
I did some testing and inbetween the instagram server killed the connection that stopps the crawling process

Can you provide some more profiles for testing, so we can try to add some *pauses* to avoid server killing the connection
"Freezing" as in the "Crawling for downloads" continues forever without doing anything, so that's the server reset, I'm guessing. Then I need to abort it. After the "freeze", I find what seems like a "bug" where trying to crawl for a new instagram profile will only pick up 12 or so files from the user. It seems that the plugin doesn't respond properly after the server reset.

Accounts that freeze the crawling:
**External links are only visible to Support Staff****External links are only visible to Support Staff** (stops at ~70% finished)
**External links are only visible to Support Staff****External links are only visible to Support Staff** (stops at ~50% finished)
**External links are only visible to Support Staff****External links are only visible to Support Staff** (stops at ~40% finished)
**External links are only visible to Support Staff****External links are only visible to Support Staff** (Untested)
**External links are only visible to Support Staff****External links are only visible to Support Staff** (Untested)
**External links are only visible to Support Staff****External links are only visible to Support Staff** (stops at ~30% finished)
**External links are only visible to Support Staff****External links are only visible to Support Staff** (stops at ~75% finished)

Aborting before it server resets (freezes) stops it from bugging

Accounts that bug out after aborting the frozen crawl:
**External links are only visible to Support Staff****External links are only visible to Support Staff** (31 files found)
This happens to every profile. I'm just providing a quick link to show it happening

The steps I take to fix the bug are (they are completely random attempts to fix it):
1. Abort crawl
2. Delete the "frozen" links folder
3. Add something new to the downloadlist
4. Restarting JDownloader many times
5. Crawl for links from a site other than Instagram
6. Crawling for Instagram profiles works again, at least the ones without too many posts.

In no particular order. I don't know which of these steps or combination of steps fixes it. It does not always fix. Maybe all you have to do is wait a while?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 04:47.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.