[goldensun87]

I know that this happens if too many API calls are made, but currently it is happening even though I copied around, maybe, 20 links? And no, I did not add all the links simulatenously to the clipboard, I added them one at a time. This should not be happening, right?

[pspzockerscene]

We can't- and never will know all block-measures of Instagram.

I'm unable to reproduce this here.
Here is what you can do/experiment with to remove the blocks:
- Add/remove Instagram account
- Enable/disable proxy/VPN (= change IP)
- Check if the same happens via browser - maybe Instagram is slow in general at this moment

-psp-

[goldensun87]

Originally Posted by pspzockerscene

We can't- and never will know all block-measures of Instagram.

I'm unable to reproduce this here.
Here is what you can do/experiment with to remove the blocks:
- Add/remove Instagram account
- Enable/disable proxy/VPN (= change IP)
- Check if the same happens via browser - maybe Instagram is slow in general at this moment

-psp-

Ok, I'll provide a url for an entire profile.

**External links are only visible to Support Staff****External links are only visible to Support Staff**

So, on my end, what exactly has happened, is that the crawler is not able to grab more than 70 items from the profile. If this happens on your end, the number of successfully grabbed files may vary. Also, the LinkGrabber remains stuck on, and I have to abort it manually.

Another thing to note, is that this problem is not causing my account to get flagged and locked for bot activity. So, Instagram is not blocking JD in this instance. My guesses are, either there is a bug in JD, or a change in Instagram's internal code is causing this problem.

[pspzockerscene]

Hi again,

I've checked this and I noticed some things:

1. Without account, I'm not able to get all items as it will simply stop after X items saying account is needed for the rest -> Not a bug (expected behavior) as this is what Instagram can do to limit requests or at least to be able to track down accounts/users that download a lot more easily than without them using an account.

2. With account,. I get about 280 items very quickly but then run into their "rate limiting".
In this case, our crawler will wait some time and then resume.
At this point it is still making progress but really slowly.
I'am now at about 450 crawled items but the wait in between increased to sometimes over 1 minute (again: normal behavior).

Summary:
I was unable to find any bugs/unexpected behavior so far.

...so maybe you just quit the process too early.

Without a log I won't be able to tell you what happened:
Please post your log-ID here | bitte poste deine Log-ID hier.

-psp-
EDIT

Now it stopped at/after 474 items.
EDIT2

Download successful -> 474 items -> 1.06GiB

[goldensun87]

I tried logging out of my account, deleting all Insta cookies, and logging in again. This time, trying to crawl the account "vikachekahair", I was able to grab 354 files, but now it has been over an hour, and JD still hasn't been able to grab the remaining files from the account.

Here is my log: 19.11.21 12.12.24 <--> 19.11.21 23.42.33 jdlog://6790925302851/

Edit:So, around 5:30 AM, I noticed that JD finally finished grabbing all the links. But, I cannot say exactly when the grabbing finished, because I wasn't paying attention. Regardless, taking several hours to grab only 475 items, is not a "rate-limiting" issue.

Fast forward to the current time, and JD is taking forever to grab only 3 links, and this is a few hours after the previous grabbing was done. So I think rate limiting should be ruled out at this point. I'm thinking Instagram itself is having issues. Recently, I have seen reports of many Instagram users suffering major downtime with Instagram. Even though I am able to access the site on my browser, perhaps some of their other server components are experiencing difficulties. Hopefully, my log will give you better insight into what is going on.

[pspzockerscene]

I can see about 40 minutes of waiting in your log but that's it.

No idea why it took so long.

-psp-

[goldensun87]

Originally Posted by pspzockerscene

I can see about 40 minutes of waiting in your log but that's it.

No idea why it took so long.

-psp-

Is there any way you can confirm whether it is rate limiting or not? I am thinking this is similar to when a change in Instagram's website code, was causing videos to download very slowly. Well, I'm not sure whether the cause of that was on Instagram's side or JD's side.

[pspzockerscene]

Youg log is very big thus harder to analyze.
Please provide a new log and do the following in beforehand to make sure that it only contains relevant activities:
- Disable all accounts you don't need (e.g. all except IG)
- Disable all URLs you don't need (all except IG)
- Restart JD

-psp-

[goldensun87]

Originally Posted by pspzockerscene

Youg log is very big thus harder to analyze.
Please provide a new log and do the following in beforehand to make sure that it only contains relevant activities:
- Disable all accounts you don't need (e.g. all except IG)
- Disable all URLs you don't need (all except IG)
- Restart JD

-psp-

Disabling all other accounts is simple enough. Can you elaborate on the "disable all urls"? What exactly should I do regarding that? Should my "Downloads" tab be cleared of all downloaded files? Finally, when exactly should I capture a log? Will any Instagram grab attempt do? Or, should it only be after a lagging crawl?

[TexasDude]

I'm having the exact same problem, and for FWIW, I have a temp fix until it gets resolved........

I grab my links, for instance 4 photos in one post, and if the crawler takes its time instead of grabbing them right away, I restart JD.

I regrab and most of the time it will grab them right away. Then it seems as though I can grab a few more from other posts. When it keeps crawling again, I repeat the above.

Just thought I'd point this out for the time being.

[pspzockerscene]

Originally Posted by goldensun87

Can you elaborate on the "disable all urls"?

I'm sorry indeed that was confusing and unnecessary:
Just make sure JDownloader is not downloading anything and all it does is run that crawl process.

Originally Posted by goldensun87

Or, should it only be after a lagging crawl?

Yes that's what we need.
Then again when uploading logs, rather upload "too much" than not enough but make sure that you select timeframes in which your problem occured.

Originally Posted by TexasDude

I grab my links, for instance 4 photos in one post, and if the crawler takes its time instead of grabbing them right away, I restart JD.

So you can even reproduce this with small "image albums"?
Any example URLs/logs available?

As said I was not able to reproduce this issue.
Currently I'am also waiting for Jiaz to provide feedback on another possible issue with our Instagram crawler...

-psp-

[goldensun87]

Ok, I hope this log is not too big.

25.11.21 05.31.16 <--> 25.11.21 05.36.30 jdlog://0711925302851/

[pspzockerscene]

Please wait for our developer Jiaz to find time to investigate this.

-psp-

[pspzockerscene]

I've just fixed another Instagram plugin issue.
Please re-check after the next set of CORE-updates, if that has also fixed the issue you were having.

Bitte auf das nächste CORE-Update warten!

Please wait for the next CORE-Update!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[goldensun87]

This issue does seem to be fixed. Thank you.

[pspzockerscene]

We haven't made any changes though...

Thanks for your feedback anyways!

-psp-