JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 24.10.2020, 09:14
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default download my own Instagram saved posts

Hello guys, I have been trying to download all my *saved* posts in Instagram, it's like the personal favorite bookmarks. But Jdownloader can only download my *own* posts, not the posts from others that I collected.

Once you log in to your Instagram, you can go to:
**External links are only visible to Support Staff****External links are only visible to Support Staff**[username]/saved/
or go to your app's profile and see it there.

Also, I am getting an Instagram plugin outdated error message recently. But I can't download the *saved* posts even when this plugin was still up to date.

Thank you so much.
Reply With Quote
  #2  
Old 25.10.2020, 03:12
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

This post is duplicated, can someone delete it for me, please? I can't find the function to delete it. Thanks.
Reply With Quote
  #3  
Old 26.10.2020, 13:51
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Hi,
You need to add your Instagram account to JD and then you can add the /username/saved" URL to let JD grab all saved items.
If you cannot add your Instagram account, please post a DEBUG-log:
Please post your log-ID here | bitte poste deine Log-ID hier.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #4  
Old 30.10.2020, 22:26
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

Thanks. I just tested it, somehow I can add Instagram account successfully through simple username/password this time.

I tried to copy the URL of saved posts just now: **External links are only visible to Support Staff****External links are only visible to Support Staff**
Jdownloader only grabbed a very small amount of the posts I saved, including only 2 Instagramers, 18 pictures in total. But I have a few hundred posts saved at least. It also grabbed my own posts which are not a part of my saved posts at all.

This log below includes both the events of me adding my instagram account successfully and partially failing to grab all my saved posts:
30.10.20 16.13.12 <--> 30.10.20 16.25.16 jdlog://6979325302851/

Thank you so much for your help.
Reply With Quote
  #5  
Old 02.11.2020, 16:13
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

I won't be able to fix this without having the credentials to either your- or a similar test-account.
I cannot use my private account for testing either - it doesn't contain that manu saved items and here, all are grabbed ...

It is not easy to create a legit instagram account for testing as they will usually want a phone number for verification.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #6  
Old 02.11.2020, 20:31
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

Can I send you my password in a private message?
Reply With Quote
  #7  
Old 02.11.2020, 21:34
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

you can send him via support@jdownloader.org or as private message
__________________
JD-Dev & Server-Admin
Reply With Quote
  #8  
Old 05.11.2020, 06:25
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

Just sent an email to the support email address with the username password. Thank you.
Reply With Quote
  #9  
Old 05.11.2020, 12:56
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

I've informed pspzocker about it
__________________
JD-Dev & Server-Admin
Reply With Quote
  #10  
Old 06.11.2020, 14:36
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Hi again,

here a small update on this:
I was finally able to successfully login into your instagram account.
However, I was not able to fix the underlying issue although I consider it "nearly working".
Jiaz will have to take a look at this - he might be able to fix what I've overlooked.

One important note:
Once we are done with this, you can change the password of your Instagram account!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #11  
Old 08.11.2020, 04:04
kdean kdean is offline
Black Hole
 
Join Date: Jul 2010
Posts: 282
Default

Did you guys happen to just break instagram completely with a recent update? Worked before the update when linkgrabbing an account url. Now urls don't get pulled into linkgrabber and I see the host and decrypter were updated and this thread appears to be the most recent discussion of changes, so I thought something may have gone wrong.
Reply With Quote
  #12  
Old 09.11.2020, 13:36
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

__________________
JD-Dev & Server-Admin
Reply With Quote
  #13  
Old 09.11.2020, 13:37
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

@kdean: we will look into this
__________________
JD-Dev & Server-Admin
Reply With Quote
  #14  
Old 09.11.2020, 15:28
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Quote:
Originally Posted by kdean View Post
Did you guys happen to just break instagram completely with a recent update?
Yes - sorry for that!

I've just fixed it.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #15  
Old 09.11.2020, 20:57
kdean kdean is offline
Black Hole
 
Join Date: Jul 2010
Posts: 282
Default

Quote:
Originally Posted by pspzockerscene View Post
Yes - sorry for that!

I've just fixed it.

-psp-
Confirmed working again. Thanks.
Reply With Quote
  #16  
Old 09.11.2020, 21:06
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Thanks for the feedback!
__________________
JD-Dev & Server-Admin
Reply With Quote
  #17  
Old 14.11.2020, 11:55
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

Quote:
Originally Posted by pspzockerscene View Post
Yes - sorry for that!

I've just fixed it.

-psp-
Hi, dear pspzockerscene, no rush, just wonder if there's anything else I can help you with. Thank you for your hard work!
Reply With Quote
  #18  
Old 14.11.2020, 12:07
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

Just realized that Instagram finally released a long-waited update for their web version, finally their saved page has collections, which used to be only available in mobile apps.

So now, you have not only links like this:
**External links are only visible to Support Staff****External links are only visible to Support Staff**[username]/saved/

You also have this:
**External links are only visible to Support Staff****External links are only visible to Support Staff**[username]/saved/all-posts/
**External links are only visible to Support Staff****External links are only visible to Support Staff**[username]/saved/[collection name]/[collection ID]/

Maybe we can support this as well?

Thanks guys! You are the best!!!:P
Reply With Quote
  #19  
Old 14.11.2020, 18:26
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Please send example links to pspzockerscene via pm. Those collections sound more useful than generic tag support?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #20  
Old 15.11.2020, 03:37
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

Quote:
Originally Posted by Jiaz View Post
Please send example links to pspzockerscene via pm. Those collections sound more useful than generic tag support?
Thank you, I just PMed him.:P
Reply With Quote
  #21  
Old 15.11.2020, 18:56
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Do not thank me, thank him
__________________
JD-Dev & Server-Admin
Reply With Quote
  #22  
Old 16.11.2020, 18:38
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Very interesting that there are collections of saved items available similar to Facebook (well - it's Facebook anyways^^).

At this moment these ones are not supported and also I don't have a test-account available with such collections.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #23  
Old 19.11.2020, 03:37
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

Thanks for the update, you can feel free to poke around in my account that I shared with you, or you can do it in your own account. It's OK if the collections are not supported, as long as the general saved posts can be downloaded, that will be GREAT! Cheers :p
Reply With Quote
  #24  
Old 19.11.2020, 18:53
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Got it working.
Feel free to test it once the next update gets released.

I'll leave your account in my test JD for now.
You can kill my session in your Instagram account whenever you've tested the changes and are fine with them.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #25  
Old 19.11.2020, 20:52
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

It's lovely to hear this great news, I can't thank you enough! But for some reason, the link grabber can't grab anything. (it can still grab all the content under a specific account, just not the saved posts.) My Instagram account is in the account manager and everything looks fine. I made sure to update JD and even restarted it after the update. Anything I might have missed?

Last edited by onyxlee; 19.11.2020 at 20:57.
Reply With Quote
  #26  
Old 19.11.2020, 21:52
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Which URL did you add?
If you want to crawl all items of a user, you need to add:
instagram.com/username
If you want to grab your own saved stuff you have to add:
instagram.com/YourOwnUsername/saved/
To grab your own saved items, you of course need to have your own instagram account added to JD first.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #27  
Old 20.11.2020, 06:51
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

Quote:
Originally Posted by pspzockerscene View Post
If you want to grab your own saved stuff you have to add:
instagram.com/YourOwnUsername/saved/
To grab your own saved items, you of course need to have your own instagram account added to JD first.
-psp-
Hi PSP. this one doesn't work for me:
instagram.com/MyOwnUsername/saved/
I already have my account added to JD.

I also tested instagram.com/MyOwnUsername/saved/all-posts/, which is the "collection of all the saved post", also didn't work.

It seems like the link grabber worked for a second, but without any error messages, it just gave up without getting any links.
Reply With Quote
  #28  
Old 20.11.2020, 13:46
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

@onyxlee: Please provide a debug log, see https://support.jdownloader.org/Know...d-session-logs
Enable Settings->Advanced Settings->Log.debugmodeenabled
restart JDownloader, reproduce the issue, create log and post shown logID here
__________________
JD-Dev & Server-Admin
Reply With Quote
  #29  
Old 20.11.2020, 14:50
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Fixed - sorry, my mistake!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #30  
Old 22.11.2020, 10:25
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

Omg!!! It worksssss!!!!!!!!!!!!!!!! Tearssssssss!!! Waited for this for my whole life!!!

Thank you psp!!! Thank you jiaz!!! Merry Christmas, hahaha!

Last edited by onyxlee; 22.11.2020 at 10:28.
Reply With Quote
  #31  
Old 23.11.2020, 10:45
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Thanks for the feedback
All thanks goes to psp
__________________
JD-Dev & Server-Admin
Reply With Quote
  #32  
Old 24.11.2020, 00:25
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Quote:
Originally Posted by onyxlee View Post
Omg!!! It worksssss!!!!!!!!!!!!!!!! Tearssssssss!!! Waited for this for my whole life!!!
Wohoo I've never seen such a crazy reaction in our forum

Quote:
Originally Posted by onyxlee View Post
Thank you psp!!! Thank you jiaz!!! Merry Christmas, hahaha!
No problem - thanks for trusting me and thanks for your feedback

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #33  
Old 12.12.2020, 00:03
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

Hi, psp, just had a very crucial discovery that I have to share with you right away.

So my most likely theory is, Instagram very likely disabled my account after I downloaded hundreds of Instagram posts with my own account.

The details:
After I shared with you my spare account (account B), Instagram only sent an email asking me whether the login in your country was permitted. Since then Instagram didn't bother me on this account at all.

After the function worked, I signed in my primary account (account A) on JDownloader, crawled my saved posts, then downloaded all those saved posts with my account A. Then, probably a few days or a week after that, my account was disabled. I can't log in, nor see my posts, so I can't be sure whether someone hacked into my account and did something terrible.

(Instagram said I violated the community guideline, but they didn't mention using a crawler to go through your own account and download posts. They did mention using bots or 3rd party services to massively like your posts to boost the traffic, which is a violation to their rules, which I never did as well.)

Instagram asked me to verify my identity, I did so. Nothing happened. (Let's skip how awful Instagram's customer support is.) I had to report my issue to them as a business owner, not just a typical user. They came back to me with the weirdest request, asking me to handwrite some specific number, hold it against my face with both hands, and then take a picture (I guess with the third hand) and send it back to them. I haven't heard from them since then.

I have been going through all the possibilities. If I ignore the possibility of a hacker compromising my account, the only possible reason might be me downloading my posts with my own account.

The solution is pretty simple. Just use the function you developed recently as is to crawl all the links. Before you download them, disable the Instagram account in the account manager. That's it. Then you can download all you want as anonymous.

Of course, this is only my theory. Without my account, I can't further confirm it. But it's so far the one that makes the most sense.

I would like to share it with you and hopefully everyone who might use this function, which is a lifesaver for me and many people. Sincerely hope nobody has to deal with the worst customer support of the world, Instagram.
Reply With Quote
  #34  
Old 14.12.2020, 14:14
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

I've created a ticket for this,
optional try to download without account first and only fallback to account if required
that should help a lot in this situation
__________________
JD-Dev & Server-Admin
Reply With Quote
  #35  
Old 14.12.2020, 17:23
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

Quote:
Originally Posted by Jiaz View Post
I've created a ticket for this,
optional try to download without account first and only fallback to account if required
that should help a lot in this situation
Yes, can't believe how tight Instagram is running their service right now. I will update you whenever I have some new discoveries.
Reply With Quote
  #36  
Old 14.12.2020, 18:01
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

I've tried to optimize this handling without the need of another setting.
The following information is simplified:

The host plugin will only login whenever needed e.g. if a direct-URL times out and needs to be re-freshed.
The crawler will still always login at this moment but if wanted, that could also be optimized more.

Most of all times and especially for users who do not use JD to mass-download Instagram items, JD will really only need the account for the initial crawling - it will store the picture direct-URLs for downloading.
And again:
In theory, Instagram should e.g. put a token in those direct-URLs so that when downloading them, they can assign these http requests to the initial account -> Maybe unlikely but possible!

Also, login is required for the "original quality download" so to minimize login requests furthermore, deactivate this in Settings -> Plugins -> instagram.com (default = off already)

Everything I've written here has certain edge cases e.g.:
- If crawling your own saved posts, best quality will always be grabbed without requiring extra http requests
- Refreshing direct-URLs of user story elements always requires an active account

Also, no matter how far we optimize this, you can always trigger their rate-limits or an account ban by simply adding tons of items - there is no permanent solution for this!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 14.12.2020 at 18:04.
Reply With Quote
  #37  
Old 14.12.2020, 18:59
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Hm okay I've optimized the crawler too.
Now it will only login if needed and if the situation is unclear e.g. profile URL, it will try without account first and retry with account if the profile is private and an account is available.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #38  
Old 14.12.2020, 19:16
onyxlee onyxlee is offline
JD Alpha
 
Join Date: Sep 2020
Posts: 21
Default

Thanks, PSP, you are right, in order not to upset Instagram and trigger some rate limits, I will now only download 50 items or so every time.

Too bad that Instagram doesn't list these limits anywhere, I used to use some chrome extension to grab my saved posts, every time I download thousands of posts without triggering any bans. Seems like those days are gone. (These extensions can skip a huge part of the posts without letting you know, so I haven't used them for a long time.)
Reply With Quote
  #39  
Old 14.12.2020, 19:31
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Quote:
Originally Posted by onyxlee View Post
Too bad that Instagram doesn't list these limits anywhere, ...
They have no reason to do this

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #40  
Old 15.12.2020, 15:29
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Quote:
Originally Posted by onyxlee View Post
Too bad that Instagram doesn't list these limits anywhere, I used to use some chrome extension to grab my saved posts, every time I download thousands of posts without triggering any bans. Seems like those days are gone. (These extensions can skip a huge part of the posts without letting you know, so I haven't used them for a long time.)
They want you to use their platform, not to crawl it
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 23:14.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.