JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 03.11.2020, 15:58
Kryat Kryat is offline
Baby Loader
 
Join Date: Nov 2020
Posts: 6
Default Usernames when downloading Instagram hashtags

Hi!

Maybe this is a feature that already exists (couldn't find it, though), but if it doesn't, it would be a huge help to me right now if it could be implemented any time soon. May be useful to many others as well.

When downloading images from a specific Instagram account, all files come with the account name included in the file name. But when downloading all images under a hashtag, all account names are replaced with the word "Data" instead.

Would it be possible to make it so that when downloading hashtag feeds each file could parse the account names into the file as it does with individual accounts?

I need this because I do work that entails saving images posted as part of art contests on Instagram, under particular hashtags, to pick a random winner later. Normally I do this all manually, and it is hard work, especially when there are way too many entries. If Jdownloader could ID the user of all the posts for me that way, it would be a blessing!

Many thanks for considering my request.
Reply With Quote
  #2  
Old 03.11.2020, 19:02
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 69,669
Default

Can you please provide example links so we can reproduce this?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 03.11.2020, 22:22
Kryat Kryat is offline
Baby Loader
 
Join Date: Nov 2020
Posts: 6
Default

Quote:
Originally Posted by Jiaz View Post
Can you please provide example links so we can reproduce this?
Sure!

When I feed JDownloader this link: **External links are only visible to Support Staff****External links are only visible to Support Staff**
It gives me the result in file "JDownloader_example_01.png", with the name of the account the post belongs to in the file name.

But when I feed it a hashtag like this one: **External links are only visible to Support Staff****External links are only visible to Support Staff**
It gives me the result in "JDownloader_example_02.png", where all account names show up as the word 'Data'. What I would like is for the files to contain the correct account name each file belongs to instead.

Is this doable?
Attached Images
File Type: png JDownloader_example_01.png (117.8 KB, 2 views)
File Type: png JDownloader_example_02.png (91.3 KB, 2 views)
Reply With Quote
  #4  
Old 04.11.2020, 12:27
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 69,669
Default

The 2nd URL (instagram.com/explore...) is not supported by JDownloader! When JDownloader does NOT auto crawl via copying to clipboard is a hint that JDownloader does not support the copied URL.

When crawling via deep decrypt, JDownloader finds
Quote:
/data/shared
as relative URL in html and combines to data user profile on instagram (instagram.com/data)
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 04.11.2020, 12:29
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,213
Default

Hmm I guess hashtags can lead to a kind of infinite number of results which is why we should rather not add crawler functionality for that.
I suggest crawling the single "/p/" urls manually - you can add those to JD then and, if you want to have the hashtag in these filenames, you could do this via Packagizer rule.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #6  
Old 04.11.2020, 12:35
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 69,669
Default

@pspzocker: we could add decrypter and only add the first x items
__________________
JD-Dev & Server-Admin
Reply With Quote
  #7  
Old 04.11.2020, 12:40
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,213
Default

Here is a ticket:

Don't expect any implementation of this in the near future.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #8  
Old 04.11.2020, 13:02
Kryat Kryat is offline
Baby Loader
 
Join Date: Nov 2020
Posts: 6
Default

Thank you!

Yes, I understand it could cause a big problem with some hashtags that have a near infinite amount of posts attached to them. The idea of a limiter sounds great to me, as normally I believe most users would only be interested in the most recent posts.

In my case, the hashtags I work with are specific, so the number of posts is controlled. Maybe the limiter could be based on either an 'X' number of items or a date of how far back to crawl? That would be neat.

Thank you for considering my request!
Reply With Quote
  #9  
Old 04.11.2020, 13:17
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,213
Default

As said, you can already add those URLs manually at this moment.
Just scroll as far down as you want (in your Browser on the Instagram page) -> Copy all URLs and add them to JD.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist

Last edited by pspzockerscene; 04.11.2020 at 18:10. Reason: Added hint on where to scroll down ...
Reply With Quote
  #10  
Old 04.11.2020, 14:49
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 69,669
Default

He means, scroll down in browser , then select all + copy clipboard will contain the links to the individual posts then
__________________
JD-Dev & Server-Admin
Reply With Quote
  #11  
Old 05.11.2020, 14:35
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,213
Default

Added simple "hashtag crawler" (not yet finished).
Notes:
- Default = limited to 25 items (1 item = defined as one "instagram.com/p/XXX" URL [can contain multiple media items])
- At this moment, pagination does not yet work which means even with no limit set, it would max. crawl the ~100 latest items
- If you want to have the hashtag in your filenames, use the packagizer to get it from the source URL (untested). If that doesn't work, I may add a packagizer property.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #12  
Old 09.11.2020, 21:38
Kryat Kryat is offline
Baby Loader
 
Join Date: Nov 2020
Posts: 6
Smile

This is already a great start! Thank you!
I'll make sure to send you guys a donation this month!
Reply With Quote
  #13  
Old 10.11.2020, 10:18
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 69,669
Default

@Kryat: have you tested the new plugin? does it work? please only donate if you really want to and of course the feature works
__________________
JD-Dev & Server-Admin
Reply With Quote
  #14  
Old 10.11.2020, 12:33
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,213
Default

Yap as said - the new functionality is unfinished and will only grab the "first page" at this moment no matter what you set in the plugin setting.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #15  
Old 11.11.2020, 13:34
Kryat Kryat is offline
Baby Loader
 
Join Date: Nov 2020
Posts: 6
Default

@Jiaz Yes, I tested it and it does work, but partially.

Pagination issues aside, while it does grab everything on the first page, only some of the items have been able to identify the original poster. The others merely come with the name of the link. I've attached an image of the downloaded files I got:

Reply With Quote
  #16  
Old 11.11.2020, 15:44
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 69,669
Default

@Kryat: Would be good to provide example links so we can check/fix the plugin
__________________
JD-Dev & Server-Admin
Reply With Quote
  #17  
Old 11.11.2020, 17:02
pspzockerscene's Avatar
pspzockerscene pspzockerscene is online now
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 57,213
Default

@Kryat
This is not really a bug - I just haven't updated the plugin to contain the hashtag inside filenames, see also my post #11 here.

I've updated that for the next update!

Please keep in mind that, if you're letting JD crawl these "hashtag URLs", it will put the hashtag into the filename but not the name of the uploader as this is not accessible during this new crawl process.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #18  
Old 12.11.2020, 13:10
Kryat Kryat is offline
Baby Loader
 
Join Date: Nov 2020
Posts: 6
Default

@pspzockerscene
Yes, I was referring to the name of the uploader. But I don't understand one thing... How is the name of the uploader not accessible if half the files I crawled picked them up in the file name?
While having the hashtag in the filename is a cool addition, the uploader name is what is most useful in this case.

Throughout the thread I will always be using this hashtag link in my tests: **External links are only visible to Support Staff****External links are only visible to Support Staff**

It is a small hashtag and the one I am currently working with.
Reply With Quote
  #19  
Old 12.11.2020, 15:23
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 69,669
Default

@Kryat: posts with multiple images(gallery) are crawled/process in an additional step and this one fetches the username.
but single image/video posts are not crawled again (to avoid many requests and speed up the crawling process) and those have no username available
__________________
JD-Dev & Server-Admin
Reply With Quote
  #20  
Old 12.11.2020, 15:28
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 69,669
Default

I've updated ticket and let pspzocker know about an idea how to solve this (requires more requests = might be slower)
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 11:30.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.