JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 23.09.2023, 05:42
Hull2366 Hull2366 is offline
Baby Loader
 
Join Date: Sep 2023
Posts: 6
Question I'm failing to download retweets

I want to download all retweets with image from a profile after a specific date but I'm failing. All links I tried to add only found the profile's media and apparently ignoring the date.
The Twitter plugin option to allow crawl retweets is on and the software is updated. I'm using windows 11.
Log:
18.09.23 18.10.53 <--> 22.09.23 00.30.32 jdlog://7197311370661/
Reply With Quote
  #2  
Old 23.09.2023, 11:40
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,047
Default

Your log doesn't contain any twitter activity.
Please enable debug mode (instructions) and provide a fresh log.

Also keep in mind that the twitter special feature allows you to crawl tweets from now to the given date and noit "from the given date to the end".
Reference: Twitter JD help article
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 23.09.2023 at 11:40. Reason: Fixed typo
Reply With Quote
  #3  
Old 27.09.2023, 14:30
Hull2366 Hull2366 is offline
Baby Loader
 
Join Date: Sep 2023
Posts: 6
Default

Here is the new log: 27.09.23 09.12.10 <--> 27.09.23 09.28.12 jdlog://0208311370661/
There is some way to ask for all retweets images from, as example, September first, until now?
Reply With Quote
  #4  
Old 05.10.2023, 14:40
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,047
Default

Hi,
1. the URL you added contains parameter "retweet=true".
Ths one does not exist. All supported parameters can be found here:
https://support.jdownloader.org/Know...anced-features
If you want JD to crawl retweets, you need to enable this feature in the twitter.com plugin settings.

2. You can only crawl "all from now until max date", see:
https://support.jdownloader.org/Know...anced-features
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #5  
Old 10.10.2023, 19:47
Hull2366 Hull2366 is offline
Baby Loader
 
Join Date: Sep 2023
Posts: 6
Default

Still can't find the retweets.
The crawl retweets option is already enabled.
New log:
10.10.23 14.33.40 <--> 10.10.23 14.45.39 jdlog://8138311370661/
Reply With Quote
  #6  
Old 11.10.2023, 10:48
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,047
Default

Hi Hull2366,
the reason why you get no results is an operator error:
Your made use of the "max_date" parameter inside the URL you added.
You parameter is: 2023-08-15
First [public visible] post in this profile is from date: 2023-05-10
This means that the first date is lower than your given date and thus all results get skipped.

Possible solutions:
- Remove max_date parametrer from URL
- Change max_date parameter in URL to a much lower value such as 2023-01-01
- Use maxitems instead
Reference: Twitter support article:
https://support.jdownloader.org/Know...anced-features

I've updated the plugin to display a bubble notifications in situations where the crawler returns nothing due to bad user given parameters or user plugin settings.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #7  
Old 15.10.2023, 23:35
Hull2366 Hull2366 is offline
Baby Loader
 
Join Date: Sep 2023
Posts: 6
Default

But I already tried without any parameter. And it gives me a result, but this result have only the images posted by the profile, never retweets.

15.10.23 18.22.20 <--> 15.10.23 18.32.43 jdlog://9048311370661/
Reply With Quote
  #8  
Old 16.10.2023, 13:29
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,047
Default

Two things that I found:
1. Looks like twitter is hiding re-tweets when you're not logged in so make sure to add your twitter account to JDownloader.

2. Looks like re-tweets are crawled successfully but you are looking for the wrong tweet-IDs in JD: Twitter is assigning a separate tweet-ID to retweets internally.
Example what JD crawls/shows:
Code:
twitter.com/Lutina_shock/status/1713732752388833717
-> Equals this in browser:
Code:
twitter.com/bobby_riko/status/1713587476043030654
I can try to make you get the same IDs you see in browser but the content you get in the end will be the same and the re-tweets are already crawled successfully.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #9  
Old 16.10.2023, 15:28
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,047
Default

I see that this was supposed to be returning the same IDs the browser shows.
Fixed.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 17.10.2023 at 11:45. Reason: Fixed typo
Reply With Quote
  #10  
Old 17.10.2023, 05:03
Hull2366 Hull2366 is offline
Baby Loader
 
Join Date: Sep 2023
Posts: 6
Default

Looks like it's returning everything now, thank you!
Just one more question, there is some way for me to set the filename from retweets with the username from who post it?
As the example you gave it, could I get bobby_riko in the filename, instead of Lutina_shock? Some rule I could create?
Reply With Quote
  #11  
Old 17.10.2023, 12:12
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,047
Default

Changes for the next update:
- Fixed re-tweets being labled with incorrect user information
- Added bubble notification: Warning when user runs profile crawler without Twitter account as results may be incomplete
- Added bubble notification: Warning when user runs profile crawler with enabled "Crawl re-tweets" setting without Twitter account as results may be incomplete

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #12  
Old 17.10.2023, 23:16
Hull2366 Hull2366 is offline
Baby Loader
 
Join Date: Sep 2023
Posts: 6
Default

Thank you for the support!
Reply With Quote
  #13  
Old 18.10.2023, 08:43
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,047
Default

Thanks for your feedback.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 19:01.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.