[accelheim]

The user has 1625 media files. JD2 only analyzed 1008 files.

18.12.22 22.38.12 <--> 19.12.22 17.20.38 jdlog://4091311370661/

[pspzockerscene]

Please post the link to the profile you want to crawl.
Does that profile contain adult content?
If so: You need to add a twitter account to JD to be able to crawl adult content!

Also:
Maybe only re-tweets were missing?
If you want retweets to be crawled too, enable this:
Settings -> Plugins -> twitter.com -> Crawl retweets

[accelheim]

Originally Posted by pspzockerscene

Please post the link to the profile you want to crawl.
Does that profile contain adult content?
If so: You need to add a twitter account to JD to be able to crawl adult content!

Also:
Maybe only re-tweets were missing?
If you want retweets to be crawled too, enable this:
Settings -> Plugins -> twitter.com -> Crawl retweets

Thanks. Here is the profile

**External links are only visible to Support Staff****External links are only visible to Support Staff**

[Jiaz]

@accelheim: have you added your account? have you checked if the missing tweets may be retweets?

[accelheim]

Originally Posted by Jiaz

@accelheim: have you added your account? have you checked if the missing tweets may be retweets?

Yes, the account is added and I only analyzed the "media" tweets.

[pspzockerscene]

Originally Posted by pspzockerscene

Maybe only re-tweets were missing?
If you want retweets to be crawled too, enable this:
Settings -> Plugins -> twitter.com -> Crawl retweets

In my tests I definitely got more items after enabling re-tweet crawling.

I won't test this all the way though so if you want us to analyze this please provide a debug log and let JD crawl the profile all the way so we can see in that log, how far our plugin is able to crawl.

[accelheim]

Originally Posted by pspzockerscene

In my tests I definitely got more items after enabling re-tweet crawling.

I won't test this all the way though so if you want us to analyze this please provide a debug log and let JD crawl the profile all the way so we can see in that log, how far our plugin is able to crawl.

Thanks for the answer. Here's a profile:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Only 10 images got analyzed, although there are 22 media files in his profile.

Here's the log:

31.05.23 14.17.14 <--> 02.06.23 14.31.35 jdlog://3275311370661/

[pspzockerscene]

That profile contains 18+ content.

Add a twitter account with 18+ permissions to JDownloader and try again.
I'm getting at least 26 images with an active twitter account.

[accelheim]

Originally Posted by pspzockerscene

That profile contains 18+ content.

Add a twitter account with 18+ permissions to JDownloader and try again.
I'm getting at least 26 images with an active twitter account.

Yes, I already have a 18+ Twitter account added in JD2. I tried with this user again and did get 26 images for some reason too. Then I tried with this user here:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

I only got 990 files, although this user has 2,197 media files. I don't know what the issue here might be.

[pspzockerscene]

Hm doesn't look like a JD bug to me.
The twitter.com website also doesn't let you scroll all the way to the end (see other posts in our forum).
That is also reflected in the log when adding your link:

219|twitter.com_jd.plugins.decrypter.TwitterComCrawler.log 06.06.23, 00:17:58 - INFO [ jd.plugins.decrypter.TwitterComCrawler(crawlUserViaAPI) ] -> Crawled page 42 | Tweets crawled so far: 817/2197 | lastCreatedAtDateStr = null | last nextCursor = CENSORED_rT5syQAAA==
219|twitter.com_jd.plugins.decrypter.TwitterComCrawler.log 06.06.23, 00:17:58 - INFO [ jd.plugins.decrypter.TwitterComCrawler(crawlUserViaAPI) ] -> Current page (42) didn't contain any results --> Probably it contained only explicit content and we're lacking permissions to view that!
219|twitter.com_jd.plugins.decrypter.TwitterComCrawler.log 06.06.23, 00:17:58 - INFO [ jd.plugins.decrypter.TwitterComCrawler(crawlUserViaAPI) ] -> Current page contained only 0 of max. 20 expected objects --> Reached the end?
219|twitter.com_jd.plugins.decrypter.TwitterComCrawler.log 06.06.23, 00:17:58 - INFO [ jd.plugins.decrypter.TwitterComCrawler(crawlUserViaAPI) ] -> Stopping because: We've already crawled current cursor: CENSORED_rT5syQAAA==
219|twitter.com_jd.plugins.decrypter.TwitterComCrawler.log 06.06.23, 00:17:58 - INFO [ jd.plugins.decrypter.TwitterComCrawler(crawlUserViaAPI) ] -> Done after 42 pages | last nextCursor = CENSORED_rT5syQAAA==

I've tested it with some other twitter profiles and the end was also after about 42 pages.

If you ask me, this is a twitter website limitation/bug.

[accelheim]

Originally Posted by pspzockerscene

Hm doesn't look like a JD bug to me.
The twitter.com website also doesn't let you scroll all the way to the end (see other posts in our forum).
That is also reflected in the log when adding your link:

219|twitter.com_jd.plugins.decrypter.TwitterComCrawler.log 06.06.23, 00:17:58 - INFO [ jd.plugins.decrypter.TwitterComCrawler(crawlUserViaAPI) ] -> Crawled page 42 | Tweets crawled so far: 817/2197 | lastCreatedAtDateStr = null | last nextCursor = CENSORED_rT5syQAAA==
219|twitter.com_jd.plugins.decrypter.TwitterComCrawler.log 06.06.23, 00:17:58 - INFO [ jd.plugins.decrypter.TwitterComCrawler(crawlUserViaAPI) ] -> Current page (42) didn't contain any results --> Probably it contained only explicit content and we're lacking permissions to view that!
219|twitter.com_jd.plugins.decrypter.TwitterComCrawler.log 06.06.23, 00:17:58 - INFO [ jd.plugins.decrypter.TwitterComCrawler(crawlUserViaAPI) ] -> Current page contained only 0 of max. 20 expected objects --> Reached the end?
219|twitter.com_jd.plugins.decrypter.TwitterComCrawler.log 06.06.23, 00:17:58 - INFO [ jd.plugins.decrypter.TwitterComCrawler(crawlUserViaAPI) ] -> Stopping because: We've already crawled current cursor: CENSORED_rT5syQAAA==
219|twitter.com_jd.plugins.decrypter.TwitterComCrawler.log 06.06.23, 00:17:58 - INFO [ jd.plugins.decrypter.TwitterComCrawler(crawlUserViaAPI) ] -> Done after 42 pages | last nextCursor = CENSORED_rT5syQAAA==

I've tested it with some other twitter profiles and the end was also after about 42 pages.

If you ask me, this is a twitter website limitation/bug.

If it's a limitation to scroll down, why does this happen with profiles with few media files while with other profiles with lots of images and far more scrolling
JD2 finds even more media? Like with this user:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

Twitter says 392 files, while JD2 finds over 2000.

[pspzockerscene]

I'm sorry but only twitter can know.

I will not test dozens of links here for you.
Please provide debug-logs instead.

Also you may want to try other tools, specifically designed to download from twitter.
They might use other APIs which can avoid those limitations or they will run into the same issue JD does.
You can find such projects for example on github.com.

Please post your log-ID here | bitte poste deine Log-ID hier.

-psp-