JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 22.08.2021, 16:56
Fakline Fakline is offline
Baby Loader
 
Join Date: Apr 2021
Posts: 5
Default Danbooru.donmai.us - Fetching posts interrupted mid-way

Hello.

The danbooru plugin works rather well in most circumstances, but sometimes it simply stops fetching the downloadable content. For example, JDownloader stops at the 57th result for this search:
**External links are only visible to Support Staff****External links are only visible to Support Staff**
... while it should find over 2100.

I suspect some specific posts to provoke an interruption in the plugin, due to unaivalable danbooru posts or something along those lines.

System info:
JRE 1.8.0 32bit
JDownload build #44876
Windows 10 Home 64 bits
Avast antivirus (deactivated)

Log:
Spoiler:
22.08.21 15.46.38 <--> 22.08.21 16.44.54 jdlog://4776825302851/


Sorry if some info is missing or something, I'm not quite experienced in using those forums. For instance, I don't know where to see the plugin's version nor how to setup the log level. Thanks in advance !
Reply With Quote
  #2  
Old 23.08.2021, 21:41
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 59,285
Default

Works fine here.
Is it possible that a single page inbetween the search results can contain less than 15 or 20 items?

Can you reproduce the issue also when using this host without an account?

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #3  
Old 23.08.2021, 23:05
Fakline Fakline is offline
Baby Loader
 
Join Date: Apr 2021
Posts: 5
Default

Without using my ID, I get 1223 results, which is still not the over 2100 that should be returned, according to the searched tag.
With my Gold account API key properly setup in the plugin, I get 61 results right now, but then it still stops after the same ID.
Given how Danbooru works, yes it is possible that a single page can return less than the theoric amount of results displayed per page. Typically when some posts are hidden (usually banned artists). For instance, in the given example, witout any account, browsing danbooru displays less than 20 results in the first 4 pages, and only tells that some are not displayed at the artist request in the 4th page of results.
I don't even get all of the results even with my Gold account anyway.

While I'm at it, I've just noticed that JDownloader only gets the post IDs in the filenames when the plugin doesn't use any API key.
Reply With Quote
  #4  
Old 24.08.2021, 13:19
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 59,285
Default

Quote:
Originally Posted by Fakline View Post
With my Gold account API key properly setup in the plugin, I get 61 results right now, but then it still stops after the same ID.
This issue will be fixed after the next update.
Seems like there is a bug in their API so sometimes the "id" field is missing.
This lead to an NPE in our plugin.

Quote:
Originally Posted by Fakline View Post
Given how Danbooru works, yes it is possible that a single page can return less than the theoric amount of results displayed per page. Typically when some posts are hidden (usually banned artists). For instance, in the given example, witout any account, browsing danbooru displays less than 20 results in the first 4 pages, and only tells that some are not displayed at the artist request in the 4th page of results.
I've updated the website handling but I'm still unable to reproduce the issues you had.
Number of items via website: 2165
Number of items via API: 2198

Quote:
Originally Posted by Fakline View Post
While I'm at it, I've just noticed that JDownloader only gets the post IDs in the filenames when the plugin doesn't use any API key.
This is not a bug.
Via website the parser works a bit different and sets temporary filenames.
Once you either let JD manually check them or start downloading, they will change to the ones you see right away when crawling via API

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #5  
Old 24.08.2021, 23:32
Fakline Fakline is offline
Baby Loader
 
Join Date: Apr 2021
Posts: 5
Default

It seems to be able to fetch all the results now, thanks a bunch!
I'll donate a little something later, because the work done here is amazing.
Reply With Quote
  #6  
Old 25.08.2021, 10:56
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 72,672
Default

@Fakline: Thanks for the feedback. Please know that we will also help without a donation but of course we say thank you Always remember only to donate if you are happy with JDownloader
__________________
JD-Dev & Server-Admin
Reply With Quote
  #7  
Old 25.08.2021, 13:42
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 59,285
Default

@Fakline
Is there an official support E-Mail for danbooru?
I'd like to report the bug I've found in their API

Thanks for your feedback!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 00:24.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.