[Solved] Bunkr.is/Bunkr.ru/Bunkr.su crawler issue

[mikk88]

Those sound fine to me about the wording. I guess if the actual download names one gets in the browser are the default settings then no one will wonder. But who uses this plugin more often sure will be happy to get those fixes as option. I know I do!

What would the Auto setting do then?

[pspzockerscene]

Originally Posted by mikk88

What would the Auto setting do then?

Placeholder for the future so we developers can change defaults without getting blamed by users

[mikk88]

got it!

[mjolnir870]

For filenames I would always prefer the "original", which on bunkr seems to be the ones displayed on the album page.

Broken link: this link has a file with an '&' character, which breaks JD's download.

album: **External links are only visible to Support Staff****External links are only visible to Support Staff**
JD's link for the '&' file: **External links are only visible to Support Staff****External links are only visible to Support Staff**
actual link from album: **External links are only visible to Support Staff****External links are only visible to Support Staff**

[pspzockerscene]

Next CORE-update will include a plugin setting called "Fix filename" which is enabled by default.

@mjolnir870
Works fine here.
Your JD might be outdated.
EDIT
Fixed!

Bitte auf das nächste CORE-Update warten!

Please wait for the next CORE-Update!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[mikk88]

Next CORE-update will include a plugin setting called "Fix filename" which is enabled by default.

Sounds great .. I will definitely try it.
One thing I noticed with the unavailable files: the file gets skipped and set on a timer to wait. So this works fine for me. The problem is the check seems to take a while too and the delayed files still stays at top of the download queue(?)

So what happens in my queue is that downloads sometimes stop for quite some time because unavailable files cluster up at the top of the queue and get retried again and again instead of finishing the other working downloads instead.

Is there a setting how we can influence this behavior? What would work good is probably if the unavailable files get a priority downgrade so that they get retried only after the working files are finished.

[pspzockerscene]

Originally Posted by mikk88

So what happens in my queue is that downloads sometimes stop for quite some time because unavailable files cluster up at the top of the queue and get retried again and again instead of finishing the other working downloads instead.

Is there a setting how we can influence this behavior? What would work good is probably if the unavailable files get a priority downgrade so that they get retried only after the working files are finished.

That is basically a design flaw/limitation of JDownloader which existed since the beginning but in most situations, users won't notice it.
The ideal- or at least a better way to cope with this would be to "shadow skip" files which are running into such timers and try the rest of the downloadlist until the end is reached once, then rollover.
...but I don't see us implementing this atm soo: The only other way to improve this is to set higher wait times.
In order to do this I need to know which error in particular you mean.

[mikk88]

It was the "temporary not available file" problem. Maybe its a possible timeout setting in the plugin even? Maybe sometimes the file comes online again only hours later so I probably would set this really high if I could manually change it.

The "shadow skip" sure sounds best yeah, a priority downgrade would probably get close to that too. If my whole list has "default priority" and all skipped files get set to "low priority" it would automatically mean they get tried only again after the rest of the download list was handled once.

[mjolnir870]

I think the timeout for the "temporarily not available" for bunkr need to be set much higher. It's at 2 minutes right now, but it usually takes bunkr days to perform server maintenance. Setting the default for the timeout to something like 6 hours or something would probably solve the problem of lots of unavailable files taking up the queue.

For now I've just disabled those downloads entirely to prevent hammering bunkr with requests every 2 minutes.

[pspzockerscene]

@mikk88
If the (new) default settings won't satisfy our users, I'am willing to add extra settings but for now I don't see the need to add more settings.

There are multiple reasons for "temporarily unavailable" for bunkr downloads:
I've made the following adjustments:

• Server error 403 from 10 minutes to 30 minutes
• Error 503 too many connections from 30 seconds to 60 seconds
• File broken or temporarily unavailable from 30 minutes to 2 hours -> THIS IS A DIFFERENT ERROR THAN "Media temporarily not available due to ongoing server maintenance."!
• Media temporarily not available due to ongoing server maintenance. 2 minutes to 2 hours

Bitte auf das nächste CORE-Update warten!

Please wait for the next CORE-Update!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[mikk88]

I'm sure these settings will be a big improvement. Thanks and looking forward to test it!

[pspzockerscene]

By the way you can always test changes earlier in our dev version.
Instructions:
https://support.jdownloader.org/Know...up-ide-eclipse

Either way I will inform you in this thread once our next CORE-update gets released.

[mikk88]

Originally Posted by pspzockerscene

By the way you can always test changes earlier in our dev version.
Instructions:

the last time I touched Java+Eclipse is like 15 years ago in university .. I'm not too sure about how successful I would be with trying it today once more

Either way I will inform you in this thread once our next CORE-update gets released.

appreaciate it! thanks)

[pspzockerscene]

No worries those instructions are made in a way that even non-programmers should get it to work.

[pspzockerscene]

CORE-Updates have been released!
All announced bugfixes and features are live!
Please update your JDownloader and report any issues you find asap.
If this thread gets marked as "[Solved]" by our forum staff you can still post in it and we will read- and reply to it!

CORE-Updates wurden released!
Alle angekündigten Bugfixes/Features sind nun verfügbar!
Bitte JDownloader updaten und eventuelle Bugs schnellstmöglich an uns melden.
Falls dieser Thread vom Team als "[Erledigt]" markiert wird, kannst du weiterhin darin antworten und wir lesen/beantworten auch solche Threads!

-psp-

[mikk88]

thanks for the update!

on first glance it seems to work fine as expected with the new longer timeouts. the filenames also are shortened as planned

[pspzockerscene]

Thanks for your feedback!

[mjolnir870]

The new filenames for bunkr are not working properly. The original filenames are correctly added with linkgrabber, but when a download starts all the hyphens in the original filename are replaced with spaces.

Log here: 25.08.23 15.43.23 <--> 25.08.23 15.47.42 jdlog://6137311370661/
Link here: **External links are only visible to Support Staff****External links are only visible to Support Staff**

[mjolnir870]

another issue: cannot download the 7z.00* files from the following link:
**External links are only visible to Support Staff****External links are only visible to Support Staff**

I think it must be the same issue that happened with rar files.

[pspzockerscene]

Changes for the next update:

• Improved detection of "Video not available" dummy-videos
• Fixed filenames being fixed in a wrong way on 2nd linkcheck (such items need to be deleted and re-added if filenames are already displayed wrong in JD
• Fixed download of some files not working

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[mjolnir870]

Another "maintenance" video downloaded instead of real file. This one has a message about moving servers. I can't add a log right now as I have an uninterruptible download going right now, but here is a link that currently returns a maintenance video:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

I will try to upload a log tomorrow if I can find a link that returns this maintenance video.

[pspzockerscene]

Added.

[mjolnir870]

Another placeholder file. This is an image instead of a video. I've only found it for image links so far. It's a redirect to a placeholder on pixl.li.

**External links are only visible to Support Staff****External links are only visible to Support Staff**

[pspzockerscene]

Do you have the "original" link to this fine?
E.g. bunkrr.su/d/filename.ext ?
...or an album which contains this file?

Without this it will be impossible to add any kind of detection since this link is simply redirecting to a static image hosted on another imagehoster called "pixl.li".

[mjolnir870]

Originally Posted by pspzockerscene

Do you have the "original" link to this fine?
E.g. bunkrr.su/d/filename.ext ?
...or an album which contains this file?

Without this it will be impossible to add any kind of detection since this link is simply redirecting to a static image hosted on another imagehoster called "pixl.li".

This is the album. The image is no longer in the album, I guess it was removed between when I loaded the album in JD and when JD actually tried to download. But based on the other images in the same album, the link I gave above is the same as any other.

**External links are only visible to Support Staff****External links are only visible to Support Staff**

[pspzockerscene]

Yes I know but still I need the album-context to properly implement that.
If it happens again please provide a link to that album and a log.

I think I will also implement something like "if the files size is smaller than half of the expected size, display an error".

[pspzockerscene]

Added generic "protection against downloading dummy files":
If the file-size of the final file is smaller than "expected_filesize * 0.75", error "File is too small: File under maintenance?" will be displayed.

While this may still fail for smaller files, it should work fine to prevent the download of "Video under maintenance" files or similar.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[mjolnir870]

**External links are only visible to Support Staff****External links are only visible to Support Staff**

bunkr seems to have added some new link types or hosts. All the images in this album are not detected by JD. There should be 735 files total with a mix of images and videos, but there are only 111 videos in LinkGrabber.

[pspzockerscene]

Added support for new single file pattern:

http.../i/<filename>

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[mjolnir870]

new files don't seem to download successfully

log: 14.09.23 02.10.08 <--> 14.09.23 02.14.08 jdlog://8177311370661/
album link: **External links are only visible to Support Staff****External links are only visible to Support Staff**

[pspzockerscene]

Fixed.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[mjolnir870]

downloads are working but the filenames are being prefixed with a _

[pspzockerscene]

Hit the update button again in ~10 minutes.
If it's not fixed then, please provide fresh example URLs.

[mjolnir870]

works fine now ��

[pspzockerscene]

Thanks for your feedback.

[strangerload]

Hi all.. I've the app up to latest update, but trying bunkr links I obtain a "416 error".
What does it mean? Thanks, StrangerLoad.

[pspzockerscene]

Does it work fine via browser?

Please post your log-ID here
If your bugreport is about a specific website which JD supports via plugin, please also provide example URLs which can be used to reproduce the issue you are having.

Bitte poste deine Log-ID hier.
Falls dein gemeldetes Problem ein Problem mit einer Webseite ist, die per Plugin unterstützt wird, stelle bitte zusätzlich Testlinks zur Verfügung, mit denen sich dein Problem nachstellen lässt.

-psp-

[mjolnir870]

03.10.23 14.04.22 <--> 03.10.23 14.08.45 jdlog://1418311370661/

zip and rar files no longer download again.

album with 1 rar: **External links are only visible to Support Staff****External links are only visible to Support Staff**
album with 2 zip: **External links are only visible to Support Staff****External links are only visible to Support Staff**

[mjolnir870]

04.10.23 13.00.31 <--> 04.10.23 13.03.41 jdlog://8518311370661/

gif files without an extension also don't download (maybe gifs don't work at all but I don't have an album with gifs).

album: **External links are only visible to Support Staff****External links are only visible to Support Staff**

[pspzockerscene]

Fixed.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-