JDownloader Community - Appwork GmbH
 

Closed Thread
 
Thread Tools Display Modes
  #1  
Old 06.12.2021, 17:34
rypz79 rypz79 is offline
JD Addict
 
Join Date: Oct 2009
Location: Israel
Posts: 172
Exclamation [LinkCrawler Rule] bunker.is video gallery, offline file errors

Hi,

Cannot DL the entire video gallery at **External links are only visible to Support Staff****External links are only visible to Support Staff**
I get offline file(s) error(s)
Copying a single video link e.g the top left [right click on the thumbnail -> copy link] **External links are only visible to Support Staff****External links are only visible to Support Staff** or entering it [left click on the thumbnail] and copying the url from the browser **External links are only visible to Support Staff****External links are only visible to Support Staff** still gives me an offline file error.

Using latest JD2 on Win10x64

TNX
  #2  
Old 06.12.2021, 18:17
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,076
Default

Hi,
we don't have a plugin for this website.

Their URLs have a file-extension which looks like those are direct-URLs.
JD checks those then, sees that there is no file and displays them as offline.

You could easily "fix" this using a LinkCrawler rule of type "REWRITE" to e.g. change the "/v/" in video URLs to "/d/" to be able to download those.

Alternatively or additionally, you could create another rule to change the URLs from inside that album (starting with "cdn.bunkr.is/") to those with "/d/" right away.

Let us know if you need help with that.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #3  
Old 06.12.2021, 20:32
rypz79 rypz79 is offline
JD Addict
 
Join Date: Oct 2009
Location: Israel
Posts: 172
Thumbs up nice workaround

@psp I used a Firefox extension to extract all the vids links from the gallery
then as you suggested I replaced the 266 strings headers to **External links are only visible to Support Staff****External links are only visible to Support Staff**... then I had to change max simultaneous DLs to 2 because it gets stuck at offline if you try to DL more then 2 at the same time.

TNX
  #4  
Old 06.12.2021, 20:39
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,076
Default

Thanks for your feedback.
There are always multiple ways to achieve your goal

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #5  
Old 23.02.2022, 02:43
inso inso is offline
Baby Loader
 
Join Date: Nov 2021
Posts: 7
Default

Quote:
Originally Posted by rypz79 View Post
@psp I used a Firefox extension to extract all the vids links from the gallery
then as you suggested I replaced the 266 strings headers to **External links are only visible to Support Staff**...... then I had to change max simultaneous DLs to 2 because it gets stuck at offline if you try to DL more then 2 at the same time.

TNX
What Firefox-plugin are you referring to?
  #6  
Old 23.02.2022, 14:13
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,076
Default

That should work automatically after the release of our next CORE-update as we found out that we got a plugin for a very similar site "cyberdrop.me".

Regarding his addon:
I don't know which one he means but for Chrome I sometimes use "Link Gopher" which is also available for Firefox.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #7  
Old 13.03.2022, 13:55
inso inso is offline
Baby Loader
 
Join Date: Nov 2021
Posts: 7
Default

Where are we on this in terms of ETA? The problem at hand can be solved with a very simple link conversion from:

Code:
https //cdn.bunkr.is/video_<video_id>.mp4
to

Code:
https //stream.bunkr.is/v/video_<video_id>.mp4
This seems fairly simple, especially because as mentioned, there is already a plugin for cyberdrop which is basically identical (the sites even have a similar layout).

I'd really appreciate the effort to solve this simple but frustrating problem.
  #8  
Old 13.03.2022, 16:23
inso inso is offline
Baby Loader
 
Join Date: Nov 2021
Posts: 7
Default

^^https //stream.bunkr.is/v/video_<video_id>.mp4 ^^

EDIT: Sorry, correction: when you click the video in the bunkr.is folder it first goes to the stream.bunkr.is link above, then there you can click "Download media" which leads to the ultimate link that JDownloader needs;

Code:
https //media-files.bunkr.is/<filename>.mp4

Last edited by inso; 13.03.2022 at 16:25.
  #9  
Old 14.03.2022, 15:30
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,076
Default

Quote:
Originally Posted by inso View Post
Where are we on this in terms of ETA? The problem at hand can be solved with a very simple link conversion from:
?
The mentioned update has been released a while ago.

I'm not able to reproduce any issues with it.
If you're able to do so, please provide example URLs and a log.

Please post your log-ID here | bitte poste deine Log-ID hier.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #10  
Old 14.03.2022, 15:46
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,076
Default

Update:
Indeed our crawler output the wrong direct-URLs and also did not "fix" added single video URLs.
Next CORE-Update will fix this issue.

Bitte auf das nächste CORE-Update warten!

Please wait for the next CORE-Update!

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!


-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #11  
Old 16.03.2022, 17:43
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,076
Default

CORE-Updates have been released!
Please update your JDownloader and report any issues you find asap.

CORE-Updates wurden released!
Bitte JDownloader updaten und eventuelle Bugs schnellstmöglich an uns melden.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #12  
Old 17.03.2022, 00:10
academyoffetishes academyoffetishes is offline
Linkgrabbing Monster
 
Join Date: Feb 2018
Posts: 86
Default

Somehow they are set up to only let one download happen at a time, and jdownloader normally respects this. But if multiple "simultaneous download" threads try to start a different bunkr.is file at the same time, this triggers DDoS protection. I suspect this will happen over and over again unless you reduce your simultaneous downloads to 1, which is an inconvenient setting, because this host downloads very very slow.

To reproduce this, add 6+ bunkr.is files to your download tab and have ~6 "simultaneous download". Restart so that it tries to start downloads on all bunkr.is files simultaneously.

16.03.22 16.04.03 <--> 16.03.22 16.05.30 jdlog://5296925302851/

Last edited by academyoffetishes; 17.03.2022 at 00:22.
  #13  
Old 17.03.2022, 11:27
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,516
Default

@academyoffetishes: you can setup DomainRule for this generic http and limit it to 1 concurrent download
Settings->Advanced Settings->GeneralSettings.domainrules

Code:
[ {
"domainPattern" : "bunkr.is",
"enabled" : true,
"maxSimultanDownloads" : 1,
"allowToExceedTheGlobalLimit" : false
} ]
__________________
JD-Dev & Server-Admin
  #14  
Old 17.03.2022, 22:01
academyoffetishes academyoffetishes is offline
Linkgrabbing Monster
 
Join Date: Feb 2018
Posts: 86
Default

okay thanks a lot. I think this would be useful to every person who downloads multiple files from "bunkr.is". At what point does it cross the threshold to be worthy of its own plug-in?
  #15  
Old 17.03.2022, 22:07
academyoffetishes academyoffetishes is offline
Linkgrabbing Monster
 
Join Date: Feb 2018
Posts: 86
Default

From another perspective: if jdownloader users have a tendency to trigger bunkr.is' DDOS attacks, doesn't it make it more likely that bunkr.is (and perhaps cyberdrop.me, if they share the same codebase) will look into what's happening and try to find a way to ban jdownloader, causing more work for you in the long run?
  #16  
Old 17.03.2022, 22:11
academyoffetishes academyoffetishes is offline
Linkgrabbing Monster
 
Join Date: Feb 2018
Posts: 86
Default

Edit: nevermind, please ignore this msg.

Last edited by academyoffetishes; 17.03.2022 at 22:48.
  #17  
Old 18.03.2022, 10:52
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,516
Default

@academyoffetishes: I will think of a way to set those limits automatically
__________________
JD-Dev & Server-Admin
  #18  
Old 18.03.2022, 21:46
academyoffetishes academyoffetishes is offline
Linkgrabbing Monster
 
Join Date: Feb 2018
Posts: 86
Default

Thanks!
  #19  
Old 19.03.2022, 12:36
inso inso is offline
Baby Loader
 
Join Date: Nov 2021
Posts: 7
Default

Quote:
Originally Posted by Jiaz View Post
@academyoffetishes: you can setup DomainRule for this generic http and limit it to 1 concurrent download
Settings->Advanced Settings->GeneralSettings.domainrules

Code:
[ {
"domainPattern" : "bunkr.is",
"enabled" : true,
"maxSimultanDownloads" : 1,
"allowToExceedTheGlobalLimit" : false
} ]
How/where do I enter this? When I try and paste it in the "Value" field beneath the standard code ( "domainPattern": ".*jdownloader\\.org",) and hit enter, a window, "An error has occurred" pops up.
  #20  
Old 19.03.2022, 15:32
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,516
Default

@inso:
Quote:
Originally Posted by inso View Post
where do I enter this?
Settings->Advanced Settings->GeneralSettings.domainrules

Quote:
Originally Posted by inso View Post
How do I enter this?
It's a JSON Object array
Code:
[ {
"domainPattern" : "bunkr.is",
"enabled" : true,
"maxSimultanDownloads" : 1,
"allowToExceedTheGlobalLimit" : false
} ]
Quote:
Originally Posted by inso View Post
When I try and paste it in the "Value" field beneath the standard code ( "domainPattern": ".*jdownloader\\.org",) and hit enter, a window, "An error has occurred" pops up.
The value must be properly JSON syntax
__________________
JD-Dev & Server-Admin
  #21  
Old 19.03.2022, 16:46
inso inso is offline
Baby Loader
 
Join Date: Nov 2021
Posts: 7
Default

Yes, I posted the code you provided - without modifying anything in terms of syntax - in the "Value" field under :

- "Settings->Advanced Settings->GeneralSettings.domainrules", beneath the standard code that was already written in that field ( "domainPattern": ".*jdownloader\\.org",.......etc.)

But when I hit Enter as to try and save it, a pop-up box saying "An error has occurred" showed up..

You did not say where to paste it exactly; so the "Value" field was the only place that seemed obvious to me. Tell me if that was wrong, and where I should put it.

Please give me a simple but complete, user friendly instruction on how to actually make it work, as I'm not a programmer.

Last edited by inso; 19.03.2022 at 23:17.
  #22  
Old 21.03.2022, 02:05
academyoffetishes academyoffetishes is offline
Linkgrabbing Monster
 
Join Date: Feb 2018
Posts: 86
Default

Quote:
Originally Posted by inso View Post
Yes, I posted the code you provided - without modifying anything in terms of syntax - in the "Value" field under :

- "Settings->Advanced Settings->GeneralSettings.domainrules", beneath the standard code that was already written in that field ( "domainPattern": ".*jdownloader\\.org",.......etc.)
What you would put in that value field changes if you already have something there. In order for anyone to tell you what to put there for your setting, you'll have to tell us what is already in it.
  #23  
Old 21.03.2022, 02:09
academyoffetishes academyoffetishes is offline
Linkgrabbing Monster
 
Join Date: Feb 2018
Posts: 86
Default

I don't know if this is a bug or not, but sometimes when I start a bunkr.is download, the status changes from downloading, to nothing, to starting, to downloading again. You can see this within the last 10 minutes of my log. Is this the website's DDOS protection, or something jdownloader can prevent? I've often seen this happen multiple times within a short period (a minute) before leading to "service temporarily unavailable."

20.03.22 04.04.23 <--> 20.03.22 18.06.51 jdlog://9317925302851/

Is there any way for me as a user to see jdownloader logs about what's happening when this occurs?
  #24  
Old 21.03.2022, 13:56
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,076
Default

Quote:
Originally Posted by academyoffetishes View Post
At what point does it cross the threshold to be worthy of its own plug-in?
We do not have a defined point for that.
In general we prefer to use our generic plugins so in this case we do have a crawler plugin for galleries but no hoster plugin for downloading.
If a lot of users are using a particular website and that website has particular limits or similar in place these factors can make it worth to add a host plugin.

Quote:
Originally Posted by inso View Post
Yes, I posted the code you provided - without modifying anything in terms of syntax - in the "Value" field under :

- "Settings->Advanced Settings->GeneralSettings.domainrules", beneath the standard code that was already written in that field ( "domainPattern": ".*jdownloader\\.org",.......etc.)

But when I hit Enter as to try and save it, a pop-up box saying "An error has occurred" showed up..
Works fine here.
You can use webtools like the following tool to easily test/verify json structures:
jsoneditoronline.org
(Paste the json into the left side, then click on the "Copy >" Button, then in the right column, left side click on the button "tree".)

Quote:
Originally Posted by inso View Post
You did not say where to paste it exactly; so the "Value" field was the only place that seemed obvious to me. Tell me if that was wrong, and where I should put it.
The "Value" field in the column "Value" is correct.

Quote:
Originally Posted by inso View Post
Please give me a simple but complete, user friendly instruction on how to actually make it work, as I'm not a programmer.
The given instructions should be fine.
You've either copied something else along the json structure provided ba Jiaz and/or another application running on your system is messing around with your clipboard content.

Quote:
Originally Posted by academyoffetishes View Post
What you would put in that value field changes if you already have something there. In order for anyone to tell you what to put there for your setting, you'll have to tell us what is already in it.
I disagree but I'll still answer that for you:

1. The default value is:
Code:
[
 {
  "enabled"                     : false,
  "pluginPattern"               : null,
  "domainPattern"               : ".*jdownloader\\.org",
  "allowToExceedTheGlobalLimit" : false,
  "filenamePattern"             : "\\.png$",
  "accountPattern"              : "myUsername",
  "maxSimultanDownloads"        : 20
 }
]
Once changed, the arrow-symbol in the right column "Actions" turns yellow which indicates that the setting has been modified.
By clicking it you can reset a setting to its default.
This is consistent through most of JDownloaders GUI too (e.g. also inside our plugin settings).

2. The default value is only an example so it's fine to select it (CTRL + A) and overwrite it with the json provided by Jiaz.

Quote:
Originally Posted by academyoffetishes View Post
I don't know if this is a bug or not, but sometimes when I start a bunkr.is download, the status changes from downloading, to nothing, to starting, to downloading again. You can see this within the last 10 minutes of my log. Is this the website's DDOS protection, or something jdownloader can prevent? I've often seen this happen multiple times within a short period (a minute) before leading to "service temporarily unavailable."
20.03.22 04.04.23 <--> 20.03.22 18.06.51 jdlog://...
According to your log the website is answering with "HTTP/1.1 403 Forbidden" instead of a file.
Set max. simultaneous downloads and max. chunks to 1 and try again.
If that doesn't help, manually check the failing files in your browser and/or wait and try again later.

Quote:
Originally Posted by academyoffetishes View Post
Is there any way for me as a user to see jdownloader logs about what's happening when this occurs?
Sure.
As described here, you can find your logs in the "logs" folder which is located in your JD install dir.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 22.03.2022 at 11:36. Reason: Fixed quotes
  #25  
Old 21.03.2022, 19:09
academyoffetishes academyoffetishes is offline
Linkgrabbing Monster
 
Join Date: Feb 2018
Posts: 86
Default

Thanks!

Just want to state that most of the things you were replying to weren't said by me, but the person I was replying to.
  #26  
Old 22.03.2022, 11:36
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,076
Default

You're right.
I've fixed that.
I hope my answer was helpful nevertheless

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #27  
Old 23.03.2022, 01:36
academyoffetishes academyoffetishes is offline
Linkgrabbing Monster
 
Join Date: Feb 2018
Posts: 86
Default

Quote:
Originally Posted by pspzockerscene View Post
Quote:
Originally Posted by academyoffetishes View Post
What you would put in that value field changes if you already have something there. In order for anyone to tell you what to put there for your setting, you'll have to tell us what is already in it.
I disagree but I'll still answer that for you:
Thanks for the help. I don't think this is too important, but I think you are saying it doesn't matter if they replace the whole thing because the default value isn't enabled anyway. But I was assuming the value was enabled. If that were the case, I don't think they could just paste over, or they'd change some functionality.
  #28  
Old 23.03.2022, 12:07
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,076
Default

Umm not sure what to answer:
Yes, the default value doesn't do anything - you can safely overwrite it.
Whenever you're playing around with any settings, I suggest to create a backup of your config in beforehand.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #29  
Old 12.10.2022, 23:17
phpremium9000 phpremium9000 is offline
Modem User
 
Join Date: Aug 2022
Posts: 1
Default

I've been having an issue when trying to download .mkv video files from bunkr.

downloading mp4 files works fine, but when it's an .mkv video file, jdownloader says the file is missing or unavailable even though it's actually there and playable in a web browser.

is there a workaround or fix for this issue?
  #30  
Old 13.10.2022, 12:38
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,076
Default

Hi phpremium9000,
I believe that we've already fixed it but haven't released the fix yet.
Please provide example URLs so I can check this.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
  #31  
Old 10.11.2022, 02:41
academyoffetishes academyoffetishes is offline
Linkgrabbing Monster
 
Join Date: Feb 2018
Posts: 86
Default

I'm having some difficulty figuring out if the 4 bunkr threads are about the same issue, or different ones:

this one
https://board.jdownloader.org/showth...ighlight=bunkr
https://board.jdownloader.org/showth...ighlight=bunkr
https://board.jdownloader.org/showth...ighlight=bunkr

I have more logs that reproduce the issue. Is this helpful, or do you already have enough information?

09.11.22 17.01.02 <--> 09.11.22 17.35.38 jdlog://9001311370661/

Here is the URL I downloaded: **External links are only visible to Support Staff****External links are only visible to Support Staff** 5/6 videos said "file not found." In my experience, if I go directly to the video and download that url instead, it works correctly. I tried that with this URL: **External links are only visible to Support Staff****External links are only visible to Support Staff** That's the last video in the gallery.
Old 10.11.2022, 07:30
rachelgomez123
Message deleted by pspzockerscene. Reason: Bot spam
  #32  
Old 10.11.2022, 14:43
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 71,076
Default

@academyoffetishes
Indeed the status of this was somehow chaotically spread over multiple threads.
I've merged thread 90961 into 91374 and closed thread 91629.

I've posted a (hopefully) clear status update here.

This thread is closed now as it is outdated as Link Crawler rules are not needed anymore more bunkr.is/cyberdrop.me websites because we do have plugins now although (as explained) they're still partially broken in the current stable version of JDownloader.
Please use this thread if you want to leave a comment about this topic.

*Thread closed*
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 10.11.2022 at 14:45. Reason: Added one more hyperlink to new/open bunkr.is/cyberdrop.me related thread.
Closed Thread

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 07:54.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.