JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 16.09.2024, 17:38
svArtist's Avatar
svArtist svArtist is offline
DSL User
 
Join Date: Mar 2015
Posts: 39
Lightbulb Custom User Agent map?

I'm sad to see that you won't implement changing the default User Agent used by JD.
(FWIW, It seems to me it would introduce very little trouble that couldn't be left up to the user and just be included in the log files; it seems weird to simply not do anything for such an important feature request because of some potential issues. You could say that about any feature 😅)

My website (that I started using for supplementary DLs) host recently started blocking requests from JD with a 503 status, whereas the same request sent with a different UA goes through.

JS test, showcasing the User Agent being the only variable leading to a different outcome:
Code:
var agents = {
	jd: {
		label: "JDownloader",
		str: "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:76.0) Gecko/20100101 Firefox/76.0"
	},
	ch: {
		label: "Chrome",
		str: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
	}
};
var url = "**External links are only visible to Support Staff**;
var ua = agents.jd;
// ua = agents.ch;
var headers = new Headers({
	"User-Agent": ua.str
});
console.log("Using", ua.label + ":", ua.str);
fetch(new Request(url, {
	headers
})).then(async (res)=>{
	console.log(res.status);
});
Spoiler:


I'm talking to them trying to make them undo what ever they did, but I'm not too hopeful that they will.

In the mean time, we need something to specify how requests are made in some way, outside the few plugins that support a custom UA.
I'd like to suggest a custom map of host URL patterns to UA strings, something like:
Code:
{
	"example.com": "My neat UA",
	"my-RegExp-url[_-]pattern\\.\\w{2,5}(\\/path\\/(.+))?": "Other cool UA",
	...
}
Won't you please look into this?

Thank you!

Last edited by pspzockerscene; 16.09.2024 at 17:44. Reason: Added SPOILER tag for image
Reply With Quote
  #2  
Old 16.09.2024, 17:49
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 72,961
Default

Quote:
Originally Posted by svArtist View Post
it seems weird to simply not do anything for such an important feature request...
How do you come to the conclusion that this is an "important feature"?
I can't find more than a hand full of threads of users who have requested this feature within the last 10 years.

Also JDownloader is open source so anyone could just implement said feature and ask us to merge the changes.
JDownloader source code:
https://support.jdownloader.org/de/k...up-ide-eclipse
Even if you are not a programmer, you could just grab the code and change the default value for your local copy.

Quote:
Originally Posted by svArtist View Post
I'd like to suggest a custom map of host URL patterns to UA strings, something like:
I'm against this (your map idea) but adding support for custom headers in LinkCrawler rules is already on our TODO list, see:
https://board.jdownloader.org/showth...382#post476382
As the user-agent value is also just a header, this would automatically allow for custom user-agent values.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?

Last edited by pspzockerscene; 16.09.2024 at 17:51. Reason: Fixed typo
Reply With Quote
  #3  
Old 16.09.2024, 18:20
svArtist's Avatar
svArtist svArtist is offline
DSL User
 
Join Date: Mar 2015
Posts: 39
Default

Quote:
Originally Posted by pspzockerscene View Post
How do you come to the conclusion that this is an "important feature"?
I meant important as in "fundamental", I suppose. I was thinking rather than handling any UA related issues individually, there could simply be one general point of configuration everything is piped through

Quote:
Originally Posted by pspzockerscene View Post
[...]adding support for custom headers in LinkCrawler rules is already on our TODO list
Oh, neato! 👌
I'm sorry, I wasn't aware.

Thank you for the quick reply! 🙌
Reply With Quote
  #4  
Old 16.09.2024, 18:28
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 72,961
Default

Quote:
Originally Posted by svArtist View Post
I was thinking rather than handling any UA related issues individually, there could simply be one general point of configuration everything is piped through
I do agree that the current way of having individual user-agent settings for some plugins is not the best design choice especially if, over time, more are needed.
This should be something that can be done globally and/or via the advanced settings "GeneralSettings.domainrules".

Quote:
Originally Posted by svArtist View Post
I'm sorry, I wasn't aware.
No worries.

I just started to work on it.
This is not any kind of promise and I will not provide any ETA!
You can see the development progress by monitoring our SVN changes and/or by watching the tickets' progress value:
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #5  
Old 16.09.2024, 18:35
svArtist's Avatar
svArtist svArtist is offline
DSL User
 
Join Date: Mar 2015
Posts: 39
Default

Quote:
Originally Posted by pspzockerscene View Post
[...] You can see the development progress by monitoring our SVN changes [...]
Thank you 👍
Though I don't think I (and other non-admins?) have access to the tracker. I get a login prompt, but my username and PW don't work there
Reply With Quote
  #6  
Old 16.09.2024, 18:40
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 72,961
Default

Quote:
Originally Posted by svArtist View Post
Thank you 👍
Though I don't think I (and other non-admins?) have access to the tracker.
It is private but you should see this yellow ticket embedded in my last forum post.
This is all you need to see.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #7  
Old 16.09.2024, 18:43
svArtist's Avatar
svArtist svArtist is offline
DSL User
 
Join Date: Mar 2015
Posts: 39
Default

Quote:
Originally Posted by pspzockerscene View Post
It is private but you should see this yellow ticket embedded in my last forum post.
This is all you need to see.
Ah, okie dokie
Reply With Quote
  #8  
Old 24.09.2024, 19:06
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 72,961
Default

The ability to add custom headers via LinkCrawler rules has been added, see changelog:
https://board.jdownloader.org/showth...766#post539766
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 21:35.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.