JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 21.05.2011, 18:23
JoeH
Guest
 
Posts: n/a
Cool Guide: How to modify JDownloader's user-agent string

I recently found myself needing to change JDownloader’s user-agent string, as I am oftentimes behind a ZScaler filter, which doesn’t work well with JDownloader using the default user-agent string (if you are interested in knowing why, cf. this thread).

If you search around on the forum here, you will see that every once in a while this feature request has come up for one reason or another. Regardless of what the reason is, if you need to modify JDownloader’s user-agent, this guide is for you. If JDownloader is working just fine, or you don’t know what a user-agent string is, feel free to move on – it’s something you don’t need to worry about.

What we are going to do is install a local proxy and send JDownloader’s requests through the local proxy to the Internet. The local proxy will modify the user-agent string before sending out the requests. Thanks to Jiaz for the idea.

I am using Squid proxy in this tutorial, and running it on Windows. This tutorial is for Windows, but if you need to run it on Linux or some other operating system, just head over to **External links are only visible to Support Staff****External links are only visible to Support Staff** (that's h-t-t-p://wiki.squid-cache.org/SquidFaq/BinaryPackages for you non-supporters out there!) and download the version you need. The instructions below should be easy to adapt to other operating systems.

1. For Windows, download Squid from **External links are only visible to Support Staff****External links are only visible to Support Staff** (for non-supporters: h-t-t-p://squid.acmeconsulting.it/download/squid-2.7.STABLE8-bin.zip). This is the latest stable Windows release.
2. Unzip to c:\squid. It is highly recommended you use this directory. If you are going to use a different directory, you will need to modify all the configuration files to point to that directory, as well as the instructions below to adjust to that fact.
3. In the c:\squid\etc directory copy and rename the following files:
Code:
squid.conf.default ==> squid.conf
mime.conf.default ==> mime.conf
cachemgr.conf.default ==> cachemgr.conf
4. Open up an MS-DOS command prompt (as administrator) to c:\squid\sbin and run the following commands:
Code:
squid -i –f "c:/squid/etc/squid.conf" –n “Squid”
squid -z -f "c:/squid/etc/squid.conf"
squid -O servicecommandline –n “Squid”
5. Open c:\squid\etc\squid.conf in a text editor. Delete everything in the file (make sure you have the backup copy in case you need to reference it) and past in the following:

Code:
#These first lines identify "access control lists". 
#We are going to identify some IPs (all and localhost) we will later allow or block
acl all src all
acl localhost src 127.0.0.1/255.255.255.255

#Now we will identify ports we want to later allow or block
acl SSL_ports port 443
acl Safe_ports port 80		# http
acl Safe_ports port 21		# ftp
acl Safe_ports port 443		# https
acl Safe_ports port 70		# gopher
acl Safe_ports port 210		# wais
acl Safe_ports port 1025-65535	# unregistered ports
acl Safe_ports port 280		# http-mgmt
acl Safe_ports port 488		# gss-http
acl Safe_ports port 591		# filemaker
acl Safe_ports port 777		# multiling http
acl CONNECT method CONNECT

#We block unsafe ports
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

#We allow access to the localhost, but block everyone else
http_access allow localhost
http_access deny all

#In this case we don't need icp access (access to the Squid cache), so we'll block that for everyone
icp_access deny all

#Allow htcp access to localhost, block everyone else
htcp_access allow localhost
htcp_access deny all

#Indicate the port you want the proxy (127.0.0.1) to listen on. By default Sqid uses 3128
#In JDownloader you will then set your proxy to 127.0.0.1 and your port to whatever port you indicate here
http_port 3128

#These next lines are necessary if you are going to use a proxy. 
#The first line indicates the proxy, the second tells Squid to forward everything it receives to the proxy
#Replace “YOUR_PROXY_IP_ADDRESS” with your proxy number.
#Replace “YOUR_PROXY_PORT_NUMBER” with your proxy port number
#If you aren't going to use a proxy you must delete or put a # before these lines
cache_peer YOUR_PROXY_IP_ADDRESS parent YOUR_PROXY_PORT_NUMBER 0 no-query no-digest
never_direct allow all

#Some default Squid configuration lines. Not sure what they all do, but they are recommended and don't seem to hurt
hierarchy_stoplist cgi-bin ?
cache_dir ufs c:/squid/var/cache 100 16 256
access_log c:/squid/var/logs/access.log squid
refresh_pattern ^ftp:		1440	20%	10080
refresh_pattern ^gopher:	1440	0%	1440
refresh_pattern -i (/cgi-bin/|\?) 0	0%	0
refresh_pattern .		0	20%	4320
acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9]
upgrade_http0.9 deny shoutcast
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache

#Here's where the magic happens. 
#Squid denies all user-agents and then replaces them with another user agent, in this case "JDownloader"
#If you want to use a different user-agent, just replace "JDownloader" with whatever agent you want to use
header_access User-Agent deny all
header_replace User-Agent JDownloader
6. In your new squid.conf file change the proxy number and port number as needed. If you don’t use a proxy, you must delete the two lines in the proxy section, or put a # infront of them so that Squid ignores them. You may also want to edit the default port that Squid listens on. The default is 3128, but you may want to use 8080 or some other port.
7. Notice the last two lines of the squid.conf file. Those are the lines that modify the user-agent string. They are currently set to use the user-agent string "JDownloader". If you want to use a different user-agent, change the text “JDownloader” to whatever user-agent you want to use. Save and close the file.
8. Open up “Services” and start the Squid service, or restart it if it is already running.
9. In the JDownloader options, configure JDownloader to use the proxy 127.0.0.1 and the port number 3128 (or whatever port number you chose). This option can be found in "Settings" - "Download and Connections" - "Internet and Network".

That’s it! Now Squid will intercept all the traffic that comes from JDownloader and modify the user-agent to whatever you tell it to, then send the traffic out to the Internet.

So far I have tested it with the update function, DLC files, FileServe and RapidShare and everything has worked perfectly. In theory it could cause problems with some plugins, however, so I would only recommend using it if you need to.

Enjoy!

Last edited by JoeH; 21.05.2011 at 20:57. Reason: Modify external links so they will be visible to all
Reply With Quote
  #2  
Old 21.05.2011, 18:32
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Didn't try it but thanks, your thread is now a sticky;)

GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #3  
Old 22.05.2011, 12:28
JoeH
Guest
 
Posts: n/a
Default

Great, thanks.
Reply With Quote
Old 12.09.2011, 14:18
remi
Message deleted by Think3r. Reason: useless
  #4  
Old 16.09.2011, 15:47
JoeH
Guest
 
Posts: n/a
Default

Uhmm... publicity for what? For Squid? If anyone know an easier solution, please post it! Squid is complicated. For ZScaler? I would recommend ANY other Internet filter over ZScaler, since ZScaler breaks most programs (like JDownloader) that need to access the Internet but aren't browsers. If you want to understand why, just read the other thread which I linked to in my first message!
Reply With Quote
  #5  
Old 16.09.2011, 16:14
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

re: publicity
I believe there used to be another message, which has been deleted. Virus/spam related.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #6  
Old 03.04.2016, 07:19
pumplex
Guest
 
Posts: n/a
Thumbs up

Sorry for reviving this old thread but I thought I would add some info that might help others, Thanks to the OP for the useful info

I had to add the following to the end of squid.conf for it to work with some hosts who use php proxys and .htaccess tricks to filter out downloaders etc..

This helps to prevent the host knowing that you are using a local proxy:

Code:
header_access VIA deny all
header_access DNT deny all
header_access X-FORWARDED-FOR deny all

And for some hosts you may have to set the referrer to the domain name of the host.

Code:
header_access REFERER deny all
header_replace REFERER **External links are only visible to Support Staff**

Last edited by pumplex; 03.04.2016 at 10:00.
Reply With Quote
  #7  
Old 26.07.2016, 12:05
DAC324 DAC324 is offline
Wind Gust
 
Join Date: Nov 2015
Posts: 44
Default Why not...

...implement a way to change the user agent in jDownloader itself?

Searching on this topic, I found a RandomUserAgent.java
here:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

How can I incorporate that in jDownloader?

Thank you very much and kind regards,

DAC324
Reply With Quote
  #8  
Old 26.07.2016, 12:19
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

@DAC324
I wouldn't look at github repos we don't use it. People set them up for themselves and they are are never in sync. If you want to look at our current source please follow the getting started guide http://jdownloader.org/knowledge/wik...nt/get-started

by default jd has fixed user-agents, though plugins can override this (depending on the given plugin).

if you want to override this yourself, easiest solution is run a proxy and change via the proxy.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #9  
Old 26.07.2016, 12:50
DAC324 DAC324 is offline
Wind Gust
 
Join Date: Nov 2015
Posts: 44
Default

Quote:
Originally Posted by raztoki View Post
@DAC324
by default jd has fixed user-agents, though plugins can override this (depending on the given plugin).

if you want to override this yourself, easiest solution is run a proxy and change via the proxy.

raztoki
Although it surely works, I would not consider it an easy solution.
Problem is that sometimes, it seems that the hosters detect jDownloader and send captchas which cannot be solved (see https://board.jdownloader.org/showthread.php?t=65851 for example).

Kind regards,

DAC324
Reply With Quote
  #10  
Old 26.07.2016, 12:52
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Recaptcha is not sent by host/service. It is sent by google and user-agent has no real impact on quality of recaptcha
__________________
JD-Dev & Server-Admin
Reply With Quote
  #11  
Old 26.07.2016, 13:20
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 17,611
Default

user-agent has very little influence on captcha types. I would go to say stuff all. Referenced thread is very old, and we have solutions to provide browser biased captchas... please see https://support.jdownloader.org/Know...splay-captchas. Ensure browser loop is enabled for recaptchav1.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #12  
Old 26.07.2016, 18:14
DAC324 DAC324 is offline
Wind Gust
 
Join Date: Nov 2015
Posts: 44
Default

Hello raztoki, hello all,

thank you very much for your hints. Of course, browser loop is enabled here but very often, I get these irresolvable captchas anyway. As they do not appear when accessing the download links directly in the browser, there must be something which still allows recaptcha to distinguish between jD and a true browser, no matter if browser loop is enabled or not.

Edit: I just read the page you referred to (https://support.jdownloader.org/Know...splay-captchas).

Interestingly enough, JD opens the browser even if there are Google accounts supplied in jD account manager.

Kind regards,

DAC324

Last edited by DAC324; 26.07.2016 at 18:18.
Reply With Quote
  #13  
Old 26.07.2016, 18:23
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

JD opens the browser for browser loop, as a real browser is always better then no/headless browser. Google can use much more information in a real browser (cookies, sessions, advanced javascript, history, browser ids, browser behaviour....) than in no/headless browser. That's why a real browser will always give better results!
__________________
JD-Dev & Server-Admin
Reply With Quote
  #14  
Old 26.07.2016, 18:39
DAC324 DAC324 is offline
Wind Gust
 
Join Date: Nov 2015
Posts: 44
Default

Then, it seems to be better to not provide any Google Recaptcha account in jD in order to always use the browser loop.

Thank you very much for that information. Unfortunately, Google still delivers those unreadable captchas if browser loop is used only.

Kind regards,

DAC324
Reply With Quote
  #15  
Old 26.07.2016, 18:43
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Browser Loop != Solving Captchas in Browser
__________________
JD-Dev & Server-Admin
Reply With Quote
  #16  
Old 26.07.2016, 19:06
DAC324 DAC324 is offline
Wind Gust
 
Join Date: Nov 2015
Posts: 44
Default

I know.

But I thought browser loop was able to fool Google into thinking you were downloading with a real browser instead of jD.
Obviously, this is not the case. Using jD, I still get irresolvable captchas (just as described here: https://board.jdownloader.org/showthread.php?t=42165 )

Seems there is nothing I can do. But thank you anyway for your quick help.

Kind regards,

DAC324
Reply With Quote
  #17  
Old 03.09.2019, 15:32
Unknown123
Guest
 
Posts: n/a
Default

Some servers are blocking any non-browser user-agent string. By changing our download manager user-agent string into browser string we can get rid of the problem.

It has been 8 years since OP post this, pardon, but I couldn't find in the program settings to change it. Could someone enlighten me? Even Free Download Manager already had it from a long time ago. I prefer JDownloader though. That's why I want it to be better.
Reply With Quote
  #18  
Old 03.09.2019, 16:49
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

@Unknown123: JDownloader is using a browser user-agent. Only very few use different ones where we know that those are okay or got asked for this change by server holder themselves.
Can you provide any non working site/link that is blocked due to user-agent?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #19  
Old 03.09.2019, 16:50
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Quote:
Originally Posted by Unknown123 View Post
It has been 8 years since OP post this, pardon, but I couldn't find in the program settings to change it. Could someone enlighten me? Even Free Download Manager already had it from a long time ago.
JDownloader doesn't (yet) support this and in all those years, it seems the demand/need for this is nearly none existing.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #20  
Old 23.01.2021, 07:05
inzzzomnia inzzzomnia is offline
JD Fan
 
Join Date: Apr 2017
Posts: 77
Default

Time to bump this one again

I found the Google: User Agent in advanced settings so would it be a big thing getting this to work with other sites too?

Instagram is beginning to get incredibly picky and temporarily close the account and force a password change when you login using two different user agents with to short time in between.

Would be nice to be able to change it cause I'm running out of ideas for passwords :D
Reply With Quote
  #21  
Old 25.01.2021, 16:54
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Hi,

at this moment the User-Agent can only be changed for specific plugins if they have an option for this functionality.
If you want to change it globally, you can grab our source code and change it in there.

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #22  
Old 27.01.2021, 04:06
inzzzomnia inzzzomnia is offline
JD Fan
 
Join Date: Apr 2017
Posts: 77
Default

I know that it only can be changed if there's an option for it that's why I asked if it would be a big thing to make it possible on a wider scale since you already do it partially.
Fixing it by my self is kind of out of my league since I'm no programmer.

So what are the user agent that JD uses? Guess I'll have to figure out how to do it the other way around.
Reply With Quote
  #23  
Old 27.01.2021, 16:07
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Quote:
Originally Posted by inzzzomnia View Post
I know that it only can be changed if there's an option for it that's why I asked if it would be a big thing to make it possible on a wider scale since you already do it partially.
No idea.

Quote:
Originally Posted by inzzzomnia View Post
Fixing it by my self is kind of out of my league since I'm no programmer.
It's not a pending bugfix - what you've requested would be a new feature.

Quote:
Originally Posted by inzzzomnia View Post
So what are the user agent that JD uses?
Depends on various factors.
See JDBrowser/src/jd/http/Request.java
Starting from line 436 "getSuggestedUserAgent()".

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
  #24  
Old 01.08.2021, 04:17
artenox artenox is offline
I will play nice!
 
Join Date: Jul 2021
Posts: 1
Default

There are three ways to change User Agent.

1. Modify source code and compile your JDownloader build using Eclipse
The default User Agent is set in JDBrowser/src/jd/http/Request.java source code file
getSuggestedUserAgent() method
Instructions to compile https://jdownloader.org/knowledge/wi...nt/get-started
But it's pretty complicated (it didn't work for me) and you won't be able to get updates.
Instructions also don't tell you how to create a package (so you don't have to use Eclipse every time), it uses "ant" for that.
The size of source code is 1.4 GiB.
However you don't have to use your own build. You can try to take from it only one file JDHttp.jar which you need.

2. Modify binary code (patch/hack)
The default User Agent is located in file JDownloader/libs/JDHttp.jar and then after unpacking /jd/http/Request.class
JDHttp.jar is a zip archive. You can extract and add files to it using any archive manager (7Zip, WinRAR, Engrampa, etc).
Request.class is a binary file and should be edited in a binary editor (for example, Flexhex). Don't use a text editor, you can break.
Extract file Request.class, edit it and drag it back into archive.
But the length of User Agent cannot exceed the existing length. You won't be able to put a long User Agent (e.g. Chromium) without damaging it. In this case you will have to use method 1.
If the length of new User Agent is shorter, add spaces to it. Normal whitespaces (in binary representation 20 20), not null characters (00 00). Null characters are not allowed in that section.
You have to disable JDownloader updates (in advanced settings search for the word Update), otherwise file will be overwritten.
After each update, patching will have to be repeated. Although, you can create a script to automate it.

3. Use a local HTTPS proxy (aka MITM proxy)
The squid mentioned before is no longer relevant, since it doesn't support decrypting HTTPS traffic (which is predominant type on the Internet nowadays).
You can use HandyCache (there is an extension which lets you change User Agent, google it), Proximitron or Fiddler.
I personally tested HandyCache, it works. But the HTTPS decrypting feature is paid (free only first 30 minutes after each launch). And it is quite difficult to buy this program, especially for non-Russian users, although it is possible.
The disadvantage of this way: you have to use an additional program (and maybe more than one).
Advantage: you can continue to receive JDownloader updates, disable and change User Agent in realtime.
Proximitron is an old application and needs an addon to support HTTPS.
Local HTTPS proxies will most likely require a self-signed certificate to be installed on the system (this is not difficult). Although, I'm not sure, maybe JDownloader will work without it.
For Linux there is a console mitmproxy (python based, works by intercepting traffic with iptables). It supports changing User Agent but it's quite hard to configure, I guess. HandyCache is easier and works in Wine, but unfortunately free mode is limited.
Also note that not all of these proxies support upstream SOCKS proxy (in case you need it, of course).
SOCKS is supported in HandyCache. Not supported in: Proximitron, Fiddler, mitmproxy.
These proxy servers ignore HTTPS errors (e.g. outdated server certificate).

I personally use method 2.

Update: There is a class editor application for safe editing class files (requires Java installed on the system) **External links are only visible to Support Staff****External links are only visible to Support Staff**

Update 2 (mods, you can merge messages): Don't use class editor, it breaks file. Use a binary editor instead (Flexhex). But be careful. Don't modify characters that don't belong to User Agent. Make a backup.

Last edited by raztoki; 01.08.2021 at 16:45. Reason: user can't edit posts just yet, merging on behalf.
Reply With Quote
  #25  
Old 14.07.2023, 04:19
djones djones is offline
Junior Loader
 
Join Date: Nov 2022
Posts: 10
Default

Quote:
Originally Posted by artenox View Post
There are three ways to change User Agent.

<snip>
Thanks for outlining this, they're the same conclusions I came to before finding this thread, as I'm encountering more and more blocks of JD's global UA string now, and it's super frustrating that the workarounds require so much brute force (compiling the source code yourself, or hex-hacking existing binaries which will immediately break after an update).

I hope JD devs reconsider their hard stance on not being able to change UA - I understand the fear of their tech support burden potentially increasing if a percentage of idiots change the UA and then complain when a plugin breaks, but surely there must be a middle ground option so we don't have to use insane workarounds or just other downloaders that support custom UA strings.

Last edited by djones; 14.07.2023 at 04:31.
Reply With Quote
  #26  
Old 14.07.2023, 09:59
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 70,922
Default

Quote:
Originally Posted by djones View Post
Thanks for outlining this, they're the same conclusions I came to before finding this thread, as I'm encountering more and more blocks of JD's global UA string now
Please provide a list of websites/plugins which are not working due to a suspected User-Agent block.

At this moment we're not planning to add a setting to change the User-Agent.
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 01:21.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.