#1
|
|||
|
|||
Guide: How to modify JDownloader's user-agent string
I recently found myself needing to change JDownloader’s user-agent string, as I am oftentimes behind a ZScaler filter, which doesn’t work well with JDownloader using the default user-agent string (if you are interested in knowing why, cf. this thread).
If you search around on the forum here, you will see that every once in a while this feature request has come up for one reason or another. Regardless of what the reason is, if you need to modify JDownloader’s user-agent, this guide is for you. If JDownloader is working just fine, or you don’t know what a user-agent string is, feel free to move on – it’s something you don’t need to worry about. What we are going to do is install a local proxy and send JDownloader’s requests through the local proxy to the Internet. The local proxy will modify the user-agent string before sending out the requests. Thanks to Jiaz for the idea. I am using Squid proxy in this tutorial, and running it on Windows. This tutorial is for Windows, but if you need to run it on Linux or some other operating system, just head over to **External links are only visible to Support Staff****External links are only visible to Support Staff** (that's h-t-t-p://wiki.squid-cache.org/SquidFaq/BinaryPackages for you non-supporters out there!) and download the version you need. The instructions below should be easy to adapt to other operating systems. 1. For Windows, download Squid from **External links are only visible to Support Staff****External links are only visible to Support Staff** (for non-supporters: h-t-t-p://squid.acmeconsulting.it/download/squid-2.7.STABLE8-bin.zip). This is the latest stable Windows release. 2. Unzip to c:\squid. It is highly recommended you use this directory. If you are going to use a different directory, you will need to modify all the configuration files to point to that directory, as well as the instructions below to adjust to that fact. 3. In the c:\squid\etc directory copy and rename the following files: Code:
squid.conf.default ==> squid.conf mime.conf.default ==> mime.conf cachemgr.conf.default ==> cachemgr.conf Code:
squid -i –f "c:/squid/etc/squid.conf" –n “Squid” squid -z -f "c:/squid/etc/squid.conf" squid -O servicecommandline –n “Squid” Code:
#These first lines identify "access control lists". #We are going to identify some IPs (all and localhost) we will later allow or block acl all src all acl localhost src 127.0.0.1/255.255.255.255 #Now we will identify ports we want to later allow or block acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT #We block unsafe ports http_access deny !Safe_ports http_access deny CONNECT !SSL_ports #We allow access to the localhost, but block everyone else http_access allow localhost http_access deny all #In this case we don't need icp access (access to the Squid cache), so we'll block that for everyone icp_access deny all #Allow htcp access to localhost, block everyone else htcp_access allow localhost htcp_access deny all #Indicate the port you want the proxy (127.0.0.1) to listen on. By default Sqid uses 3128 #In JDownloader you will then set your proxy to 127.0.0.1 and your port to whatever port you indicate here http_port 3128 #These next lines are necessary if you are going to use a proxy. #The first line indicates the proxy, the second tells Squid to forward everything it receives to the proxy #Replace “YOUR_PROXY_IP_ADDRESS” with your proxy number. #Replace “YOUR_PROXY_PORT_NUMBER” with your proxy port number #If you aren't going to use a proxy you must delete or put a # before these lines cache_peer YOUR_PROXY_IP_ADDRESS parent YOUR_PROXY_PORT_NUMBER 0 no-query no-digest never_direct allow all #Some default Squid configuration lines. Not sure what they all do, but they are recommended and don't seem to hurt hierarchy_stoplist cgi-bin ? cache_dir ufs c:/squid/var/cache 100 16 256 access_log c:/squid/var/logs/access.log squid refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9] upgrade_http0.9 deny shoutcast acl apache rep_header Server ^Apache broken_vary_encoding allow apache #Here's where the magic happens. #Squid denies all user-agents and then replaces them with another user agent, in this case "JDownloader" #If you want to use a different user-agent, just replace "JDownloader" with whatever agent you want to use header_access User-Agent deny all header_replace User-Agent JDownloader 7. Notice the last two lines of the squid.conf file. Those are the lines that modify the user-agent string. They are currently set to use the user-agent string "JDownloader". If you want to use a different user-agent, change the text “JDownloader” to whatever user-agent you want to use. Save and close the file. 8. Open up “Services” and start the Squid service, or restart it if it is already running. 9. In the JDownloader options, configure JDownloader to use the proxy 127.0.0.1 and the port number 3128 (or whatever port number you chose). This option can be found in "Settings" - "Download and Connections" - "Internet and Network". That’s it! Now Squid will intercept all the traffic that comes from JDownloader and modify the user-agent to whatever you tell it to, then send the traffic out to the Internet. So far I have tested it with the update function, DLC files, FileServe and RapidShare and everything has worked perfectly. In theory it could cause problems with some plugins, however, so I would only recommend using it if you need to. Enjoy! Last edited by JoeH; 21.05.2011 at 19:57. Reason: Modify external links so they will be visible to all |
#2
|
||||
|
||||
Didn't try it but thanks, your thread is now a sticky;)
GreeZ pspzockerscene
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#3
|
|||
|
|||
Great, thanks.
|
12.09.2011, 13:18 |
remi |
Message deleted by Think3r.
Reason: useless
|
#4
|
|||
|
|||
Uhmm... publicity for what? For Squid? If anyone know an easier solution, please post it! Squid is complicated. For ZScaler? I would recommend ANY other Internet filter over ZScaler, since ZScaler breaks most programs (like JDownloader) that need to access the Internet but aren't browsers. If you want to understand why, just read the other thread which I linked to in my first message!
|
#5
|
||||
|
||||
re: publicity
I believe there used to be another message, which has been deleted. Virus/spam related.
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#6
|
|||
|
|||
Sorry for reviving this old thread but I thought I would add some info that might help others, Thanks to the OP for the useful info
I had to add the following to the end of squid.conf for it to work with some hosts who use php proxys and .htaccess tricks to filter out downloaders etc.. This helps to prevent the host knowing that you are using a local proxy: Code:
header_access VIA deny all header_access DNT deny all header_access X-FORWARDED-FOR deny all And for some hosts you may have to set the referrer to the domain name of the host. Code:
header_access REFERER deny all header_replace REFERER **External links are only visible to Support Staff** Last edited by pumplex; 03.04.2016 at 09:00. |
#7
|
|||
|
|||
Why not...
...implement a way to change the user agent in jDownloader itself?
Searching on this topic, I found a RandomUserAgent.java here: **External links are only visible to Support Staff****External links are only visible to Support Staff** How can I incorporate that in jDownloader? Thank you very much and kind regards, DAC324 |
#8
|
||||
|
||||
@DAC324
I wouldn't look at github repos we don't use it. People set them up for themselves and they are are never in sync. If you want to look at our current source please follow the getting started guide http://jdownloader.org/knowledge/wik...nt/get-started by default jd has fixed user-agents, though plugins can override this (depending on the given plugin). if you want to override this yourself, easiest solution is run a proxy and change via the proxy. raztoki
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#9
|
|||
|
|||
Quote:
Problem is that sometimes, it seems that the hosters detect jDownloader and send captchas which cannot be solved (see https://board.jdownloader.org/showthread.php?t=65851 for example). Kind regards, DAC324 |
#10
|
||||
|
||||
Recaptcha is not sent by host/service. It is sent by google and user-agent has no real impact on quality of recaptcha
__________________
JD-Dev & Server-Admin |
#11
|
||||
|
||||
user-agent has very little influence on captcha types. I would go to say stuff all. Referenced thread is very old, and we have solutions to provide browser biased captchas... please see https://support.jdownloader.org/Know...splay-captchas. Ensure browser loop is enabled for recaptchav1.
raztoki
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#12
|
|||
|
|||
Hello raztoki, hello all,
thank you very much for your hints. Of course, browser loop is enabled here but very often, I get these irresolvable captchas anyway. As they do not appear when accessing the download links directly in the browser, there must be something which still allows recaptcha to distinguish between jD and a true browser, no matter if browser loop is enabled or not. Edit: I just read the page you referred to (https://support.jdownloader.org/Know...splay-captchas). Interestingly enough, JD opens the browser even if there are Google accounts supplied in jD account manager. Kind regards, DAC324 Last edited by DAC324; 26.07.2016 at 17:18. |
#13
|
||||
|
||||
JD opens the browser for browser loop, as a real browser is always better then no/headless browser. Google can use much more information in a real browser (cookies, sessions, advanced javascript, history, browser ids, browser behaviour....) than in no/headless browser. That's why a real browser will always give better results!
__________________
JD-Dev & Server-Admin |
#14
|
|||
|
|||
Then, it seems to be better to not provide any Google Recaptcha account in jD in order to always use the browser loop.
Thank you very much for that information. Unfortunately, Google still delivers those unreadable captchas if browser loop is used only. Kind regards, DAC324 |
#15
|
||||
|
||||
Browser Loop != Solving Captchas in Browser
__________________
JD-Dev & Server-Admin |
#16
|
|||
|
|||
I know.
But I thought browser loop was able to fool Google into thinking you were downloading with a real browser instead of jD. Obviously, this is not the case. Using jD, I still get irresolvable captchas (just as described here: https://board.jdownloader.org/showthread.php?t=42165 ) Seems there is nothing I can do. But thank you anyway for your quick help. Kind regards, DAC324 |
#17
|
|||
|
|||
Some servers are blocking any non-browser user-agent string. By changing our download manager user-agent string into browser string we can get rid of the problem.
It has been 8 years since OP post this, pardon, but I couldn't find in the program settings to change it. Could someone enlighten me? Even Free Download Manager already had it from a long time ago. I prefer JDownloader though. That's why I want it to be better. |
#18
|
||||
|
||||
@Unknown123: JDownloader is using a browser user-agent. Only very few use different ones where we know that those are okay or got asked for this change by server holder themselves.
Can you provide any non working site/link that is blocked due to user-agent?
__________________
JD-Dev & Server-Admin |
#19
|
||||
|
||||
JDownloader doesn't (yet) support this and in all those years, it seems the demand/need for this is nearly none existing.
__________________
JD-Dev & Server-Admin |
#20
|
|||
|
|||
Time to bump this one again
I found the Google: User Agent in advanced settings so would it be a big thing getting this to work with other sites too? Instagram is beginning to get incredibly picky and temporarily close the account and force a password change when you login using two different user agents with to short time in between. Would be nice to be able to change it cause I'm running out of ideas for passwords :D |
#21
|
||||
|
||||
Hi,
at this moment the User-Agent can only be changed for specific plugins if they have an option for this functionality. If you want to change it globally, you can grab our source code and change it in there. -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#22
|
|||
|
|||
I know that it only can be changed if there's an option for it that's why I asked if it would be a big thing to make it possible on a wider scale since you already do it partially.
Fixing it by my self is kind of out of my league since I'm no programmer. So what are the user agent that JD uses? Guess I'll have to figure out how to do it the other way around. |
#23
|
||||
|
||||
Quote:
Quote:
Depends on various factors. See JDBrowser/src/jd/http/Request.java Starting from line 436 "getSuggestedUserAgent()". -psp-
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#24
|
|||
|
|||
There are three ways to change User Agent.
1. Modify source code and compile your JDownloader build using Eclipse The default User Agent is set in JDBrowser/src/jd/http/Request.java source code file getSuggestedUserAgent() method Instructions to compile https://jdownloader.org/knowledge/wi...nt/get-started But it's pretty complicated (it didn't work for me) and you won't be able to get updates. Instructions also don't tell you how to create a package (so you don't have to use Eclipse every time), it uses "ant" for that. The size of source code is 1.4 GiB. However you don't have to use your own build. You can try to take from it only one file JDHttp.jar which you need. 2. Modify binary code (patch/hack) The default User Agent is located in file JDownloader/libs/JDHttp.jar and then after unpacking /jd/http/Request.class JDHttp.jar is a zip archive. You can extract and add files to it using any archive manager (7Zip, WinRAR, Engrampa, etc). Request.class is a binary file and should be edited in a binary editor (for example, Flexhex). Don't use a text editor, you can break. Extract file Request.class, edit it and drag it back into archive. But the length of User Agent cannot exceed the existing length. You won't be able to put a long User Agent (e.g. Chromium) without damaging it. In this case you will have to use method 1. If the length of new User Agent is shorter, add spaces to it. Normal whitespaces (in binary representation 20 20), not null characters (00 00). Null characters are not allowed in that section. You have to disable JDownloader updates (in advanced settings search for the word Update), otherwise file will be overwritten. After each update, patching will have to be repeated. Although, you can create a script to automate it. 3. Use a local HTTPS proxy (aka MITM proxy) The squid mentioned before is no longer relevant, since it doesn't support decrypting HTTPS traffic (which is predominant type on the Internet nowadays). You can use HandyCache (there is an extension which lets you change User Agent, google it), Proximitron or Fiddler. I personally tested HandyCache, it works. But the HTTPS decrypting feature is paid (free only first 30 minutes after each launch). And it is quite difficult to buy this program, especially for non-Russian users, although it is possible. The disadvantage of this way: you have to use an additional program (and maybe more than one). Advantage: you can continue to receive JDownloader updates, disable and change User Agent in realtime. Proximitron is an old application and needs an addon to support HTTPS. Local HTTPS proxies will most likely require a self-signed certificate to be installed on the system (this is not difficult). Although, I'm not sure, maybe JDownloader will work without it. For Linux there is a console mitmproxy (python based, works by intercepting traffic with iptables). It supports changing User Agent but it's quite hard to configure, I guess. HandyCache is easier and works in Wine, but unfortunately free mode is limited. Also note that not all of these proxies support upstream SOCKS proxy (in case you need it, of course). SOCKS is supported in HandyCache. Not supported in: Proximitron, Fiddler, mitmproxy. These proxy servers ignore HTTPS errors (e.g. outdated server certificate). I personally use method 2. Update: There is a class editor application for safe editing class files (requires Java installed on the system) **External links are only visible to Support Staff****External links are only visible to Support Staff** Update 2 (mods, you can merge messages): Don't use class editor, it breaks file. Use a binary editor instead (Flexhex). But be careful. Don't modify characters that don't belong to User Agent. Make a backup. Last edited by raztoki; 01.08.2021 at 15:45. Reason: user can't edit posts just yet, merging on behalf. |
#25
|
|||
|
|||
Thanks for outlining this, they're the same conclusions I came to before finding this thread, as I'm encountering more and more blocks of JD's global UA string now, and it's super frustrating that the workarounds require so much brute force (compiling the source code yourself, or hex-hacking existing binaries which will immediately break after an update).
I hope JD devs reconsider their hard stance on not being able to change UA - I understand the fear of their tech support burden potentially increasing if a percentage of idiots change the UA and then complain when a plugin breaks, but surely there must be a middle ground option so we don't have to use insane workarounds or just other downloaders that support custom UA strings. Last edited by djones; 14.07.2023 at 03:31. |
#26
|
||||
|
||||
Quote:
At this moment we're not planning to add a setting to change the User-Agent.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
|
|