JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 21.05.2011, 18:23
JoeH
Guest
 
Posts: n/a
Cool Guide: How to modify JDownloader's user-agent string

I recently found myself needing to change JDownloader’s user-agent string, as I am oftentimes behind a ZScaler filter, which doesn’t work well with JDownloader using the default user-agent string (if you are interested in knowing why, cf. this thread).

If you search around on the forum here, you will see that every once in a while this feature request has come up for one reason or another. Regardless of what the reason is, if you need to modify JDownloader’s user-agent, this guide is for you. If JDownloader is working just fine, or you don’t know what a user-agent string is, feel free to move on – it’s something you don’t need to worry about.

What we are going to do is install a local proxy and send JDownloader’s requests through the local proxy to the Internet. The local proxy will modify the user-agent string before sending out the requests. Thanks to Jiaz for the idea.

I am using Squid proxy in this tutorial, and running it on Windows. This tutorial is for Windows, but if you need to run it on Linux or some other operating system, just head over to **External links are only visible to Support Staff****External links are only visible to Support Staff** (that's h-t-t-p://wiki.squid-cache.org/SquidFaq/BinaryPackages for you non-supporters out there!) and download the version you need. The instructions below should be easy to adapt to other operating systems.

1. For Windows, download Squid from **External links are only visible to Support Staff****External links are only visible to Support Staff** (for non-supporters: h-t-t-p://squid.acmeconsulting.it/download/squid-2.7.STABLE8-bin.zip). This is the latest stable Windows release.
2. Unzip to c:\squid. It is highly recommended you use this directory. If you are going to use a different directory, you will need to modify all the configuration files to point to that directory, as well as the instructions below to adjust to that fact.
3. In the c:\squid\etc directory copy and rename the following files:
Code:
squid.conf.default ==> squid.conf
mime.conf.default ==> mime.conf
cachemgr.conf.default ==> cachemgr.conf
4. Open up an MS-DOS command prompt (as administrator) to c:\squid\sbin and run the following commands:
Code:
squid -i –f "c:/squid/etc/squid.conf" –n “Squid”
squid -z -f "c:/squid/etc/squid.conf"
squid -O servicecommandline –n “Squid”
5. Open c:\squid\etc\squid.conf in a text editor. Delete everything in the file (make sure you have the backup copy in case you need to reference it) and past in the following:

Code:
#These first lines identify "access control lists". 
#We are going to identify some IPs (all and localhost) we will later allow or block
acl all src all
acl localhost src 127.0.0.1/255.255.255.255

#Now we will identify ports we want to later allow or block
acl SSL_ports port 443
acl Safe_ports port 80		# http
acl Safe_ports port 21		# ftp
acl Safe_ports port 443		# https
acl Safe_ports port 70		# gopher
acl Safe_ports port 210		# wais
acl Safe_ports port 1025-65535	# unregistered ports
acl Safe_ports port 280		# http-mgmt
acl Safe_ports port 488		# gss-http
acl Safe_ports port 591		# filemaker
acl Safe_ports port 777		# multiling http
acl CONNECT method CONNECT

#We block unsafe ports
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

#We allow access to the localhost, but block everyone else
http_access allow localhost
http_access deny all

#In this case we don't need icp access (access to the Squid cache), so we'll block that for everyone
icp_access deny all

#Allow htcp access to localhost, block everyone else
htcp_access allow localhost
htcp_access deny all

#Indicate the port you want the proxy (127.0.0.1) to listen on. By default Sqid uses 3128
#In JDownloader you will then set your proxy to 127.0.0.1 and your port to whatever port you indicate here
http_port 3128

#These next lines are necessary if you are going to use a proxy. 
#The first line indicates the proxy, the second tells Squid to forward everything it receives to the proxy
#Replace “YOUR_PROXY_IP_ADDRESS” with your proxy number.
#Replace “YOUR_PROXY_PORT_NUMBER” with your proxy port number
#If you aren't going to use a proxy you must delete or put a # before these lines
cache_peer YOUR_PROXY_IP_ADDRESS parent YOUR_PROXY_PORT_NUMBER 0 no-query no-digest
never_direct allow all

#Some default Squid configuration lines. Not sure what they all do, but they are recommended and don't seem to hurt
hierarchy_stoplist cgi-bin ?
cache_dir ufs c:/squid/var/cache 100 16 256
access_log c:/squid/var/logs/access.log squid
refresh_pattern ^ftp:		1440	20%	10080
refresh_pattern ^gopher:	1440	0%	1440
refresh_pattern -i (/cgi-bin/|\?) 0	0%	0
refresh_pattern .		0	20%	4320
acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9]
upgrade_http0.9 deny shoutcast
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache

#Here's where the magic happens. 
#Squid denies all user-agents and then replaces them with another user agent, in this case "JDownloader"
#If you want to use a different user-agent, just replace "JDownloader" with whatever agent you want to use
header_access User-Agent deny all
header_replace User-Agent JDownloader
6. In your new squid.conf file change the proxy number and port number as needed. If you don’t use a proxy, you must delete the two lines in the proxy section, or put a # infront of them so that Squid ignores them. You may also want to edit the default port that Squid listens on. The default is 3128, but you may want to use 8080 or some other port.
7. Notice the last two lines of the squid.conf file. Those are the lines that modify the user-agent string. They are currently set to use the user-agent string "JDownloader". If you want to use a different user-agent, change the text “JDownloader” to whatever user-agent you want to use. Save and close the file.
8. Open up “Services” and start the Squid service, or restart it if it is already running.
9. In the JDownloader options, configure JDownloader to use the proxy 127.0.0.1 and the port number 3128 (or whatever port number you chose). This option can be found in "Settings" - "Download and Connections" - "Internet and Network".

That’s it! Now Squid will intercept all the traffic that comes from JDownloader and modify the user-agent to whatever you tell it to, then send the traffic out to the Internet.

So far I have tested it with the update function, DLC files, FileServe and RapidShare and everything has worked perfectly. In theory it could cause problems with some plugins, however, so I would only recommend using it if you need to.

Enjoy!

Last edited by JoeH; 21.05.2011 at 20:57. Reason: Modify external links so they will be visible to all
Reply With Quote
  #2  
Old 21.05.2011, 18:32
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 48,912
Default

Didn't try it but thanks, your thread is now a sticky;)

GreeZ pspzockerscene
__________________

Ad-free installers || Werbefreie Installer
Windows Setup<--JD2 BETA-->Linux Setup x86 || Linux Setup x64 || Mac Setup
-----=>Support Chat<=-----
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
That's true James
Quote:
Originally Posted by James
Die Leute verstehen einfach nicht dass nur weil man mit einer Waffe auch auf Menschen schießen kann dass ein Schützenver​ein kein Ort für Amoklaufide​en ist
Reply With Quote
  #3  
Old 22.05.2011, 12:28
JoeH
Guest
 
Posts: n/a
Default

Great, thanks.
Reply With Quote
Old 12.09.2011, 14:18
remi
Message deleted by Think3r. Reason: useless
  #4  
Old 16.09.2011, 15:47
JoeH
Guest
 
Posts: n/a
Default

Uhmm... publicity for what? For Squid? If anyone know an easier solution, please post it! Squid is complicated. For ZScaler? I would recommend ANY other Internet filter over ZScaler, since ZScaler breaks most programs (like JDownloader) that need to access the Internet but aren't browsers. If you want to understand why, just read the other thread which I linked to in my first message!
Reply With Quote
  #5  
Old 16.09.2011, 16:14
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,288
Default

re: publicity
I believe there used to be another message, which has been deleted. Virus/spam related.
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #6  
Old 03.04.2016, 07:19
pumplex pumplex is offline
Modem User
 
Join Date: Apr 2016
Posts: 1
Thumbs up

Sorry for reviving this old thread but I thought I would add some info that might help others, Thanks to the OP for the useful info

I had to add the following to the end of squid.conf for it to work with some hosts who use php proxys and .htaccess tricks to filter out downloaders etc..

This helps to prevent the host knowing that you are using a local proxy:

Code:
header_access VIA deny all
header_access DNT deny all
header_access X-FORWARDED-FOR deny all

And for some hosts you may have to set the referrer to the domain name of the host.

Code:
header_access REFERER deny all
header_replace REFERER **External links are only visible to Support Staff**

Last edited by pumplex; 03.04.2016 at 10:00.
Reply With Quote
  #7  
Old 26.07.2016, 12:05
DAC324 DAC324 is offline
DSL Light User
 
Join Date: Nov 2015
Posts: 30
Default Why not...

...implement a way to change the user agent in jDownloader itself?

Searching on this topic, I found a RandomUserAgent.java
here:

**External links are only visible to Support Staff****External links are only visible to Support Staff**

How can I incorporate that in jDownloader?

Thank you very much and kind regards,

DAC324
Reply With Quote
  #8  
Old 26.07.2016, 12:19
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,288
Default

@DAC324
I wouldn't look at github repos we don't use it. People set them up for themselves and they are are never in sync. If you want to look at our current source please follow the getting started guide http://jdownloader.org/knowledge/wik...nt/get-started

by default jd has fixed user-agents, though plugins can override this (depending on the given plugin).

if you want to override this yourself, easiest solution is run a proxy and change via the proxy.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #9  
Old 26.07.2016, 12:50
DAC324 DAC324 is offline
DSL Light User
 
Join Date: Nov 2015
Posts: 30
Default

Quote:
Originally Posted by raztoki View Post
@DAC324
by default jd has fixed user-agents, though plugins can override this (depending on the given plugin).

if you want to override this yourself, easiest solution is run a proxy and change via the proxy.

raztoki
Although it surely works, I would not consider it an easy solution.
Problem is that sometimes, it seems that the hosters detect jDownloader and send captchas which cannot be solved (see https://board.jdownloader.org/showthread.php?t=65851 for example).

Kind regards,

DAC324
Reply With Quote
  #10  
Old 26.07.2016, 12:52
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 66,134
Default

Recaptcha is not sent by host/service. It is sent by google and user-agent has no real impact on quality of recaptcha
__________________
JD-Dev & Server-Admin
Reply With Quote
  #11  
Old 26.07.2016, 13:20
raztoki's Avatar
raztoki raztoki is offline
English Supporter
 
Join Date: Apr 2010
Location: Australia
Posts: 16,288
Default

user-agent has very little influence on captcha types. I would go to say stuff all. Referenced thread is very old, and we have solutions to provide browser biased captchas... please see https://support.jdownloader.org/Know...splay-captchas. Ensure browser loop is enabled for recaptchav1.

raztoki
__________________
raztoki @ jDownloader reporter/developer
http://svn.jdownloader.org/users/170

Don't fight the system, use it to your advantage. :]
Reply With Quote
  #12  
Old 26.07.2016, 18:14
DAC324 DAC324 is offline
DSL Light User
 
Join Date: Nov 2015
Posts: 30
Default

Hello raztoki, hello all,

thank you very much for your hints. Of course, browser loop is enabled here but very often, I get these irresolvable captchas anyway. As they do not appear when accessing the download links directly in the browser, there must be something which still allows recaptcha to distinguish between jD and a true browser, no matter if browser loop is enabled or not.

Edit: I just read the page you referred to (https://support.jdownloader.org/Know...splay-captchas).

Interestingly enough, JD opens the browser even if there are Google accounts supplied in jD account manager.

Kind regards,

DAC324

Last edited by DAC324; 26.07.2016 at 18:18.
Reply With Quote
  #13  
Old 26.07.2016, 18:23
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 66,134
Default

JD opens the browser for browser loop, as a real browser is always better then no/headless browser. Google can use much more information in a real browser (cookies, sessions, advanced javascript, history, browser ids, browser behaviour....) than in no/headless browser. That's why a real browser will always give better results!
__________________
JD-Dev & Server-Admin
Reply With Quote
  #14  
Old 26.07.2016, 18:39
DAC324 DAC324 is offline
DSL Light User
 
Join Date: Nov 2015
Posts: 30
Default

Then, it seems to be better to not provide any Google Recaptcha account in jD in order to always use the browser loop.

Thank you very much for that information. Unfortunately, Google still delivers those unreadable captchas if browser loop is used only.

Kind regards,

DAC324
Reply With Quote
  #15  
Old 26.07.2016, 18:43
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 66,134
Default

Browser Loop != Solving Captchas in Browser
__________________
JD-Dev & Server-Admin
Reply With Quote
  #16  
Old 26.07.2016, 19:06
DAC324 DAC324 is offline
DSL Light User
 
Join Date: Nov 2015
Posts: 30
Default

I know.

But I thought browser loop was able to fool Google into thinking you were downloading with a real browser instead of jD.
Obviously, this is not the case. Using jD, I still get irresolvable captchas (just as described here: https://board.jdownloader.org/showthread.php?t=42165 )

Seems there is nothing I can do. But thank you anyway for your quick help.

Kind regards,

DAC324
Reply With Quote
  #17  
Old 03.09.2019, 15:32
Unknown123 Unknown123 is offline
Modem User
 
Join Date: Jul 2018
Posts: 1
Default

Some servers are blocking any non-browser user-agent string. By changing our download manager user-agent string into browser string we can get rid of the problem.

It has been 8 years since OP post this, pardon, but I couldn't find in the program settings to change it. Could someone enlighten me? Even Free Download Manager already had it from a long time ago. I prefer JDownloader though. That's why I want it to be better.
Reply With Quote
  #18  
Old 03.09.2019, 16:49
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 66,134
Default

@Unknown123: JDownloader is using a browser user-agent. Only very few use different ones where we know that those are okay or got asked for this change by server holder themselves.
Can you provide any non working site/link that is blocked due to user-agent?
__________________
JD-Dev & Server-Admin
Reply With Quote
  #19  
Old 03.09.2019, 16:50
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 66,134
Default

Quote:
Originally Posted by Unknown123 View Post
It has been 8 years since OP post this, pardon, but I couldn't find in the program settings to change it. Could someone enlighten me? Even Free Download Manager already had it from a long time ago.
JDownloader doesn't (yet) support this and in all those years, it seems the demand/need for this is nearly none existing.
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 10:08.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.