JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 02.07.2024, 19:02
nogaff nogaff is offline
I will play nice!
 
Join Date: Sep 2019
Posts: 3
Default FTP plugin doesn't handle certain special characters in filenames properly

The FTP plugin is discarding parts of filenames following certain special characters and is generating invalid download URLs as a result. I've confirmed the issue is triggered by ampersands, but I suspect it will also occur with question marks.

As an example, take the URL "ftp://example.com/some/directory/foo%20%26%20bar.txt" which corresponds to a file named "foo & bar.txt".

When the FTP plugin processes that URL it ends up in the LinkGrabber as "ftp://example.com/some/directory/foo%20" and then fails to download.

Having inspected the source code and done some debugging, it looks like the following is occurring:
  1. The checkLinkFile() method in the decrypter plugin swaps "ftp://" for "ftpviajd://" and sends the URL off to the PluginForDecrypt.createDownloadlink() method.
  2. The createDownloadlink() method by default sends it off to the Encoding.urlDecode() method since it is not explicitly told whether to decode the URL or not.
  3. The Encoding.urlDecode() method checks whether the URL starts with "**External links are only visible to Support Staff** or "**External links are only visible to Support Staff** and since it does not, it proceeds to replace the "%26" with a "&" and returns a partially decoded URL of "ftpviajd://example.com/some/directory/foo%20&%20bar.txt".
  4. Eventually that URL gets handed over to the PluginForHost.getDownloadLinks() method which parses the URL using the regex "ftpviajd://.*?\.[\p{L}\p{Nd}a-zA-Z0-9]{1,}(:\d+)?/([^\?&\"\r\n ]+|$)" specified in the hoster plugin and therefore strips everything after the decoded ampersand.
  5. The hoster plugin's convert() method then swaps the "ftpviajd://" back to "ftp://", leaving us with a broken URL of "ftp://example.com/some/directory/foo%20".

I believe the same issue would occur with question marks in the filename since the Encoding.urlDecode() method also decodes those and they are likewise stripped by the hoster plugin's regex.

Given that the issue seems to be a result of multiple string transformations across 3 or 4 classes, I'm not entirely sure what the preferred solution would be.

Presumably the Encoding.urlDecode() method shouldn't contain custom logic for the "ftpviajd://" protocol, so I feel like the decrypter plugin either needs its own decoding logic, or needs to not decode the URL at all?

Or perhaps the hoster plugin's regex needs to be adjusted?

I also note the the hoster plugin's source includes the following comment at the top:
Code:
// DEV NOTES:
// - ftp filenames can contain & characters!
So somebody obviously thought about it at some point!
Reply With Quote
  #2  
Old 03.07.2024, 13:01
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,046
Default

@nogaff: can you provide an example test link, so I don't have to setup test environment by myself? you can also send link to support@jdownloader.org
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 03.07.2024, 15:45
nogaff nogaff is offline
I will play nice!
 
Join Date: Sep 2019
Posts: 3
Default

Well the only private FTP server I have access to is my employer's corporate website so I can't give you that, but I found some random text file on a public FTP server that has an ampersand in the filename: ftp://ftp.scene.org/pub/resources/docs/telephony/at%26tfil.txt

Will that do?

When the LinkGrabber scans that URL it ends up as "ftp://ftp.scene.org/pub/resources/docs/telephony/at" and then I get a "File not found" error when I attempt to start the download.
Reply With Quote
  #4  
Old 03.07.2024, 16:53
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,046
Default

thanks, will check/work on this
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 04.07.2024, 13:25
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 81,046
Default

@nogaff: Thanks for the report. It also helped to find a bug in LIST parser
please check again with next update
__________________
JD-Dev & Server-Admin
Reply With Quote
  #6  
Old 04.07.2024, 16:43
pspzockerscene's Avatar
pspzockerscene pspzockerscene is offline
Community Manager
 
Join Date: Mar 2009
Location: Deutschland
Posts: 73,132
Default

CORE-Updates have been released!
All announced bugfixes and features are live!
Please update your JDownloader and report any issues you find asap.
If this thread gets marked as "[Solved]" by our forum staff you can still post in it and we will read- and reply to it!

CORE-Updates wurden released!
Alle angekündigten Bugfixes/Features sind nun verfügbar!
Bitte JDownloader updaten und eventuelle Bugs schnellstmöglich an uns melden.
Falls dieser Thread vom Team als "[Erledigt]" markiert wird, kannst du weiterhin darin antworten und wir lesen/beantworten auch solche Threads!

-psp-
__________________
JD Supporter, Plugin Dev. & Community Manager

Erste Schritte & Tutorials || JDownloader 2 Setup Download
Spoiler:

A users' JD crashes and the first thing to ask is:
Quote:
Originally Posted by Jiaz View Post
Do you have Nero installed?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 22:13.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.