[nogaff]

The FTP plugin is discarding parts of filenames following certain special characters and is generating invalid download URLs as a result. I've confirmed the issue is triggered by ampersands, but I suspect it will also occur with question marks.

As an example, take the URL "ftp://example.com/some/directory/foo%20%26%20bar.txt" which corresponds to a file named "foo & bar.txt".

When the FTP plugin processes that URL it ends up in the LinkGrabber as "ftp://example.com/some/directory/foo%20" and then fails to download.

Having inspected the source code and done some debugging, it looks like the following is occurring:

1. The checkLinkFile() method in the decrypter plugin swaps "ftp://" for "ftpviajd://" and sends the URL off to the PluginForDecrypt.createDownloadlink() method.
2. The createDownloadlink() method by default sends it off to the Encoding.urlDecode() method since it is not explicitly told whether to decode the URL or not.
3. The Encoding.urlDecode() method checks whether the URL starts with "**External links are only visible to Support Staff** or "**External links are only visible to Support Staff** and since it does not, it proceeds to replace the "%26" with a "&" and returns a partially decoded URL of "ftpviajd://example.com/some/directory/foo%20&%20bar.txt".
4. Eventually that URL gets handed over to the PluginForHost.getDownloadLinks() method which parses the URL using the regex "ftpviajd://.*?\.[\p{L}\p{Nd}a-zA-Z0-9]{1,}(:\d+)?/([^\?&\"\r\n ]+|$)" specified in the hoster plugin and therefore strips everything after the decoded ampersand.
5. The hoster plugin's convert() method then swaps the "ftpviajd://" back to "ftp://", leaving us with a broken URL of "ftp://example.com/some/directory/foo%20".

I believe the same issue would occur with question marks in the filename since the Encoding.urlDecode() method also decodes those and they are likewise stripped by the hoster plugin's regex.

Given that the issue seems to be a result of multiple string transformations across 3 or 4 classes, I'm not entirely sure what the preferred solution would be.

Presumably the Encoding.urlDecode() method shouldn't contain custom logic for the "ftpviajd://" protocol, so I feel like the decrypter plugin either needs its own decoding logic, or needs to not decode the URL at all?

Or perhaps the hoster plugin's regex needs to be adjusted?

I also note the the hoster plugin's source includes the following comment at the top:

// DEV NOTES:
// - ftp filenames can contain & characters!

So somebody obviously thought about it at some point!

[Jiaz]

@nogaff: can you provide an example test link, so I don't have to setup test environment by myself? you can also send link to support@jdownloader.org

[nogaff]

Well the only private FTP server I have access to is my employer's corporate website so I can't give you that, but I found some random text file on a public FTP server that has an ampersand in the filename: ftp://ftp.scene.org/pub/resources/docs/telephony/at%26tfil.txt

Will that do?

When the LinkGrabber scans that URL it ends up as "ftp://ftp.scene.org/pub/resources/docs/telephony/at" and then I get a "File not found" error when I attempt to start the download.

[Jiaz]

thanks, will check/work on this

[Jiaz]

@nogaff: Thanks for the report. It also helped to find a bug in LIST parser
please check again with next update

[pspzockerscene]

CORE-Updates have been released!
All announced bugfixes and features are live!
Please update your JDownloader and report any issues you find asap.
If this thread gets marked as "[Solved]" by our forum staff you can still post in it and we will read- and reply to it!

CORE-Updates wurden released!
Alle angekündigten Bugfixes/Features sind nun verfügbar!
Bitte JDownloader updaten und eventuelle Bugs schnellstmöglich an uns melden.
Falls dieser Thread vom Team als "[Erledigt]" markiert wird, kannst du weiterhin darin antworten und wir lesen/beantworten auch solche Threads!

-psp-