JDownloader Community - Appwork GmbH
 

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 29.10.2017, 19:23
fred_gaou's Avatar
fred_gaou fred_gaou is offline
DSL User
 
Join Date: Mar 2016
Location: France
Posts: 39
Exclamation [EventScripter] weird issue about accents in output file

Hi,

In some cases when I gather names with myDownloadLink.getName() and then use writeFile() to output them to a file, the accents are corrupted.

Example:
Code:
[Lama Faché] 17.10.18 - 10 FONCTIONS CACHÉES DANS VOTRE CLAVIER QUE VOUS NE CONNAISSIEZ PAS Lama Faché.mp4
is turned into:
Code:
[Lama Faché] 17.10.18 - 10 FONCTIONS CACHÉES DANS VOTRE CLAVIER QUE VOUS NE CONNAISSIEZ PAS Lama Faché.mp4

Steps to help dev to reproduce

1. If you know a link with such accents in name, download it. Else go to Mister Buzz channel on Dailymotion where to find short videos with french title.

2. Run this test accents script manually:

Code:
/*********************************************************************
                         GLOBAL DECLARATIONS
 *********************************************************************/

/* OS environment */
var myEnv = getEnvironment(); // Get OS environment
var sep = myEnv.getPathSeparator(); // Specific OS path separator

/* PACKAGES */
var a_packages = getAllFilePackages(); //Get a list of all packages that are in download tab

/* -------------   A C C E N T    T E S T   --------------- */
var s_test_accent_works = "",
    s_test_accent_fails = "";
var s_test_path_works = getEnv("temp") + sep + "test accent works.log";
var s_test_path_fails = getEnv("temp") + sep + "test accent fails.log";

/*********************************************************************
                              P R O C E E D
 *********************************************************************/
/* -----------   A C C E N T    T E S T   F A I L S  --------------- */
// IIFE
(function() {
    //loop through all packages in download list tab.
    for (var i = 0; i < a_packages.length; i++) {
        var a_parts = a_packages[i].getDownloadLinks(); // get all parts of current package in loop

        // get name for each parts of current package
        for (var j = 0; j < a_parts.length; j++) {
            s_test_accent_fails += a_parts[j].getName() + "\r\n"; // accents gets corrupted in file
            //alert(a_parts[j].getName());
        }

    } // end of packages loop
})(); // end of IIFE

/* -----------   A C C E N T    T E S T   W O R K S  ------------- */
// IIFE
(function() {
    var o_package = a_packages[0]; // test first package
    var a_parts = o_package.getDownloadLinks(); // get all parts of current package in loop

    // get name for each part of first package
    for (var i = 0; i < a_parts.length; i++) {
        s_test_accent_works += a_parts[i].getName() + "\r\n"; // accents are fine in output file
    }
})(); // end of IIFE

/* ----------   W R I T E   A C C E N T    T E S T   ------------- */
deleteFile(s_test_path_works, false);
writeFile(s_test_path_works, s_test_accent_works, false);

deleteFile(s_test_path_fails, false);
writeFile(s_test_path_fails, s_test_accent_fails, false);

I really don't understand why it fails in the 1st case if I loop through all my packages while it works in second case where I just gather links name from 1st package (or any other) only.

There is no issue if use alert() to display name line 31.

What is wrong with my code? Would it be a JD or Java bug?

My setup
  • JD 2.0 core 38121
  • JAVA 1.8.0.152 x64
  • Windows 10 pro x64

Here is my logs just after I manually run the script on my system:

29.10.17 17.57.12 <--> 29.10.17 18.06.03 jdlog://5887814015941/
Reply With Quote
  #2  
Old 30.10.2017, 13:26
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

I guess your text editor does not(not correct) auto detect UTF-8.
JDownloader writes out Strings UTF-8 encoded.

You can enforce to write BOM via
Code:
var string="Lama Faché";
string="\uFEFF" + string;
BOM informs text editor about UTF-8 encoding
__________________
JD-Dev & Server-Admin

Last edited by Jiaz; 30.10.2017 at 13:29.
Reply With Quote
  #3  
Old 30.10.2017, 15:25
fred_gaou's Avatar
fred_gaou fred_gaou is offline
DSL User
 
Join Date: Mar 2016
Location: France
Posts: 39
Default

Thanks.

After having removed packages and downloaded new ones, the issue is gone. I guess one of the part.name had a weird character that corrupted something.

I don't thing the cause is the BOM. My text editor never had issues to open utf-8 with or without BOM. Unfortunately I didn't keep the "test accent fails.log" from yesterday with corrupted characters. I could have force it to reload with utf-8 encoding with my text editor. Today, such "test accent fails.log" open directly as utf-8 with no character corruption issue.

If I encounter this issue again, I will come back with new test and data.
Reply With Quote
  #4  
Old 30.10.2017, 15:51
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Without BOM the texteditor has to auto detect encoding and this might fail. So when it happens again, try to add BOM to string before writing the file.

Thanks for the feedback!
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 30.10.2017, 22:53
fred_gaou's Avatar
fred_gaou fred_gaou is offline
DSL User
 
Join Date: Mar 2016
Location: France
Posts: 39
Thumbs up

Quote:
Originally Posted by Jiaz View Post
You can enforce to write BOM via
Code:
var string="Lama Faché";
string="\uFEFF" + string;
Thanks for this trick. It solved the issue I could reproduce. It was caused by japanese characters.
Reply With Quote
  #6  
Old 02.11.2017, 10:26
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 79,290
Default

Thanks for the feedback!
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 03:24.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.