JDownloader Community - Appwork GmbH
 

Reply
 
Thread Tools Display Modes
  #1  
Old 14.10.2018, 14:56
malcor
Guest
 
Posts: n/a
Thumbs up CRC errors in large encrypted downloads

Hi, I am having trouble downloading encrypted files from Mega NZ using JDownloader. The files almost always have CRC errors. There are no CRC errors when downloading files directly from Mega NZ via a browser. The problem only occurs when using JDownloader.

I have read through the forum and tried various things. My hardware is fine, and I have thoroughly checked and stress tested it (RAM, HDD, SSD).

I found that Bitdefender (paid version) was responsible for some issues. Even with Bitdefender disabled, it was still causing issues, so I uninstalled and removed all trace of it. However, I still get CRC errors with encrypted files larger than 500MB.

The errors occur randomly, as the locations will be different each time the same file is downloaded. However, there is a definite pattern to the errors, and they are just single bit errors. The pattern enables easy repair of the files, however, the file needs to be downloaded twice in order to repair it.

The following are examples of the types of errors that I am getting in the downloaded files. In the examples, the location of the errors are given as well as the bad byte (Bad) and what it should be (Good).

Any suggestions on how to overcome these errors would be greatly appreciated.

Thank you!

Good = correct byte
Bad = bad byte

File size 9.2GB
Location Good Bad
1240401C: F7 F6
19F2C01C: 8F 8E
2573701C: F5 F4
4DD3201C: 6D 6C
613A401C: 57 56
737DE01C: 15 14
750F201C: 63 62
77AA101C: D9 D8
9876FE66: 30 31
A7FA401C: 53 52
BDD5501C: C3 C2
C2A2D01C: CD CC
1128E901C: DF DE
14030BD6C: 23 22
14334C01C: 13 12
180BD901C: E1 E0
18242701C: 25 24
188CFE01C: 5B 5A
188D9C01C: 9D 9C
19280701C: 6D 6C
1B786E01C: FB FA
1C070401C: 7F 7E
1E1C7401C: 75 74
20941A01C: 57 56

File size 2.5GB
Location Good Bad
A90801C: 91 90
4578201C: 11 10
4CC9401C: 35 34
57FCA01C: B7 B6
7260501C: 15 14
767CFED6: 30 31

File Size 1.95GB
Location Good Bad
17FCC96C: 74 75
1A77201C: 9F 9E
5794C04C: 89 88

File size 9.1GB
Location Good Bad
1108BEC4: 9B 9A
40D5801C: 65 64
46DC401C: 0D 0C
48E5A01C: F3 F2
72E6201C: 1F 1E
93A7201C: 77 76
94CFB01C: 0F 0E
9949D01C: A1 A0
9DB6201C: 03 02
A4E3601C: FB FA
BB48701C: F3 F2
C79E501C: B7 B6
DF3D201C: E3 E2
E49C201C: 27 26
E4A5E01C: 03 02
F3FA601C: 1F 1E
12B26601C: 65 64
12D4D601C: 09 08
13230401C: D1 D0
1346D801C: 25 24
1407D901C: D3 D2
15377101C: 4F 4E
15B6D901C: 67 66
17A9C201C: 61 60
18BFEE01C: 41 40
19108801C: B9 B8
19AF5701C: D1 D0
1A314701C: AB AA
1ABB9301C: 63 62
1B108701C: 33 32
1DCE1201C: 8F 8E
1FD38F01C: AB AA
208C2501C: 37 36
21180C01C: 8F 8E
2286CB01C: C5 C4
Reply With Quote
  #2  
Old 16.10.2018, 15:44
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 82,983
Default

Toggled bits is 99,999% caused by Hardware issues. There is huge difference between browser and java (memory usage in technical view) and that can easily explain why it works in browser and hardware issues show up in java.


Did you check memory with memtest86 tool? Don't use/trust windows tool. In my testing for damaged memory it didn't find damaged memory while memtest86 did. Also memtets86 not finding any issue doesn't mean there is non, because even with good memory, memory corruption can happen , for example row hammer. so access pattern can also cause bit toggle.

This can also be caused by bitrot on disk or issues when connected to NAS.
do you download to internal/external/usb drive? did you test downloads to different locations? eg instead to external/usb, try to download to internal drive?

How much memory does your system have?

random locations of single bit toggles is very strong sign for either memory issues or bitrot on disk.

And just to make it clear. I'm not saying that JDownloader is error free, but I can tell that this sort of error isn't caused by JDownloader because either the downloaded file is already *damaged* which would require the damage to NOT detected by TCP and still single bit change or the decryption would be flawed which would result in reproducible error masks/positions and not random bit toggles.
I did encounter bit toggle errors myself with servers and desktop computers three times and two times memtest86 wasn't able to detect broken memory bar. So I had to remove all and do extended testing on different combinations until I found the damaged one.
Same for disks. Smart values were all good and yet I was able to detect bitrot on SSD thanks to hashchecks that run on every file.
__________________
JD-Dev & Server-Admin
Reply With Quote
  #3  
Old 17.10.2018, 13:57
malcor
Guest
 
Posts: n/a
Default

Yes, I tested with memtest86, on several different occasions. Also stress tested CPU, RAM, PSU and file transfer integrity between my local HDDs as well as to a NAS on a local network and to a remote server. That's when I discovered issues with Bitdefender, even when it was fully disabled.

During this testing I ended up replacing two HDDs (from 2x 2TB to 2x 4TB) and replaced the SSD for my OS.

Memory is 8GB. I tried removing two RAM DIMMs and alternated the remaining two DIMMs with the ones that I had removed. I thought that there may have been an unusual issue with one of the DIMMs but it made no difference.

If there is a problem with my RAM or hardware, then it would have to be an unusual fault, as all the testing shows no fault, and yet every large encrypted Mega download that I do has a CRC error.

I really like JDownloader, and it works great for non-encrypted downloads. Just wish I could get it to work with encrypted downloads!

I suppose there may be some sort of software/hardware conflict or compatibility issue on my system with JDownloader, if no-one else is having an issue.

Thank you for your suggestions and the reply.
Reply With Quote
  #4  
Old 17.10.2018, 14:20
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 82,983
Default

Can you please open the about dialog of JDownloader and tell me the 3 numbers shown for memory usage?
If you want we can take a look at this together via Teamviewer, just contact me via support@jdownloader.org

Then we can do some testing to find out if the issue happens while decrypting or already present when downloading the encrypted file. eg create hash of encrypted file and compare with several downloads and check if they are the same or not.
I can also provide simple test that just decrypts a file from disk, so we can run decryption several times
__________________
JD-Dev & Server-Admin
Reply With Quote
  #5  
Old 18.10.2018, 14:33
malcor
Guest
 
Posts: n/a
Default

Thanks for the help. I'll contact you privately.
Reply With Quote
  #6  
Old 18.10.2018, 14:43
Jiaz's Avatar
Jiaz Jiaz is offline
JD Manager
 
Join Date: Mar 2009
Location: Germany
Posts: 82,983
Default

support@jdownloader.org is private
__________________
JD-Dev & Server-Admin
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT +2. The time now is 14:30.
Provided By AppWork GmbH | Privacy | Imprint
Parts of the Design are used from Kirsch designed by Andrew & Austin
Powered by vBulletin® Version 3.8.10 Beta 1
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.