#1
|
|||
|
|||
Hitomi.la plugin broken
Jdownloader reports plugin defekt for hitomi.la galleries since yesterday.
Last edited by damo; 01.01.2022 at 12:26. |
#2
|
||||
|
||||
wait for next plugin update
__________________
JD-Dev & Server-Admin |
#3
|
|||
|
|||
Unfortunately, it is broken again.
|
#4
|
||||
|
||||
Either cat/mouse or random change. Wait for next plugin update
__________________
JD-Dev & Server-Admin |
#5
|
|||
|
|||
Not here to complain, just to say I can confirm issue since today for me (last time I downloaded from hitomi was about one week ago)
As this thread is marked as Solved, should I create a new one ? |
#6
|
||||
|
||||
@Etshy: wait for another update
__________________
JD-Dev & Server-Admin |
#7
|
|||
|
|||
Sure
|
#8
|
|||
|
|||
Broken again :(
|
#9
|
|||
|
|||
Plugin still broken.
Last edited by Cesra250; 11.02.2022 at 00:08. |
#10
|
||||
|
||||
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#11
|
|||
|
|||
Hitomi stopped working since a few days (I didn't try to download for a few days and today every link are "offline" instead of detecting the pages.).
I'll try to see why, but I don't know sh*t about java, so I won't fix it, just try to help find the problem. EDIT : Here are my notes from quick look ProgressController argument used is deprecated (just saying, I don't thing that's the problem) I guess the "Offline Files" in Hitomi plugin comes from these lines Code:
decryptedLinks.add(createOfflinelink(parameter)); return decryptedLinks; Code:
final String js = br.getRegex("src\\s*=\\s*\"([^\"]+" + gallery_id + "\\.js)\"").getMatch(0); if (js == null) { logger.info("Seems like this is no downloadable/supported content"); decryptedLinks.add(createOfflinelink(parameter)); return decryptedLinks; } From what I see in galleries and reader page this script tag doesn't exist. On the Reader page, they load "*gallerie_id*.js" from jquery at this url : Code:
**External links are only visible to Support Staff** Code:
var domain = (/^dev\./.test(document.location.hostname.toString()) ? 'dev' : 'ltn')+'.hitomi.la'; Code:
var result = window.location.href.match(/\/reader\/([0-9]+)\.html/); var galleryid = result[1]; $.getScript('//'+domain+'/galleries/'+galleryid+'.js' ... example what I mean with "reader page" : **External links are only visible to Support Staff****External links are only visible to Support Staff** example what I mean with "galleries page" : **External links are only visible to Support Staff****External links are only visible to Support Staff** There are also some changes in method "copied" from their js Code:
private String url_from_url_from_hash(String galleryid, Map<String, String> image, String dir, String ext, String base) throws Exception { //NEW (JS) if ('tn' === base) { return url_from_url('**External links are only visible to Support Staff**+dir+'/'+real_full_path_from_hash(image.hash)+'.'+ext, base); } //ENDNEW return url_from_url(url_from_hash(galleryid, image, dir, ext), base); } Code:
function real_full_path_from_hash(hash) { return hash.replace(/^.*(..)(.)$/, '$2/$1/'+hash); } Code:
private String url_from_hash(String galleryid, Map<String, String> image, String dir, String ext) throws Exception { //CHANGE JS - not sure the "split" part is equivalent, guess the java part shoud be just "split(".")" too ? ext = ext || dir || image.name.split('.').pop(); //CHANGE JAVA (existing) ext = isNotBlank(ext) ? ext : (isNotBlank(dir) ? dir : last(image.get("name").split("\\."))); //ENDCHANGE dir = isNotBlank(dir) ? dir : "images"; return "**External links are only visible to Support Staff** + dir + '/' + full_path_from_hash(image.get("hash")) + '.' + ext; } Also it seems the "base" variable is always "a" and "dir" variable is either "webp" or "avif" now, but seems "webp" will always work. (from what I can see in JDownloader sources, it was null before) (see in reader.js, search for "url_from_url_from_hash(", the only calls without "a" as base variable That was a really quick look though, I hope I wasn't too confusing and this will help. Last edited by Etshy; 14.04.2022 at 11:42. |
#12
|
||||
|
||||
@Etshy:They have changed again, wait for next plugin update. Can't promise jpeg support as they move to webp and avif image format.
Please check again with next update. I'm sorry but they stopped jpg and only have avif and webp. in case browser doesn't support webp, they use javascript webp decoder
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 14.04.2022 at 12:29. |
#13
|
|||
|
|||
Yeah we already saw that they stopped jpeg support a few months ago and was transitionning to webp.
from their JS I guess they completely change from jpeg to webp as almost everything seems to use webp. Thanks a lot for the quick update |
#14
|
||||
|
||||
Wait for update and thank me once it's working for you I've updated plugin to prefer webp over avif, okay?
__________________
JD-Dev & Server-Admin |
#15
|
|||
|
|||
Okay I'll wait to really thank then ^^
For me webp is fine, no problem. |
#16
|
||||
|
||||
@Etshy: Update will be available in about 5 mins
__________________
JD-Dev & Server-Admin |
#17
|
|||
|
|||
Seems they changed something else about the url generation
All files appears as not Found and have 403 when i go to them https://i.imgur.com/JeBkhY5.png Example url generated by jdownloader : **External links are only visible to Support Staff****External links are only visible to Support Staff** Even on their reader page ( **External links are only visible to Support Staff****External links are only visible to Support Staff** ) the webp seems to be in 403, so that's a bit weird (their json for this manga show this manga have webp though.) **External links are only visible to Support Staff****External links are only visible to Support Staff** The subdomain seems to be wrongly guessed too I'll make some more research tonight. Edit : webp url from Jdownloader (not working) : **External links are only visible to Support Staff****External links are only visible to Support Staff** webp url from hitomi donwloader : **External links are only visible to Support Staff****External links are only visible to Support Staff** I found something about the "gg" variable, a part of the process that find the subdomain and first path part. there is a file "gg.js" that doesn't seem to be loaded in Jdownloader. It's just an object with with 3 properties/methods (it should be simple to remake them I think) Some process about finding a number and then transforming this number into a letter to generate the subdomain That should fix the problem with the subdomain And then there is another problem with the first part of the path (after the '/webp') in js Code:
function full_path_from_hash(hash) { return gg.b+gg.s(hash)+'/'+hash; } //gg methods s: function(h) { var m = /(..)(.)$/.exec(h); return parseInt(m[2]+m[1], 16).toString(10); }, b: '1649948402/' I guess this is not loaded, or it's loading some old script from their server The actual gg.js is here : **External links are only visible to Support Staff****External links are only visible to Support Staff** I guess the "engine" in HitomiPlugin is executing the js from the page it loaded ? so just loading the correct gg.js should be enough I think. Last edited by Etshy; 14.04.2022 at 17:42. Reason: More information |
#18
|
||||
|
||||
@Etshy: I will have to check again. My example links did work fine. I'm sure something I just have overseen
__________________
JD-Dev & Server-Admin |
#19
|
||||
|
||||
Plugin is already using the same one
__________________
JD-Dev & Server-Admin |
#20
|
|||
|
|||
I really don't know what's wrong with the URL generation whthin JDownloader.
But if you can exec the js script, why not call their "url_from_url_from_hash" directly ? Code:
if (haswebp == 1) { url = url_from_url_from_hash(gallery_id, picInfo, "webp", null, "a"); } else if (hasavif == 1) { url = url_from_url_from_hash(gallery_id, picInfo, "avif", null, "a"); } else { url = url_from_url_from_hash(gallery_id, picInfo, null, null, null); } Not sure if the following is good I wrote that from example in the source Code:
//create a variable containing the object image in js engine.eval("var image=JSON.parse(" + JavaObjectToJSON(picInfo)+ ");"); if (haswebp == 1) { engine.eval("var url = url_from_url_from_hash(" + gallery_id + ",image, 'webp', null, 'a');"); } else if (hasavif == 1) { engine.eval("var url = url_from_url_from_hash(" + gallery_id + ",image, 'avif', null, 'a');"); } else { engine.eval("var url = url_from_url_from_hash(" + gallery_id + ",image);"); } final String result = engine.get("url").toString(); |
#21
|
||||
|
||||
@Etshy: check again with next plugin update
__________________
JD-Dev & Server-Admin |
#22
|
|||
|
|||
It seems to be working now.
I'll try with more links Thanks a lot. |
#23
|
||||
|
||||
@Etshy: Thanks for the feedback and you're welcome!
__________________
JD-Dev & Server-Admin |
#24
|
|||
|
|||
after trying a lot of links, I git a few "File not found" issue.
but only a few pages per manga, not the whole chapter. I guess there is still somethong wrong, I'll keep the manga link that have issues and search why some pages doesn't have the good url Edit : I'm starting to think they have some kind of hidden/backend security. I re-added manga that had errors, and now it works fine (some "temporary unavailable" show but retrying a few times, it work) Got some weird page names too : https://i.imgur.com/GkaKRZn.png Werid names come after a "Temporary unavailable". it loose the page number somehow. Edit : found on HDoujinDownloader that Hitomi started rate-limiting sinces a few months. That should be the cause of the "Temporary unavailable" that occasionnaly lead to a "File Not Found" after some time. Edit : Oh and another thing, the name of the package is not quite right, it always containe "manga" or "doujinshi" at the start. If that part could be removed, it'd be great. Last edited by Etshy; 17.04.2022 at 10:56. |
#25
|
||||
|
||||
will be fixed with next update
__________________
JD-Dev & Server-Admin |
#26
|
||||
|
||||
Rate-limit in meaning of requests/time or overall number of requests? I will add rate limit this week and waiting for new feedback from you then, okay?
__________________
JD-Dev & Server-Admin |
#27
|
||||
|
||||
With next plugin update, new added links will have a request limit of 2 request/sec
__________________
JD-Dev & Server-Admin |
#28
|
|||
|
|||
We seem to be missing something else. It works for freshly imported gallery, but after a while I they start to all fail with 403.
There seems to be some time component, that doesn't affect the download url itself. Maybe a dynamic cookie or something? This could probably be defeated, by storing the gallery url with the package and retrieve the gallery.js if the imported timestamp is older than x-minutes. |
#29
|
||||
|
||||
@damo: the links are only valid for specific time, right now you should not collect and download later.
to support this I need to add more plugin code to refresh expired links I will check and rewrite to support refresh
__________________
JD-Dev & Server-Admin |
Thread Tools | |
Display Modes | |
|
|