[Etshy]

Seems like hitomi.la is broken.

After a little investigation, they added AVIF file format in their sources tag, but not sure if that is what cause the problem.

In their sources, the code hasn't changed, so I guess the problem comes from their AVIF sources ?

[pspzockerscene]

Hi,

as always, please post example URLs.

-psp-

[pspzockerscene]

Found the issue and "fixed" webp handling.
I'm not really sure about this but it works for the URLs I've tested ...

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[damo]

They have slightly changed the code from the version I posted in Post https://board.jdownloader.org/showpo...8&postcount=16

IMO it still should grab the original .jpg/.png, although this could be a configurable option for the plugin, as some people might prefer the smaller webp versions. BTW they have added .avif as well as with webp there is a "hasavif: 1" entry in the <galleryid>.js file. Furthermore, I'd really appreciate a way to keep the original file names, e.g., "105_chapter10_8.jpg" instead of just "60.jpg".

Here is what has changed

 function subdomain_from_url(url, base) {
-        var retval = 'a';
+        var retval = 'b';

// ...
         var m = r.exec(url);
         if (!m) {
-                return retval;
+                return 'a';
         }

Here are the changed functions, further down is the full version.

  String subdomain_from_url(String url, String base) {
    String retval = "b";
    if (base != null) {
      retval = base;
    }

    int number_of_frontends = 3;

    Matcher m = SUBDOMAIN_FROM_URL_PATTERN.matcher(url);
    if (!m.find()) {
      return "a";
    }

    try {
      int g = Integer.parseInt(m.group(1), 16);
      if (g < 0x30) {
        number_of_frontends = 2;
      }
      if (g < 0x09) {
        g = 1;
      }
      retval = subdomain_from_galleryid(g, number_of_frontends) + retval;
    } catch (NumberFormatException ignore) {}

    return retval;
  }

  String url_from_hash(String galleryid, Map<String, String> image, String dir, String ext) {
    ext = isNotBlank(ext) ? ext : (isNotBlank(dir) ? dir :
        image.get("name").split("\\.")[1]);
    dir = isNotBlank(dir) ? dir : "images";

    return "**External links are only visible to Support Staff** + dir + '/' + full_path_from_hash(image.get("hash")) + '.' + ext;
  }

The best way to get Test Data is to open a gallery and set this breakpoint in the DevTools in common.js:65

'Assert.assertEquals("'+url_from_url(url_from_hash(galleryid, image, dir, ext), base)+'", url_from_url_from_hash("'+galleryid+'", mapOf("'+ image.hash+'", "'+ image.name+'"), '+ (dir ? '"'+dir+'"' : 'null') +', '+ (ext ? '"'+ext+'"' : 'null') +', '+ (base ? '"'+base+'"' : 'null') +'));'

This will log out assert statements that can be copy pasted into a test and then you only need this helper function to test the current code.

  Map<String, String> mapOf(String hash, String name) {
    HashMap<String, String> m = new HashMap<>();
    m.put("hash", hash);
    m.put("name", name);
    return m;
  }

    Assert.assertEquals("**External links are only visible to Support Staff**, url_from_url_from_hash("1760873", mapOf("7c3d8d36f3a73375b1194207d95127d62002f0ce94d1d9a6045c21a9bed2c78e", "105_chapter10_8.jpg"), null, null, null));
    Assert.assertEquals("**External links are only visible to Support Staff**,  url_from_url_from_hash("1760873", mapOf("7c3d8d36f3a73375b1194207d95127d62002f0ce94d1d9a6045c21a9bed2c78e", "105_chapter10_8.jpg"), "avif", null, "a"));
    Assert.assertEquals("**External links are only visible to Support Staff**,  url_from_url_from_hash("1760873", mapOf("7c3d8d36f3a73375b1194207d95127d62002f0ce94d1d9a6045c21a9bed2c78e", "105_chapter10_8.jpg"), "webp", null, "a"));

As you can see, you need to pass null for dir, ext, and base for the original image and "avif"/"webp" for dir plus "a" for base for the avif/webp versions.

This is the current working version:

  public static final Pattern SUBDOMAIN_FROM_URL_PATTERN = Pattern.compile("/[0-9a-f]/([0-9a-f]{2})/");
  public static final Pattern URL_FROM_URL_PATTERN = Pattern.compile("//..?\\.hitomi\\.la/");
  public static final Pattern FULL_PATH_FROM_HASH_PATTERN = Pattern.compile("^.*(..)(.)$");



  String subdomain_from_galleryid(int g, int number_of_frontends) {
    int o = g % number_of_frontends;

    return String.valueOf((char)(97 + o));
  }

  String subdomain_from_url(String url, String base) {
    String retval = "b";
    if (base != null) {
      retval = base;
    }

    int number_of_frontends = 3;

    Matcher m = SUBDOMAIN_FROM_URL_PATTERN.matcher(url);
    if (!m.find()) {
      return "a";
    }

    try {
      int g = Integer.parseInt(m.group(1), 16);
      if (g < 0x30) {
        number_of_frontends = 2;
      }
      if (g < 0x09) {
        g = 1;
      }
      retval = subdomain_from_galleryid(g, number_of_frontends) + retval;
    } catch (NumberFormatException ignore) {}

    return retval;
  }

  String url_from_url(String url, String base) {
    return URL_FROM_URL_PATTERN.matcher(url).replaceAll("//" + subdomain_from_url(url, base) + ".hitomi.la/");
  }


  String full_path_from_hash(String hash) {
    if (hash.length() < 3) {
      return hash;
    }
    return FULL_PATH_FROM_HASH_PATTERN.matcher(hash).replaceAll("$2/$1/" + hash);
  }

  String url_from_hash(String galleryid, Map<String, String> image, String dir, String ext) {
    ext = isNotBlank(ext) ? ext : (isNotBlank(dir) ? dir :
        image.get("name").split("\\.")[1]);
    dir = isNotBlank(dir) ? dir : "images";

    return "**External links are only visible to Support Staff** + dir + '/' + full_path_from_hash(image.get("hash")) + '.' + ext;
  }

  String url_from_url_from_hash(String galleryid, Map<String, String> image, String dir, String ext, String base) {
    return url_from_url(url_from_hash(galleryid, image, dir, ext), base);
  }

  boolean isNotBlank(String str) {
    return str != null && !str.isEmpty();
  }

[pspzockerscene]

Sorry I'm still not so much into js stuff.
I've tested your code and it doesn't seem to work.
Here is the random URL I've used for testing:
hitomi.la/cg/宇崎ちゃんと遊びたい！-日本語-1761639.html
The .jpg URLs which your code has returned were invalid, resulting in 403 errors instead of files.

Thanks for working on this - I for sure would not have found the time to properly fix this!

-psp-

[damo]

I've checked the urls they are correct, I assume that you have to set the correct referrer when requesting the image. When I opened the calculated urls directly it also gave me 403, but then I opened it via the gallery it worked and it was the same url.

If you open the gallery you can switch to the original by pressing "w" that should toggle the file type between jpg/png and webp.

[Etshy]

Sorry I was'nt available these days and didn't had time to look more into their codes etc.

When I posted the first post, links that had the new AVIF Source (in html) didn't worked.

Tried today, The comics are now downloaded (with or without the new AVIF extension) in WebP.

not really a problem for me as I installed the webP codec but might cause problem for others (I don't know if WebP can be opened without the webp codec and without using a browser)

Edit :
Seems like old links without webp or avif doesn't work
example : **External links are only visible to Support Staff****External links are only visible to Support Staff**
Jdownloader try to download the jpg/png and fail with "File not found" message
maybe the referrer error damo talked about.

little question, is there a way to see the exact download url jdownloader detect ?
When I try to right click on an image > open in browser (or show download URL) it returns the comic URL not the image URL.

Edit2 : I updated my js tampermonkey script to download hitomi images.
It works with all links I tested (without webp AND avif, with webp AND without avif, with both webp AND avif) and it download in png

I still don't know a thing about JAVA so I can't translate this to JAVA, but here is a link for my script **External links are only visible to Support Staff****External links are only visible to Support Staff**.

I use an img tag to download the image but I guess JAVA have many tools to do better than that.
The img tag crossOrigin="Anonymous" parameter is mandatory to download.


Hope this can help.

[pspzockerscene]

@damo
Are you sure they're checking the referer?
Why would they do this for .jpg images but not for .webp?

@Etshy
That doesn't help me.
To answer your question though:

Originally Posted by Etshy

little question, is there a way to see the exact download url jdownloader detect ?

Settings -> User Interface -> Downoadlink address display -> Move "Data" to the top or deactivate all other options

-psp-

[damo]

@pspzockerscene
I just verified it you need to include the Referer header, actually for both jpg/png and webp. My guess is that if you tried it from the browser, you might have had the webp already in the cache.
As for which url to use as referer, afaics it should be

**External links are only visible to Support Staff**

So with the updated algorithm, it should work again for jpg/png.

Any thoughts on my other requests in regards to using original names and choosing which image version to download?

[pspzockerscene]

Fixed - the issue was not caused by the missing referer.

Wartest du auf einen angekündigten Bugfix oder ein neues Feature?
Updates werden nicht immer sofort bereitgestellt!
Bitte lies unser Update FAQ! | Please read our Update FAQ!
---
Are you waiting for recently announced changes to get released?
Updates to not necessarily get released immediately!
Bitte lies unser Update FAQ! | Please read our Update FAQ!

-psp-

[damo]

Nice, but could you elaborate a bit more?

[Jiaz]

small misstake, he added protocol twice

[pspzockerscene]

Originally Posted by damo

Nice, but could you elaborate a bit more?

I was about to check it - I've just made it work with your changes before although I knew something else had caused the issue.

Thanks to Jiaz for re-checking it.

-psp-