#1
|
|||
|
|||
![]()
In order to avoid bans and restrictions from Instagram I think the global request limit could have a random range input feature. Crawling on one defined time limit will ultimately be detected and result in account restrictions, I've tried long limits and it still gives me account issues with Instagram. If the limit can be randomized in order to avoid automated activity detection and imitate normal user usage then I think it will be less likely and probably impossible for it to be detected. An option under Global request limit to select [ Random ] with two input boxes. For example the request limit between a user defined range [ 100 ] → [ 999999 ] keeping the requests as random as possible, the algorithm that jdownloader uses to randomize request will be crucial.
|
#2
|
||||
|
||||
![]()
I'am not planning to add any more "bot detection avoid" features for instagram since that would be a waste of time, leading to an endless "cat & mouse" game.
See also: https://board.jdownloader.org/showthread.php?t=92962 Adding to this: - So far, the idea that such a randomized delay would help is only a speculation - There are other tools, specifically made to download from Instagram which may do a better job. Example: github.com/instaloader/instaloader - JDownloader is open source. Feel free to implement/test this change yourself and if it helps, we can still officially add it. Source code: https://support.jdownloader.org/know...up-ide-eclipse
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#3
|
|||
|
|||
![]()
I'm almost certain it would work. If there are any experienced coders out there who want to give this a shot, try this, I think it will definitely work. Ive been inputting random numbers manually and it doesn't give me any Instagram issue but the moment I stop it, it detects it. It's long and tedious doing it manually especially when you want to crawl lots of accounts but this will definitely work.
|
#4
|
||||
|
||||
![]()
Sorry, I'am not willing to invest the time.
Also, as explained in other posts, IG has so many possibilities to detect tools like JDownloader. If they want, they will detect such tools and this will end up in an never ending and time consuming arms race. Therefore, as explained either conduct tests on your own or try other tools that are specifically designed to download from IG.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#5
|
|||
|
|||
![]()
Use eventscripter to automate.
|
#6
|
||||
|
||||
![]()
* this can help him with downloading but not with crawling.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#7
|
|||
|
|||
![]()
Not bad eh?
|
#8
|
||||
|
||||
![]()
Nice screenshots.
Here they are publicly visible:
Spoiler:
![]() ![]() Is that a mockup or did you actually implement said functionality yourself?
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#9
|
|||
|
|||
![]()
They are mockups but I'm hoping someone will come across this and be inspired to try it before I attempt to haha, I think the hardest part will be choosing the right algorithm to use for the randomizer, I've searched a few and it's tricky.
|
#10
|
||||
|
||||
![]() Quote:
Code:
new Random().nextInt(userDefinedRange) The range is user-defined and the above line of code returns a random value in the pre defined range.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#11
|
|||
|
|||
![]() Quote:
![]() |
#12
|
||||
|
||||
![]()
Okay I didn't get this at all.
It's a sketchy idea but it will probably work :D
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#13
|
|||
|
|||
![]() Quote:
Quote:
![]() Quote:
![]() ![]() ![]() Last edited by roddy; 05.02.2025 at 22:41. |
#14
|
||||
|
||||
![]() Quote:
- Even a more sophisticated "randomness" does not simulate normal users' behavior at all - The instagram website has much much more trackiong abilities: It can fingerprint the users' hardware, it can track mouse movements -> All of this is missing when you are using JD -> IG will always know that you are using download tools unless you are using something that is remote-controlling a real browser (such as "Selenium") No, you mis-understood me: All I said is: You could realize this via EventScripter script without changing the plugin. If you want this, go ahead and inform yourself about how EventScripter scripts work and write yourself a script: EventScripter subforum: https://board.jdownloader.org/forumdisplay.php?f=52 EventScripter help article: https://support.jdownloader.org/know...event-scripter No, not planned.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#15
|
|||
|
|||
![]()
Yes agree ig has plenty of trackers on users nowadays, but when it comes to simply fetching data from their servers on a single constant time limit then it will most certainly detect this and take action, I've had to recover my ig account from being suspended so many times I lost count, and like I've said adjusting the fetch times in a random manor can effectively stop detections and suspensions from happening, that's why I'm still able to use ig.
And yeah I got excited earlier today but then later I thought you might have just been referring to mgpai's post about the eventscripter idea, and I was correct haha to good to be true. But anyway, I will be definitely trying the eventscripter method now until there is an official ig plugin update to randomize the request limit. Thanks mgpai for the suggestion and thanks to you in advance if you ever decide to create this feature. |
#16
|
||||
|
||||
![]() Quote:
Quote:
I know I'm repeating myself but I highly recommend trying alternatives such as "instaloader", even if you only do it to see, which actions they have eventually taken to avoid account bans and also to see, how it performs compared to JDownloader. See: github.com/instaloader/instaloader
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#17
|
|||
|
|||
![]() Quote:
|
#18
|
||||
|
||||
![]()
It doesn't:
Hoster plugin (InstaGramCom.java): Code:
@Override public void init() { InstaGramComDecrypter.setRequestIntervalLimitGlobal(); } Code:
public static void setRequestIntervalLimitGlobal() { final int limit = PluginJsonConfig.get(InstagramConfig.class).getGlobalRequestIntervalLimitMilliseconds(); if (limit > 0) { Browser.setRequestIntervalLimitGlobal("instagram.com", limit); Browser.setRequestIntervalLimitGlobal("cdninstagram.com", limit); } } Code:
github.com/mycodedoesnotcompile2/jdownloader_mirror/blob/main/svn_trunk/src/jd/plugins/decrypter/InstaGramComDecrypter.java#L152 Code:
github.com/mycodedoesnotcompile2/jdownloader_mirror/blob/main/svn_browser/src/jd/http/Browser.java
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#19
|
|||
|
|||
![]()
Script:
Code:
/* ig random request limit trigger : new crawler job */ var min = 1000; var max = 9999; var int = Math.floor(Math.random() * (max - min + 1)) + min; callAPI( "config", "set", "org.jdownloader.plugins.components.config.InstagramConfig", "cfg/plugins/HOSTER/instagram.com", "GlobalRequestIntervalLimitMilliseconds", int ); note: can be used with any trigger or multiple event triggers. For e.g. interval, a download started etc. |
#20
|
|||
|
|||
![]() Quote:
Quote:
Quote:
I have absolutely no issues with downloading, it can be 0 with no problems, it's just crawling– that's the issue ig is just super sensitive to crawls for some reason. |
#21
|
||||
|
||||
![]()
I don't understand.
I will just re-phrase my last reply: I was just talking about how the randomization would be implemented in JDownloaders' code if I wanted to implement it. My sentence was unrelated to the script idea. I don't understand. Quote:
Fyi: If a previously crawled direct-URL expires, the hoster-plugin can internally call the crawler plugin again. This means, fresh crawl processes can happen at any point of time [yes, also "during"/before downloading] without you knowing. This will especially happen when you crawl a lot of items, leave them in your linkgrabberlist/downloadlist and crawl them a long time later. Code reference: Code:
github.com/mycodedoesnotcompile2/jdownloader_mirror/blob/main/svn_trunk/src/jd/plugins/hoster/InstaGramCom.java#L530
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#22
|
|||
|
|||
![]() Quote:
When I do start the downloads I disable my ig account in the account manager and log out of ig completely before downloading. This works fine with no issues. |
#23
|
||||
|
||||
![]()
Yes.
Quote:
I was just thinking out loud about what would need to be done to integrate the feature into the plugin. That stentence was completely unrelated to how the script solution could eventually perform. Quote:
Then such items will fail with error "skipped - account required". Also, the instagram plugin is already heavily optimized to really only perform logged-in requests when necessary so you should be good with just leaving your IG account enabled all the time [of course on your own risc]. In 99% of all cases, all necessary information for downloading is available after the crawl process (also the direct-urls to the media files) so the download requests will happen without login cookies.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download Last edited by pspzockerscene; 06.02.2025 at 15:39. Reason: Formatting++ |
#24
|
|||
|
|||
![]() Quote:
Quote:
|
#25
|
||||
|
||||
![]()
That was just to illustrate that no extra algo is needed to generate a random number since a Java function for this already exists aka "no need to reinvent the wheel".
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#26
|
|||
|
|||
![]()
Agreed, no need to "reinvent the wheel" but when a wheel breaks down or blows out it needs to be changed or fixed i.e. when the default randomizer algorithm is no longer sufficient or obsolete; then it needs a new wheel or tire i.e. a new randomizer algorithm. But we haven't even got close to such a point and may never, dependent upon how well the standard random algorithm can perform against ig's automated activity detection.
Last edited by roddy; 07.02.2025 at 05:09. |
#27
|
|||
|
|||
![]() Quote:
|
#28
|
||||
|
||||
![]()
@roddy: choose different trigger then. the script is executed once for each "new crawler job". Maybe change to interval and use something like 10 seconds and thus the script will be executed every 10 seconds.
__________________
JD-Dev & Server-Admin |
#29
|
|||
|
|||
![]() Quote:
![]() |
#30
|
||||
|
||||
![]()
@roddy: Thanks for the feedback
![]() ![]()
__________________
JD-Dev & Server-Admin |
![]() |
Thread Tools | |
Display Modes | |
|
|