[DukeM]

I tend to jump VPN servers every so often and I just wondered if I'm missing out on anything or possibly losing some data/files during the few seconds that I have no internet connection while switching.

The Linkgrabber Activity icons on the bottom right doesn't disappear or ends when I do so at least it continues but what happens to the files/links that JD2 was currently checking at the moment of switching? Does it skip them or does it somewhat pause crawling and wait for the internet to come back before continuing?

Thanks!

[raztoki]

it highly depends on many factors.. for example (I haven't covered all scenarios)
- the plugin itself? is login used? You could get multiple ip access violations, some services this triggers account sharing/abuse.
- Browser, if there is a request sent out over old IP and it changes thus lost, the crawler would most likely abort due to error (timeout), which throws exception, most plugins do not retry! they abort on first failure.
- Browser, if no requests where taking place at the time of change over it would most likely continue working. The only time this could still fail if the website had ip checking, as in tied to your session to an IP address additionally to or exclusively to.

I personally would advise not changing IP whilst completing any crawling, specially in complicated plugins, else lost time is involved when you effectively have to re-add link and start over.

To over come this whilst still progressing workflow, you could set up connection management rules and multiple network interfaces and or proxies to still have access to multiple IPs without interrupting the typical network wide VPN. You could do this based on host.

[DukeM]

Originally Posted by raztoki

it highly depends on many factors.. for example (I haven't covered all scenarios)
- the plugin itself? is login used? You could get multiple ip access violations, some services this triggers account sharing/abuse.
- Browser, if there is a request sent out over old IP and it changes thus lost, the crawler would most likely abort due to error (timeout), which throws exception, most plugins do not retry! they abort on first failure.
- Browser, if no requests where taking place at the time of change over it would most likely continue working. The only time this could still fail if the website had ip checking, as in tied to your session to an IP address additionally to or exclusively to.

Thank you for the clarifications!

For the website I specifically had in mind when asking this question, as far as I know, it doesn't use any JD2 plugins. Also doesn't require any account to use. I do have a Linkgrabber filter made by psp in use for it. And as I understand, JD2 just crawls the pages' html source for a specific direct link so there should be no limitations/restrictions when using the website. At least for now, haha.

Here's some example links that maybe could help if you want to check:

Spoiler:

**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**
**External links are only visible to Support Staff****External links are only visible to Support Staff**

Originally Posted by raztoki

I personally would advise not changing IP whilst completing any crawling, specially in complicated plugins, else lost time is involved when you effectively have to re-add link and start over.

Unfortunately, it's unavoidable for me to have to switch servers to do some other stuff on the same PC. Crawling these links could take hours depending on quantity.

To be fair, I know I'm causing this to myself and it's my fault I even have to worry about such trivial things but I can't stop. I just compulsively have the need to download everything.

Originally Posted by raztoki

To over come this whilst still progressing workflow, you could set up connection management rules and multiple network interfaces and or proxies to still have access to multiple IPs without interrupting the typical network wide VPN. You could do this based on host.

Thanks I'll look into it. Not sure if it'll work though because as I've observed, my entire connection seems to be unavailable whenever I switch. Like my VPN allows me to whitelist software/apps so they can use my internet connection directly and whenever I switch server, I still notice those programs to lose connection.

[raztoki]

Well with linkcrawler rules it could fail due to connection issue, and move onto the next in the queue. Depending on the time it takes to change the VPN service, will reflect on how many that fail.

You would have to setup your VPN a little differently with multiple interfaces, or utilise socks proxies (some VPN providers offer that service also) which is an easier solution.