#1
|
||||
|
||||
Issue with filtered links in LinkGrabber
After grabbing links you see, that x links have been filtered. You can choose to have those links being displayed, but nothing happens.
Before I provide logs, whatsoever… Could you please explain what the expected behavior should be and which criteria lead to filtering (such as duplicate links). Maybe my interpretration/expectation is not correct. |
#2
|
||||
|
||||
@StefanM:
The number disabled in front of a quick filter means how many links are *matching* this condition/filter in the current setup of filters When you check(to make them visible) then please know that they still may not show up due to other filters still filtering them out.
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 02.06.2022 at 18:27. |
#3
|
||||
|
||||
Do you mean *Already in Downloadlist*? If so, this view/filter means that a link is already in Downloads
__________________
JD-Dev & Server-Admin |
#4
|
||||
|
||||
Settings->Linkgrabber Filter->
Filter: do not process/add links to Linkgrabber that match any of the rules (except there is a matching View filter) Views: provide quick on/off filtering of links in Linkgrabebr that match any of the rules
__________________
JD-Dev & Server-Admin |
#5
|
||||
|
||||
In case you've got more questions about this,please just ask
__________________
JD-Dev & Server-Admin |
#6
|
||||
|
||||
Not sure, if I explained it good enough:
At the end or during LinkGrabber is grabbing, you get a popup window showing you the grabbing process (number of files grabbed so far) and next to that figure there is a funnel, and next to that you see the number of filtered links. Now, I would like to see what has been filtered. And no matter if there are other filters active or not: When I click to make them visible, there is no change. I don't see any additional links. Only the number of filtered links is reset to zero. Last edited by StefanM; 02.06.2022 at 19:57. |
#7
|
||||
|
||||
@StefanM: That is not supported.
Filtered links are *filtered* and either will go into the lower bottom "Restore" button (default on, Settings->Advanced Settings->LinkgrabberSettings.restorebuttonenabled)(allows you to restore/add them later on) or are gone for good. An idea would be to also add the *filtered* links as normal links into the Linkgrabber and just mark them as filtered, similiar to those *dummy* entries when a plugin error occurs
__________________
JD-Dev & Server-Admin |
#8
|
||||
|
||||
Quote:
__________________
JD-Dev & Server-Admin |
#9
|
||||
|
||||
Quote:
And I made a mistake: Actually, I was referring to the "Restore" button. And when I click on that button, the number of 'filtered links' is reset to zero. But I still don't see any files that were restored. That was the problem, I actually wanted to report. Sorry about the confusion... |
#10
|
||||
|
||||
Quote:
We've just recently talked about this. In the other case this happened because the filtered entries were folderwatch jobs(?) so upon "restore" nothing appeared in the linkgrabber which can be distracting. Also, wouldn't it be more consistent to simply add another view for filtered items instead of the "restore" button? Though I guess there is a reason why it has been done like that...
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download Last edited by pspzockerscene; 03.06.2022 at 13:27. |
#11
|
||||
|
||||
I will read/answer, no time yet
__________________
JD-Dev & Server-Admin |
#12
|
||||
|
||||
That would be much better :-)
|
#13
|
||||
|
||||
That said there must be a reason why it hasn't yet been done like that.
I've added this information/idea to our internal ticket.
__________________
JD Supporter, Plugin Dev. & Community Manager
Erste Schritte & Tutorials || JDownloader 2 Setup Download |
#14
|
||||
|
||||
__________________
JD-Dev & Server-Admin |
#15
|
||||
|
||||
Quote:
1. The restore feature and the automatic filtering that comes along with it - when not used directly after crawling has finished, i.e. before any other action - will lead to automatic deletion of those dupes. If this is the intention, that feature at least shouldn't be active by default. 2. As already reported, dupes among links in the LinkGrabber pane are not being treated the same way as dupes in the Download pane. 3. One reason - as you already assumed - why filtered links do not show up after restoration, can be other active filters. (Had this case myself) |
#16
|
||||
|
||||
Quote:
The links *behind* restore button are NOT stored across sessions and that's intended behaviour. What exactly do you mean? Do you mean 1.) enable filter -> add link -> link is filtered to restore button 2.) disable filter -> add link -> doesn't show up because it's already in restore button? if so, yes this is a bug. If you mean something else, please explain better
__________________
JD-Dev & Server-Admin |
#17
|
||||
|
||||
Quote:
Quote:
No it can't. When restoring links, they are no longer checked against your filter rules because you actual want to *restore* them. The whole process of filtering does not happen again else you would have to restore it again and again and again, depending on how many filters do match on the link.
__________________
JD-Dev & Server-Admin Last edited by Jiaz; 11.06.2022 at 08:49. |
#18
|
||||
|
||||
Would it be an idea to have an additional window to see whats behind the restored button? and which filter it matched against (first or all matches), kinda like the test button within rule creation? So you don't actually have to restore ALL behind it, to see whats actively there. Then a function to retrieve desired (selected) links from that window or all button pull them out?
Why I propose this, is specially from (heavy) users who crawl multiple links(jobs) concurrently or basically scrape large components of websites which then return subsequent links(jobs), curiosity to see whats behind filtered links vs restoring in actual linkgrabber then having manually remove them post is annoying! Or the easier method deleting all links, and re-adding all jobs. raztoki
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#19
|
||||
|
||||
@raztoki: instead of making the restore button more complex,eg showing the links *behind it*, I would prefer to include the *filtered* links as dummy within the list, so you can see/work with them directly (eg delete, or retry without filter or keep for later (now part of the list)). similar to the *linkcrawlerretry* links. that way the dummy link could show which rule matched (eg in comment) and easier to see what got filtered
your opinion on this? of course an additional window would be possible but just putting them into the list with all others, you can better work with them, eg quickfilter on right side or move/retry/delete/keep.
__________________
JD-Dev & Server-Admin |
#20
|
||||
|
||||
Guess there are pros and cons to everything in life and this program isn't going to escape that!
One of the only issues is with power users, they can have hundreds of thousands of links already within linkgrabber, "finding them" would/could be needle in hay stack situation (like it is now) if you hit that restore function. If you design it with that in mind, making it easy would be nice. I assumed clicking a button & new window appears as its fast and current design doesn't have to change, peoples custom UI layouts still work. You can quickly and efficiently see the results either 'select, restore, clear, close' with ease. Should be quick to implement in the scheme of things. The tables/gui already exists within test functions in package customer && linkfilter, just don't have to run jobs, just display exisiting data held in memory. One would need to store which rule it matched against and also display that data, based on my previous post. With a customs views filter; would this require you to deselect all view checkboxes && select 'restore' views rule? As in just show the filtered results. If you just wanted to view filtered links within linkgrabber and/or apply additional view rules it would be nice, and very powerful overlay. The main downside to this would be additional rules hiding results, thus requiring users to disable multiple rules (if they exist), and an additional minor issue could be the support requirement this would cause (say very minor). Personally I see with long lists one would need to scroll and toggle the checkbox to identify if you didn't have like a colour identifier behind the filtered view, else how would you know, without toggling checkboxes while scrolling. or do we have a different state, and include all links again! and like offline or dupes, they just don't add to the downloadlist unless user interaction (ask window) or user default actions (remembered state). Different colour shading in table view do easily can be identified. or offline package equivalent, then the DownloadLinks wouldn't be within there proper packages etc. (early day JD design) those are the current frame works we are using currently to previously from current JD2 back to older versions
__________________
raztoki @ jDownloader reporter/developer http://svn.jdownloader.org/users/170 Don't fight the system, use it to your advantage. :] |
#21
|
|||
|
|||
Hi JD team,
Filtered links are not showing up as 'clear filtered links' in Linkgrabber anymore (since a few updates ago). I have set rule in linkgrabber filter as: downloadURL contains .*\.(jpg|jpeg|png|gif) and it used to collect into filtered links before a few days ago. Log: 19.06.22 02.08.35 <--> 19.06.22 02.13.32 jdlog://5001035302851/ Please have a look. Regards. |
#22
|
||||
|
||||
@reddevil: Thank you so much for the report. Issue was caused by a recent change of mine and only happens in filtered situation. will be fixed with next core update later today
__________________
JD-Dev & Server-Admin |
#23
|
|||
|
|||
@Jiaz, it's fixed now, Thanks.
|
#24
|
||||
|
||||
@reddevil: Thanks for the feedback and thank you for the report
__________________
JD-Dev & Server-Admin |
Thread Tools | |
Display Modes | |
|
|