Feature request: Fragment-based filtering
Posted: February 2, 2014 Filed under: discussion 3 CommentsPiggydb user Novbert posted his thought on the problems of Piggydb’s filters on the issue tracker. I think his feedback is to the point and am beginning to think about removing or overhauling the current filter feature in the coming versions of Piggydb.
I’d love to hear from you on this topic, so please post a comment on this article if you have any opinions.
I don’t particularly like the way filters are integrated into the piggydb concept. Actually I think they are part unnecessary and part redundant. One could argue, that they are there as shorthands for any meaningful set of concepts, but we have tags for the very same purpose, rigth? Even if we argue, that there are useful sets of tags we want to leave untagged, that set should probably also be a concept in itself, thus filters should be fragments themselves, shouldn’t they?
Actually this way of thinking also works in the opposite direction. Fragments could be used as filters for the sum of tags assigned to them. Would that be the case, having filters as a separate entity would be totally unnecessary.
Let me tell an example. I’m building a DB on all types of fiction, movies, books, comics, etc. I categorize them based on genre, media, author, etc. Naturally I have tags like Movies, Books, and also Sci-fi, Fantasy. Though having a category for “Sci-fi movies” sounds rather intuitive, how would I do that in piggydb? I don’t want to set it up as a tag, as I already have a set of tags for this, so it would make my tag system redundant by assigning both “Sci-fi” “Movies” and “Sci-fi Movies” tags to all the relevant fragmens. But if I set it up as a filter, I lose the ability to make general notes on the subject. Of course I could solve the problem by creating a fragment called “Sci-fi movies” and assign all the specific movies as child fragments, but that would result an organized list of movies, while I’d like to see these as a set, while making general notes on them (e.g. : the history of Sci-fi movies, notable directors, etc.) I’d like to look at this summary fragment, and edit it, while browsing through all the specific movies.
Which brings me to my next point: the uselessness of the middle and right pane when focusing (clicking) on a non-tag fragment. If I do that, the fragment content appears on the left, leaving a useless view of all my fragments in the middle (which can be filtered by keyword, but not by tags), and a similarly useless right pane, without tag-based filtering ability.
I think it would make much more sense if it would work like this:
– Clicking on any non-tag fragment wouldn’t only show the contents on the left, but it would also make the middle and right pane behave as if I’d have set up a filter with all the tags assigned to that segment. Changing filter parameters would only affect the middle and right pane, the chosen segment would remain active on the left as long as I don’t click on another one.In the above scenario it would result this:
I could set up a non-tag fragment called “Sci-fi movies” and assign tags “Sci-fi” and “Movies” to it. If I click on that fragment, it would appear with all the details on the left, while all the actual sci-fi movies with tags “Sci-fi” and “Movies” assigned would appear on the right, with ability for further filtering based on both keywords, and tags. I could overview all of them, while looking at my summary fragment, and I could even try to find relevant fragments in other categories e.g. by removing Sci-fi from the filter criteria and add another related tag like “James Cameron”. All this, while still looking at my summary fragment called “Sci-fi movies”
Similarly if I’d click on any actual sci-fi movie (e.g. Avatar, with tags “Sci-fi”, “Movies” and “James Cameron” assigned), in the middle pane there would appear all the movies falling in exactly the same category (e.g. Alines, The Abyss and Terminator 2), and I could even try to find related concepts e.g. by removing Cameron from the tag filter criteria and inserting another director.)
As I see this way of working might as well make Filters as separate entities entirely unnecessary for piggydb, as users would have the ability to filter fragments based on tags (as it is right now), and fragments themselves would behave just like filters, thus if the user finds any important pattern/set in the database, he could create a fragment with the relevant tags assigned, making himself able to query that set later on at any time. That later step would even be unnecessary as any set of fragments with the same tags applied would behave like a filter for itself.
What do you think?
Piggydb 
First of all, I think that your insight into the problems of filters makes a very good point.
Since the introduce of tag-fragments, Piggydb has been heading toward a more minimalistic “everything is a fragment” model. Filters were introduced when tags were just like plain Web 2.0 tags, and as you pointed out, I also feel filters are not well in harmony with the current model. And in fact, I myself rarely use filters.
This kind of a combination of tags can be realized with hierarchical tagging. In the above case, you can solve the problem by creating a tag “Sci-fi movies” and attaching the two tags “Sci-fi” “Movies” to it. The fragments tagged with”Sci-fi movies” can be found by any of the tags: “Sci-fi”, “Movies”, or “Sci-fi movies”.
I’ve been thinking about an idea similar to yours about this. When a non-tag fragment is displayed in the left pane, it would be useful to display, in the middle pane, the fragments that share the some of the same tags with the focused fragment. I call this feature “Related Fragments”, which shows indirect relationships between fragments.
I agree that the feature of filters as separate entities would be redundant, so I will consider removing the feature and compensating it by making the list view (or the middle pane) smarter.
I’m glad to hear that you like my idea. To be honest I also barely ever use filters, I tend to use tag-based filtering all the time.
About your notes on the Sci-fi movies example: Though your proposed solution would solve my problems in the short term, I’m reluctant to introduce such redundant tags in my tag system. Generally speaking I think there’s no added value in adding a tag to mark a concept which is readily available as the intersection of two or more other concepts (otherwise I’d soon end up with tags like “Sci-fi Movies directed by James Cameron, featuring Sigourney Weaver in a lead role” which is definitely not the way tags are supposed to be used). I think it would be more intuitive if access to these intersections – or Related Fragments, as you’ve called them – would be simpler.
On a second thought what my problem really is, that piggydb currently offers very limited support to compare one part of the database to the other. Recently I’ve started using piggydb with this chrome extension: https://chrome.google.com/webstore/detail/split/ikjbemhhmfojpgaphgfogkpfibcjhfgp which makes me able to do that but I consider it rather a workaround than an actual solution to the problem.
To solve that, and the above mentioned “Sci-fi movies” problem, I think two functions would help a lot:
1) “Split screen” – Making me able to view details of a fragment on the left, while freely filtering other fragments on the right. Clicking on a fragment wouldn’t affect the already established filter criteria.
2) “Related fragments” – just as you’ve mentioned. Maybe not as a default view, but at least as an option/button on each fragment’s menu. If the fragment is in focus on the left, clicking this button would create a filter based on the tags assigned on the right.
Actually I think these two would work together well. Sticking to the original example:
– I’m browsing my Sci-Fi fragments (by chosing the tag “Sci-Fi”)
– I click on “Avatar” which is categorized as “Movies” and “Sci-fi”.
-> The fragment detail view appears on the left side, the filter criteria remains intact on the right, still showing all my “Sci-Fi” fragmens.
– Now I click on the “Show Related Fragments” button on Avatar
-> The filter criteria gets updated to show only “Sci-Fi” “Movies” on the right (even Avatar itself), while still showing Avatar on the left side.
– Now I remove the filter criteria “Sci-fi” from the right
-> All my “Movies” fragments appear on the right, while I’m still looking at all the details of the Avatar page on the left.
And at this point I can start looking for similar movies and linking them to Avatar, or tagging them if I feel so. I could as well remove the filter criteria “Movies” from the right side, replacing it with “Novels” and try to find similarities there.
On the other hand I’m also thinking about making the split screen a “full” split screen, making all the piggydb functionality available on both sides of the screen (like when I use that chrome addon I’ve linked), but I’m not sure how that would work out – maybe it would make the user interface too cluttered. Though that could be solved by rearranging the interface into a vertical layout… What do you think about that?
I am tinkering a bit with Piggydb, as I am looking for a replacement for BasKet which stalled at KDE3, and my favorite Wiki on a Stick that is on its way to discontinuation of development for some years now (unfortunately, pretty common in one-man-developement FOSS). So far, Piggydb looks extremely promising. To get to know it, I wrote a little knowledge base of its user interface, parts of which are undocumented, and that made me read the documentation base thoroughly.
Concerning redundancy of filtering, I can only confirm it from the standpoint of purity of concept. However, Piggydb is just a tool, so the true concept is its projected usage. I see filters as saved searches, and when the base is growing huge, one needs every help in finding fragments. Using tags, that is, concepts for this purpose makes tags themselves impure, or rather misused. Besides, no matter how do we approach tagging, possibility of tagging overflow is luring all the time, the more so as one pays more attention to structuring one’s knowledge base. We learn about it from almost any science, particularly from humanities, as the extent of conceptual superstructure might seriously challenge any basic influx of data, and the literary criticism is a very good example.
So, the only thing I feel is wrong with filters is impossibility of filtering by keywords. In other words, I’d like to see the filtering tool as another search tool with additional naming and saving. This, of course, can be an option added to the main search tool.
As for the third, the left frame or column, in the beginning it seemed to me as wasting of precious page space, once there is this very fine quick preview available. But I very soon learned to appreciate the possibility of viewing simultaneously entire single fragment and the list of other ones. Particularly in the initial phase of looking for regularities it is good to have as few limitations as necessary. So I’d prefer not having the list (or the tree) in the central frame automatically filtered by the chosen fragment tags, because instead of sparing few clicks it is often forcing few extra. I am regularly comparing the fragment to various sets of other fragments, so I am doing some searching in the central frame, having the focused fragment exposed in the left one. Of course, when one already knows one’s base hierarchy and has it built, then present behavior is perfectly OK, but as I got it, Piggydb is about finding a structure rather than documenting it, and about relation network rather than hierarchical tree.
Actually, the only thing I believe Piggydb is missing at this moment is graphical, vectorial presentation of relations network on demand, with fragments represented by their names (possibly links) interconnected by arrowed lines, maybe some sort of SVG. It would make a huge help in examining (and possibly even editing?) already established relationships. But again, bearing in mind probable number of fragments of any serious base, I am fully aware this might be too much to ask for. Pity, because this one feature would make Piggydb a killer app.