Piggydb V5.0-dev2 released
Posted: August 14, 2011 Filed under: uncategorized 25 CommentsFinally, the most important feature in V5.0 has arrived!
This release introduces Tag-Fragment, which allows you to use a knowledge fragment as a tag. It is simple to say, but it enhances the flexibility and expressiveness of Piggydb quite a lot.
In comparison to the impact on the internal model, the change of the user interface is trivial, just one magical button:
If you activate the “As a tag” toggle button when creating/updating a fragment, the fragment will be created/updated as a tag:
↓
You may notice the tag icon at the head of the tag-fragment title instead of a number link.
The tag-fragment page shows the information of both tag-side and fragment-side of a tag-fragment.
You can attach an “as a tag” attribute to a fragment anytime you feel necessary and remove it from a fragment just by deleting the tag:
I have been thinking about this update since last year, and finally released it as V5.0-dev2. Altougth it may seem like a trivial update, I expect that it will change the process of knowledge creation dramatically.
The releases of the V5.0-dev branch are experimental. So the default download remains version 4.23. If you want to try out V5.0-dev2, please download it from the links below:
Piggydb 
Hello Daisuke,
Congrats on the new dev-release, but….
I fail to see the real advantage of this change as it seems that the only real difference with the current system is that I can now add information to a tag.
That a tag is also a fragment is, as far as I can see, the same as my current solution in making ‘organizational’ fragments. Other than that, it seems to me that this actually complicates working with the datamodel (although I guess you can still use tags in the ‘old’ way as well).
So I fail to see how this will help me in managing information better then version 4.x. But maybe you can give a better example of an information structure utilizing this change ?
Regards,
Hans Peter Willems.
Hi Hans,
Thank you for your comment.
I think there is the advantage of the new model in terms of both “model” and “process”.
>the only real difference with the current system is that I can now add information to a tag.
The new version of a tag (a tag-fragment) can have not only information, but also relationships (or links) to other concepts with which it is closely associated.
The screenshot of the tag-fragment page (“Social Networking Service”) in this entry would be a good example. The tag “Social Networking Service” has the relationships to “Social Links”, “User Profile”, “Privacy” while it classifies the subordinate things as a tag: “Myspace”, “Orkut”, “LinkedIn”, “Facebook”, “Twitter”.
If there isn’t any difference between links and taggings, the tag-fragments is the same as your “organizational fragments” as you wrote, but there are differences, or at least I’m going to update Piggydb to make the difference more clear (ex. making the direction of a link optional).
In terms of the “process” of knowledge creation, I wrote a little about it before. With the capability of Tag-Fragment, you don’t need to make all the concepts as tags at first. In the course of growing your knowledge, only concepts you feel are important in your database should be turned into tags from fragments.
Best regards,
Daisuke
Thanks for your elaboration, this makes things a little bit more clear. I will have to see how I am going to use this functionality in my own use of PiggyDB.
However, what I was hoping to see in v5 was that ‘tags’ would gain functionality that would help me to organize fragments ‘within’ a certain hierarchy. As it is now, tags are still grouping fragments on a global level but not on the level of fragments in a hierarchy. I somehow was looking for that functionality in your examples above, but didn’t see it (as I think it’s not there).
Is there any chance that you are going to leverage tags for organizing sub-levels in a hierarchy?
Hans Peter.
I don’t quite understand your idea of tag functionality: “organizing fragments within a certain hierarchy”. Could you explain it in more detail or give me an example?
> Could you explain it in more detail or give me an example?
Sure can do 😀
The most logical use for tags is to categorize things. However, this categorization is currently global: I can see ALL fragments that are tagged (for example) as ‘research paper’. But I can not see ONLY the fragments that are tagged ‘research paper’ AND are linked under a certain fragment like the author of those papers.
The problem that emerges in my use of PiggyDB is that I have fragments that have lots of linked fragments below them that have different tags, but no simple way to ‘group’ those linked fragments by their tags. For example; a fragment for a specific researcher has links to research papers, books, participated projects, universities and departments, etc. As I described before, I now have to make intermediate fragments to organize those lists. These organizational fragments are complicating my data-model and are a pain-in-the-neck to setup for each ‘researcher’ (in this example).
The solution could be simple and beautiful to my opinion: PiggyDB should be able, while rendering the hierarchy-view for a fragment, to collect the tags used on the connected fragments and then render those tags similar as a fragment in the hierarchy and thus grouping the connected fragments by tag.
This in combination with your current implementation of the tag-as-fragment would be grand.
Hans Peter.
OK, I got it. Thanks for the explanation.
Why not use a tag for a specific researcher? Then you can see the fragments that are tagged with both a specific researcher and “research paper” using the filter feature.
In my experience, a person’s name or proper name is a good candidate for a tag.
>I have fragments that have lots of linked fragments below them
In that case, you should think about using tagging instead of fragment-links. Fragment-links are more suitable for physically (or structurally) close relationships.
>Why not use a tag for a specific researcher? Then you can see the fragments that are tagged with both a specific researcher and “research paper” using the filter feature.
In my experience, a person’s name or proper name is a good candidate for a tag.
That would mean that most of my fragments would go into tags instead, and I don’t see the tag-hierarchy as an alternative to the actual data-model. The ‘researchers’ are just an example, I have this model with almost every other aspect of my data.
Besides that, having everything grouped under a researcher-tag instead of a researcher-fragment doesn’t do anything towards my described problem; I still see the whole list of connected items either way with no clear organization into specific sub-topics (where tags actually should do their job).
Filters are also not an option because either I have to build the filter every time I need it OR I end up with a very huge list of pre-defined filters that are actually replacing the linked hierarchy. So I don’t see that as an option either.
>I that case, you should think about using tagging instead of fragment-links. Fragment-links are more suitable for physically (or structurally) close relationships.
That is my exact point; those items DO have a very close relationship, so they are linked instead of tagged. I want/need to see those relations when looking at any particular fragment. The point is that a certain fragment could (and in my case many fragments DO) have lots of close relationships. I store EVERYTHING I know and work with inside PiggyDB just for that particular feature; to find, and document, all those interwoven relationships between loads of seemingly unrelated fragments. THIS is where PiggyDB actually shines. The only other program that can do this is PersonalBrain and that is, besides being commercial, slow, graphically bloated and not even as good as PiggyDB in most other areas.
I have, over the last few months of using PiggyDB, stored close to 4000 fragments (this is just from daily manual input). This count is already being skewed by lots of fragments that doesn’t hold any information but are needed for organization of hierarchies (as described before). Finding PiggyDB was close to my dream come true for managing my knowledge. PiggyDB is already suited (mre then most alternatives) in holding very big datasets but seriously need ‘something’ to manage this information better inside hierarchies.
Hans Peter.
>Filters are also not an option because either I have to build the filter every time I need it
For example, you select a main category tag (ex. a specific researcher) jumping to its page, and on the page, there is the “Related Tags” section in the sidebar. You can regard the related tags as sub-categories (or sub-topics, sub-group as you wrote). The list should contain the tags such as “research papers”, “books”, “participated projects”, etc, if you have done tagging correctly. Then you select one of them and click the [+] button of it to drill down the fragment list.
I think, basically, your problem could be solved in the above scenario, without too much effort (just one click actually). And you don’t have to save the filters.
Or am I misunderstanding your point?
> Or am I misunderstanding your point?
I don’t think you misunderstood. Your suggestions are certainly valid in some scenarios. However, I need to bring some sort of organization into my data-model because it is really big and diverse. For that reason I don’t like the ‘related tags’ menu at all because that way I need to remember how I categorized (tagged) stuff when I’m looking for something. That is exactly what I don’t want to do. So I have the related-tags menu always collapsed.
Also, relying on tags instead of links negates the possibility to traverse the data by following links. And that is the biggest thing fore me in using a tool like PiggyDB; when I look at at a certain fragment I can see the context of that fragment by seeing all the linked items to it (both up and down the hierarchy).
But don’t sweat it, the way I currently do it (with organizational fragments) does the job although being a bit cumbersome at times. Nevertheless, having linked fragments in a hierarchy automatically grouped by tag would seriously be a killer feature 🙂
Hans Peter.
I have begun to understand your data-model, a hierarchy that has concepts or categories in middle of it to group sub-fragments. With it, you almost don’t need tags, right?
I think that this structure was designed to fit the way you browse your knowledge. You want to view your hierarchy in a format like the fragment-tree view.
Basically it is no problem at all and I don’t want to impose any rules on users, but this kind of data-model might not suitable for Piggydb (at least the current version). If you normalize your data-model separating how you browse your information from the conceptual structure of your data-model, then Piggydb would show its ability. For example, distinguish between is-a relationships and has-a relationships, map the former to taggings and the latter to fragment-links.
As I wrote before, I think the current fragment-link feature is confusing especially with taggings. It should be improved to make the difference more clear.
I still feel the need to make things a bit more clear:
> I have begun to understand your data-model, a hierarchy that has concepts or categories in middle of it to group sub-fragments. With it, you almost don’t need tags, right?
Not specifically. I want to have only information in my datamodel and rather NOT use fragments as ‘categories’. However, there is currently no way in PiggyDB to use tags as a mechanism to group fragments WITHIN the scope of the higher-level fragment that they are connected to. Currently tags only group on the global level of the whole database. This is fine if you have just a small amount of fragments under a specific tag, but as soon as a tag holds hundreds or more related items it’s organizational model fails miserably. Especially if those hundreds of fragments are each related to different things.
> I think that this structure was designed to fit the way you browse your knowledge. You want to view your hierarchy in a format like the fragment-tree view.
Not necessarily. I don’t care much HOW the relations are presented (I actually would like to see a graphical network view). But I do need to be able to traverse the data-structure.
> Basically it is no problem at all and I don’t want to impose any rules on users, but this kind of data-model might not suitable for Piggydb (at least the current version).
I still think you missunderstand my use of PiggyDB; PiggyDB is a perfect fit for my datamodel. It is still the only tool I’ve found that can manage all the relations between my knowledge fragments.
> If you normalize your data-model separating how you browse your information from the conceptual structure of your data-model, then Piggydb would show its ability. For example, distinguish between is-a relationships and has-a relationships, map the former to taggings and the latter to fragment-links.
Again, this is exactly what I do; my fragments are of the form ‘has-a-relationship’. I only need to introduce fragments that ‘are-a-relationship’ (in fact my ‘organizational fragments’) because PiggyDB does not facilitate this kind of organization with it’s tag-model.
If PiggyDB would facilitate using tags as filters WITHIN a certain domain of relations (in PiggyDB being mainly a hierarchy) then I could get rid of those pesky ‘are-a-relation’ fragments 😉
I get the feeling you don’t seem to see the scenario where a specific fragment has tens or even hundreds of relations and you need to group those related fragments WHILE looking at this main fragment and all it’s related items. I mean, isn’t that the main goal of PiggyDB, to see how a certain fragment relates to the surrounding information and by being able to see that, to find new insights in the accumulated information. That is certainly what I use PiggyDB for and apart from this organizational issue it performs wonderfully for me.
Hans Peter.
OK, let me explain the intention of the Piggydb model.
First of all, taggings and fragment-links have common in their structure: A => B. Thus, in terms of traversing your fragment network, there’s no difference between them other than how they are presented in the UI.
With this in mind, fragment-links are intended for a relatively small number of relationships, and taggings are intended for a large number of relationships. This intention is reflected in the UI.
>but as soon as a tag holds hundreds or more related items it’s organizational model fails miserably.
This is not true because a fragment can have multiple tags so that you can drill down through your fragments with a filter.
>I get the feeling you don’t seem to see the scenario where a specific fragment has tens or even hundreds of relations and you need to group those related fragments WHILE looking at this main fragment and all it’s related items.
Maybe so, but in order for me to understand your problem correctly, I need concrete examples. Could you give me some of the examples where a specific fragment has tens or even hundreds of relations?
> With this in mind, fragment-links are intended for a relatively small number of relationships, and taggings are intended for a large number of relationships. This intention is reflected in the UI.
OK, this explains a LOT. But I also think it shows a flaw in your design philosophy. Because when you look at a very large data-set of dissimilar but related items, most relations will be very specific to narrow scoped domains of information (i.e. links between them) and on average a small amount can be categorized by tagging. This is because tags are not specific to the scope in where they are used. So to use tags in the way you describe, but with a large and complex data-set means that a large and complex tag-taxonomy is needed as well. And this I think completely negates the utility of the program.
> Maybe so, but in order for me to understand your problem correctly, I need concrete examples. Could you give me some of the examples where a specific fragment has tens or even hundreds of relations?
Well, I’ve given examples before; I have inter-related information on researchers, projects, papers, books, concepts (mostly links to pages like Wikipedia), institutions, events, notes, quotes. There are relations from each type of item to other types, but also links between same-type items; concepts can link to other concepts, papers link to other related papers, projects link to other related projects and so on. And this is just a small part of my database, those things are related to points of interest, non scientific books, movies and other media, and so on.
So a ‘researcher’ could link to several projects, all papers they have written, all papers they did peer-review on, faculties they worked with, books they have written, lectures they’ve given, symposiums they’ve attended, etc. And I think it’s clear that each and every of those linked items are linking to other researchers as well as ‘other’ items of information.
I hope you can see that you can not build such relational information by tagging. I’ve been thinking about your suggestion of moving ‘researchers’ into tags, but I think it’s clear from this example that I then have to move projects into tags as well, and papers also, and…… I think you can see where this is going.
All in all I’ve come to the conclusion that having fragments act as a tag is not a good idea as it makes things worse; it promotes the effect of moving large parts of the actual data (the fragments) into the tag-hierarchy, thereby breaking (to a certain extend) the power of linking fragments on a scoped level (when you make something a tag, it’s no longer scoped). Now, I can leave that functionality alone, but when it means that ‘other’ functionality will suffer from it, then it’s bad.
What I fear from your reply though is that PiggyDB will emphasize more on using tags and less on the relational links. What would be sad about that is that there is a whole assortment of information and notes management software available that uses tags to catalog things (some even smarter then PiggyDB) but there is NO OTHER alternative (besides the commercial PersonalBrain) that has a focus on information-networks.
Hans Peter.
I just came across another good example:
I’m now setting up several corporate software projects inside PiggyDB to track everything around them. For each project a very elaborate set of requirements (among many other things) are to be described. There are 71 categories for the requirements. These categories are prime candidates to be utilized as tags. However, to be able to find all requirements specific to a project AND a requirement-category, I now have to make each project a tag (otherwise I can’t filter it).
OK, so let’s say I make projects tags as well, now for each requirement there are related user-cases, test-scenarios, examples and online information, related development tools, data-model descriptions, linked documents and so on. So now the problem cascades down to each requirement and to mitigate this problem I now will have to make all the requirements into tags as well (as I need filtering to find specific related items). So slowly but surely I’m moving almost ALL my data into the tag-tree and the complex hierarchy I want to prevent in my fragments-view is now starting to show inside the tag-tree.
Imagine a tag-system where thousands of items have been promoted to tags to be able to manage them as data-sets. Soon you are forced to think about yet another way to manage such a large set of tags in a way that it still functions.
Very, very good example. I think I finally understand your point. Thank you 😉
In that case, “organizational fragments” are naturally useful for grouping sub-fragments, and I think that it is important to distinguish between “grouping (locally)” and “categorizing”.
For example, a fragment of a specific project has a “requirement” fragment as an organizational fragment. And in addition to it, you might want to create a “requirement” tag if you want to view requirement-related fragments across different projects. The “requirement” fragment could be tagged with the “requirement” tag. These two entities have different roles.
You wrote before:
The “requirement” fragment can not only group sub-fragments, but also contain the project-specific information (ex. summary) which should not be placed in a tag. So I think organizational fragments are valuable to setup for each project.
On the other hand, tags are global concepts as you wrote, and should be used when you want to collect fragments across different fragment-link networks.
> Very, very good example. I think I finally understand your point. Thank you 😉
GREAT 😀
> In that case, “organizational fragments” are naturally useful for grouping sub-fragments, and I think that it is important to distinguish between “grouping (locally)” and “categorizing”.
I totally agree 🙂
> For example, a fragment of a specific project has a “requirement” fragment as an organizational fragment. And in addition to it, you might want to create a “requirement” tag if you want to view requirement-related fragments across different projects. The “requirement” fragment could be tagged with the “requirement” tag. These two entities have different roles.
Yes, but tagging those items as ‘requirement’ has very little value; I don’t gain anything by listing all requirements within all projects at once as there is no useful correlation between requirements of different projects.
> The “requirement” fragment can not only group sub-fragments, but also contain the project-specific information (ex. summary) which should not be placed in a tag. So I think organizational fragments are valuable to setup for each project.
The ‘problem with setting up these is that, because these organizational fragments (they are actually connection nodes) have to be unique for each project to be able to group WITHIN the project, I have to enter them again and again. We are talking hundreds for EACH project (only for managing the requirements of one project I have 71 nodes).
Thinking of this I came to another possible solution that might enrich PiggyDB’s functionality AND solve my problem: introduce a ‘node-fragment’ that does not hold any information (no title and no content) but can link up and down like normal fragments AND be tagged. I can already see possibilities in combination with your ner tag-mode as well.
Now here’s the use-case for this:
Per my previous example I would be able to setup organization-nodes by simply add a node into the hierarchy and tag it for categorization. Setting up loads of nodes this way would be just a few mouseclicks per node.
OK, so I tested this while writing this post because I remembered I can add fragments without entering a title. This works so I’m going to test this in combination with tags. Now if you could somehow make such fragments stand out somehow, that would be great. Another suggestion I’ve been wanting to make is ‘custom icons for tags’, that might be a solution for this 😉
This development in insight is fantastic, thanks for engaging in this 🙂
Thank you, too. It’s been really interesting 😉
Another interesting issue would be the process of knowledge creation. Your problem about setting up the same structure for each project came from the process you adopted. You introduced the NASA taxonomy to me before, and I think that kind of taxonomy also came from the same process.
I will write about this issue on this blog in the near future.
> I will write about this issue on this blog in the near future.
I’m looking forward to your ideas and insights on this.
To give some more input to you about my ‘model’ (I think this is important): The organizational model I’ve described so far is obviously meant to categorize large amounts of inter-related information that I gather for my projects and interests. The second and most important step in using PiggyDB is that I connect notes to all the collected information. Then I reference these notes (by linking) from other fragments in my database. This way I create a documented (in the note) connection between related items.
Example: When I read a research paper I document my insights and thoughts in notes that are connected to the research paper. Now when working on a certain part of a project, I can link relevant notes to, for example, a specific task or requirement for the project. The linked note in turn links to the research paper that instigated the note. This kind of linking goes on everywhere in my database and is the actual ‘knowledge’ that is accumulating around the collected information. But in the same way a youtube video or anything else can act as a documented connection and those in turn can have notes that connect them more granular to other concepts and stuff.
I am in fact using PiggyDB as a concept mapper, like you described in another blog-post 😉
Hans Peter.
Greetings Hans Peter and Daisuke
I had a similar response to the fragment-as-tag capability in 5.0.
Right now, we can mark a fragment as a tag, and it will show up on the tag list. If we drag it from the tag list on to another fragment, it behaves as a tag. Whereas if we drag its pointer to create a relationship, it performs as a context parent. As these functions ultimately combine, I’m hoping that the fragment-as-tag will appear in the wrapping row with the other tags, rather than stacked with the context parents. And that if we place the fragment-as-tag in a hierarchy, and expand it, Piggydb will attempt to show all fragments that carry the tag, indistinguishably from those that share the parent.
As of Dev 2, the fragments with the f-a-t as parent, and those with the f-a-t as tag, are different sets. I think the great benefit of the feature will come with the merger of these sets.
Regards
Jerome
Hello Jerome, and thanks for your insights.
> And that if we place the fragment-as-tag in a hierarchy, and expand it, Piggydb will attempt to show all fragments that carry the tag, indistinguishably from those that share the parent.
Maybe I didn’t understand you correctly, but my point seems to be the opposite; have the tag-as-fragment inside the hierarchy and when expanded it shows the fragments that share the tag AND are linked to the parent fragment.
I see no advantage in showing ALL fragments that share the tag inside a specific hierarchy. The focal point (for me) is this: there is, and needs to remain, a distinct difference between ‘globally tagged fragments’ and hierarchically linked fragments as they don’t share a common scope other then the actual tag.
Hans Peter.
Greetings Daisuke
As bugs are expected in a dev release, I don’t know if you’re interested in bug reports just yet, or where you’d prefer them logged. Just three, thus far.
1. If we activate the “reorder” handle, then go into a quick edit, the edit window opens and the line cursor appears, but the edit capability is locked out.
2. If we delete a tag while editing, we lose the edit changes.
3. If we edit and successfully update a fragment title, then create a relationship to the updated fragment, the confirmation screen shows the old title.
All Best
Jerome
Hi Jerome,
Thank you very much for your reports. Bug reports are always welcome. I don’t know why, but there are less bug reports than I expected.
I have posted them to the ticket system.
Best regards,
Daisuke
Thanks for logging them. Dev2 has been amazingly tight and responsive, like a production release.
Speaking of modal confirmation windows, I see you’re using two different types in Piggydb, as in “Create a relationship” vs “Delete a relationship.” “Create” is hand coded: showConfirmDialog in js.vm. “Delete” uses window.confirm(messages) and appears to be fired in the operating system.
The “Delete” modal window is a little nicer in my opinion because we can set it in Windows to snap the mouse pointer to the confirmation button, and thus complete an action in two unbroken clicks. Something to know if you should ever decide to align these routines
All Best
Jerome
Thanks for your feedback.
The modal confirmation dialogs used in such as “Create a relationship” are not implemented with window.confirm() because they need to show more detailed messages.
I didn’t know about the behavior of the standard dialog in Windows as you wrote, but it is a little meaningless to make the confirmation process too easy, isn’t it? If you feel the confirmation is not necessary, it would be better to provide an option to skip it.
Best regards,
Daisuke
Hi Daisuke
I believe Firefox deliberately turned off its cursor snap-to a few versions back because of the dangers of defaults too easily accepted. A policy that makes perfect sense when a mouse click is used to confirm a payment, it slows us down a little in a familiar local application used every day.
I’d probably keep these confirmation windows even if they were optional. But the cursor positioning does make for a smoother flow, particularly as tag and relationship operation sare low-risk and reversible. Fortunately, a little experimentation reveals that space bar also works to activate a default selection, so the user can cultivate a fluid motion without repositioning either hand.
Thanks for engaging!
Best Regards
Jerome