Improve references by taking sections/slices into account

[YakovL]

Currently, the getReferringTiddlers method of store (1) (which is used to provide references (2)) uses links provided by Tiddler.prototype.changed (3) (through TiddlyWiki.prototype.reverseLookup (4)). The changed method recognizes wikiwords, bracketed links etc via the config.textPrimitives.tiddlerAnyLinkRegExp and config.textPrimitives.tiddlerForcedLinkRegExp (5) RegExps. The latter don't distinguish [[some tiddler##section]] link-like expressions which can be used in tiddler and other macros. This causes the situation when some tiddler's references don't show the tiddler where the transclusion is used (if there's a tiddler named some tiddler##section then its references list shows the macro-containing tiddler).

So, to fix this, either changed method or getReferringTiddlers method sholud be changed. It seems reasonable to change changed method by adding a helper inside it:

var getTiddlerName = function(linkText) {
    var pos = linkText.indexOf(config.textPrimitives.sectionSeparator);
    if(pos == -1)
        pos = linkText.indexOf(config.textPrimitives.sliceSeparator);
    if(pos != -1)
        return linkText.substr(0,pos);
    return linkText;
}

and use it each time before this.links.pushUnique (like this: this.links.pushUnique(getTiddlerName(...))). This way, tiddler.links will always contain only tiddler names, not sections/slices and the references lists (as well as missing or orphans lists) will be correct.

[1] https://github.com/TiddlyWiki/tiddlywiki/blob/master/js/TiddlyWiki.js#L511
[2] https://github.com/TiddlyWiki/tiddlywiki/blob/master/js/Commands.js#L75
[3] https://github.com/TiddlyWiki/tiddlywiki/blob/master/js/Tiddler.js#L126
[4] https://github.com/TiddlyWiki/tiddlywiki/blob/master/js/TiddlyWiki.js#L520
[5] https://github.com/TiddlyWiki/tiddlywiki/blob/master/js/Config.js#L192

[tobibeer]

Anyone picking up on your well documented requests, @YakovL?

[YakovL]

@tobibeer
Hi Tobias,
well, in 2.9.0 beta 3, @ericshulman has accepted a good portion of my requests, but that was long ago (and right after that, I added several other issues). I guess he has lots of things to do these days, so.. (actually, I don't read all the posts in google.groups these days since there's too many about TW5, so I don't know all the news). I proposed once that I get the rights to edit the core, so that I can apply those patches that I have tested thoroughly, but that discussion didn't get much further.

[tobibeer]

Mhhh, I hope Eric will pick up on those issues. No idea about the traction of TW2 these days. So long as there is one who is willing and able to maintain, even further enhance, I see no obstacles to doing so... even if still a competitive product to TW5, especially with TiddlySpace being all TW2 (still!)... and TW5 not being in the Templates on TiddlySpot... as far as basic users are concerned that don't build via node.js.

[YakovL]

By the way, I've created a fixing plugin. Changing core requires fewer lines and can be done by either adding separate RegExps for the Tiddler.prototype.changedmethod (instead of "hijacking" existing ones) [this way is simple and effective] or investigating config.textPrimitives.tiddlerForcedLinkRegExp and config.textPrimitives.tiddlerAnyLinkRegExp usages (I suspect that those shouldn't be changed globaly, but may be I'm wrong). So, here's my solution: the 2 RegExps are "temporarily hijacked":

var orig_BRP_changed = Tiddler.prototype.changed;
// this recalcs links according to config.textPrimitives.tiddlerAnyLinkRegExp and
// config.textPrimitives.tiddlerForcedLinkRegExp , so temporarily hijack them
Tiddler.prototype.changed = function()
{
    var brackettedLinkRE = config.textPrimitives.brackettedLink,
//      titledBrackettedLinkRE = config.textPrimitives.titledBrackettedLink,
        sectionOrSliceAddition =
        "(?:(?:(?:"+config.textPrimitives.sliceSeparator+"[^\\|\\n\\]]+)|"+ //::
           "(?:"+config.textPrimitives.sectionSeparator+"[^\\n\\]]+))?)",   //##
        tiddlerForcedLinkRegExp = config.textPrimitives.tiddlerForcedLinkRegExp,
        tiddlerAnyLinkRegExp    = config.textPrimitives.tiddlerAnyLinkRegExp;

    // hijack REs
    config.textPrimitives.brackettedLink = "\\[\\[([^\\]]+?)"+ // extra "?" is important here
        sectionOrSliceAddition+
        "\\]\\]";
        // core definition: "\\[\\[([^\\]]+)\\]\\]";

//  config.textPrimitives.titledBrackettedLink = "\\[\\[([^\\[\\]\\|]+)\\|([^\\[\\]\\|]+?)"+
//      sectionOrSliceAddition+
//      "\\]\\]";
        // core definition: "\\[\\[([^\\[\\]\\|]+)\\|([^\\[\\]\\|]+)\\]\\]";
    // recalc, as in the core:
    config.textPrimitives.tiddlerForcedLinkRegExp = new RegExp("(?:" +
        config.textPrimitives.titledBrackettedLink + ")|(?:" +
        config.textPrimitives.brackettedLink + ")|(?:" +
        config.textPrimitives.urlPattern + ")","mg");
    config.textPrimitives.tiddlerAnyLinkRegExp = new RegExp("("+
        config.textPrimitives.wikiLink + ")|(?:" +
        config.textPrimitives.titledBrackettedLink + ")|(?:" +
        config.textPrimitives.brackettedLink + ")|(?:" +
        config.textPrimitives.urlPattern + ")","mg");

    var result = orig_BRP_changed.apply(this,arguments);

    // unhijack REs
    config.textPrimitives.brackettedLink = brackettedLinkRE;
//  config.textPrimitives.titledBrackettedLink = titledBrackettedLinkRE;
    // recalc again
    config.textPrimitives.tiddlerForcedLinkRegExp = new RegExp("(?:" +
        config.textPrimitives.titledBrackettedLink + ")|(?:" +
        config.textPrimitives.brackettedLink + ")|(?:" +
        config.textPrimitives.urlPattern + ")","mg");
    config.textPrimitives.tiddlerAnyLinkRegExp = new RegExp("("+
        config.textPrimitives.wikiLink + ")|(?:" +
        config.textPrimitives.titledBrackettedLink + ")|(?:" +
        config.textPrimitives.brackettedLink + ")|(?:" +
        config.textPrimitives.urlPattern + ")","mg");

    return result; // in fact, there's no result, this is for possible future extensions
};

PS commented out stuff seems to be unnecessary in all cases. The point of this plugin is making stuff like <<tiddler [[tidName::sliceName]]>> be interpreted as a link to the tiddler and be shown in the references list.

@tobibeer , you can see some datails at [1]. Basically, I need some help with GitHub/Git and cooking process (or study them myself, which requires some extra time), but in terms of development yes, I can add some tested improvements. Not sure if I can manage the whole project (issues, stuff like TiddlySaver) though.

[1] https://groups.google.com/forum/#!topic/tiddlywikidev/Wan8E9Q9b1U