Dynamic lookup can completely shadow Bindings in the REPL

[scolsen]

Consider the following in the REPL:

(Pair.init 1 2)

Should this evaluate to Dynamic.Pair.init or just Pair.init? Currently it does the former as we prefer and privilege dynamic code by silently looking for all symbols in the dynamic module first.

The question here is more philosophical—it's currently impossible to evaluate a Static pair in the repl without using it, or forcing its typing. Otherwise, the Dynamic Pair is always found instead. Is this the behavior we want, or should our lookup be slightly more sophisticated in this case?

[eriksvedang]

Hmm, I thought we always looked at static code first..?

[hellerve]

We have a flag for that in eval, which tells the evaluator whether to prefer dynamic or static code. The REPL will prefer dynamic code; whether that’s the right behavior I don’t know.

[eriksvedang]

OK, maybe it changed at some point. And perhaps it's leading to too many bugs and we should just wrap dynamic code in a (with Dynamic ...) instead? It will be somewhat painful in the REPL though, but maybe we can work around that somehow.

[hellerve]

Funnily enough, I think that flag was introduced somewhat recently by @scolsen to address bugs that were creeping in; I can’t seem to find the PR and/or issues it addressed anymore, though. We should probably look at that to see why it is there and why it’s needed before thinking of alternative approaches?

[scolsen]

If we find a sequence of lookups that makes sense, we can probably get rid of the special flags in the evaluator. I think something like the following might make sense in terms of lookup preference:

1. If the symbol is a known internal command/primitive (car, with, etc. (anything in the starting env) append Dynamic to it and look in that module first.
2. Look in local envs
3. Look in used envs
4. Look up the parent chain
5. Look in the global env

This would introduce a special case for commands/primitives, but remove the special case that currently causes this issue (appending dynamic to all forms when the evalDynamic flag is set (most cases) first and looking there). It would also mean that, in a context in which we have:

(use Dynamic)

we'd find Dynamic.Pair.init for the case in the issue, whereas if we had no use and only Pair.init we'd find the static init. In the case of (use-all Dynamic Pair) the init would resolve to a multisym and need to be determined by the type system.

This would mean that using dynamic functions in the Repl is a bit more tedious, as you might need to prefix them, but at least we can still resolve commands/primitives without prefixes.

[hellerve]

I guess you could also leverage profile.carp to always (use Dynamic) if you wanted to, making this a configuration rather than a design issue (that we could also document somewhere).

[eriksvedang]

Yeah that's an interesting idea.

[scolsen]

So, a recent fix required another lookup preference datum.

If we do expose the envs in carp, we probably should have a signature like:

find: Context -> LookupOrder -> Binding

For the moment, the way eval is set up allows us to be "smart" about it. e.g. we can prefer Local bindings when evaling lets, dynamic when evalling dynamics etc.

The tricky bit about this case though is that selecting the dynamic preference makes the static one totally inaccessible at the top level of the repl, whereas, contrarily, preferring the static still leaves the dynamic one accessible via qualification Dynamic.Pair.init.

I don't think we'll ever get rid of the lookup preferences, actually, but it does help to augment them with lists of bindings that reduce the scope of relevant matches (e.g. the let binding names for preferLocal)

[hellerve]

Thanks for that write-up, that makes sense. I’ll make sure to get your input when I get to work on exposing envs in Carp again (unless someone beats me to it, of course)!