Add Analytic Quotient and another protected log

[jmmcd]

gplearn already has a protected division: np.where(np.abs(x2) > 0.001, np.divide(x1, x2), 1.) and protected log: np.where(np.abs(x1) > 0.001, np.log(np.abs(x1)), 0.).

Both of these are common but both introduce discontinuities which can be undesirable (see eg Keijzer; Ni, Drieberg and Rockett; Nicolau and Agapitos). Alternatives include:

• The analytic quotient (Ni, J., Drieberg, R.H., Rockett, P.I.: The use of an analytic quotient operator in
  genetic programming. Transactions On Evolutionary Computation 17(1) (2013)) is defined as: $AQ(a, b) = \frac{a}{\sqrt{1 + b^2}}$.

• Another protected log np.log(1 + np.abs(x1)).

Of course these would be of interest as alternatives, not replacements. If this addition would be useful I can generate a PR. Would these names be suitable:

• aq = def _analytic_quotient(x1, x2)
• cpl = def _continuous_protected_log(x1)

[trevorstephens]

Can you provide a link to the paper(s) please?

[jmmcd]

• Keijzer https://link.springer.com/chapter/10.1007/3-540-36599-0_7
• Nicolau and Agapitos https://dl.acm.org/doi/pdf/10.1145/3205651.3205773?casa_token=ibAlv6UdJ70AAAAA:VP78XazPwLbSCvgFZanwgr76pKH0qeoeFWjGF6L028EJpLr3Hc3jkyMZNEAvNN_t_ICN7ZsKE_Kq
• Ni et al https://dev.heuristiclab.com/trac.fcgi/raw-attachment/ticket/2915/06186815.pdf

Short summary: they don't like discontinuities.

[trevorstephens]

Thanks! 😄 Hope I'll have a chance to flick through these soon. If it's related to work you wish to do personally you should be able to use the custom functions to do what you need to but I think it's worth considering putting them in as a first class function set if there's enough research supporting it

[hwulfmeyer]

See #131 and #130

[trevorstephens]

I'm thinking of how to move on this after years of user demand. Initial thoughts are to have a (yes another API bloat addition) parameter where the user can set something like closure_method=('koza'/'smooth') or something like that. And still specify the default log, div in their function sets. But also allow specifying via log-koza or similar that if they want to mix and match.

[jmmcd]

closure_method is slightly ambiguous, as there could be multiple smooth log operators, for example. It does create extra complexity, as the code would have to deal with the closure_method setting being over-ridden by the individual operators. (Plus default values for both.)

I think having default operators and allowing the user to over-ride them one-by-one is fine.

Just my 2 cents.

[trevorstephens]

Yeah could be a bit excessive perhaps. All that parsing would be done up front though so the overhead wouldn't add anything much to processing time, just some more code in the parameter parsing/checking. Was kind of thinking about the printed outputs being clean and readable.

[trevorstephens]

Other ways to address the printed outputs though. But could lead to ambiguity. Are users allowed to use both types of division (for example) in one program? 😬

[jmmcd]

I would be in favour of putting the onus on the user here. If they specify two division operators, give it to them! Would require a lot of new code to prevent that.