Can ECCO be used for sentence similarity?

[mkovalova]

I would like to understand what words or tokens make the biggest impact on similarity score between two texts.

[jalammar]

That's really interesting. I'm not sure that can be done with Ecco, but likely the same method can work. Conceptually it could be doing a computation like:

• Embedding the two sentences (possibly with sentence transformers)
• cosine similarity on the embeddings
• backprop from the cosine result to the embeddings
• That would generate a saliency score, can probably be improved by multiplying with the input vectors (see: input X gradients)