I just set up a test using the seesaw component for an Adafruit ANO Rotary Encoder to I2C Adapter, the rotary encoder works fine but the 4 buttons as well as the center button don't respond.
If it helps here are the pinouts from Adafruit
https://learn.adafruit.com/adafruit-ano-rotary-navigation-encoder-to-i2c-stemma-qt-adapter/pinouts