Fix: Revert Chrome upgrade

[rafaeljusto]

The latest stable release of Chrome has a breaking change that affected the bot flow to join a meeting:

Failed to join meeting and the UiCouldNotLocateElementException exception is
retryable so retrying

Related to:
#10

[noah-duncan]

Hi @rafaeljusto, just saw this fork. You're doing some good work here. This tweak got Google Meet working on Chrome 144: https://github.com/attendee-labs/attendee/pull/614/changes

I am waiting on our suite of end to end tests to be finished before merging the change to main, but if you want to try it and let me know if there are any problems, feel free to.

[rafaeljusto]

Ohh, amazing news! Thank you Noah, I will give it a go and will report back here.

[rafaeljusto]

Hey @noah-duncan 👋 Checked your changes, and the bot doesn't join the Google Meet.

Here are the logs:

web driver server initialized at port 35359
Waiting for the name input field...
Could not find name input. Timed out. Raising UiCouldNotLocateElementException
Failed to join meeting and the UiCouldNotLocateElementException exception is retryable so retrying
Chrome sandboxing is enabled
Using Chrome user data directory: /tmp/chrome_user_data_9slyzwy9
web driver server initialized at port 42857

[noah-duncan]

Hi @rafaeljusto, I checked out this repo and tried introducing the changes myself and it worked on my machine. I don't have permission to push to this repo and the changes aren't big enough to warrant forking your fork imo.

So just going to attach the two files I changed. Paste these in and rebuild your dockerfile and let me know if it works.

Dockerfile: https://gist.github.com/noah-duncan/fe31d5b2e9e009658db2d887ee2ca7e9
google_meet_chromedriver_payload.js: https://gist.github.com/noah-duncan/7487b1ffa59b3007264d9cb7f8a8a195

[rafaeljusto]

Hey @noah-duncan , still no luck with these changes. The bot joins the meeting, and it exits after around 30 seconds. The bot state remains "joining" during the entire period. It will attempt to join again after many minutes (once "Still waiting for captions button" attempts are done), but the same behaviour will happen.

Logs:

web driver server initialized at port 41167
Waiting for the name input field...
Google is blocking us for whatever reason, but we can retry. Element text: 'You can't join this video call'. Raising UiGoogleBlockingUsException
Failed to join meeting and the UiGoogleBlockingUsException exception is expected so not incrementing num_retries, but 1 expected exceptions have occurred
Chrome sandboxing is enabled
Using Chrome user data directory: /tmp/chrome_user_data_t8r7bale
web driver server initialized at port 50513
Waiting for the name input field...
name input found
Waiting for the microphone button...
Clicking the microphone button...
Waiting for the camera button...
Clicking the camera button...
Waiting for the 'Ask to join' or 'Join now' button...
Clicking the join button...
Waiting for captions button...
Still waiting for captions button... Attempt 30/600, elapsed 35s. Checking for denial/blocked patterns.
Still waiting for captions button... Attempt 60/600, elapsed 68s. Checking for denial/blocked patterns.
Still waiting for captions button... Attempt 90/600, elapsed 102s. Checking for denial/blocked patterns.
Still waiting for captions button... Attempt 120/600, elapsed 135s. Checking for denial/blocked patterns.
Still waiting for captions button... Attempt 150/600, elapsed 168s. Checking for denial/blocked patterns.
Still waiting for captions button... Attempt 180/600, elapsed 201s. Checking for denial/blocked patterns.
...
Could not find captions button. Timed out. Raising UiCouldNotLocateElementException
Failed to join meeting and the UiCouldNotLocateElementException exception is retryable so retrying
Chrome sandboxing is enabled
Using Chrome user data directory: /tmp/chrome_user_data_44h064zr
web driver server initialized at port 32869
Waiting for the name input field...
name input found
Waiting for the microphone button...
Clicking the microphone button...
Waiting for the camera button...
Clicking the camera button...
Waiting for the 'Ask to join' or 'Join now' button...
Clicking the join button...
Waiting for captions button...
Still waiting for captions button... Attempt 30/600, elapsed 34s. Checking for denial/blocked patterns.
Still waiting for captions button... Attempt 60/600, elapsed 67s. Checking for denial/blocked patterns.
Still waiting for captions button... Attempt 90/600, elapsed 100s. Checking for denial/blocked patterns.
Still waiting for captions button... Attempt 120/600, elapsed 135s. Checking for denial/blocked patterns.
...

It may be related to Google blocking the IP address, as you mentioned in Slack.

The use of the following config when creating the bot didn't help:

{
  ...
  "debug_settings": {
    "create_debug_recording": true,
  }
}

Full API request:

curl -i -X POST "https://example.com/api/v1/bots" \
  -H "Authorization: Token <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "meeting_url": "https://meet.google.com/<code>",
    "bot_name": "Meeting Bot",
    "recording_settings": {
      "format": "none"
    },
    "debug_settings": {
      "create_debug_recording": true
    }
  }'

[noah-duncan]

Hi @rafaeljusto You need to let the bot run to completion for the debug recording to get generated. If it encounters a UiCouldNotLocateElementException it will retry several times, so this can take a while.

If you want to prevent it from retrying, lower this number here:

attendee/bots/web_bot_adapter/web_bot_adapter.py

Line 663 in 26c34d5

max_retries = 3

Then we can see what the bot is seeing.

[rafaeljusto]

Yep, you are right! After the attempts, we got more data there:

debug_screen_recording_shot_M458Cdxcd5HurSXk.mp4

The long video appears that the bot wasn't accepted, but it was. Stayed in the call for around 30 seconds and exited.

[noah-duncan]

Ok, I think this screen is the key thing

We haven't seen this screen before. Do you still get this when you downgrade chrome to the original version? (134)

[rafaeljusto]

Do you still get this when you downgrade chrome to the original version? (134)

The original version is working fine:

debug_screen_recording_shot_mrxzFQrOWeo8nS9Y.mp4

[noah-duncan]

Hey @rafaeljusto I was just able to reproduce on my machine when testing on a google meet that had the waiting room enabled. Will try to find the cause. Thanks for reporting!

[noah-duncan]

Hey quick update @rafaeljusto the "Your network is having problems" issue does not happen for my branch here: attendee-labs#614

So it's likely due to one of the changes in the fork. I tested it repeatedly to confirm it only happened with the fork. When I have some more time I can try to see what within the fork is causing the problem.

[rafaeljusto]

Thanks for checking Noah! Hopefully it's not the websocket reconnection change I made 😬
#21

[noah-duncan]

I think it is or at least it's something to do with bots/google_meet_bot_adapter/google_meet_chromedriver_payload.js. When I replaced that file with the version from attendee-labs#614, the issue stopped.

Well, good to know that Google Meet seems very sensitive to websocket behavior in Chrome 144, but not Chrome 134.

[rafaeljusto]

I've tested the Chrome upgrade again:

• Applied your patch
• Reverted the web-socket changes
• Changed to execute in K8s mode

The deployed pod shows the following error:

{"value":{"error":"no such element","message":"no such element: Unable to locate element: {"method":"css selector","selector":"button[jsname="CQylAd"][aria-label="Leave call"]"}\n (Session info: chrome=144.0.7559.96)","stacktrace":"#0 0x55d2fa9baafa \u003Cunknown>\n#1 0x55d2fa3d460f \u003Cunknown>\n#2 0x55d2fa429a16 \u003Cunknown>\n#3 0x55d2fa429c51 \u003Cunknown>\n#4 0x55d2fa475404 \u003Cunknown>\n#5 0x55d2fa47262b \u003Cunknown>\n#6 0x55d2fa41b842 \u003Cunknown>\n#7 0x55d2fa41c631 \u003Cunknown>\n#8 0x55d2fa980689 \u003Cunknown>\n#9 0x55d2fa983581 \u003Cunknown>\n#10 0x55d2fa96cca9 \u003Cunknown>\n#11 0x55d2fa98413e \u003Cunknown>\n#12 0x55d2fa9533a0 \u003Cunknown>\n#13 0x55d2fa9a7fc8 \u003Cunknown>\n#14 0x55d2fa9a819b \u003Cunknown>\n#15 0x55d2fa9b9e68 \u003Cunknown>\n#16 0x7fa7691a0ac3 \u003Cunknown>\n"}} | headers=HTTPHeaderDict({'Content-Length': '833', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'}

And the bot appears to be stuck on the Google homepage:

debug_screen_recording_shot_zhNYMwTkUa3hrt9I.mp4

The next step is probably switching to the official repo, but it will require the Postgres setup.

[rafaeljusto]

Ah, ignored me... I didn't fully apply your patch 🤦 One sec.

[rafaeljusto]

... and 🥁🥁🥁 ... it worked! 😄 Thanks Noah