[6/25] Rate limit part ???

[sigaloid]

Today some rate limit changes seemingly occurred. I'll take a look later, for now, if a public instance operator that is currently rate limited could test something for me. Restart the container and check right after to see if the rate limit appears, or if it resolves for the next few minutes (the exact time span depends on your traffic, but right after a restart should be soon enough). This will tell me how deep the issue is. For now please don't open any issues for this, as i'll keep the investigation here.

PS to the reddit SWE in charge of monitoring this repo:

[enizor]

Rate limit instantly reappears after recreating the docker instance (official quay.io image, 1 commit old).

Starting Redlib...
ERROR redlib::client > Got an invalid response from reddit expected value at line 1 column 1. Status code: 403 Forbidden
Rate limit check failed: Failed to parse page JSON data: expected value at line 1 column 1 | /r/reddit/hot.json?&raw_json=1
This may cause issues with the rate limit.
Please report this error with the above information.
https://github.com/redlib-org/redlib/issues/new?assignees=sigaloid&labels=bug&title=%F0%9F%90%9B+Bug+Report%3A+Rate+limit+mismatch
Running Redlib v0.36.0 on [::]:8080!

[LucifersCircle]

I let it run for 5m after starting a container and it didn’t “start working” in that short time frame. I do get a decent amount of traffic. I’ll let it run even longer to see. As a side note, after about an hour of it not working last night, I restarted the container and it worked for ~7 hours before the error showed again.

Edit: After the container was live for 10m it looks like it re-created oauth and immediately started giving errors again.

Edit 2: After the container was live for 26m, oauth refreshed again. Still getting invalid responses so I tried a container restart and it did not help. I’ll start checking every hour to see if it starts working randomly.

Edit 3: After the container was live for about 2 hours, I got this message in the logs, and the instance started working again:

TRACE redlib::client > Ratelimit remaining: Header says 74.0, we have 16.
Resets in 8.
Rollover: no.
Ratelimit used: 26

Edit 4: Wanted to share a screenshot of the logs, the top log being what’s shared in edit 3, seems to be running normally again.

Edit 5: Still running normally an hour later, will make one last edit with the logs if it stops again.

Edit 6: Issue is closed now because it may be a random server issue, but since my instance stopped working again I figured I would at least post a screenshot of the logs when it stopped, although there doesn’t seem to be any helpful information in it.

[sigaloid]

This seems to have just been a sporadic server issue... closing for now

[GBH]

Not sure what is happening with rate limiting, but my IP just got banned. I use redlib from my phone/tablet and this morning I discovered that it's not longer working. Hopefully ban is temporary.

[gigirassy]

Just want to note to you guys that if your server supports IPV6 with subnets, IPV6 rotators easily fix it.
Edit: ...Temporarily. I think this issue needs to be worked on a bit more even if it's for select servers.

[LucifersCircle]

Just want to note to you guys that if your server supports IPV6 with subnets, IPV6 rotators easily fix it. Edit: ...Temporarily. I think this issue needs to be worked on a bit more even if it's for select servers.

By temporarily do you mean your server is still having the issue with ipv6 rotation?

My instances have been “down” for 25% of the day the last 3 days on my 99%+ uptime server and there’s no oddities on the reddit status site. I’ll have to dig into my server and see if I can rotate ipv6 but I may not even be using it anymore. Thank you for the info!

[erusc]

I modified some environment variables about 2 hours ago and got the same error after redeployed the docker container.

Starting Redlib...
ERROR redlib::client > Got an invalid response from reddit expected value at line 1 column 1. Status code: 403 Forbidden
Rate limit check failed: Failed to parse page JSON data: expected value at line 1 column 1 | /r/reddit/hot.json?&raw_json=1
This may cause issues with the rate limit.
Please report this error with the above information.
https://github.com/redlib-org/redlib/issues/new?assignees=sigaloid&labels=bug&title=%F0%9F%90%9B+Bug+Report%3A+Rate+limit+mismatch
Running Redlib v0.36.0 on [::]:8080!

I visit my instance again now and it is working fine, able to load newest posts.

Now here is the weird part.
I run another container with the same image and this error pops again.
So I am now having a working instance and a not-working instance with the same IP address, no proxy or VPN whatsoever.
I am pretty sure they run on the same IP address because IPv6 is disabled.

[gigirassy]

Sometimes my instance is down, sometimes it isn't. I feel like it has nothing to do with IP in that case.

[crispiboi]

Atleast in my case I found in my nginx logs that openai is scraping my private instance. I banned the IP until I can figure out a better solution. It is still trying even though it's getting 4xx responses. Example logs contain the following, atleast they're not hiding who they are (yet):
"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)" "-"

[LucifersCircle]

There’s only about 3 instances without issue the rest have been down for a majority of the last few days.