Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

500 internal server error on GET /passes/<norad-id> #12

Closed
viconnex opened this issue Apr 24, 2024 · 9 comments
Closed

500 internal server error on GET /passes/<norad-id> #12

viconnex opened this issue Apr 24, 2024 · 9 comments

Comments

@viconnex
Copy link

viconnex commented Apr 24, 2024 •
edited
Loading

Hello! Thank you for this project!
I get 500 errors when calling the GET /passes/<norad-id> endpoint.
Did you also notice this? Do you know what is happening?
Or am I making a mistake in my query?

Below a sample of the performed query:

curl 'https://satellites.fly.dev/passes/25544?lat=-34.911221&lon=-57.9372988&limit=100&days=7&visible_only=false' \
  -H 'accept: application/json' \
  -H 'accept-language: fr,en-US;q=0.9,en;q=0.8,fr-FR;q=0.7,es;q=0.6,de;q=0.5,it;q=0.4' \
  -H 'cache-control: no-cache' \
  -H 'pragma: no-cache' \
  -H 'priority: u=1, i' \
  -H 'referer: https://satellites.fly.dev/docs/' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-fetch-dest: empty' \
  -H 'sec-fetch-mode: cors' \
  -H 'sec-fetch-site: same-origin'

Result:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or
    there is an error in the application.</p>

Thank you for your help!
@viconnex viconnex changed the title 500 internal server error on GET /passes/<norad-id> 500 internal server error on GET /passes/<norad-id> Apr 24, 2024
@viconnex viconnex changed the title 500 internal server error on GET /passes/<norad-id> 500 internal server error on GET /passes/<norad-id> Apr 24, 2024
@redraw
Copy link
Owner

redraw commented Apr 24, 2024

Hi, thanks for alerting. Reading the logs and seems like the redis cache reached the daily limit 🥵

2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] cache.set(cache_key, tle, CACHE_TIMEOUT)

2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] File "/usr/local/lib/python3.8/site-packages/redis/client.py", line 1801, in set

2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] return self.execute_command('SET', *pieces)

2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] File "/usr/local/lib/python3.8/site-packages/redis/client.py", line 901, in execute_command

2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] return self.parse_response(conn, command_name, **options)

2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] File "/usr/local/lib/python3.8/site-packages/redis/client.py", line 915, in parse_response

2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] response = connection.read_response()

2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] File "/usr/local/lib/python3.8/site-packages/redis/connection.py", line 747, in read_response

2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] raise response

2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] redis.exceptions.ResponseError: max daily request limit exceeded. Limit: 10000, Usage: 10000. See https://upstash.com/docs/redis/troubleshooting/max_daily_request_limit for details

2024-04-24T14:58:35.836 app[9080e205a22087] gru [info] 172.16.136.66 - - [24/Apr/2024:14:58:35 +0000] "GET /passes/25544?lat=-34.911221&lon=-57.9372988&limit=100&days=7&visible_only=false HTTP/1.1" 500 290 "https://satellites.fly.dev/docs/" "curl/8.4.0"

I guess I could skip cache writes if hitting this limit, I'll take a look.

@redraw
Copy link
Owner

redraw commented Apr 24, 2024

I could add Cloudflare on front to cache cached requests, but that will require a domain change.

@redraw
Copy link
Owner

redraw commented Apr 25, 2024

Done. Now use https://sat.terrestre.ar and let's wait until tomorrow to reset the daily quota. I blocked a heavy bot that was consuming the quota.

@redraw redraw closed this as completed Apr 25, 2024
@viconnex
Copy link
Author

Hello @redraw! Thanks for setting this up! However I get 403 Forbidden errors now 😅
I am blocked by the Cloudflare protection. The error message invites me to share my Cloudflare Ray ID: 88394dda2a0a9e8d. Does it help for debugging?
Thanks! 🚀

@redraw
Copy link
Owner

redraw commented May 14, 2024 •
edited
Loading

@viconnex based on the Cloudflare Ray ID, you are THE one I've blocked 👀. You were flooding the server, exceeding the redis operations quota. I can unblock your IP if you can fix your code. Seems like you were hitting the /passes endpoint in a while True loop maybe?

Screenshot 2024-05-14 at 12 42 37

@viconnex
Copy link
Author

LOL indeed! Sorry! My code calls the /passes endpoint every 10 seconds. What would be an acceptable delay for you?

@redraw
Copy link
Owner

redraw commented May 20, 2024

No prob. Why every 10 seconds? passes don't change. Calculations tends to drift after 15 days of prediction, as those will require updated TLEs in the future. Every 1 day you should be fine.

@redraw
Copy link
Owner

redraw commented May 20, 2024

Just unblocked your IP 👍

@viconnex
Copy link
Author

Thank you! Yes you're right, I will change the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@redraw @viconnex