[Serve] Require traffic weights to sum more closely to 1.

[robertnishihara]

This fixes the following.

import ray
from ray import serve

class Foo:
    def __call__(self, request):
        return 1
 
ray.init()
serve.init()
serve.create_backend("foo", Foo)
serve.create_endpoint("foo", route="/foo")
serve.set_traffic("foo", {"foo": 0.99})  # This SUCCEEDS!

import requests
requests.get("http:https://127.0.0.1:8000/foo").text  # This returns an ERROR!

The last line returns

Internal Error. Traceback: \x1b[36mray::Router.enqueue_request()\x1b[39m (pid=82306, ip=192.168.1.104)\n File "python/ray/_raylet.pyx", line 457, in ray._raylet.execute_task\n with ray.worker._changeproctitle(title, next_title):\n File "python/ray/_raylet.pyx", line 458, in ray._raylet.execute_task\n outputs = function_executor(*args, **kwargs)\n File "python/ray/_raylet.pyx", line 410, in ray._raylet.execute_task.function_executor\n return future.result()\n File "/Users/rkn/opt/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 428, in result\n return self.__get_result()\n File "/Users/rkn/opt/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result\n raise self._exception\n File "/Users/rkn/anyscale/ray/python/ray/serve/router.py", line 220, in enqueue_request\n await self.flush_endpoint_queue(endpoint)\n File "/Users/rkn/anyscale/ray/python/ray/serve/router.py", line 308, in flush_endpoint_queue\n self.endpoint_queues[endpoint], self.backend_queues)\n File "/Users/rkn/anyscale/ray/python/ray/serve/policy.py", line 71, in flush\n p=self.backend_weights).squeeze()\n File "mtrand.pyx", line 924, in numpy.random.mtrand.RandomState.choice\nValueError: probabilities do not sum to 1.

[AmplabJenkins]

Can one of the admins verify this patch?

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/25998/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/26013/
Test PASSed.

[simon-mo]

LGTM, thanks for the fix @robertnishihara