Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add integration for FastAPI, Starlette and ASGI & enhance async support #373

Merged
merged 174 commits into from
May 26, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
174 commits
Select commit Hold shift + click to select a range
c14e719
Add plain ASGIMiddleware for ASGI v3 spec
bxsx Mar 18, 2021
f340e3b
Add ASGI hook with correct framework name
bxsx Mar 18, 2021
5f55302
Add typing support for Starlette-based apps
bxsx Mar 18, 2021
7331393
Format code
bxsx Mar 19, 2021
3d5a700
Add support for type hints if Starlette package installed
bxsx Mar 19, 2021
50b1702
Restrict object exporting to ASGIMiddleware only
bxsx Mar 19, 2021
63f7539
retrigger checks
bxsx Apr 6, 2021
083a094
Add basic FastAPIMiddleware based on ASGIMiddleware
bxsx Mar 19, 2021
1dc1bb9
Make sure the `type hints` support is working for FastAPI
bxsx Mar 19, 2021
6877682
Add test helper: FailingTestASGIApp
bxsx Apr 9, 2021
7d58223
Check if ASGI app is HTTP type
bxsx Apr 9, 2021
770bbf8
Refactor async wrapper. Use asyncio.run() for Py3.7
bxsx Apr 9, 2021
dc2cdc9
Refactor Python version checker
bxsx Apr 9, 2021
8a68f8b
Add FastAPI version to payload
bxsx Apr 10, 2021
d7cf2fd
Refactor tests
bxsx Apr 10, 2021
9aece56
Add StarletteMiddleware
bxsx Apr 10, 2021
4bd51c6
Refactor tests. Make sure each test reloads hooks.
bxsx Apr 10, 2021
7562df9
FastAPIMiddleware inherits from StarletteMiddleware
bxsx Apr 10, 2021
34c7caa
Make __call__ wrapper more private: _asgi_app()
bxsx Apr 10, 2021
7f8bb5d
Refactor tests helper. Run coro directly from __call__
bxsx Apr 10, 2021
2e77aba
Remove obsolate docstring
bxsx Apr 10, 2021
db0dec8
Add Starlette Request data builder
bxsx Apr 12, 2021
9354d46
Build request data for FastAPI
bxsx Apr 12, 2021
f548a26
Add Starlette user ip extractor
bxsx Apr 12, 2021
ac75dee
Add user ip to payload for Starlette and FastAPI
bxsx Apr 12, 2021
0e492f9
Remove unused data from tests
bxsx Apr 12, 2021
161a573
Move ASGI tests helper to async test helper
bxsx Apr 12, 2021
d163644
Add wrapper for async receive event handler
bxsx Apr 12, 2021
bd13b26
Remove unused imports
bxsx Apr 12, 2021
ef0d597
Use unique identifier name for ASGIApp type
bxsx Apr 12, 2021
7a6040b
Add async handler based on HTTPX library
bxsx Apr 19, 2021
bc307f4
Normalize string quotes to project standard
bxsx Apr 19, 2021
7abec78
Make tests more independent from the used handler
bxsx Apr 19, 2021
04566bc
Add consuming request body in Starlette middleware
bxsx Apr 20, 2021
ca1e42a
Add tests for sending payload with correct request data
bxsx Apr 20, 2021
3be7bdc
Refactor tests. Make them more flat.
bxsx Apr 20, 2021
745359c
Add support for type hints for all ASGI apps
bxsx Apr 20, 2021
fb6591f
Set __all__ to export middlewares only
bxsx Apr 20, 2021
e82da48
Add FastAPIVersionError exception
bxsx Apr 20, 2021
ecc3a9c
Add decorator to check minimum required FastAPI version
bxsx Apr 20, 2021
fb0e3c8
Move FastAPI tests to fastapi_tests package
bxsx Apr 21, 2021
9ebbab2
Reorganize FastAPI tests
bxsx Apr 21, 2021
bfe3a5a
Add add_to() that inject RollbarLoggingRoute as FastAPI route class
bxsx Apr 21, 2021
803a237
Check if required FastAPI version is installed
bxsx Apr 21, 2021
43a83de
Refactor tests
bxsx Apr 21, 2021
014b54f
Load router handler only if added before adding other routes to the app
bxsx Apr 21, 2021
788a52e
Add RollbarLoggingRoute support for APIRouter
bxsx Apr 21, 2021
995d64c
Refactor tests
bxsx Apr 21, 2021
52637ff
Add test for multiple routers
bxsx Apr 21, 2021
2b9047d
If middleware is loaded, warn about possible duplicates
bxsx Apr 21, 2021
a491dd0
Extract FastAPI implementation to modules
bxsx Apr 21, 2021
0be8c86
Refactor tests: skip tests if FastAPI version is to low
bxsx Apr 21, 2021
1eb411a
Add BareMiddleware to test helpers
bxsx Apr 21, 2021
31cc750
Add get_installed_middlewares(app) to FastAPI utils
bxsx Apr 21, 2021
6f8db54
Refactor by extracting method
bxsx Apr 21, 2021
d2e9747
Refactor FastAPI route handler
bxsx Apr 21, 2021
9dfa7e8
Rename rollbar.contrib.fastapi.route to .routing
bxsx Apr 21, 2021
4fba2cd
Add path params to payload
bxsx Apr 22, 2021
f1bf6bc
Do not fill payload with empty request data
bxsx Apr 22, 2021
4965ebd
Move StarletteMiddleware to separate module
bxsx Apr 22, 2021
4ffc1f5
Add Starlette requests storage handler
bxsx Apr 22, 2021
c544f0d
Refactor tests. Use coroutines instead of functions
bxsx Apr 23, 2021
d578579
Export get_current_request()
bxsx Apr 23, 2021
6155d86
Add current request to context storage
bxsx Apr 23, 2021
5a46194
Reformat code
bxsx Apr 23, 2021
ab1012a
Include Starlette request object in get_return()
bxsx Apr 23, 2021
4e6f855
Add FastAPI to get_request()
bxsx Apr 23, 2021
bf6fe88
Add get_current_request to FastAPI integration
bxsx Apr 23, 2021
e6668ae
Extract Starlette test suite
bxsx Apr 23, 2021
2f5aefc
Add support for wraped Request parameter in store_current_request()
bxsx Apr 23, 2021
675c3f8
Store current request when FastAPI route handler is used
bxsx Apr 23, 2021
fa44af7
Add Starlette LoggerMiddleware to store current request only
bxsx Apr 23, 2021
e02bd23
Add LoggerMiddleware to Starlette API
bxsx Apr 23, 2021
474237c
Add LoggerMiddleware to FastAPI API
bxsx Apr 23, 2021
f6276bc
Refactor imports
bxsx Apr 23, 2021
0c59ade
Add support to build person data for Starlette
bxsx Apr 24, 2021
a4e392a
Change default handler to `default`.
bxsx Apr 26, 2021
c20c603
Add report_exc_info and report_message coroutines
bxsx Apr 26, 2021
d24f364
Make sure SETTINGS are fresh for each testcase
bxsx Apr 26, 2021
bdd914b
Make sure to use async handler in case of calling coroutines
bxsx Apr 26, 2021
8b299cb
Update linting
bxsx Apr 26, 2021
831995d
Add FastAPI util func `has_bare_routing()`
bxsx Apr 27, 2021
aa562f0
Refactor: compose methods
bxsx Apr 27, 2021
396c28c
Return context handler for async_handler ctx mananager
bxsx Apr 27, 2021
78c821e
Make sure SETTINGS are fresh for each test case
bxsx Apr 27, 2021
fe3d164
Use async handlers for StarletteMiddleware by default
bxsx Apr 27, 2021
fa576fe
Make sure SETTINGS are fresh for each test case
bxsx Apr 27, 2021
7b519fe
Make sure FastAPIMiddleware behaves similar to StarletteMiddleware
bxsx Apr 27, 2021
dc3488f
Refactor tests: be more strict with mocks
bxsx Apr 27, 2021
fb56d1b
Rafactor tests
bxsx Apr 27, 2021
3b28d58
Apply fallback async handler algorithm to RollbarLoggingRoute
bxsx Apr 27, 2021
3559dde
Extract fallback to sync reporter to try_report()
bxsx Apr 27, 2021
2f819ed
Add _async.call_later() to schedule coroutines
bxsx Apr 27, 2021
4af967f
Use call_later() instead of using asyncio module directly
bxsx Apr 27, 2021
9cd78fb
Lint code
bxsx Apr 27, 2021
4b84489
Refactor global request provider
bxsx Apr 27, 2021
85b56bc
Remove warning when FastAPI/Starlette report via sync handler
bxsx Apr 27, 2021
d73b3bf
Fix _post_api_httpx
bxsx Apr 28, 2021
e40aa26
Move ASGIMiddleware to middleware.py module
bxsx Apr 28, 2021
e2b9470
Fix tests warnings
bxsx Apr 28, 2021
7247979
Refactor imports: use relative namespace
bxsx Apr 28, 2021
f55a7bf
Remove unused code
bxsx Apr 28, 2021
b4ead20
Add AsyncIO classifier
bxsx Apr 28, 2021
868f806
Add Starlette and FastAPI to the CI matrix
bxsx Apr 28, 2021
db386e3
Rename middleware to ReporterMiddleware and LoggerMiddleware
bxsx Apr 28, 2021
1a11c3f
Refactor docstrings and comments
bxsx Apr 28, 2021
19906f8
Clean up code
bxsx Apr 28, 2021
e561e22
Fix annotation
bxsx Apr 28, 2021
274976b
Reorder tests
bxsx Apr 28, 2021
a2170cf
Make sure SETTINGS are fresh for each testcase
bxsx Apr 29, 2021
19195b3
Clean unused imports
bxsx Apr 29, 2021
70031b8
Fetch stream data into request object (body, form, json).
bxsx Apr 29, 2021
cfdae6d
Update comment and optional instructions
bxsx Apr 29, 2021
ceed956
Refactor ASGI integration
bxsx Apr 29, 2021
569f450
Remove print from the test
bxsx Apr 29, 2021
25b9460
Clean up
bxsx Apr 30, 2021
e3325a6
Add httpx and python-multipart packages for Starlette and FastAPI builds
bxsx Apr 30, 2021
c2edf70
Skip tests if missing packages
bxsx Apr 30, 2021
deafd8c
Move async test helpers to rollbar.lib.async
bxsx Apr 30, 2021
b5ae265
Update jobs exclusion
bxsx Apr 30, 2021
3801abb
Exclude non-compatible tests from test loader
bxsx May 4, 2021
0d1e3d5
Reformat code
bxsx May 4, 2021
1507489
Move bare coroutine to rollbar.lib._async
bxsx May 4, 2021
fa2cd51
Verify status code
bxsx May 4, 2021
b4c6edb
Dynamically inject required annotations
bxsx May 4, 2021
e1725ac
Backport AsyncMock for Python3.7-
bxsx May 4, 2021
fe9ffbd
Exclude async tests for Python < 3.5
bxsx May 4, 2021
07c7822
Make sure rollbar.init() works for each Flask testcase
bxsx May 4, 2021
b175005
Export LoggerMiddleware to API only for Python3.7+
bxsx May 4, 2021
1098b3c
Make HTTPX package optional
bxsx May 4, 2021
425798d
Do not import rollbar.contrib.fastapi if FastAPI is not installed
bxsx May 4, 2021
c2310a2
Skip UtilsBareRoutingTest if FastAPI not installed
bxsx May 4, 2021
7fb60fd
Do not import rollbar.contrib.starlette if Starlette is not installed
bxsx May 4, 2021
e97acac
Import correct middleware
bxsx May 4, 2021
a103d91
Mock if Python version is sufficient
bxsx May 4, 2021
b8b7580
Skip FastAPI and Starlette tests for Python v3.5-
bxsx May 4, 2021
95c50f1
Explicitly set disabled logging level
bxsx May 4, 2021
f5406a8
Do not use `mock.assert_called_once()` for Python3.5-
bxsx May 4, 2021
4c06ea3
Fall back to starlette.Request for FastAPI v0.50.0-
bxsx May 4, 2021
29e0a36
Make tests compatible with Starlette v0.12.13-
bxsx May 4, 2021
04efa7d
Async report_*() requires Py3.6+
bxsx May 4, 2021
f6a204e
Reformat code
bxsx May 4, 2021
800ebaa
Refactor code, fix comment
bxsx May 4, 2021
4de7f00
Add HTTPX to tests dependency. Skip related tests for Py3.5-
bxsx May 4, 2021
71f3977
Fall back to Starlette Response class for FastAPI v0.50.0-
bxsx May 4, 2021
a8e3757
Do not pass Send message to Request
bxsx May 4, 2021
ae0e46d
Fall back to Starlette classes in case of too old FastAPI
bxsx May 4, 2021
93a40e9
Fall back to Starlette classes in case of too old FastAPI
bxsx May 4, 2021
9fbad99
Reformat code
bxsx May 4, 2021
f194a88
Make sure tests work for older versions of Starlette
bxsx May 4, 2021
4a00662
Fix get_installed_middlewares() for older FastAPI versions
bxsx May 4, 2021
ecd3098
Revert "Refactor code, fix comment"
bxsx May 5, 2021
e7edee0
Refactor get_installed_middlewares()
bxsx May 5, 2021
7a18d20
Reformat code
bxsx May 5, 2021
a1bdcd6
Merge branch 'master' into bxsx/fastapi-starlette-integration
bxsx May 6, 2021
a14a74c
Add comment regarding optional python-multipart package
bxsx May 6, 2021
fe2bde8
Export the full API of Starlette and FastAPI regardless of current sy…
bxsx May 6, 2021
0cc3326
Raise the exception when incompatible middleware is trying to be used
bxsx May 6, 2021
f47ec5d
Integrate only when the object has been initialized
bxsx May 17, 2021
ae2daea
Remove unused logger
bxsx May 17, 2021
6d87324
Refactor docs
bxsx May 17, 2021
c1bb16a
Remove unused test-function
bxsx May 17, 2021
535d6f5
Add send_payloa() docstring: add 'async' and 'httpx'
bxsx May 17, 2021
4cd225e
Fix test for Python3.5
bxsx May 17, 2021
9901ca0
Raise with the actual min_version required instead of hard-coded
bxsx May 18, 2021
93d84eb
Merge branch 'master' into bxsx/fastapi-starlette-integration
bxsx May 20, 2021
86ebeb4
Refactor test
bxsx May 21, 2021
bc85fec
Use 'thread' instead of 'default' handler
bxsx May 21, 2021
1f66fd9
Add comment to workaround Python2 SyntaxError
bxsx May 21, 2021
35da296
Use local context when using async_handler ctx manager
bxsx May 21, 2021
140e3ee
Backport contextvars for Python 3.6
bxsx May 22, 2021
d6eb992
Unlock LoggerMiddleware for Python 3.6
bxsx May 22, 2021
9955666
Add comment about optional requirements
bxsx May 22, 2021
72f306b
Rename async_handler ctx manager to AsyncHandler
bxsx May 25, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add Starlette Request data builder
  • Loading branch information
bxsx committed Apr 12, 2021
commit db0dec8a0d4d573e3107c3a82af10bfe376df493
22 changes: 21 additions & 1 deletion rollbar/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,11 @@
except ImportError:
AppEngineFetch = None

try:
from starlette.requests import Request as StarletteRequest
except ImportError:
StarletteRequest = None


def passthrough_decorator(func):
def wrap(*args, **kwargs):
Expand Down Expand Up @@ -877,7 +882,8 @@ def _build_person_data(request):
else:
return None

if hasattr(request, 'user'):
# TODO: Check if AuthenticationMiddlawre installed for Startlette
if not StarletteRequest and hasattr(request, 'user'):
user_prop = request.user
user = user_prop() if callable(user_prop) else user_prop
if not user:
Expand Down Expand Up @@ -1120,6 +1126,10 @@ def _build_request_data(request):
if isinstance(request, dict) and 'wsgi.version' in request:
return _build_wsgi_request_data(request)

# Starlette
if StarletteRequest and isinstance(request, StarletteRequest):
return _build_starlette_request_data(request)

return None


Expand Down Expand Up @@ -1299,6 +1309,16 @@ def _build_wsgi_request_data(request):

return request_data

def _build_starlette_request_data(request):
request_data = {
'url': str(request.url),
'GET': dict(request.query_params),
'headers': dict(request.headers),
'method': request.method,
}

return request_data


def _filter_ip(request_data, capture_ip):
if 'user_ip' not in request_data or capture_ip == True:
Expand Down
16 changes: 13 additions & 3 deletions rollbar/contrib/starlette/__init__.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,27 @@
import sys

from starlette import __version__
from starlette.types import ASGIApp
from starlette.requests import Request
from starlette.types import Receive, Scope, Send

import rollbar
from rollbar.contrib.asgi import ASGIMiddleware


class StarletteMiddleware(ASGIMiddleware):
...
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
try:
await self.app(scope, receive, send)
except Exception:
if scope["type"] == "http":
request = Request(scope, receive, send)
exc_info = sys.exc_info()
rollbar.report_exc_info(exc_info, request)
raise


def _hook(request, data):
data["framework"] = f"starlette {__version__}"


rollbar.BASE_DATA_HOOK = _hook

44 changes: 44 additions & 0 deletions rollbar/test/test_rollbar.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,50 @@ def test_wsgi_request_data(self):
self.assertDictEqual(data['GET'], {'format': 'json', 'param1': 'value1', 'param2': 'value2'})
self.assertDictEqual(data['headers'], {'Connection': 'close', 'Host': 'example.com', 'User-Agent': 'Agent'})

def test_starlette_request_data(self):
try:
from starlette.requests import Request
except ImportError:
self.skipTest("Requires Starlette to be installed")

scope = {
"type": "http",
"client": ("127.0.0.1", 1453),
"headers": [
(b"accept", b"*/*"),
(b"content-type", b"application/x-www-form-urlencoded"),
(b"host", b"example.com"),
(b"user-agent", b"Agent"),
],
"http_version": "1.1",
"method": "GET",
"path": "/api/test",
"query_params": {
"format": "json",
"param1": "value1",
"param2": "value2",
},
"query_string": b"format=json&param1=value1&param2=value2",
"scheme": "http",
"server": ("example.com", 80),
"url": {"path": "example.com", "root_path": "xxxx"},
}
request = Request(scope)
data = rollbar._build_starlette_request_data(request)

self.assertEqual(data["url"], "https://example.com/api/test?format=json&param1=value1&param2=value2")
self.assertEqual(data["method"], "GET")
self.assertDictEqual(data["GET"], {"format": "json", "param1": "value1", "param2": "value2"})
self.assertDictEqual(
data["headers"],
{
"accept": "*/*",
"content-type": "application/x-www-form-urlencoded",
"host": "example.com",
"user-agent": "Agent",
},
)

@mock.patch('rollbar.send_payload')
def test_report_exception(self, send_payload):

Expand Down
25 changes: 25 additions & 0 deletions rollbar/test/test_starlette.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,31 @@ def root(request):
self.assertEqual(exc_type, ZeroDivisionError)
self.assertIsInstance(exc_value, ZeroDivisionError)

def test_should_report_with_request_data(self):
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.requests import Request
from starlette.routing import Route
from starlette.testclient import TestClient
from rollbar.contrib.starlette import StarletteMiddleware

def root(request):
1 / 0

routes = [ Route("/", root) ]
middleware = [ Middleware(StarletteMiddleware) ]
app = Starlette(routes=routes, middleware=middleware)

client = TestClient(app)
with mock.patch("rollbar.report_exc_info") as mock_report:
with self.assertRaises(ZeroDivisionError):
client.get("/")

mock_report.assert_called_once()
request = mock_report.call_args[0][1]

self.assertIsInstance(request, Request)

def test_should_support_type_hints(self):
from starlette.types import ASGIApp, Receive, Scope, Send

Expand Down