Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds logger filter for TOKENS when logging in DEBUG MODE #539

Draft
wants to merge 9 commits into
base: master
Choose a base branch
from
2 changes: 2 additions & 0 deletions AUTHORS.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,5 @@ Maintainers & Contributors
- Sylvain Marie <[email protected]>
- Craig Anderson <[email protected]>
- Hugo van Kemenade <https://github.com/hugovk>
- Jacques Troussard <https://github.com/jtroussard>
- Erland Vollset <https://github.com/erlendvollset>
19 changes: 19 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,22 @@ To install requests and requests_oauthlib you can use pip:
:alt: Documentation Status
:scale: 100%
:target: https://requests-oauthlib.readthedocs.io/

Advanced Configurations
-----------------------

### Logger Configuration Framework

`requests-oauthlib` now includes a flexible framework for applying custom filters and configurations to the logger, enhancing control over logging behavior and improving security.

#### Custom Filters

- **Debug Mode Token Filter**: To enhance security and provide more control over logging of sensitive information, requests-oauthlib introduces the Debug Mode Token Filter. This feature is controlled via the DEBUG_MODE_TOKEN_FILTER environment variable, allowing the suppression or masking of sensitive data in logs.

##### Configuring the Debug Mode Token Filter

- **Environment Variable**: `DEBUG_MODE_TOKEN_FILTER`
- **Options**:
- `DEFAULT`: No alteration to logging behavior.
- `MASK`: Masks sensitive tokens in logs.
- `SUPPRESS`: Prevents logging of potentially sensitive information. (logger ignores these logs entirely)
10 changes: 10 additions & 0 deletions requests_oauthlib/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
from .oauth2_auth import OAuth2
from .oauth2_session import OAuth2Session, TokenUpdated

from .log_filters import DebugModeTokenFilter

__version__ = "2.0.0"

import requests
Expand All @@ -18,3 +20,11 @@
raise Warning(msg % requests.__version__)

logging.getLogger("requests_oauthlib").addHandler(logging.NullHandler())
logging.getLogger("requests_oauthlib").addFilter(DebugModeTokenFilter())

for filter_ in logging.getLogger("requests_oauthlib").filters:
if isinstance(filter_, DebugModeTokenFilter):
if filter_.mode == 'DEFAULT':
msg = "Your logger, when in DEBUG mode, will log TOKENS"
logging.warning(msg)
break
40 changes: 40 additions & 0 deletions requests_oauthlib/log_filters.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import os
import re
import logging

class DebugModeTokenFilter(logging.Filter):
"""
A logging filter that while in DEBUG mode can filter TOKENS dependent on configuration.

This filter uses an environment variable to determine its mode,
which can either mask sensitive tokens in log messages, suppress logging,
or default to standard logging behavior with a warning.

Attributes:
mode (str): The mode of operation based on the environment variable
'DEBUG_MODE_TOKEN_FILTER'. Can be 'MASK', 'SUPPRESS', or 'DEFAULT'.
"""
def __init__(self):
"""
Initializes the DebugModeTokenFilter with the 'DEBUG_MODE_TOKEN_FILTER'
environment variable.
"""
super().__init__()
self.mode = os.getenv('DEBUG_MODE_TOKEN_FILTER', 'DEFAULT').upper()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest prefixing the envvar with the library name to reduce likelihood of collisions with other environment variables.

Suggested change
self.mode = os.getenv('DEBUG_MODE_TOKEN_FILTER', 'DEFAULT').upper()
self.mode = os.getenv('REQUESTS_OAUTHLIB_DEBUG_MODE_TOKEN_FILTER', 'DEFAULT').upper()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love this comment. In fact I was running into this issue in another project. This project then variable name pattern is clean. I'm going to borrow and apply this pattern there as well. 🙏 Thank you!


def filter(self, record):
"""
Filters logs of TOKENS dependent on the configured mode.

Args:
record (logging.LogRecord): The log record to filter.

Returns:
bool: True if the record should be logged, False otherwise.
"""
if record.levelno == logging.DEBUG:
if self.mode == "MASK":
record.msg = re.sub(r'Bearer\s+([A-Za-z0-9\-._~+\/]+)', '[MASKED]', record.getMessage())
Copy link

@erlendvollset erlendvollset Apr 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I would suggest compiling the regex pattern once to avoid doing that for every log message.
  2. I would suggest putting this behind an LRU cache as we'll be logging a lot of the same statements repeatedly. re.sub is expensive enough that we would want to avoid doing it each time.

elif self.mode == "SUPPRESS":
return False

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe only suppress log messages which match the above regex? Otherwise this equivalent to disabling the logger entirely.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am hoping a single space would suffice as a replacement? Or maybe just pass None to the msg?

return True # if mode is not MASKED then DEFAULT is implied
30 changes: 30 additions & 0 deletions tests/test_log_filters.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import unittest
from unittest.mock import patch
import logging
from requests_oauthlib.log_filters import DebugModeTokenFilter

class TestDebugModeTokenFilter(unittest.TestCase):

def setUp(self):
self.record = logging.LogRecord(name="test", level=logging.DEBUG, pathname=None, lineno=None, msg="Bearer i-am-a-little-token-here-is-my-scope-and-here-is-my-signature", args=None, exc_info=None)

@patch.dict('os.environ', {'DEBUG_MODE_TOKEN_FILTER': 'MASK'})
def test_mask_mode(self):
filter = DebugModeTokenFilter()
filter.filter(self.record)
self.assertIn('[MASKED]', self.record.msg)

@patch.dict('os.environ', {'DEBUG_MODE_TOKEN_FILTER': 'SUPPRESS'})
def test_suppress_mode(self):
filter = DebugModeTokenFilter()
result = filter.filter(self.record)
self.assertFalse(result) # No logging

@patch.dict('os.environ', {'DEBUG_MODE_TOKEN_FILTER': 'DEFAULT'})
def test_default_mode_raises_warning(self):
filter = DebugModeTokenFilter()
result = filter.filter(self.record)
self.assertTrue(result)

if __name__ == '__main__':
unittest.main()
Loading