Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Code Reviews

Welcome to Software Development on Codidact!

Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.

Comments on JSON log formatter

Post

JSON log formatter

+6
−0

Here's a JSON log formatter for Python. I want to be able to log details of exceptions (and have some capability to debug-by-logs). I want to be able to log extra data in JSON format (in addition to a message string).

Is my approach sound, in general?

I distinguish between regular logging and exception logging by checking for presence of exc_info. It works, but is it the right way to do it?

Am logging the exception info that I should be logging? Am I missing anything useful?

Here's what I wrote so far. (Work in progress, and I'm not married to this design.)

class JsonLogFormatter(logging.Formatter):
	def __init__(self) -> None:
		super().__init__()

	def format(self, record):
		log_record_dict = { "time": self.formatTime(record), "level": record.levelname, "message": record.message }

		if record.exc_info is not None:
			log_record_dict.update( {"exception": self.formatException(record.exc_info)} )
		
		log_record_dict.update(record.args)
		return json.dumps(log_record_dict, indent=None)

Log entry in the unhandled exception hook.

class UncaughtExceptionHook:

	def __init__(self, logger):
		self.logger = logger
		sys.excepthook = self.exception_hook    # It seemd sonvenient to register the hook with the system in the ctor.  Future will show if this is future-proof.

	def exception_hook(self, ex_type, ex_value, ex_traceback):
		self.logger.critical(f"Unhandled exception.",  exc_info=(ex_type, ex_value, ex_traceback))
		sys.exit(1)

Calling code.

logger.info("hello JSON log entry")
logger.info("now with an attachment", {"foo": "Lorem ipsum"} )

#deliberate exception to check the UncaughtExceptionHook
raise ValueError("deliberate exception to check the UncaughtExceptionHook" )

Resulting log entries.

{"time": "2023-05-02 15:12:46,735", "level": "INFO", "message": "hello JSON log entry"}
{"time": "2023-05-02 15:12:53,643", "level": "INFO", "message": "now with an attachment", "foo": "Lorem ipsum"}
{"time": "2023-05-02 15:12:56,086", "level": "CRITICAL", "message": "Unhandled exception.", "exception": "Traceback (most recent call last):\n  File \"C:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.11_3.11.1008.0_x64__qbz5n2kfra8p0\\Lib\\runpy.py\", line 198, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.11_3.11.1008.0_x64__qbz5n2kfra8p0\\Lib\\runpy.py\", line 88, in _run_code\n    exec(code, run_globals)\n  File \"c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2023.6.1\\pythonFiles\\lib\\python\\debugpy\\adapter/../..\\debugpy\\launcher/../..\\debugpy\\__main__.py\", line 39, in <module>\n    cli.main()\n  File \"c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2023.6.1\\pythonFiles\\lib\\python\\debugpy\\adapter/../..\\debugpy\\launcher/../..\\debugpy/..\\debugpy\\server\\cli.py\", line 430, in main\n    run()\n  File \"c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2023.6.1\\pythonFiles\\lib\\python\\debugpy\\adapter/../..\\debugpy\\launcher/../..\\debugpy/..\\debugpy\\server\\cli.py\", line 284, in run_file\n    runpy.run_path(target, run_name=\"__main__\")\n  File \"c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2023.6.1\\pythonFiles\\lib\\python\\debugpy\\_vendored\\pydevd\\_pydevd_bundle\\pydevd_runpy.py\", line 321, in run_path\n    return _run_module_code(code, init_globals, run_name,\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2023.6.1\\pythonFiles\\lib\\python\\debugpy\\_vendored\\pydevd\\_pydevd_bundle\\pydevd_runpy.py\", line 135, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"c:\\Users\\User\\.vscode\\extensions\\ms-python.python-2023.6.1\\pythonFiles\\lib\\python\\debugpy\\_vendored\\pydevd\\_pydevd_bundle\\pydevd_runpy.py\", line 124, in _run_code\n    exec(code, run_globals)\n  File \"C:\\Users\\User\\source\\repos\\KiCAD netlist reader\\test01.py\", line 59, in <module>\n    raise ValueError(\"deliberate exception to check the UncaughtExceptionHook\" )\nValueError: deliberate exception to check the UncaughtExceptionHook"}
History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.
Why should this post be closed?

1 comment thread

existing packages? (2 comments)
existing packages?
Monica Cellio‭ wrote over 1 year ago

Someone on Mastodon posted a comment there that I'm relaying here as a starting point:

It's not a bad approach at all. But I would say back up a step, and don't necessarily reinvent the wheel. There are packages that already do everything desired here - structlog, for example - with varying levels of flexibility and functionality.

I hope the author will come here to expand on that, but I wanted to pass this along in case it helps.

luser‭ wrote over 1 year ago

The problem with structlog is it's not build above the existing logging infrastructure at all, it's completely separate and (sort of) incompatible.