logging.rst

Overview

This documentation file provides a comprehensive guide on how to manage and customize logging in pytest, the Python testing framework. It explains the built-in features of pytest's logging capture mechanism, how to configure logging output formats and levels, and how to interact with logging inside tests using the `caplog` fixture. It also covers live logging to the console, customizing log colors, and important changes introduced in pytest versions 3.3 and 3.4.

This file serves as a user-facing explanation rather than source code. It documents the logging functionality integrated into pytest, helping users configure and utilize logging effectively during test runs.

Detailed Explanation

Logging Capture in pytest

Default behavior: pytest automatically captures log messages of level WARNING or higher and displays them in a dedicated section for each failed test alongside captured stdout and stderr.
Captured log message format: By default, each log message displays the module, line number, log level, and message.
Customization:
- You can specify log message and date formats using --log-format and --log-date-format command line options, or through the pytest.ini configuration file.
- Example command line:
```
pytest --log-format="%(asctime)s %(levelname)s %(message)s" \
       --log-date-format="%Y-%m-%d %H:%M:%S"
```
Disabling specific loggers: Use --log-disable={logger_name} multiple times to exclude certain loggers from capturing.
Disabling captured output reporting: --show-capture=no disables all captured content display for failed tests.

`caplog` Fixture

The `caplog` fixture provides powerful ways to interact with logging inside test functions.

Key Features:

Changing log level during tests:

def test_example(caplog):
    caplog.set_level(logging.INFO)  # Sets root logger level to INFO
    caplog.set_level(logging.CRITICAL, logger="root.baz")  # Sets specific logger level

Context manager support for temporary level change:

def test_context(caplog):
    with caplog.at_level(logging.INFO):
        # Logs captured at INFO level within this block
        pass

    with caplog.at_level(logging.CRITICAL, logger="root.baz"):
        pass

Accessing captured logs:
- caplog.records: List of logging.LogRecord instances captured during the current test phase (setup, call, or teardown).
- caplog.text: Concatenated log messages as plain text.
- caplog.record_tuples: List of (logger_name, level, message) tuples for quick assertions.

Clearing captured logs:

def test_clearing(caplog):
    caplog.clear()  # Resets captured log records

Multi-phase log access: Use caplog.get_records(when) to fetch logs from other test phases ("setup", "call", "teardown").

Example usage to fail on warnings during fixture setup and call:

@pytest.fixture
def window(caplog):
    window = create_window()
    yield window
    for when in ("setup", "call"):
        warnings = [r.message for r in caplog.get_records(when) if r.levelno == logging.WARNING]
        if warnings:
            pytest.fail(f"Warnings found during testing: {warnings}")

Important Notes:

The fixture adds a handler to the root logger to capture logs.
If the root logger configuration is changed during a test (e.g., via logging.config.dictConfig), the handler might be removed and logs will not be captured.
Avoid removing root logger handlers during tests to maintain capture functionality.

Live Logs

pytest supports *live* logging output to the console during test execution.

Enable by setting the configuration option log_cli to true.
Control the minimum log level printed with --log-cli-level.
Customize live log formatting with --log-cli-format and --log-cli-date-format.
These CLI options correspond to INI options: log_cli_level, log_cli_format, and log_cli_date_format.

Example enabling live logs:

[pytest]
log_cli = true
log_cli_level = INFO

Logging to file:
- Use --log-file=/path/to/file to write all logs to a file.
- By default, the file is overwritten each test run; use --log-file-mode=a to append instead.
- Control file log level and format with --log-file-level, --log-file-format, and --log-file-date-format.
- Corresponding INI options: log_file, log_file_mode, log_file_level, log_file_format, log_file_date_format.
- The experimental set_log_path() function allows dynamic runtime changes to the log file path.

Customizing Log Colors

Log levels are colorized if colored output is enabled.
You can customize colors or add colors for custom log levels using the add_color_level() method on the CLI log formatter.

Example:

@pytest.hookimpl(trylast=True)
def pytest_configure(config):
    logging_plugin = config.pluginmanager.get_plugin("logging-plugin")
    logging_plugin.log_cli_handler.formatter.add_color_level(logging.INFO, "cyan")
    logging_plugin.log_cli_handler.formatter.add_color_level(logging.SPAM, "blue")  # Custom level

This feature is experimental and subject to change without notice.

Release Notes & Compatibility

The logging integration replaced the external pytest-catchlog plugin starting pytest 3.3.
It conflicts with pytest-catchlog; to disable the internal logging feature, add -p no:logging to your pytest.ini.
In pytest 3.4, some incompatible changes were made:
- Log levels are no longer forcibly changed unless explicitly configured.
- Live logs disabled by default, enabled via log_cli=true.
- Live logs now output to sys.stdout and do not require -s.
To partially restore 3.3 behavior, configure:
```
[pytest]
log_cli = true
log_level = NOTSET
```

Interactions with Other Parts of pytest

The logging functionality is tightly integrated with pytest's test execution phases (setup, call, teardown).
The caplog fixture interacts with Python's standard logging module to capture logs during tests.
CLI options and configuration file options allow users to customize logging behavior globally.
The live logging feature outputs logs in real time to the console, interacting with pytest's output capturing and reporting mechanisms.
The logging plugin can be accessed via pytest's plugin manager (logging-plugin) in hooks like pytest_configure.

Visual Diagram

The following Mermaid class diagram illustrates the main entity exposed by this feature: the `caplog` fixture, represented by the `LogCaptureFixture` class, showing its key methods and attributes that tests interact with.

classDiagram
    class LogCaptureFixture {
        +set_level(level: int, logger: Optional[str] = None)
        +at_level(level: int, logger: Optional[str] = None)
        +clear()
        +get_records(when: str) List[LogRecord]
        +records List[LogRecord]
        +record_tuples List[Tuple[str, int, str]]
        +text str
    }

    class LogRecord {
        +levelname: str
        +levelno: int
        +message: str
        +name: str
        +pathname: str
        +lineno: int
        +created: float
        +args: tuple
        +exc_info: Optional[tuple]
    }

    LogCaptureFixture --> LogRecord : "captures"

Usage Examples Summary

Changing log level during a test:

def test_example(caplog):
    caplog.set_level(logging.INFO)

Using context manager for temporary level:

def test_example(caplog):
    with caplog.at_level(logging.CRITICAL, logger="root.baz"):
        # test code
        pass

Asserting log messages:

def test_log_assert(caplog):
    some_function()
    assert ("root", logging.INFO, "expected message") in caplog.record_tuples