Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 43 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,37 @@
# pytest-httpdbg

A pytest plugin to record HTTP(S) requests with stack trace.

![](ui.png)
A pytest plugin for recording HTTP(S) requests and saving them in your test report.

## installation

```
pip install pytest-httpdbg
```

## usage

### pytest custom options

```
--httpdbg record HTTP(S) requests
--httpdbg-dir=HTTPDBG_DIR
save httpdbg traces in a directory
--httpdbg-no-clean do not clean the httpdbg directory
--httpdbg-initiator=HTTPDBG_INITIATOR
add a new initiator (package) for httpdbg
```
### option httpdbg

Enables the record of the HTTP requests.
## Allure report

### option httpdbg-dir
If you use the [allure-pytest](https://pypi.org/project/allure-pytest/) plugin to generate an [Allure](https://allurereport.org/docs/pytest/) report, you can use [pytest-httpdbg](https://pypi.org/project/pytest-httpdbg/) to include HTTP request traces in your test report without any code modifications.

Indicates where to save the log files.
All you need to do is add the `--httpdbg-allure` option to your pytest command line:

### option httpdbg-no-clean

Does not clean existing log files if the log directory is not empty.
```
pytest ../httpdbg-docs/examples/ --alluredir=./allure-results --httpdbg-allure
```

### option http-initiator
If an HTTP request is made by the test (or within a fixture, during the setup or teardown phase), the request will be saved in the Allure report under a step called `httpdbg`.

An initiator is the function/method that is at the origin of the HTTP requests. By default, we already support some packages but you can add your own initiators.
![](https://github.com/cle-b/pytest-httpdbg/blob/main/pytest-httpdbg-allure-0.8.0.png?raw=true)

To add a new package in the list of initiators, you can use the `http-initiator` command line argument.

You can use any package as an initiator, this is not limited to HTTP requests.
## Custom test report

## test report
You can add HTTP traces to any test report of your choice. To do this, you can use the HTTP traces saved by the plugin in Markdown format.

When the test is finished (teardown step included), one log file in markdown format is written. The path to this log file is stashed in the item when the test starts (before the setup step), even if the file not exists yet.
When a test finishes (including the teardown step), a log file in Markdown format is generated. The path to this log file is stored in the test item when the test starts (before the setup step), even if the file does not yet exist.

### pytest-html

You can copy the following code in your top-level `conftest.py` to include the logs into your pytest-html report.
You can copy the following code in your top-level `conftest.py` to include the logs into your `pytest-html` report.

```python
import os
Expand Down Expand Up @@ -82,6 +65,36 @@ This example works if you use the same directory for the html test report file a

If this is not the case, you must adapt it to your configuration.

![](https://github.com/cle-b/pytest-httpdbg/blob/main/ui.png?raw=true)

## pytest command line options

```
reporting:

--httpdbg record HTTP(S) requests
--httpdbg-dir=HTTPDBG_DIR save httpdbg traces in a directory
--httpdbg-no-clean do not clean the httpdbg directory

--httpdbg-allure save HTTP(S) traces into the allure report
--httpdbg-no-headers save the HTTP headers
--httpdbg-no-binary do not save the HTTP payload if it's a binary content
--httpdbg-only-on-failure save the HTTP requests only if the test failed

--httpdbg-initiator=HTTPDBG_INITIATOR add a new initiator (package) for httpdbg

```

## httpdbg

This plugin is based on the [httpdbg](https://pypi.org/project/httpdbg/) Python tool. You can use it to trace all HTTP requests in your tests and view them in a more detailed user interface using the `pyhttpdbg` command.

```
pyhttpdbg -m pytest -v examples/
```

![](https://github.com/cle-b/pytest-httpdbg/blob/main/httpdbg-pytest-1.2.1.png?raw=true)

## documentation

https://httpdbg.readthedocs.io/en/latest/pytest/
Binary file added httpdbg-pytest-1.2.1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added pytest-httpdbg-allure-0.8.0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion pytest_httpdbg/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# -*- coding: utf-8 -*-
from pytest_httpdbg.plugin import httpdbg_record_filename # noqa F401

__version__ = "0.7.0"
__version__ = "0.8.0"
203 changes: 196 additions & 7 deletions pytest_httpdbg/plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import glob
import os
import time
import traceback
from typing import Optional

import pytest
Expand Down Expand Up @@ -62,27 +63,66 @@ def record_to_md(record, initiators):


def pytest_addoption(parser):
parser.addoption("--httpdbg", action="store_true", help="record HTTP(S) requests")
parser.addoption(

reporting_group = parser.getgroup("reporting")

# mode custom
reporting_group.addoption(
"--httpdbg", action="store_true", help="record HTTP(S) requests"
)

reporting_group.addoption(
"--httpdbg-dir", type=str, default="", help="save httpdbg traces in a directory"
)
parser.addoption(

reporting_group.addoption(
"--httpdbg-no-clean",
action="store_true",
default=False,
help="do not clean the httpdbg directory",
)
parser.addoption(

# mode allure
reporting_group.addoption(
"--httpdbg-allure",
action="store_true",
help="save HTTP(S) traces into the allure report",
)

reporting_group.addoption(
"--httpdbg-no-headers",
action="store_true",
default=False,
help="save the HTTP headers",
)

reporting_group.addoption(
"--httpdbg-no-binary",
action="store_true",
default=False,
help="do not save the HTTP payload if it's a binary content",
)

reporting_group.addoption(
"--httpdbg-only-on-failure",
action="store_true",
default=False,
help="save the HTTP requests only if the test failed",
)

reporting_group.addoption(
"--httpdbg-initiator",
action="append",
help="add a new initiator (package) for httpdbg",
)


def pytest_configure(config):
# add a flag to indicates to HTTPDBG to not set specific initiator
if config.option.httpdbg:
os.environ["HTTPDBG_PYTEST_PLUGIN"] = "1"

if config.option.httpdbg is True and config.option.httpdbg_allure is True:
pytest.exit(
"Error: --httpdbg and --httpdbg-allure are mutually exclusive. Please specify only one."
)

# clean logs directory
httpdbg_dir = config.option.httpdbg_dir
Expand Down Expand Up @@ -124,3 +164,152 @@ def pytest_runtest_protocol(item: pytest.Item, nextitem: Optional[pytest.Item]):
f.write(f"{record_to_md(record, records.initiators)}\n")
else:
yield


# Allure mode: HTTP requests are recorded throughout the entire session and
# saved in the Allure report at the test level.
def pytest_sessionstart(session):
if session.config.option.httpdbg_allure:
session.httpdbg_recorder = httprecord(
initiators=session.config.option.httpdbg_initiator
)
session.httpdbg_records = session.httpdbg_recorder.__enter__()


def pytest_sessionfinish(session, exitstatus):
if session.config.option.httpdbg_allure:
session.httpdbg_recorder.__exit__(None, None, None)


def get_allure_attachment_type_from_content_type(content_type: str):
try:
import allure

for attachment_type in allure.attachment_type:
if attachment_type.mime_type == content_type:
return attachment_type
except ImportError:
pass
return None


def req_resp_steps(label, req, save_headers, save_binary_payload):
try:
import allure

# we generate the payload first because we do not want to add a step
# if there is no headers and no payload to save
content = req.preview
payload = None
if content.get("text"):
payload = content.get("text")
elif save_binary_payload:
payload = req.content

if save_headers or payload:
with allure.step(label):
if save_headers:
allure.attach(
req.rawheaders.decode("utf-8"),
name="headers",
attachment_type=allure.attachment_type.TEXT,
)
if payload:
attachment_type = get_allure_attachment_type_from_content_type(
content.get("content_type")
)
allure.attach(
payload, name="payload", attachment_type=attachment_type
)
except ImportError:
pass


@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):

outcome = yield
report = outcome.get_result()

if item.config.option.httpdbg_allure:
# we keep the information about the status of the test for all phases
item.passed = getattr(item, "passed", True) and report.passed

if report.when == "teardown":
if (not item.config.option.httpdbg_only_on_failure) or (not item.passed):
try:
import allure

with allure.step("httpdbg"):

records = item.session.httpdbg_records

for record in records:

label = ""

if record.response.status_code:
label += f"{record.response.status_code} "

if record.request.method:
label += f"{record.request.method} "

if record.request.uri:
url = record.request.uri
else:
url = record.url
if len(url) > 200:
url = url[:100] + "..." + url[-97:]
ex = (
(str(type(record.exception)) + " ")
if record.exception is not None
else ""
)
label += f"{ex}{url}"

if record.tag:
label += f" (from {record.tag})"

with allure.step(label):
details = record.url
details += f"\n\nstatus: {record.response.status_code} {record.response.message}"
details += f"\n\nstart: {record.tbegin.isoformat()}"
details += f"\nend: {record.last_update.isoformat()}"

if record.initiator_id in records.initiators:
details += f"\n\n{records.initiators[record.initiator_id].short_stack}"

if record.exception is not None:
details += (
f"\n\nException: {type(record.exception)}\n"
)
details += "".join(
traceback.format_exception(
type(record.exception),
record.exception,
record.exception.__traceback__,
)
)

allure.attach(
details,
name="details",
attachment_type=allure.attachment_type.TEXT,
)

req_resp_steps(
"request",
record.request,
not item.config.option.httpdbg_no_headers,
not item.config.option.httpdbg_no_binary,
)
req_resp_steps(
"response",
record.response,
not item.config.option.httpdbg_no_headers,
not item.config.option.httpdbg_no_binary,
)
except ImportError:
pass

item.session.httpdbg_records.reset()
3 changes: 2 additions & 1 deletion requirements-dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@ Werkzeug<2.1
flask>2
pytest-httpbin
pytest-html
pytest-xdist
pytest-xdist
allure-pytest
Loading