Rework of the URL subtraction feature

[babieiev]

What do these changes do?

Rework of the URL subtraction feature (added in #1340; removed in #1391)

Are there changes in behavior for the user?

Being able to calculate the relative path between two URLs:

from yarl import URL

target_url = URL("http://example.com/path/index.html")
base_url = URL("http://example.com/path/")

relative_url = target_url.relative_to(base_url)

print(relative_url)  # output: "index.html"

Related issue number

Resolves #1183

Checklist

• I think the code is well written
• Unit tests for the changes exist
• Documentation reflects the changes

Known issues

• Time complexity is roughly O(base_path_segments * target_path_segments)
• Empty segments are not handled correctly Reduce time complexity of URL subtraction #1388
• Potentially handling special cases with leading/trailing slashes Reduce time complexity of URL subtraction #1388

[codspeed-hq]

CodSpeed Performance Report

Merging #1392 will not alter performance

_{Comparing babieiev:subtraction (e15878f) with master (937f030)}

Summary

✅ 101 untouched benchmarks
🆕 2 new benchmarks

Benchmarks breakdown

	Benchmark	BASE	HEAD	Change
🆕	test_relative_to	N/A	829.8 µs	N/A
🆕	test_relative_to_long_urls	N/A	4.5 ms	N/A

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.84%. Comparing base (937f030) to head (e15878f).

❌ Your project check has failed because the head coverage (98.84%) is below the target coverage (100.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1392      +/-   ##
==========================================
- Coverage   98.93%   98.84%   -0.10%     
==========================================
  Files          32       25       -7     
  Lines        6091     5007    -1084     
  Branches      365      347      -18     
==========================================
- Hits         6026     4949    -1077     
+ Misses         62       56       -6     
+ Partials        3        2       -1     

Flag	Coverage Δ
CI-GHA	98.80% <100.00%> (-0.14%)	⬇️
MyPy	?
OS-Linux	98.80% <100.00%> (+0.01%)	⬆️
OS-Windows	98.83% <100.00%> (+0.01%)	⬆️
OS-macOS	98.58% <100.00%> (+0.01%)	⬆️
Py-3.10.11	98.56% <100.00%> (+0.01%)	⬆️
Py-3.10.16	98.76% <100.00%> (+0.01%)	⬆️
Py-3.11.11	98.76% <100.00%> (+0.01%)	⬆️
Py-3.11.9	98.56% <100.00%> (+0.01%)	⬆️
Py-3.12.9	98.76% <100.00%> (+0.01%)	⬆️
Py-3.13.2	98.76% <100.00%> (+0.01%)	⬆️
Py-3.9.13	98.52% <100.00%> (+0.01%)	⬆️
Py-3.9.21	98.72% <100.00%> (+0.01%)	⬆️
Py-pypy7.3.16	98.71% <100.00%> (+0.01%)	⬆️
Py-pypy7.3.19	98.73% <100.00%> (+0.01%)	⬆️
VM-macos-latest	98.58% <100.00%> (+0.01%)	⬆️
VM-ubuntu-latest	98.80% <100.00%> (+0.01%)	⬆️
VM-windows-latest	98.83% <100.00%> (+0.01%)	⬆️
pytest	98.80% <100.00%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[bdraco]

I'm going to do a 1.17.0 release so we can start preparing aiohttp 3.11.x and I want do to some more downstream cleanups. We can do a 1.18.0 for this once its ready

[bdraco]

Some conflicts happened. I think I've resolved them correctly

[asvetlov]

While I support the intention I propose more conservative approach.
Let's provide a method, name it, instead of __sub__ operator overloading.
Add tests, doc, etc.

Publish new yarl release, wait for feedback.
Tune and polish.
After some time we can return to adding __sub__.

[commonism]

I disagree with some of the unit tests, as I expect the returned relative path to add up to the complete path when concatenated as a relative path.

def test_relative_to(target: str, base: str, expected: str):
    expected_url = URL(expected)
    target_url = URL(target)
    base_url = URL(base)
    result_url = target_url.relative_to(base_url)
    assert result_url == expected_url
    combined = base_url / expected
    assert target_url == combined

Please include this in the unit tests, it'll help to align those with the idea of a relative path and harden the semantics on this operation wrt. to trailing / in base and result.

[webknjaz]

Please, avoid touching anything in the packaging/ directory.

[babieiev]

Please, avoid touching anything in the packaging/ directory.

Ok, but this was an auto-fix from pre-commit hooks. The bot added it after I had already pushed the changes 🤷‍♂️

[webknjaz]

Oh, that's weird. I wonder if the pre-commit.ci cache got invalidated randomly and it pulled in some newer transitive deps...

[babieiev]

    combined = base_url / expected
    assert target_url == combined

Please include this in the unit tests, it'll help to align those with the idea of a relative path and harden the semantics on this operation wrt. to trailing / in base and result.

@commonism this condition will only be true if we first normalize the URLs before comparing, so I've modified this code a bit:

combined_url = base_url / expected
combined_path = normalized_path(combined_url.path)
target_path = normalized_path(target_url.path)
assert combined_path == target_path

If we don't normalize them, the following can happen:

target_url = URL("/path/to")
base_url = URL("/spam")
relative_url = target_url.relative_to(base_url)
print(relative_url) # URL("../path/to")
combined_url = base_url / relative_url
print(combined_url) # URL("/spam/../path/to") => combined_url != target_url

But now everything seems to be working properly.