Skip to content

Commit e66d51f

Browse files
author
Two Dev
committed
release
1 parent 1e837ad commit e66d51f

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+6217
-2
lines changed

‎.github/workflows/ci.yml

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
name: CI/CD
2+
3+
on:
4+
push:
5+
branches: ['master', 'main', 'dev', 'develop']
6+
7+
pull_request:
8+
branches: ['master', 'main', 'dev', 'develop']
9+
10+
jobs:
11+
build:
12+
runs-on: ubuntu-latest
13+
strategy:
14+
max-parallel: 4
15+
matrix:
16+
python-version: ['3.9', '3.10', '3.11']
17+
steps:
18+
- uses: actions/checkout@v4
19+
- name: Set up Python ${{ matrix.python-version }}
20+
uses: actions/setup-python@v4
21+
with:
22+
python-version: ${{ matrix.python-version }}
23+
24+
- name: Install Dependencies
25+
run: |
26+
python -m pip install --upgrade pip
27+
pip install -r requirements.txt
28+
29+
- name: Run pre-commit
30+
uses: pre-commit/[email protected]
31+
32+
deploy:
33+
needs: build
34+
runs-on: ubuntu-latest
35+
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
36+
steps:
37+
- uses: actions/checkout@v4
38+
- name: Configure Git Credentials
39+
run: |
40+
git config user.name github-actions[bot]
41+
git config user.email 41898282+github-actions[bot]@users.noreply.github.com
42+
- uses: actions/setup-python@v5
43+
with:
44+
python-version: '3.10'
45+
- run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
46+
- uses: actions/cache@v4
47+
with:
48+
key: mkdocs-material-${{ env.cache_id }}
49+
path: .cache
50+
restore-keys: |
51+
mkdocs-material-
52+
- run: pip install -r requirements.txt
53+
- run: mkdocs gh-deploy --force

‎.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -159,4 +159,4 @@ cython_debug/
159159
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
160160
# and can be added to the global gitignore or merged into this file. For a more nuclear
161161
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
162-
#.idea/
162+
.idea/

‎.pre-commit-config.yaml

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
exclude: '^docs.sh/|scripts/'
2+
default_stages: [commit]
3+
4+
default_language_version:
5+
python: python3.10
6+
7+
repos:
8+
- repo: https://github.com/pre-commit/pre-commit-hooks
9+
rev: v4.5.0
10+
hooks:
11+
- id: trailing-whitespace
12+
- id: end-of-file-fixer
13+
- id: check-json
14+
- id: check-toml
15+
- id: check-xml
16+
- id: check-yaml
17+
- id: debug-statements
18+
- id: check-builtin-literals
19+
- id: check-case-conflict
20+
- id: check-docstring-first
21+
- id: detect-private-key
22+
23+
# run the isort.
24+
- repo: https://github.com/PyCQA/isort
25+
rev: 5.13.2
26+
hooks:
27+
- id: isort
28+
29+
# run the flake8.
30+
- repo: https://github.com/PyCQA/flake8
31+
rev: 7.0.0
32+
hooks:
33+
- id: flake8

‎CHANGELOG.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Release History
2+
===============
3+
4+
1.0.0 (2024-12-04)
5+
-------------------
6+
* First release

‎MANIFEST.in

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
include LICENSE
2+
include README.md
3+
include CHANGELOG.md
4+
recursive-include tls_requests docs Makefile *.md *.rst

‎Makefile

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
.PHONY: docs
2+
init:
3+
python -m pip install -r requirements.txt
4+
5+
test-readme:
6+
python setup.py check --restructuredtext --strict && ([ $$? -eq 0 ] && echo "README.rst and CHANGELOG.md ok") || echo "Invalid markup in README.md or CHANGELOG.md!"
7+
8+
lint:
9+
python -m black tls_requests
10+
python -m isort tls_requests
11+
python -m flake8 tls_requests
12+
13+
publish-pypi:
14+
python -m pip install 'twine>=6.0.1'
15+
python setup.py sdist bdist_wheel
16+
twine upload dist/*
17+
rm -rf build dist .egg wrapper_tls_requests.egg-info
18+
19+
docs:
20+
mkdocs serve

‎README.md

Lines changed: 112 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,112 @@
1-
# tls-requests
1+
# TLS REQUESTS
2+
**A powerful and lightweight Python library for making secure and reliable HTTP/TLS Fingerprint requests.**
3+
4+
* * *
5+
6+
**Installation**
7+
----------------
8+
9+
To install the library, you can choose between two methods:
10+
11+
#### **1\. Install via PyPI:**
12+
13+
```shell
14+
pip install wrapper-tls-requests
15+
```
16+
17+
#### **2\. Install via GitHub Repository:**
18+
19+
```shell
20+
pip install git+https://github.com/thewebscraping/tls-requests.git
21+
```
22+
23+
**Quick Start**
24+
---------------
25+
26+
Start using TLS Requests with just a few lines of code:
27+
28+
```pycon
29+
>>> import tls_requests
30+
>>> r = tls_requests.get("https://httpbin.org/get")
31+
>>> r
32+
<Response [200 OK]>
33+
>>> r.status_code
34+
200
35+
```
36+
37+
**Introduction**
38+
----------------
39+
40+
**TLS Requests** is a cutting-edge HTTP client for Python, offering a feature-rich,
41+
highly configurable alternative to the popular [`requests`](https://github.com/psf/requests) library.
42+
43+
Built on top of [`tls-client`](https://github.com/bogdanfinn/tls-client),
44+
it combines ease of use with advanced functionality for secure networking.
45+
46+
**Acknowledgment**: A big thank you to all contributors for their support!
47+
48+
### **Key Benefits**
49+
50+
* **Bypass TLS Fingerprinting:** Mimic browser-like behaviors to navigate sophisticated anti-bot systems.
51+
* **Customizable TLS Clients:** Select specific TLS fingerprints to meet your needs.
52+
* **Ideal for Developers:** Build scrapers, API clients, or other custom networking tools effortlessly.
53+
54+
55+
**Why Use TLS Requests?**
56+
-------------------------
57+
58+
Modern websites increasingly use **TLS Fingerprinting** and anti-bot tools like Cloudflare Bot Fight Mode to block web crawlers.
59+
60+
**TLS Requests** bypasses these obstacles by mimicking browser-like TLS behaviors,
61+
making it easy to scrape data or interact with websites that use sophisticated anti-bot measures.
62+
63+
### Example: Unlocking Cloudflare Bot Fight Mode
64+
![coingecko.png](https://raw.githubusercontent.com/thewebscraping/tls-requests/refs/heads/main/docs/static/coingecko.png)
65+
66+
**Example Code:**
67+
68+
```pycon
69+
>>> import tls_requests
70+
>>> r = tls_requests.get('https://www.coingecko.com/')
71+
>>> r
72+
<Response [200]>
73+
```
74+
75+
**Key Features**
76+
----------------
77+
78+
### **Enhanced Capabilities**
79+
80+
* **Browser-like TLS Fingerprinting**: Enables secure and reliable browser-mimicking connections.
81+
* **High-Performance Backend**: Built on a Go-based HTTP backend for speed and efficiency.
82+
* **Synchronous & Asynchronous Support**: Seamlessly switch between synchronous and asynchronous requests.
83+
* **Protocol Support**: Fully compatible with HTTP/1.1 and HTTP/2.
84+
* **Strict Timeouts**: Reliable timeout management for precise control over request durations.
85+
86+
### **Additional Features**
87+
88+
* **Internationalized Domain & URL Support**: Handles non-ASCII URLs effortlessly.
89+
* **Cookie Management**: Ensures session-based cookie persistence.
90+
* **Authentication**: Native support for Basic and Function authentication.
91+
* **Content Decoding**: Automatic handling of gzip and brotli-encoded responses.
92+
* **Hooks**: Perfect for logging, monitoring, tracing, or pre/post-processing requests and responses.
93+
* **Unicode Support**: Effortlessly process Unicode response bodies.
94+
* **File Uploads**: Simplified multipart file upload support.
95+
* **Proxy Configuration**: Supports Socks5, HTTP, and HTTPS proxies for enhanced privacy.
96+
97+
98+
**Documentation**
99+
-----------------
100+
101+
Explore the full capabilities of TLS Requests in the documentation:
102+
103+
* **[Quickstart Guide](https://thewebscraping.github.io/tls-requests/quickstart/)**: A beginner-friendly guide.
104+
* **[Advanced Topics](https://thewebscraping.github.io/tls-requests/advanced/client/)**: Learn to leverage specialized features.
105+
* **[Async Support](https://thewebscraping.github.io/tls-requests/advanced/async_client/)**: Handle high-concurrency scenarios.
106+
* **Custom TLS Configurations**:
107+
* **[Wrapper TLS Client](https://thewebscraping.github.io/tls-requests/tls/)**
108+
* **[TLS Client Profiles](https://thewebscraping.github.io/tls-requests/tls/profiles/)**
109+
* **[Custom TLS Configurations](https://thewebscraping.github.io/tls-requests/tls/configuration/)**
110+
111+
112+
Read the documentation: [**thewebscraping.github.io/tls-requests/**](https://thewebscraping.github.io/tls-requests/)

‎docs/advanced/async_client.md

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
Async Support in TLS Requests
2+
=============================
3+
4+
TLS Requests provides support for asynchronous HTTP requests using the `AsyncClient`. This is especially useful when working in an asynchronous environment, such as with modern web frameworks, or when you need the performance benefits of asynchronous I/O.
5+
6+
* * *
7+
8+
Why Use Async?
9+
--------------
10+
11+
* **Improved Performance:** Async is more efficient than multi-threading for handling high concurrency workloads.
12+
* **Long-lived Connections:** Useful for protocols like WebSockets or long polling.
13+
* **Framework Compatibility:** Essential when integrating with async web frameworks (e.g., FastAPI, Starlette).
14+
15+
Advanced usage with syntax similar to Client, refer to the [Client documentation](client).
16+
17+
* * *
18+
19+
Making Async Requests
20+
---------------------
21+
22+
To send asynchronous HTTP requests, use the `AsyncClient`:
23+
24+
```pycon
25+
>>> import asyncio
26+
>>> async def fetch(url):
27+
async with tls_requests.AsyncClient() as client:
28+
r = await client.get("https://www.example.com/")
29+
return r
30+
31+
>>> r = asyncio.run(fetch("https://httpbin.org/get"))
32+
>>> r
33+
<Response [200 OK]>
34+
```
35+
36+
!!! tip
37+
Use [IPython](https://ipython.readthedocs.io/en/stable/) or Python 3.8+ with `python -m asyncio` to try this code interactively, as they support executing `async`/`await` expressions in the console.
38+
39+
* * *
40+
41+
Key API Differences
42+
-------------------
43+
44+
When using `AsyncClient`, the API methods are asynchronous and must be awaited.
45+
46+
### Making Requests
47+
48+
Use `await` for all request methods:
49+
50+
* `await client.get(url, ...)`
51+
* `await client.post(url, ...)`
52+
* `await client.put(url, ...)`
53+
* `await client.patch(url, ...)`
54+
* `await client.delete(url, ...)`
55+
* `await client.options(url, ...)`
56+
* `await client.head(url, ...)`
57+
* `await client.request(method, url, ...)`
58+
* `await client.send(request, ...)`
59+
60+
* * *
61+
62+
### Managing Client Lifecycle
63+
64+
#### Context Manager
65+
66+
For proper resource cleanup, use `async with`:
67+
68+
```python
69+
import asyncio
70+
71+
async def fetch(url):
72+
async with tls_requests.AsyncClient() as client:
73+
response = await client.get(url)
74+
return response
75+
76+
r = asyncio.run(fetch("https://httpbin.org/get"))
77+
print(r) # <Response [200 OK]>
78+
```
79+
80+
#### Manual Closing
81+
82+
Alternatively, explicitly close the client:
83+
84+
```python
85+
import asyncio
86+
87+
async def fetch(url):
88+
client = tls_requests.AsyncClient()
89+
try:
90+
response = await client.get("https://www.example.com/")
91+
finally:
92+
await client.aclose()
93+
```
94+
95+
* * *
96+
97+
By using `AsyncClient`, you can unlock the full potential of asynchronous programming in Python while enjoying the simplicity and power of TLS Requests.

0 commit comments

Comments
 (0)