Merge pull request #139 from igorbenav/preparations_for_0_14

[WIP] Preparations for 0.14.0

Author: igorbenav

docs/changelog.md

@@ -5,6 +5,207 @@
 The Changelog documents all notable changes made to FastCRUD. This includes new features, bug fixes, and improvements. It's organized by version and date, providing a clear history of the library's development.
 
 ___
+## [0.14.0] - Jul 29, 2024
+
+#### Added
+- Type-checking support for SQLModel types by @kdcokenny 🚀
+- Returning clause to update operations by @feluelle
+- Upsert_multi functionality by @feluelle
+- Simplified endpoint configurations by @JakNowy, streamlining path generation and merging pagination functionalities into a unified `_read_items` endpoint, promoting more efficient API structure and usage. Details in https://github.com/igorbenav/fastcrud/pull/105
+
+#### Improved
+- Comprehensive tests for paginated retrieval of items, maintaining 100% coverage
+- Docker client check before running tests that require Docker by @feluelle
+
+#### Fixed
+- Vulnerability associated with an outdated cryptography package
+- Return type inconsistency in async session fixtures by @slaarti
+
+#### Documentation Updates
+- Cleanup of documentation formatting by @slaarti
+- Replacement of the Contributing section in docs with an include to file in repo root by @slaarti
+- Correction of links to advanced filters in docstrings by @slaarti
+- Backfill of docstring fixes across various modules by @slaarti
+- Enhanced filter documentation with new AND and OR clause examples, making complex queries more accessible and understandable.
+
+#### Models and Schemas Enhancements
+- Introduction of simple and one-off models (Batch 1) by @slaarti
+- Expansion to include models and schemas for Customers, Products, and Orders (Batch 2) by @slaarti
+
+#### Code Refinements
+- Resolution of missing type specifications in kwargs by @slaarti
+- Collapsed space adjustments for models/schemas in `fast_crud.py` by @slaarti
+
+#### Warnings
+- **Deprecation Notice**: `_read_paginated` endpoint is set to be deprecated and merged into `_read_items`. Users are encouraged to transition to the latter, utilizing optional pagination parameters. Full details and usage instructions provided to ensure a smooth transition.
+- **Future Changes Alert**: Default endpoint names in `EndpointCreator` are anticipated to be set to empty strings in a forthcoming major release, aligning with simplification efforts. Refer to https://github.com/igorbenav/fastcrud/issues/67 for more information.
+
+#### Detailed Changes
+___
+##### Simplified Endpoint Configurations
+
+In an effort to streamline FastCRUD’s API, we have reconfigured endpoint paths to avoid redundancy (great work by @JakNowy). This change allows developers to specify empty strings for endpoint names in the `crud_router` setup, which prevents the generation of unnecessary `//` in the paths. The following configurations illustrate how endpoints can now be defined more succinctly:
+
+```python
+endpoint_names = {
+    "create": "",
+    "read": "",
+    "update": "",
+    "delete": "",
+    "db_delete": "",
+    "read_multi": "",
+    "read_paginated": "get_paginated",
+}
+```
+
+Moreover, the `_read_paginated` logic has been integrated into the `_read_items` endpoint. This integration means that pagination can now be controlled via `page` and `items_per_page` query parameters, offering a unified method for both paginated and non-paginated reads:
+
+- **Paginated read example**:
+
+```bash
+curl -X 'GET' \
+  'http://localhost:8000/users/get_multi?page=2&itemsPerPage=10' \
+  -H 'accept: application/json'
+```
+
+- **Non-paginated read example**:
+
+```bash
+curl -X 'GET' \
+  'http://localhost:8000/users/get_multi?offset=0&limit=100' \
+  -H 'accept: application/json'
+```
+
+###### Warnings
+
+- **Deprecation Warning**: The `_read_paginated` endpoint is slated for deprecation. Developers should transition to using `_read_items` with the relevant pagination parameters.
+- **Configuration Change Alert**: In a future major release, default endpoint names in `EndpointCreator` will be empty strings by default, as discussed in [Issue #67](https://github.com/igorbenav/fastcrud/issues/67).
+
+###### Advanced Filters Documentation Update
+
+Documentation for advanced filters has been expanded to include comprehensive examples of AND and OR clauses, enhancing the utility and accessibility of complex query constructions.
+
+- **OR clause example**:
+
+```python
+# Fetch items priced under $5 or above $20
+items = await item_crud.get_multi(
+    db=db,
+    price__or={'lt': 5, 'gt': 20},
+)
+```
+
+- **AND clause example**:
+
+```python
+# Fetch items priced under $20 and over 2 years of warranty
+items = await item_crud.get_multi(
+    db=db,
+    price__lt=20,
+    warranty_years__gt=2,
+)
+```
+
+___
+##### Returning Clauses in Update Operations
+
+###### Description
+Users can now retrieve updated records immediately following an update operation. This feature streamlines the process, reducing the need for subsequent retrieval calls and increasing efficiency.
+
+###### Changes
+- **Return Columns**: Specify the columns to be returned after the update via the `return_columns` argument.
+- **Schema Selection**: Optionally select a Pydantic schema to format the returned data using the `schema_to_select` argument.
+- **Return as Model**: Decide if the returned data should be converted into a model using the `return_as_model` argument.
+- **Single or None**: Utilize the `one_or_none` argument to ensure that either a single record is returned or none, in case the conditions do not match any records.
+
+These additions are aligned with existing CRUD API functions, enhancing consistency across the library and making the new features intuitive for users.
+
+###### Usage Example
+
+###### Returning Updated Fields
+
+```python
+from fastcrud import FastCRUD
+from .models.item import Item
+from .database import session as db
+
+crud_items = FastCRUD(Item)
+updated_item = await crud_items.update(
+    db=db,
+    object={"price": 9.99},
+    price__lt=10,
+    return_columns=["price"]
+)
+# This returns the updated price of the item directly.
+```
+
+###### Returning Data as a Model
+
+```python
+from fastcrud import FastCRUD
+from .models.item import Item
+from .schemas.item import ItemSchema
+from .database import session as db
+
+crud_items = FastCRUD(Item)
+updated_item_schema = await crud_items.update(
+    db=db,
+    object={"price": 9.99},
+    price__lt=10,
+    schema_to_select=ItemSchema,
+    return_as_model=True
+)
+# This returns the updated item data formatted as an ItemSchema model.
+```
+
+___
+##### Bulk Upsert Operations with `upsert_multi`
+
+The `upsert_multi` method provides the ability to perform bulk upsert operations, which are optimized for different SQL dialects.
+
+###### Changes
+- **Dialect-Optimized SQL**: Uses the most efficient SQL commands based on the database's SQL dialect.
+- **Support for Multiple Dialects**: Includes custom implementations for PostgreSQL, SQLite, and MySQL, with appropriate handling for each's capabilities and limitations.
+
+###### Usage Example
+
+###### Upserting Multiple Records
+
+```python
+from fastcrud import FastCRUD
+from .models.item import Item
+from .schemas.item import ItemCreateSchema, ItemSchema
+from .database import session as db
+
+crud_items = FastCRUD(Item)
+items = await crud_items.upsert_multi(
+    db=db,
+    instances=[
+        ItemCreateSchema(price=9.99),
+    ],
+    schema_to_select=ItemSchema,
+    return_as_model=True,
+)
+# This will return the upserted data in the form of ItemSchema.
+```
+
+###### Implementation Details
+
+`upsert_multi` handles different database dialects:
+- **PostgreSQL**: Uses `ON CONFLICT DO UPDATE`.
+- **SQLite**: Utilizes `ON CONFLICT DO UPDATE`.
+- **MySQL**: Implements `ON DUPLICATE KEY UPDATE`.
+
+###### Notes
+- MySQL and MariaDB do not support certain advanced features used in other dialects, such as returning values directly after an insert or update operation. This limitation is clearly documented to prevent misuse.
+
+#### New Contributors
+- @kdcokenny made their first contribution 🌟
+- @feluelle made their first contribution 🌟
+
+**Full Changelog**: [View the full changelog](https://github.com/igorbenav/fastcrud/compare/v0.13.1...v0.14.0)
+
+
 ## [0.13.1] - Jun 22, 2024
 
 #### Added

docs/usage/endpoint.md

@@ -6,10 +6,6 @@ This section of the documentation explains how to use the `crud_router` utility
 
 Before proceeding, ensure you have FastAPI and FastCRUD installed in your environment. FastCRUD streamlines interactions with the database using SQLAlchemy models and Pydantic schemas.
 
-!!! WARNING
-
-    For now, your primary column in the database model must be named `id`.
-
 ---
 
 ## Using `crud_router`

fastcrud/crud/fast_crud.py

@@ -687,13 +687,12 @@ async def upsert_multi(
                     "MySQL does not support the returning clause for insert operations."
                 )
             statement, params = await self._upsert_multi_mysql(instances)
-        else:
+        else:  # pragma: no cover
             raise NotImplementedError(
                 f"Upsert multi is not implemented for {db.bind.dialect.name}"
             )
 
         if return_as_model:
-            # All columns are returned to ensure the model can be constructed
             return_columns = self.model_col_names
 
         if return_columns:
@@ -1975,7 +1974,6 @@ async def update(
         stmt = update(self.model).filter(*filters).values(update_data)
 
         if return_as_model:
-            # All columns are returned to ensure the model can be constructed
             return_columns = self.model_col_names
 
         if return_columns:
@@ -2007,12 +2005,12 @@ def _as_single_response(
         one_or_none: bool = False,
     ) -> Optional[Union[dict, BaseModel]]:
         result: Optional[Row] = db_row.one_or_none() if one_or_none else db_row.first()
-        if result is None:
+        if result is None:  # pragma: no cover
             return None
         out: dict = dict(result._mapping)
         if not return_as_model:
             return out
-        if not schema_to_select:
+        if not schema_to_select:  # pragma: no cover
             raise ValueError(
                 "schema_to_select must be provided when return_as_model is True."
             )
@@ -2029,14 +2027,14 @@ def _as_multi_response(
         response: dict[str, Any] = {"data": data}
 
         if return_as_model:
-            if not schema_to_select:
+            if not schema_to_select:  # pragma: no cover
                 raise ValueError(
                     "schema_to_select must be provided when return_as_model is True."
                 )
             try:
                 model_data = [schema_to_select(**row) for row in data]
                 response["data"] = model_data
-            except ValidationError as e:
+            except ValidationError as e:  # pragma: no cover
                 raise ValueError(
                     f"Data validation error for schema {schema_to_select.__name__}: {e}"
                 )

fastcrud/crud/helper.py

@@ -64,7 +64,7 @@ def _extract_matching_columns_from_schema(
         in the schema or all columns from the model if no schema is specified. These columns are correctly referenced
         through the provided alias if one is given.
     """
-    if not hasattr(model, "__table__"):
+    if not hasattr(model, "__table__"):  # pragma: no cover
         raise AttributeError(f"{model.__name__} does not have a '__table__' attribute.")
 
     model_or_alias = alias if alias else model
@@ -117,11 +117,11 @@ def _auto_detect_join_condition(
         ValueError: If the join condition cannot be automatically determined.
         AttributeError: If either base_model or join_model does not have a `__table__` attribute.
     """
-    if not hasattr(base_model, "__table__"):
+    if not hasattr(base_model, "__table__"):  # pragma: no cover
         raise AttributeError(
             f"{base_model.__name__} does not have a '__table__' attribute."
         )
-    if not hasattr(join_model, "__table__"):
+    if not hasattr(join_model, "__table__"):  # pragma: no cover
         raise AttributeError(
             f"{join_model.__name__} does not have a '__table__' attribute."
         )

fastcrud/endpoint/helper.py

@@ -83,7 +83,7 @@ def _get_primary_keys(
 ) -> Sequence[Column]:
     """Get the primary key of a SQLAlchemy model."""
     inspector_result = sa_inspect(model)
-    if inspector_result is None:
+    if inspector_result is None:  # pragma: no cover
         raise ValueError("Model inspection failed, resulting in None.")
     primary_key_columns: Sequence[Column] = inspector_result.mapper.primary_key
 
@@ -109,7 +109,7 @@ def _get_column_types(
 ) -> dict[str, Union[type, None]]:
     """Get a dictionary of column names and their corresponding Python types from a SQLAlchemy model."""
     inspector_result = sa_inspect(model)
-    if inspector_result is None or inspector_result.mapper is None:
+    if inspector_result is None or inspector_result.mapper is None:  # pragma: no cover
         raise ValueError("Model inspection failed, resulting in None.")
     column_types = {}
     for column in inspector_result.mapper.columns:
@@ -121,7 +121,7 @@ def _extract_unique_columns(
     model: ModelType,
 ) -> Sequence[KeyedColumnElement]:
     """Extracts columns from a SQLAlchemy model that are marked as unique."""
-    if not hasattr(model, "__table__"):
+    if not hasattr(model, "__table__"):  # pragma: no cover
         raise AttributeError(f"{model.__name__} does not have a '__table__' attribute.")
     unique_columns = [column for column in model.__table__.columns if column.unique]
     return unique_columns

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "fastcrud"
-version = "0.13.1"
+version = "0.14.0"
 description = "FastCRUD is a Python package for FastAPI, offering robust async CRUD operations and flexible endpoint creation utilities."
 authors = ["Igor Benav <igor.magalhaes.r@gmail.com>"]
 license = "MIT"

tests/sqlalchemy/conftest.py

@@ -285,7 +285,7 @@ class TaskRead(TaskReadSub):
     client: Optional[ClientRead]
 
 
-def is_docker_running() -> bool:
+def is_docker_running() -> bool:  # pragma: no cover
     try:
         DockerClient()
         return True
@@ -316,7 +316,7 @@ async def async_session(request: pytest.FixtureRequest) -> AsyncGenerator[AsyncS
     dialect_marker = request.node.get_closest_marker("dialect")
     dialect = dialect_marker.args[0] if dialect_marker else "sqlite"
     if dialect == "postgresql":
-        if not is_docker_running():
+        if not is_docker_running():  # pragma: no cover
             pytest.skip("Docker is required, but not running")
         with PostgresContainer(driver="psycopg") as pg:
             async with _async_session(
@@ -327,7 +327,7 @@ async def async_session(request: pytest.FixtureRequest) -> AsyncGenerator[AsyncS
         async with _async_session(url="sqlite+aiosqlite:///:memory:") as session:
             yield session
     elif dialect == "mysql":
-        if not is_docker_running():
+        if not is_docker_running():  # pragma: no cover
             pytest.skip("Docker is required, but not running")
         with MySqlContainer() as mysql:
             async with _async_session(
@@ -336,8 +336,8 @@ async def async_session(request: pytest.FixtureRequest) -> AsyncGenerator[AsyncS
                 )
             ) as session:
                 yield session
-    else:
-        raise ValueError(f"Unsupported dialect: {dialect}")
+    else:  # pragma: no cover
+        raise NotImplementedError(f"Unsupported dialect: {dialect}")
 
 
 @pytest.fixture(scope="function")

tests/sqlmodel/conftest.py

@@ -1,17 +1,22 @@
 from collections.abc import AsyncGenerator
 from typing import Optional
+from contextlib import asynccontextmanager
 
 import pytest
 import pytest_asyncio
 from datetime import datetime
 
 from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
 from sqlalchemy.orm import sessionmaker
+from sqlalchemy import make_url
 from pydantic import ConfigDict
 from sqlmodel import SQLModel, Field, Relationship
 from fastapi import FastAPI
 from fastapi.testclient import TestClient
 from sqlalchemy.sql import func
+from testcontainers.postgres import PostgresContainer
+from testcontainers.mysql import MySqlContainer
+from testcontainers.core.docker_client import DockerClient
 
 from fastcrud.crud.fast_crud import FastCRUD
 from fastcrud.endpoint.crud_router import crud_router
@@ -267,25 +272,60 @@ class TaskRead(TaskReadSub):
     client: Optional[ClientRead]
 
 
+def is_docker_running() -> bool:  # pragma: no cover
+    try:
+        DockerClient()
+        return True
+    except Exception:
+        return False
+
+
 async_engine = create_async_engine(
     "sqlite+aiosqlite:///:memory:", echo=True, future=True
 )
 
 
-@pytest_asyncio.fixture(scope="function")
-async def async_session() -> AsyncGenerator[AsyncSession]:
-    session = sessionmaker(async_engine, class_=AsyncSession, expire_on_commit=False)
-
-    async with session() as s:
-        async with async_engine.begin() as conn:
+@asynccontextmanager
+async def _setup_database(url: str) -> AsyncGenerator[AsyncSession]:
+    engine = create_async_engine(url, echo=True, future=True)
+    session_maker = sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
+    async with session_maker() as session:
+        async with engine.begin() as conn:
             await conn.run_sync(SQLModel.metadata.create_all)
+        try:
+            yield session
+        finally:
+            async with engine.begin() as conn:
+                await conn.run_sync(SQLModel.metadata.drop_all)
+    await engine.dispose()
 
-        yield s
-
-    async with async_engine.begin() as conn:
-        await conn.run_sync(SQLModel.metadata.drop_all)
 
-    await async_engine.dispose()
+@pytest_asyncio.fixture(scope="function")
+async def async_session(
+    request: pytest.FixtureRequest,
+) -> AsyncGenerator[AsyncSession]:  # pragma: no cover
+    dialect_marker = request.node.get_closest_marker("dialect")
+    dialect = dialect_marker.args[0] if dialect_marker else "sqlite"
+
+    if dialect == "postgresql" or dialect == "mysql":
+        if not is_docker_running():
+            pytest.skip("Docker is required, but not running")
+
+        if dialect == "postgresql":
+            with PostgresContainer() as postgres:
+                url = postgres.get_connection_url()
+                async with _setup_database(url) as session:
+                    yield session
+        elif dialect == "mysql":
+            with MySqlContainer() as mysql:
+                url = make_url(mysql.get_connection_url())._replace(
+                    drivername="mysql+aiomysql"
+                )
+                async with _setup_database(url) as session:
+                    yield session
+    else:
+        async with _setup_database("sqlite+aiosqlite:///:memory:") as session:
+            yield session
 
 
 @pytest.fixture(scope="function")