Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Version changelog

## Release v0.1.0

Initial release of the Databricks Zerobus Ingest SDK for Python.

### API Changes

- Added `ZerobusSdk` class for creating ingestion streams
- Added `ZerobusStream` class for managing stateful gRPC streams
- Added `RecordAcknowledgment` for blocking until record acknowledgment
- Added asynchronous versions: `zerobus.sdk.aio.ZerobusSdk` and `zerobus.sdk.aio.ZerobusStream`
- Added `TableProperties` for configuring table schema and name
- Added `StreamConfigurationOptions` for stream behavior configuration
- Added `ZerobusException` and `NonRetriableException` for error handling
- Added `StreamState` enum for tracking stream lifecycle
- Support for Python 3.9, 3.10, 3.11, 3.12, and 3.13
69 changes: 69 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
Databricks License
Copyright (2022) Databricks, Inc.

Definitions.

Agreement: The agreement between Databricks, Inc., and you governing
the use of the Databricks Services, as that term is defined in
the Master Cloud Services Agreement (MCSA) located at
www.databricks.com/legal/mcsa.

Licensed Materials: The source code, object code, data, and/or other
works to which this license applies.

Scope of Use. You may not use the Licensed Materials except in
connection with your use of the Databricks Services pursuant to
the Agreement. Your use of the Licensed Materials must comply at all
times with any restrictions applicable to the Databricks Services,
generally, and must be used in accordance with any applicable
documentation. You may view, use, copy, modify, publish, and/or
distribute the Licensed Materials solely for the purposes of using
the Licensed Materials within or connecting to the Databricks Services.
If you do not agree to these terms, you may not view, use, copy,
modify, publish, and/or distribute the Licensed Materials.

Redistribution. You may redistribute and sublicense the Licensed
Materials so long as all use is in compliance with these terms.
In addition:

- You must give any other recipients a copy of this License;
- You must cause any modified files to carry prominent notices
stating that you changed the files;
- You must retain, in any derivative works that you distribute,
all copyright, patent, trademark, and attribution notices,
excluding those notices that do not pertain to any part of
the derivative works; and
- If a "NOTICE" text file is provided as part of its
distribution, then any derivative works that you distribute
must include a readable copy of the attribution notices
contained within such NOTICE file, excluding those notices
that do not pertain to any part of the derivative works.

You may add your own copyright statement to your modifications and may
provide additional license terms and conditions for use, reproduction,
or distribution of your modifications, or for any such derivative works
as a whole, provided your use, reproduction, and distribution of
the Licensed Materials otherwise complies with the conditions stated
in this License.

Termination. This license terminates automatically upon your breach of
these terms or upon the termination of your Agreement. Additionally,
Databricks may terminate this license at any time on notice. Upon
termination, you must permanently delete the Licensed Materials and
all copies thereof.

DISCLAIMER; LIMITATION OF LIABILITY.

THE LICENSED MATERIALS ARE PROVIDED “AS-IS” AND WITH ALL FAULTS.
DATABRICKS, ON BEHALF OF ITSELF AND ITS LICENSORS, SPECIFICALLY
DISCLAIMS ALL WARRANTIES RELATING TO THE LICENSED MATERIALS, EXPRESS
AND IMPLIED, INCLUDING, WITHOUT LIMITATION, IMPLIED WARRANTIES,
CONDITIONS AND OTHER TERMS OF MERCHANTABILITY, SATISFACTORY QUALITY OR
FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. DATABRICKS AND
ITS LICENSORS TOTAL AGGREGATE LIABILITY RELATING TO OR ARISING OUT OF
YOUR USE OF OR DATABRICKS’ PROVISIONING OF THE LICENSED MATERIALS SHALL
BE LIMITED TO ONE THOUSAND ($1,000) DOLLARS. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE LICENSED MATERIALS OR
THE USE OR OTHER DEALINGS IN THE LICENSED MATERIALS.
12 changes: 12 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
include README.md
include LICENSE
include NOTICE
include CHANGELOG.md
include DCO
include Makefile
include SECURITY.md
include CONTRIBUTING.md
include pyproject.toml

recursive-exclude examples *
recursive-exclude tests *
13 changes: 13 additions & 0 deletions NEXT_CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# NEXT CHANGELOG

## Release v0.2.0

### New Features and Improvements

### Bug Fixes

### Documentation

### Internal Changes

### API Changes
26 changes: 26 additions & 0 deletions NOTICE
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
Copyright (2025) Databricks, Inc.

This Software includes software developed at Databricks (https://www.databricks.com/) and its use is subject to the included LICENSE file.

---

This Software includes the following open source components:

## Apache License 2.0

### requests
Copyright 2019 Kenneth Reitz
https://github.com/psf/requests
License: https://github.com/psf/requests/blob/main/LICENSE

### grpcio
Copyright 2014 gRPC authors
https://github.com/grpc/grpc
License: https://github.com/grpc/grpc/blob/master/LICENSE

## BSD 3-Clause License

### protobuf
Copyright 2008 Google Inc.
https://github.com/protocolbuffers/protobuf
License: https://github.com/protocolbuffers/protobuf/blob/main/LICENSE
28 changes: 19 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,18 @@
# Databricks Zerobus Ingest SDK for Python

[![Python](https://img.shields.io/badge/python-3.9+-blue)](https://www.python.org/)
[![PyPI - Downloads](https://img.shields.io/pypi/dw/databricks-zerobus-ingest-sdk)](https://pypistats.org/packages/databricks-zerobus-ingest-sdk)
[![PyPI - License](https://img.shields.io/pypi/l/databricks-zerobus-ingest-sdk)](https://github.com/databricks/zerobus-sdk-py/blob/main/LICENSE)
![PyPI](https://img.shields.io/pypi/v/databricks-zerobus-ingest-sdk)

The Databricks Zerobus Ingest SDK for Python provides a high-performance client for ingesting data directly into Databricks Delta tables using the Zerobus streaming protocol.
[Public Preview](https://docs.databricks.com/release-notes/release-types.html): This SDK is supported for production use cases and is available to all customers. Databricks is actively working on stabilizing the Zerobus Ingest SDK for Python. Minor version updates may include backwards-incompatible changes.

We are keen to hear feedback from you on this SDK. Please [file issues](https://github.com/databricks/zerobus-sdk-py/issues), and we will address them.

The Databricks Zerobus Ingest SDK for Python provides a high-performance client for ingesting data directly into Databricks Delta tables using the Zerobus streaming protocol. | See also the [SDK for Rust](https://github.com/databricks/zerobus-sdk-rs) | See also the [SDK for Java](https://github.com/databricks/zerobus-sdk-java)

## Table of Contents

- [Disclaimer](#disclaimer)
- [Features](#features)
- [Requirements](#requirements)
- [Quick Start User Guide](#quick-start-user-guide)
Expand All @@ -21,7 +28,6 @@ The Databricks Zerobus Ingest SDK for Python provides a high-performance client
- [Error Handling](#error-handling)
- [API Reference](#api-reference)
- [Best Practices](#best-practices)
- [Disclaimer](#disclaimer)

## Features

Expand Down Expand Up @@ -108,8 +114,18 @@ GRANT SELECT, MODIFY ON TABLE <catalog_name>.default.air_quality TO `<service-pr

### Installation

#### From PyPI

Install the latest stable version using pip:

```bash
pip install databricks-zerobus-ingest
```

#### From Source

Clone the repository and install from source:

```bash
git clone https://github.com/databricks/zerobus-sdk-py.git
cd zerobus-sdk-py
Expand Down Expand Up @@ -748,9 +764,3 @@ NonRetriableException(message: str, cause: Exception = None)
5. **Monitoring**: Use `ack_callback` to track ingestion progress
6. **Choose the right API**: Use sync API for low-volume, async API for high-volume ingestion
7. **Token refresh**: Tokens are automatically refreshed on stream creation and recovery

## Disclaimer

[Public Preview](https://docs.databricks.com/release-notes/release-types.html): This SDK is supported for production use cases and is available to all customers. Databricks is actively working on stabilizing the Zerobus Ingest SDK for Python. Minor version updates may include backwards-incompatible changes.

We are keen to hear feedback from you on this SDK. Please [file issues](https://github.com/databricks/zerobus-sdk-py/issues), and we will address them.
12 changes: 9 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
[build-system]
requires = ["setuptools>=61.0.0", "wheel"]
requires = ["setuptools>=77", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "databricks-zerobus-ingest-sdk"
version = "0.1.0"
dynamic = ["version"]
description = "Databricks Zerobus Ingest SDK for Python"
readme = "README.md"
requires-python = ">=3.9"
keywords = ["zerobus", "databricks", "sdk"]
license = "LicenseRef-Proprietary"
license-files = ["LICENSE"]
classifiers = [
"Development Status :: 4 - Beta",
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"Intended Audience :: Science/Research",
"Intended Audience :: System Administrators",
Expand Down Expand Up @@ -44,6 +47,9 @@ dev = [
"pytest-asyncio",
]

[tool.setuptools.dynamic]
version = {attr = "zerobus.__version__"}

[tool.setuptools.packages.find]
include = ["zerobus", "zerobus.*"]

Expand Down