Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@ Deserialization uses merge semantics: parsing into an existing message overwrite

ProtoJSON serialization and deserialization are complete, including the special encodings required for well-known types like `Timestamp`, `Any`, `Struct`, `Duration`, `FieldMask`, `Value`, and `ListValue`.

The [text format](https://protobuf.dev/reference/protobuf/textformat-spec/) is supported for debugging, tests, and config files (`.txtpb`), via free functions in `protobuf.txtpb` (`message_to_text`/`message_from_text`/`merge_from_text`) rather than `Message` methods — this is the one serialization format that is *not* exposed as a method, so that using it never expands the set of reserved attribute names on every generated message class. Output matches the canonical writer used by `google.protobuf.text_format` (two-space indentation, no colon before a message value's `{`). All three formats pass the upstream conformance suite.

## Type System

**Oneofs** are represented as `Oneof` dataclasses with a `field` and `value`, which works naturally with Python's `match` statement for type-safe dispatch:
Expand Down
1 change: 1 addition & 0 deletions docs/api/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@
- [`protobuf`](./protobuf.md) — Core primitives: messages, enums, descriptors, extensions
- [`protobuf.wkt`](./wkt.md) — Well-Known Types
- [`protobuf.plugin`](./plugin.md) — protoc plugin framework
- [`protobuf.txtpb`](./txtpb.md) — Protobuf text format (`.txtpb`) serialization
5 changes: 5 additions & 0 deletions docs/api/txtpb.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# `protobuf.txtpb`

::: protobuf.txtpb
options:
show_root_heading: false
62 changes: 62 additions & 0 deletions docs/serialization.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,3 +120,65 @@ Merge semantics follow the Protobuf specification:
- **Map fields**: source entries are added; existing keys are overwritten. Message-valued map entries are not recursively merged.
- **Unknown fields**: retained in the target unless `ignore_unknown_fields=True` is passed.

## Text Format

In addition to binary and JSON, Protobuf also provides [text format](https://protobuf.dev/reference/protobuf/textformat-spec/), a plain-text syntax used for debugging, tests, and config files.
You can use it to read and write `.txtpb` files.

```python
from protobuf.txtpb import message_from_text, message_to_text

# Serialize
text: str = message_to_text(user)

# Deserialize
user = message_from_text(User, text)
```

Serializing the example message from the [tutorial](./tutorial.md) prints:

```txtpb
first_name: "Alice"
last_name: "Smith"
active: true
locations: "NYC"
locations: "LDN"
projects {
key: "atlas"
value: "infra"
}
```

The output matches the canonical writer used by `google.protobuf.text_format` (the same style `protoc --decode` and other Protobuf implementations produce): two-space indentation, one field per line, and no colon before a message value's `{`.
Text written by `message_to_text` can be read by the buf CLI, `protoc`, and other implementations, and text they produce can be read by `message_from_text`.

### Options

`registry`: required to write and read [`google.protobuf.Any`](./well-known-types.md#any) fields in their expanded `[type.url] {...}` form, and extension fields (`[pkg.extension_name]`).
Without it, an Any is written as its raw `type_url`/`value` fields, and extensions are omitted from output and rejected on parse.

```python
message_to_text(user, registry=registry)
user = message_from_text(User, text, registry=registry)
```

`print_unknown_fields`: set to `True` to print unknown fields by their field number.
This is a debugging aid only: `message_from_text` rejects fields addressed by number, so output that includes them cannot be parsed back.

```python
message_to_text(user, print_unknown_fields=True)
```

`ignore_unknown_fields`: set to `True` to silently skip unknown fields on parse instead of raising an error.

```python
user = message_from_text(User, text, ignore_unknown_fields=True)
```

To merge into an existing message, use `merge_from_text`:

```python
from protobuf.txtpb import merge_from_text

merge_from_text(user, text)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If moving the text format docs to the last section, I think we can remove these from this one and keep them all together under Text Format

```
5 changes: 4 additions & 1 deletion src/protobuf/_field_values.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,9 @@ def is_zero_value(member: DescFieldValue | DescOneof, value: Any) -> bool:
"""
if isinstance(member, DescFieldValueEnum):
return value == member.enum.values[0].number
if isinstance(member, DescFieldValueScalar) and member.scalar == ScalarType.FLOAT:
if isinstance(member, DescFieldValueScalar) and member.scalar in (
ScalarType.FLOAT,
ScalarType.DOUBLE,
):
return value == 0.0 and copysign(1, value) == 1
return not bool(value)
103 changes: 103 additions & 0 deletions src/protobuf/txtpb/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Copyright (c) 2025-2026 Buf Technologies, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Protobuf text format (`.txtpb`) serialization.

The [text format](https://protobuf.dev/reference/protobuf/textformat-spec/)
is a plain-text syntax used for debugging, tests, and config files.

Examples:
```python
from protobuf.txtpb import message_from_text, message_to_text

text = message_to_text(user)
user = message_from_text(User, text)
```
"""

from __future__ import annotations

from typing import TYPE_CHECKING, TypeVar

from ._from_text import merge_from_text
from ._to_text import ToTextOptions, to_text as _to_text_impl

if TYPE_CHECKING:
from protobuf._message import Message
from protobuf._registry import Registry

T = TypeVar("T", bound="Message")

__all__ = ["merge_from_text", "message_from_text", "message_to_text"]


def message_to_text(
message: Message,
/,
*,
registry: Registry | None = None,
print_unknown_fields: bool = False,
) -> str:
"""Serialize a message to the protobuf text format.

Unlike standard serialization, unset required fields will not raise an error.

Args:
message: The message to serialize.
registry: A registry for resolving google.protobuf.Any messages
and extensions. Without it, an Any is written as its raw
`type_url`/`value` fields and extensions are omitted.
print_unknown_fields: If `True`, unknown fields are printed by
field number.

Returns:
The message in protobuf text format.
"""
return _to_text_impl(
message,
ToTextOptions(print_unknown_fields=print_unknown_fields, registry=registry),
)


def message_from_text(
message_type: type[T],
text: str | bytes | bytearray,
*,
registry: Registry | None = None,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While it deviates from protobuf-es a bit, I feel it is worth having the ignore_unknown_fields option here (default to False) to have symmetry between the two APIs, and it looks like upstream Python and Go both have the option.

ignore_unknown_fields: bool = False,
) -> T:
"""Create a new message by parsing the protobuf text format.

To merge into an existing message, use [`merge_from_text`][].

Args:
message_type: The type of message to create.
text: The text data to parse.
registry: Required to read `google.protobuf.Any` in its expanded
`[type.url] {...}` form, and extension fields, from text
format.
ignore_unknown_fields: If `True`, unknown fields are silently
skipped instead of raising an error.

Raises:
ValueError: If the text cannot be parsed into the message. This
includes encountering an unknown field, unless
ignore_unknown_fields is set.
RecursionError: If messages are nested deeper than the supported
limit.
"""
message = message_type()
merge_from_text(
message, text, registry=registry, ignore_unknown_fields=ignore_unknown_fields
)
return message
Loading
Loading