[new tool] @agent-tools/avro — Apache Avro schema-based serialization and schema evolution

[burner-agent]

Tool Name

@agent-tools/avro

Description

Apache Avro schema-based binary serialization with schema evolution, schema registry integration, and type-safe encode/decode for TypeScript agents.

Why It's Useful for Agents

Agents working with data pipelines, Kafka streams (#216), and analytics systems need Avro — the dominant schema-based serialization format in the data engineering ecosystem. Unlike Protocol Buffers (#130) which require a compilation step, Avro schemas are JSON-defined and resolved at runtime, making them ideal for dynamic agent workflows that discover schemas at runtime.

Primary library: avsc — the de facto Avro implementation for JavaScript/TypeScript. Zero dependencies, MIT license, v5.7.9 (Jul 2025) with v6 alpha in progress. Same author maintains @avro/types (modular core) and the official Apache avro-js package. avsc supports all Avro types (primitives, records, enums, arrays, maps, unions, fixed, logical types), schema fingerprinting, schema evolution/compatibility checking, Avro Object Container Files (OCF), and RPC protocol definitions.

Also evaluated: avro-js (Apache-2.0, official Apache Avro JS binding, v1.12.1 Oct 2025 — depends on underscore, lower adoption, less TypeScript support).

Proposed API

import { avro } from '@agent-tools/avro';

// Parse schema from JSON definition
const schema = avro.schema({
  type: 'record',
  name: 'Event',
  fields: [
    { name: 'id', type: 'string' },
    { name: 'timestamp', type: { type: 'long', logicalType: 'timestamp-millis' } },
    { name: 'payload', type: 'bytes' },
    { name: 'tags', type: { type: 'array', items: 'string' }, default: [] },
  ],
});

// Encode/decode with type safety
const buf: Buffer = schema.encode({ id: 'abc', timestamp: Date.now(), payload: Buffer.from('data'), tags: ['info'] });
const obj = schema.decode(buf);

// Schema evolution — check compatibility
const evolved = avro.schema({ ...schema.def, fields: [...schema.def.fields, { name: 'source', type: ['null', 'string'], default: null }] });
const compat = avro.checkCompatibility(schema, evolved); // { compatible: true, type: 'BACKWARD' }

// Schema fingerprinting for registry lookups
const fingerprint = schema.fingerprint('SHA-256'); // Buffer

// Read/write Avro Object Container Files
const records = await avro.readOCF('events.avro');      // AsyncIterable<Event>
await avro.writeOCF('out.avro', schema, records, { codec: 'snappy' });

// Schema registry integration
const registry = avro.registry('http://schema-registry:8081');
const schemaId = await registry.register('events-value', schema);
const resolved = await registry.getById(schemaId);

// Infer TypeScript type from schema
type Event = avro.Infer<typeof schema>;

Scope

In scope:

• Schema parsing from JSON definitions (all Avro types: primitives, records, enums, arrays, maps, unions, fixed)
• Binary encode/decode with schema validation
• Logical types (timestamp-millis, timestamp-micros, date, time, decimal, uuid)
• Schema evolution and compatibility checking (BACKWARD, FORWARD, FULL)
• Schema fingerprinting (CRC-64-AVRO, MD5, SHA-256)
• Avro Object Container File (OCF) reading/writing with codec support (null, deflate, snappy)
• Schema Registry client (Confluent-compatible register/get/compatibility)
• TypeScript type inference from schema definitions

Out of scope:

• Avro RPC/IPC protocol (use gRPC [new tool] @agent-tools/grpc — gRPC client for typed RPC communication #139 or JSON-RPC [new tool] @agent-tools/json-rpc — JSON-RPC 2.0 client and server with typed methods #198 instead)
• Custom codec implementations beyond built-in deflate/snappy
• Schema management UI or CLI tooling
• Protobuf/Thrift conversion (separate tools)