Cyolo SecOps Middleware

A lightweight middleware service that forwards Cyolo IDAC security events to Google Security Operations (SecOps/Chronicle) using the official Google SecOps SDK and UDM (Unified Data Model) format.

Architecture

┌─────────────┐     Syslog (CEF/JSON)     ┌──────────────────┐
│   Cyolo     │ ────────────────────────► │   Middleware     │
│   IDAC      │      UDP Port 514         │   (Docker)       │
└─────────────┘                           └────────┬─────────┘
                                                  │
                                                  │ Google SecOps SDK
                                                  │ (ingest_udm)
                                                  ▼
                                         ┌──────────────────┐
                                         │  Google SecOps   │
                                         │  Ingestion API   │
                                         └──────────────────┘

Features

• Official SDK: Uses Google's official secops Python SDK for reliable integration
• CEF and JSON parsing: Supports both CEF and JSON syslog formats from Cyolo
• UDM mapping: Converts 200+ Cyolo event types to Google SecOps UDM format
• Intelligent batching: Size-based and time-based batching for efficient API usage
• Retry logic: Built-in retry with exponential backoff
• Health checks: HTTP endpoint for container orchestration
• Structured logging: JSON-formatted logs for observability

Quick Start

Prerequisites

1. GCP Project linked to your Google SecOps instance
2. IAM Role: Service account with Chronicle API Admin (roles/chronicle.admin) or Chronicle API Editor role
3. Authentication - One of:
   • Application Default Credentials (ADC) via gcloud auth application-default login
   • Service Account Key file
   • GCP metadata service (when running on GCP)
4. Cyolo IDAC configured with syslog export
5. Docker and Docker Compose

1. Setup Authentication

The middleware uses the official Google SecOps SDK which supports multiple authentication methods:

Option A: Application Default Credentials (Recommended for Development)

# Login with gcloud - simplest for local development
gcloud auth application-default login

The SDK will automatically use these credentials.

Option B: Service Account Key

# 1. Create a service account
gcloud iam service-accounts create cyolo-secops-ingestion \
  --display-name="Cyolo SecOps Ingestion"

# 2. Grant the Chronicle API Admin role
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
  --member="serviceAccount:cyolo-secops-ingestion@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/chronicle.admin"

# 3. Download credentials key
gcloud iam service-accounts keys create credentials.json \
  --iam-account=cyolo-secops-ingestion@YOUR_PROJECT_ID.iam.gserviceaccount.com

Then set GOOGLE_APPLICATION_CREDENTIALS to point to the key file.

Option C: Workload Identity Federation (Production)

For production deployments, use Workload Identity Federation for keyless authentication. See GCP WIF documentation.

Required Information

You'll need:

• Project ID: Your GCP project ID (linked to SecOps)
• Instance ID: Your SecOps instance UUID (also called customer_id, found in SecOps settings)
• Region: Your SecOps instance region (e.g., us, europe)

2. Configure Environment

Create a .env file:

# Required - GCP/SecOps Configuration
SECOPS_PROJECT_ID=your-gcp-project-id
SECOPS_INSTANCE_ID=your-secops-instance-uuid  # UUID format: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
SECOPS_REGION=us  # or europe, asia-southeast1, etc.

# Optional - Authentication (if not using ADC)
GOOGLE_APPLICATION_CREDENTIALS=/app/credentials.json

# Optional - Tuning
LOG_LEVEL=INFO
SYSLOG_FORMAT=cef  # or json

3. Start the Middleware

docker-compose up -d

4. Verify Connection

# Check health
curl http://localhost:8080/health

# Verify SecOps connection
curl http://localhost:8080/verify

# View stats
curl http://localhost:8080/stats

5. Configure Cyolo

Point Cyolo's syslog export to the middleware:

Via Cyolo Admin UI:

1. Go to Settings → Syslog Servers
2. Click Add Syslog Server
3. Enter the middleware host IP and port 514
4. Select CEF format (recommended) or JSON
5. Enable for all sites or specific sites

6. Test with a Sample Event

# Send a test CEF event
echo '<134>Feb 2 13:00:00 test CEF:0|cyolo|idac|1.0|101110013|User Login|1|act=login dst=10.1.2.3 cyolosubjectname=test@example.com' | nc -u localhost 514

# Wait for batch flush (5 seconds)
sleep 6

# Check logs for result
docker logs cyolo-secops-middleware 2>&1 | tail -20

Configuration

Environment Variables

Variable	Default	Description
SYSLOG_HOST	0.0.0.0	Host to bind syslog receiver
SYSLOG_PORT	514	UDP port for syslog
SYSLOG_FORMAT	cef	Input format: cef or json
GOOGLE_APPLICATION_CREDENTIALS	auto	Path to credentials file. Optional if using ADC.
SECOPS_PROJECT_ID	required	GCP project ID linked to SecOps
SECOPS_INSTANCE_ID	required	Google SecOps instance UUID
SECOPS_REGION	us	SecOps regional endpoint
BATCH_MAX_SIZE_BYTES	4000000	Max batch size (4MB limit)
BATCH_MAX_WAIT_SECONDS	5	Max time before flushing batch
HEALTH_HOST	0.0.0.0	Health check server host
HEALTH_PORT	8080	Health check server port
LOG_LEVEL	INFO	Logging level
RETRY_MAX_ATTEMPTS	5	Max API retry attempts

Supported Regions

See Google SecOps regions for the complete list.

Region	Location
us	United States Multi-Region
europe	Europe Multi-Region
europe-west2	London
europe-west3	Frankfurt
asia-southeast1	Singapore
asia-northeast1	Tokyo
me-west1	Tel Aviv
australia-southeast1	Sydney

Event Mapping

The middleware maps Cyolo events to Google SecOps UDM event types:

Cyolo Event Category	UDM Event Type
Login Success/Failure	USER_LOGIN
Logout	USER_LOGOUT
Access Granted/Denied	USER_RESOURCE_ACCESS
File Download	FILE_READ
File Upload	FILE_CREATION
File Delete	FILE_DELETION
Remote Session	USER_RESOURCE_ACCESS
Network Access	NETWORK_CONNECTION
System Alerts	STATUS_UPDATE
Password Change	USER_CHANGE_PASSWORD
User Enrollment	USER_CREATION
WAF Events	NETWORK_HTTP

HTTP Endpoints

Endpoint	Description
GET /health	Basic health check
GET /ready	Readiness check
GET /stats	Processing statistics
GET /verify	Verify SecOps API connection

Development

Setup

# Install dependencies
poetry install

# Run tests
poetry run pytest

# Format code
poetry run black src/ tests/
poetry run ruff src/ tests/

# Type checking
poetry run mypy src/

Running Locally

# Set required environment variables
export SECOPS_PROJECT_ID=your-project-id
export SECOPS_INSTANCE_ID=your-instance-uuid

# Authenticate (simplest method for local dev)
gcloud auth application-default login

# Run the middleware
poetry run python -m cyolo_secops.main

Project Structure

cyolo-secops-middleware/
├── src/cyolo_secops/
│   ├── main.py              # Entry point
│   ├── config.py            # Pydantic settings
│   ├── syslog_receiver.py   # UDP syslog server
│   ├── batcher.py           # Event batching
│   ├── secops_client.py     # Google SecOps SDK wrapper
│   ├── models.py            # Data models
│   ├── parsers/
│   │   ├── cef_parser.py    # CEF format parser
│   │   └── json_parser.py   # JSON format parser
│   └── mapper/
│       ├── udm_mapper.py    # UDM mapping logic
│       └── event_types.py   # Event type mappings
├── tests/
├── Dockerfile
├── docker-compose.yml
└── pyproject.toml

Troubleshooting

Common Issues

"Authentication failed"

• Ensure you're authenticated: gcloud auth application-default login
• Or set GOOGLE_APPLICATION_CREDENTIALS to a valid service account key
• Verify the service account has Chronicle API Admin or Chronicle API Editor role

"API error: 403"

• Verify the service account has the required IAM role on the GCP project
• Check that the GCP project is linked to your SecOps instance
• Verify SECOPS_PROJECT_ID and SECOPS_INSTANCE_ID are correct

"API error: 400"

• Check the logs for UDM validation errors
• Ensure events have required fields (timestamp, event_type, principal)

Events not appearing in SecOps

• Check that Cyolo is sending to the correct syslog address
• Verify the syslog format matches (cef vs json)
• Check middleware logs for parsing errors
• Use /stats endpoint to see processing counts

Debug Mode

LOG_LEVEL=DEBUG docker-compose up

Security Considerations

1. Credentials: Store GCP credentials securely, never commit to source control
2. Network: Run the middleware in an isolated network segment
3. Ports: The syslog port (514) should only be accessible from Cyolo
4. Updates: Keep the middleware updated for security patches

License

Copyright (c) Cyolo Security. All rights reserved.