Skip to content

Latest commit

 

History

History
294 lines (207 loc) · 7.76 KB

README.md

File metadata and controls

294 lines (207 loc) · 7.76 KB

Analytics Engine

Analytics Engine is a microservice designed to capture and analyze user events in real-time using Redis. It aggregates and provides insights into daily, weekly, and monthly usage, offering both global and command-specific statistics.

Why Use Analytics Engine?

Challenges in Gathering Real-time Analytics

When building applications that require real-time insights, several challenges often arise:

  1. Scalability:

    • Traditional analytics solutions can struggle with large volumes of data, leading to bottlenecks. Analytics Engine scales horizontally, efficiently handling increasing loads.

  2. Data Persistence:

    • Ensuring that usage data is stored and retrievable after server restarts or crashes is crucial. By leveraging Redis, Analytics Engine provides reliable persistence for your analytics data.

  3. Resource Efficiency:

    • Analytics systems can consume significant resources if not optimized. Analytics Engine efficiently manages and stores only the necessary data, reducing overhead.

  4. Ease of Integration:

    • Integrating analytics into an application should be straightforward. Analytics Engine offers a simple API, making it easy to send and retrieve event data without additional complexity.

  5. Real-time Insights:

    • Many systems struggle with delivering real-time analytics. Analytics Engine provides immediate access to usage data, enabling you to make informed decisions quickly.

Benefits of Analytics Engine

By using Analytics Engine, you gain:

  • Scalability: Efficiently handles large volumes of events across distributed instances.
  • Reliability: Persistent data storage ensures that your analytics are accurate and durable.
  • Simplicity: A clean, easy-to-use API for tracking and aggregating event data.
  • Flexibility: Aggregate data by the day, week, or month, and get detailed insights into each command.

Table of Contents

Features

  • Track and store user events with timestamps.
  • Aggregate event data into daily, weekly, and monthly summaries.
  • Retrieve global usage statistics as well as per-command insights.
  • Easy deployment with Docker and Docker Compose.

Requirements

  • Docker installed on your server or local machine.
  • Docker Compose (optional, but recommended).

Environment Variables

Set the following environment variables before running the service:

  • REDIS_URL: URL of the Redis server (e.g., redis://localhost:6379).
  • REDIS_KEY: Redis key prefix for storing event data (default is analyticsEngine).
  • PORT: Port on which the service will run (default is 8080).
  • API_AUTH: API key for securing the service.
  • MAX_AGE: Maximum retention time for event data in days (e.g., 30).

Running with Docker

You can easily run Analytics Engine using Docker. Follow these steps:

1. Pull the Docker Image 
docker pull ghcr.io/digital39999/analytics-engine:latest
2. Run the Container

Run the container with the necessary environment variables:

docker run -d \
  -e REDIS_URL="redis://your-redis-url:6379" \
  -e PORT=8080 \
  -e MAX_AGE=30 \
  -e API_AUTH="your-api-key" \
  -p 8080:8080 \
  ghcr.io/digital39999/redis-analytics-engine:latest
3. Access the Service

The service will be available at http://localhost:8080.

Running with Docker Compose

If you prefer to use Docker Compose, follow these steps:

1. Create a `docker-compose.yml` File

Here’s an example docker-compose.yml:

version: '3.8'

services:
  analytics-engine:
    image: ghcr.io/digital39999/analytics-engine:latest
    environment:
      REDIS_URL: "redis://redis:6379"
      PORT: 8080
      API_AUTH: "your-api-key"
      MAX_AGE: 365
    ports:
      - "8080:8080"
    depends_on:
      - redis

  redis:
    image: redis:latest
    ports:
      - "6379:6379"
2. Run the Services

To start the services, use the following command:

docker-compose up -d
3. Access the Service

Once the services are up, you can access Analytics Engine at http://localhost:8080.

API Usage

Routes Overview

Record an Event

You can record an event by sending a POST request to /event. Here's an example using curl:

curl -X POST http://localhost:8080/event \
-H "Content-Type: application/json" \
-H "Authorization: your-api-key" \
-d '{
  "name": "login",
  "uniqueId": "user123",
  "createdAt": 1632960000000,
  "type": "auth"
}'
  • name: The event name (e.g., login, purchase).
  • uniqueId: A unique identifier for the user (optional).
  • createdAt: The timestamp of the event.
  • type: The type of event (e.g., auth, purchase).

Get Aggregated Analytics

To retrieve aggregated analytics, send a GET request to /analytics:

curl -X GET http://localhost:8080/analytics
  • Optional Query Parameters:
    • lookback: Number of days to look back for daily counts (default is 7).
    • type: Filter analytics by a specific type.
    • uniqueId: Filter analytics by a specific user ID.

Get System Statistics

To retrieve system statistics, send a GET request to /stats:

curl -X GET http://localhost:8080/stats
  • This will return information such as the total Redis keys, CPU and RAM usage, system uptime, and more.

Flush Analytics Data

To flush analytics data from Redis, send a DELETE request to /analytics:

curl -X DELETE http://localhost:8080/analytics
  • Optional Query Parameter:
    • type: Flush analytics data by a specific type.
Examples

Example Node.js Client

Here’s how you could integrate Analytics Engine into a Node.js project:

const apiUrl = 'http://localhost:8080/event';

async function recordEvent() {
  try {
    const response = await fetch(apiUrl, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': 'your-api-key'
      },
      body: JSON.stringify({
        name: 'login',
        uniqueId: 'user123',
        createdAt: Date.now()
        type: 'auth'
      })
    }).then(res => res.json());

    if (response.error) throw new Error(response.error);
    console.log('Event recorded successfully:', response.data);
  } catch (error) {
    console.error('Error recording event:', error.message);
  }
}

recordEvent();

Example Python Client

Here’s how you could integrate Analytics Engine into a Python project using requests:

import requests
import time

api_url = 'http://localhost:8080/event'

def record_event():
    data = {
        'name': 'login',
        'uniqueId': 'user123',
        'createdAt': int(time.time() * 1000)  # milliseconds
        'type': 'auth'
    }
    
    headers = {
        'Authorization': 'your-api-key',
        'Content-Type': 'application/json'
    }
    
    response = requests.post(api_url, json=data, headers=headers)
    if response.status_code == 200:
        print('Event recorded successfully:', response.json())
    else:
        print('Error recording event:', response.text)

record_event()

Contributing

If you'd like to contribute to this project, feel free to open a pull request or submit an issue on the GitHub repository.

License

This project is licensed under the MIT License. See the LICENSE file for more details.