Performance Optimization Guide

Overview

This guide provides comprehensive strategies for optimizing SmartGlass AI Agent performance on Meta Ray-Ban + OPPO Reno 12 deployment. The goal is to achieve < 2 second latency, < 10% battery drain per hour, and < 200MB memory footprint while maintaining high-quality multimodal AI responses.

Current vs Target Benchmarks

System Performance Metrics

Metric	Current	Target	Priority	Status
End-to-End Latency	3-5s	< 2s	🔴 Critical	In Progress
Backend Response Time	1.5-2.5s	< 1s	🔴 Critical	Optimizing
Audio Transcription	1.5s	< 1s	🟡 High	In Progress
Vision Processing (CLIP)	800ms	< 500ms	🔴 Critical	Needs Work
SNN Inference	300ms	< 200ms	🟢 Low	✅ Within Target
Frame Capture Rate	30 fps	5-10 fps (adaptive)	🟢 Low	✅ Implemented
Battery Drain (1hr)	12-15%	< 10%	🟡 High	Optimizing
Memory Footprint (App)	180-220MB	< 200MB	🟢 Low	✅ Within Target
Backend Memory	1.2-1.8GB	< 1GB	🟡 High	Optimizing
Network Bandwidth	800 KB/min	< 600 KB/min	🟢 Low	Optimizing

Component Breakdown (Current)

Total Latency: ~3.5s
├── Frame Capture: 200ms
├── Frame Upload: 300ms
├── Vision (CLIP): 800ms
├── Audio Transcription: 1500ms
├── LLM Generation: 500ms
└── Response Download: 200ms

Target Breakdown

Total Latency: < 2s
├── Frame Capture: 150ms
├── Frame Upload: 100ms (compressed)
├── Vision (CLIP): 400ms (optimized)
├── Audio Transcription: 800ms (quantized)
├── LLM Generation: 350ms (INT8 SNN)
└── Response Download: 100ms

Optimization Strategy 1: Frame Compression

Problem

Uncompressed frames from Ray-Ban camera are ~500KB each
Network upload takes 300ms+ on typical Wi-Fi
Bandwidth usage is 800 KB/min (high battery drain)

Solution: JPEG Quality Adjustment

Implementation in Android App

File: sample/src/main/kotlin/com/smartglass/sample/SmartGlassViewModel.kt

import android.graphics.Bitmap
import java.io.ByteArrayOutputStream

/**
 * Compress frame before network upload.
 * 
 * @param bitmap Original frame from Ray-Ban camera
 * @param quality JPEG quality (1-100). Lower = smaller file, faster upload
 * @return Compressed JPEG bytes
 */
private fun compressFrame(bitmap: Bitmap, quality: Int = 75): ByteArray {
    val outputStream = ByteArrayOutputStream()
    bitmap.compress(Bitmap.CompressFormat.JPEG, quality, outputStream)
    return outputStream.toByteArray()
}

/**
 * Process video frame with adaptive compression.
 * 
 * Compression quality adapts based on:
 * - Battery level (lower battery = higher compression)
 * - Network speed (slower network = higher compression)
 */
private suspend fun processVideoFrame(bitmap: Bitmap) {
    // Adaptive quality based on battery level
    val batteryLevel = getBatteryLevel()
    val quality = when {
        batteryLevel < 20 -> 60  // High compression for low battery
        batteryLevel < 50 -> 75  // Medium compression
        else -> 85               // Lower compression for good battery
    }
    
    val compressedBytes = compressFrame(bitmap, quality)
    
    Log.d(TAG, "Frame compressed: ${bitmap.byteCount} -> ${compressedBytes.size} bytes " +
               "(${compressedBytes.size * 100 / bitmap.byteCount}%, quality=$quality)")
    
    // Upload compressed frame
    uploadFrame(compressedBytes)
}

/**
 * Get current battery level percentage.
 */
private fun getBatteryLevel(): Int {
    val batteryManager = context.getSystemService(Context.BATTERY_SERVICE) as BatteryManager
    return batteryManager.getIntProperty(BatteryManager.BATTERY_PROPERTY_CAPACITY)
}

Backend Frame Handling

File: src/edge_runtime/server.py

from PIL import Image
import io
import base64

def decode_compressed_frame(base64_image: str) -> Image.Image:
    """
    Decode compressed JPEG frame from mobile app.
    
    Args:
        base64_image: Base64-encoded JPEG bytes
        
    Returns:
        PIL Image object
    """
    image_bytes = base64.b64decode(base64_image)
    image = Image.open(io.BytesIO(image_bytes))
    return image

# Usage in endpoint
@app.route('/sessions/<session_id>/frame', methods=['POST'])
def upload_frame(session_id):
    data = request.json
    image = decode_compressed_frame(data['image'])
    
    # Process with CLIP (no quality loss for inference)
    result = process_vision(image)
    return jsonify(result)

Expected Impact

Quality	File Size	Upload Time	Vision Accuracy	Recommended Use
95	~450 KB	280ms	100%	High battery, fast network
85	~200 KB	120ms	99%	✅ Default (balanced)
75	~100 KB	60ms	98%	Medium battery, typical use
60	~50 KB	30ms	95%	Low battery, slow network
40	~25 KB	15ms	90%	Emergency mode

Recommended: Start with quality=85, adapt based on battery/network

Optimization Strategy 2: SNN INT8 Quantization

Problem

SNN model (student.pt) is FP32 (~50MB)
Inference takes 300ms on OPPO Reno 12 CPU
Memory usage is 180MB during inference

Solution: INT8 Post-Training Quantization

Step 1: Export SNN Model for Mobile

Script: scripts/quantize_snn_model.py

import torch
from torch.quantization import quantize_dynamic
from src.llm_snn_backend import SNNLLMBackend

def quantize_snn_student(
    model_path: str = "artifacts/snn_student/student.pt",
    output_path: str = "artifacts/snn_student/student_int8.pt"
):
    """
    Apply INT8 dynamic quantization to SNN student model.
    
    Benefits:
    - 4x smaller model size (50MB -> 12MB)
    - 2-3x faster inference on CPU
    - Minimal accuracy loss (< 1%)
    """
    # Load FP32 model
    backend = SNNLLMBackend(model_path=model_path)
    model = backend.model
    
    # Apply dynamic quantization (INT8 weights, FP32 activations)
    quantized_model = quantize_dynamic(
        model,
        {torch.nn.Linear, torch.nn.Conv1d},  # Quantize linear layers
        dtype=torch.qint8
    )
    
    # Save quantized model
    torch.save({
        'model_state_dict': quantized_model.state_dict(),
        'tokenizer_config': backend.tokenizer_config,
        'quantized': True
    }, output_path)
    
    print(f"✅ Quantized model saved to {output_path}")
    print(f"   Original size: {os.path.getsize(model_path) / 1024 / 1024:.1f} MB")
    print(f"   Quantized size: {os.path.getsize(output_path) / 1024 / 1024:.1f} MB")

if __name__ == "__main__":
    quantize_snn_student()

Step 2: Update Android SNN Engine

File: sdk-android/src/main/kotlin/com/smartglass/runtime/llm/LocalSnnEngine.kt

/**
 * Load INT8 quantized SNN model for faster inference.
 * 
 * Update model asset path to use quantized version:
 * - Original: snn_student_ts.pt (~50MB)
 * - Quantized: snn_student_int8_ts.pt (~12MB)
 */
class LocalSnnEngine(context: Context) {
    companion object {
        // Use quantized model for production
        private const val MODEL_ASSET_PATH = "snn_student_int8_ts.pt"
        
        // Inference optimization flags
        private const val USE_NNAPI = true  // Use Android NNAPI acceleration
        private const val NUM_THREADS = 4   // CPU threads for inference
    }
    
    private val module: Module by lazy {
        val modelPath = copyAssetToFile(context, MODEL_ASSET_PATH)
        
        // Load with optimization flags
        Module.load(modelPath).apply {
            // Enable NNAPI hardware acceleration if available
            if (USE_NNAPI) {
                try {
                    // This enables Android Neural Networks API
                    // for hardware-accelerated inference on NPU/DSP
                    System.loadLibrary("pytorch_jni")
                    System.loadLibrary("nnapi_jni")
                    Log.i("SNN", "NNAPI acceleration enabled")
                } catch (e: Exception) {
                    Log.w("SNN", "NNAPI not available, using CPU")
                }
            }
        }
    }
    
    fun generate(prompt: String, maxTokens: Int = 50): String {
        val startTime = System.currentTimeMillis()
        
        // Tokenize
        val inputIds = tokenizer.encode(prompt)
        val inputTensor = Tensor.fromBlob(
            inputIds,
            longArrayOf(1, inputIds.size.toLong())
        )
        
        // Inference (quantized INT8)
        val outputTensor = module.forward(
            IValue.from(inputTensor)
        ).toTensor()
        
        val inferenceTime = System.currentTimeMillis() - startTime
        Log.d("SNN", "INT8 inference: ${inferenceTime}ms")
        
        // Decode
        return tokenizer.decode(outputTensor)
    }
}

Step 3: Build Script Updates

File: build.gradle.kts (sample module)

android {
    // ... existing config
    
    defaultConfig {
        // ...
        
        // Optimize for mobile deployment
        ndk {
            abiFilters += listOf("arm64-v8a", "armeabi-v7a")  // ARM architectures
        }
    }
    
    buildTypes {
        release {
            // Enable ProGuard to strip unused PyTorch ops
            isMinifyEnabled = true
            proguardFiles(
                getDefaultProguardFile("proguard-android-optimize.txt"),
                "proguard-rules.pro"
            )
        }
    }
}

dependencies {
    // PyTorch Android (use lite version for smaller APK)
    implementation("org.pytorch:pytorch_android_lite:1.13.1")
    implementation("org.pytorch:pytorch_android_torchvision_lite:1.13.1")
}

Expected Impact

Model Version	Size	Inference Time	Memory	Accuracy Loss
FP32 (original)	50 MB	300ms	180 MB	Baseline
INT8 (quantized)	12 MB	100-150ms	80 MB	< 1%
INT8 + NNAPI	12 MB	80-100ms	60 MB	< 1%

Expected Speedup: 2-3x faster inference, 60% memory reduction

Optimization Strategy 3: Adaptive Frame Rate

Problem

Continuous 30 fps streaming drains battery rapidly
Not all scenarios need high frame rate (e.g., static scenes)

Solution: Dynamic Frame Rate Based on Context

Implementation

File: sample/src/main/kotlin/com/smartglass/sample/SmartGlassViewModel.kt

/**
 * Adaptive frame rate controller.
 * 
 * Adjusts capture rate based on:
 * 1. Battery level
 * 2. Scene motion (detected via frame diff)
 * 3. User interaction state
 */
class AdaptiveFrameRateController {
    
    enum class FrameRateMode {
        HIGH(30),      // High motion, good battery
        MEDIUM(10),    // Normal use
        LOW(5),        // Battery saver
        MINIMAL(1);    // Critical battery
        
        val fps: Int
        constructor(fps: Int) { this.fps = fps }
    }
    
    private var currentMode = FrameRateMode.MEDIUM
    private var lastFrame: Bitmap? = null
    
    /**
     * Determine optimal frame rate for current conditions.
     */
    fun getOptimalFrameRate(
        batteryLevel: Int,
        isUserActive: Boolean
    ): Int {
        currentMode = when {
            batteryLevel < 15 -> FrameRateMode.MINIMAL
            batteryLevel < 30 -> FrameRateMode.LOW
            batteryLevel < 60 && !isUserActive -> FrameRateMode.LOW
            isUserActive && batteryLevel > 60 -> FrameRateMode.MEDIUM
            else -> FrameRateMode.MEDIUM
        }
        
        Log.d(TAG, "Frame rate mode: $currentMode (${currentMode.fps} fps)")
        return currentMode.fps
    }
    
    /**
     * Detect scene motion by comparing consecutive frames.
     * 
     * @return Motion score (0.0 = static, 1.0 = high motion)
     */
    fun detectMotion(currentFrame: Bitmap): Float {
        val prevFrame = lastFrame ?: run {
            lastFrame = currentFrame.copy(currentFrame.config, true)
            return 0f
        }
        
        // Simple pixel difference metric
        var totalDiff = 0L
        val width = minOf(currentFrame.width, prevFrame.width)
        val height = minOf(currentFrame.height, prevFrame.height)
        
        for (x in 0 until width step 10) {  // Sample every 10 pixels
            for (y in 0 until height step 10) {
                val pixel1 = currentFrame.getPixel(x, y)
                val pixel2 = prevFrame.getPixel(x, y)
                totalDiff += kotlin.math.abs(pixel1 - pixel2)
            }
        }
        
        lastFrame = currentFrame.copy(currentFrame.config, true)
        
        // Normalize to 0-1 range
        val maxDiff = (width / 10) * (height / 10) * 255L * 3  // RGB
        return (totalDiff.toFloat() / maxDiff).coerceIn(0f, 1f)
    }
    
    /**
     * Should we capture this frame based on adaptive rate?
     */
    fun shouldCaptureFrame(frameNumber: Int, targetFps: Int): Boolean {
        val interval = 30 / targetFps  // Assuming 30fps camera
        return frameNumber % interval == 0
    }
}

// Usage in ViewModel
private val frameRateController = AdaptiveFrameRateController()

private fun onNewCameraFrame(bitmap: Bitmap, frameNumber: Int) {
    // Get battery level
    val batteryLevel = getBatteryLevel()
    val isUserActive = (System.currentTimeMillis() - lastUserInteractionTime) < 10000
    
    // Determine optimal frame rate
    val targetFps = frameRateController.getOptimalFrameRate(batteryLevel, isUserActive)
    
    // Detect motion for dynamic adjustment
    val motion = frameRateController.detectMotion(bitmap)
    if (motion > 0.3 && batteryLevel > 30) {
        // Increase rate temporarily for high-motion scenes
        targetFps = (targetFps * 1.5).toInt()
    }
    
    // Capture only if needed
    if (frameRateController.shouldCaptureFrame(frameNumber, targetFps)) {
        processVideoFrame(bitmap)
    }
}

Configuration Matrix

Battery	User Active	Motion	Frame Rate	Notes
> 60%	Yes	High	15 fps	Good experience
> 60%	Yes	Low	10 fps	Default
> 60%	No	Low	5 fps	Idle optimization
30-60%	Yes	Any	5-10 fps	Battery conscious
15-30%	Any	Any	5 fps	Battery saver
< 15%	Any	Any	1 fps	Critical mode

Expected Impact

Battery savings: 30-50% reduction in battery drain
Network savings: 40-60% reduction in bandwidth
Latency: Unchanged (same processing per captured frame)
User experience: Negligible impact (5-10 fps sufficient for most queries)

Optimization Strategy 4: Backend Model Optimization

Problem

CLIP vision model is slow (800ms per frame)
Whisper transcription takes 1.5s
Backend memory usage is high (1.5GB+)

Solution: Model Optimization Pipeline

4.1: CLIP Model Optimization

# File: src/vision_clip_processor.py

import torch
from transformers import CLIPProcessor, CLIPModel
import torch.quantization

class OptimizedCLIPProcessor:
    """
    Optimized CLIP processor with quantization and caching.
    """
    
    def __init__(self, model_name="openai/clip-vit-base-patch32"):
        self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
        
        # Load model
        self.model = CLIPModel.from_pretrained(model_name).to(self.device)
        self.processor = CLIPProcessor.from_pretrained(model_name)
        
        # Apply optimizations
        self._optimize_model()
        
    def _optimize_model(self):
        """Apply model optimizations for faster inference."""
        
        # 1. TorchScript compilation
        if self.device.type == "cpu":
            self.model = torch.jit.script(self.model)
            print("✅ CLIP model compiled with TorchScript")
        
        # 2. Quantization (CPU only)
        if self.device.type == "cpu":
            self.model = torch.quantization.quantize_dynamic(
                self.model,
                {torch.nn.Linear},
                dtype=torch.qint8
            )
            print("✅ CLIP model quantized to INT8")
        
        # 3. Mixed precision (GPU only)
        if self.device.type == "cuda":
            self.model = self.model.half()  # FP16
            print("✅ CLIP model using FP16 precision")
        
        # 4. Set to eval mode and disable gradients
        self.model.eval()
        for param in self.model.parameters():
            param.requires_grad = False
    
    @torch.no_grad()
    def process_image(self, image, queries=None):
        """
        Process image with optimizations.
        
        Expected speedup: 800ms -> 400ms
        """
        start_time = time.time()
        
        # Preprocess
        inputs = self.processor(
            images=image,
            text=queries or ["a photo of"],
            return_tensors="pt",
            padding=True
        ).to(self.device)
        
        # Inference with optimizations
        with torch.cuda.amp.autocast(enabled=(self.device.type == "cuda")):
            outputs = self.model(**inputs)
        
        inference_time = (time.time() - start_time) * 1000
        print(f"CLIP inference: {inference_time:.0f}ms")
        
        return outputs

4.2: Whisper Model Optimization

# File: src/audio_whisper_processor.py

import whisper
import torch

class OptimizedWhisperProcessor:
    """
    Optimized Whisper with faster-whisper backend.
    """
    
    def __init__(self, model_size="base"):
        # Use faster-whisper for 2-3x speedup
        try:
            from faster_whisper import WhisperModel
            
            # Load with optimizations
            self.model = WhisperModel(
                model_size,
                device="cuda" if torch.cuda.is_available() else "cpu",
                compute_type="int8" if torch.cuda.is_available() else "int8",
                num_workers=4  # Parallel processing
            )
            print("✅ Using faster-whisper backend")
            
        except ImportError:
            # Fallback to standard whisper
            self.model = whisper.load_model(model_size)
            print("⚠️ faster-whisper not available, using standard whisper")
    
    def transcribe(self, audio_path: str) -> str:
        """
        Transcribe audio with optimizations.
        
        Expected speedup: 1500ms -> 800ms
        """
        start_time = time.time()
        
        if hasattr(self.model, 'transcribe'):
            # faster-whisper API
            segments, info = self.model.transcribe(
                audio_path,
                beam_size=1,  # Faster than default 5
                best_of=1,    # Faster than default 5
                vad_filter=True,  # Skip silence
                vad_parameters=dict(min_silence_duration_ms=500)
            )
            text = " ".join([seg.text for seg in segments])
        else:
            # Standard whisper API
            result = self.model.transcribe(audio_path)
            text = result["text"]
        
        transcription_time = (time.time() - start_time) * 1000
        print(f"Whisper transcription: {transcription_time:.0f}ms")
        
        return text.strip()

4.3: Install Faster-Whisper

# Add to requirements.txt
faster-whisper==0.10.0

# Install
pip install faster-whisper

# Requires:
# - CUDA 11.8+ (for GPU)
# - CTranslate2 (installed automatically)

Expected Impact

Component	Before	After	Speedup
CLIP (CPU)	800ms	400ms	2x
CLIP (GPU FP16)	800ms	300ms	2.7x
Whisper (standard)	1500ms	1500ms	1x
Whisper (faster-whisper)	1500ms	600-800ms	2-2.5x

Profiling Tools Setup

1. Android Profiler (Client-Side)

Setup in Android Studio:

Connect OPPO Reno 12 via USB (ADB)
Open Android Studio
View → Tool Windows → Profiler
Select com.smartglass.sample process

Key Metrics to Monitor:

CPU Profiler:

Target: < 50% CPU usage during streaming
Monitor: Frame processing threads, SNN inference

Memory Profiler:

Target: < 200MB heap, no memory leaks
Monitor: Bitmap allocations, model loading

Network Profiler:

Target: < 800 KB/min, < 100ms latency
Monitor: Frame uploads, API calls

Battery Profiler:

Target: < 10% drain per hour
Monitor: Background wakelocks, GPS usage

Record Performance Profile:

# Start recording
adb shell am start-profiling com.smartglass.sample

# Run test scenarios (5 minutes)
# ...

# Stop recording
adb shell am stop-profiling com.smartglass.sample

# Pull trace file
adb pull /sdcard/Android/data/com.smartglass.sample/files/profile.trace

2. py-spy (Backend Profiling)

Installation:

pip install py-spy

Profile Backend Server:

# Start backend normally
python -m src.edge_runtime.server &
SERVER_PID=$!

# Record CPU profile for 60 seconds
sudo py-spy record -o profile.svg --pid $SERVER_PID --duration 60

# View flamegraph
firefox profile.svg

Identify Bottlenecks:

Look for functions consuming > 10% CPU time:

CLIPModel.forward() - Vision processing
whisper.transcribe() - Audio transcription
SNNLLMBackend.generate() - Text generation

3. Backend Metrics Endpoint

Enable Detailed Metrics:

# File: src/edge_runtime/server.py

from flask import Flask, jsonify
import time
from collections import defaultdict

class MetricsCollector:
    def __init__(self):
        self.latencies = defaultdict(list)
        self.counters = defaultdict(int)
    
    def record_latency(self, stage: str, duration_ms: float):
        self.latencies[stage].append(duration_ms)
    
    def increment(self, counter: str):
        self.counters[counter] += 1
    
    def get_stats(self):
        stats = {}
        for stage, latencies in self.latencies.items():
            if latencies:
                stats[stage] = {
                    'count': len(latencies),
                    'avg': sum(latencies) / len(latencies),
                    'min': min(latencies),
                    'max': max(latencies),
                    'p50': sorted(latencies)[len(latencies)//2],
                    'p95': sorted(latencies)[int(len(latencies)*0.95)],
                }
        return {'latencies': stats, 'counters': dict(self.counters)}

metrics = MetricsCollector()

@app.route('/metrics')
def get_metrics():
    return jsonify(metrics.get_stats())

Query Metrics:

# Get current metrics
curl http://localhost:5000/metrics | jq .

# Sample output:
{
  "latencies": {
    "vision": {"avg": 423, "min": 380, "max": 520, "p95": 490},
    "asr": {"avg": 734, "min": 650, "max": 890, "p95": 850},
    "llm": {"avg": 312, "min": 280, "max": 410, "p95": 380}
  },
  "counters": {
    "queries": 42,
    "sessions": 3
  }
}

Load Testing Methodology

1. Single-User Load Test

Simulate realistic user behavior:

# File: tests/load_test_single_user.py

import requests
import time
import base64
from PIL import Image
import io

def simulate_user_session(server_url: str, duration_minutes: int = 10):
    """
    Simulate a single user session with realistic query pattern.
    
    Pattern:
    - 1 multimodal query per 2 minutes
    - 2 text queries per 5 minutes
    - Idle time between queries
    """
    session = requests.Session()
    
    # Create session
    resp = session.post(f"{server_url}/sessions")
    session_id = resp.json()["session_id"]
    print(f"Session created: {session_id}")
    
    start_time = time.time()
    queries = 0
    
    while (time.time() - start_time) < duration_minutes * 60:
        # Multimodal query (every 2 minutes)
        if queries % 3 == 0:
            query_start = time.time()
            
            # Load test image
            image = Image.new('RGB', (640, 480), color='blue')
            buffer = io.BytesIO()
            image.save(buffer, format='JPEG', quality=85)
            image_b64 = base64.b64encode(buffer.getvalue()).decode()
            
            resp = session.post(
                f"{server_url}/sessions/{session_id}/multimodal",
                json={"text": "What do you see?", "image": image_b64}
            )
            
            latency = (time.time() - query_start) * 1000
            print(f"Query {queries+1}: {latency:.0f}ms")
            
        # Text query
        else:
            query_start = time.time()
            
            resp = session.post(
                f"{server_url}/sessions/{session_id}/query",
                json={"text": "Hello"}
            )
            
            latency = (time.time() - query_start) * 1000
            print(f"Query {queries+1}: {latency:.0f}ms")
        
        queries += 1
        
        # Wait 30-60 seconds (realistic idle time)
        time.sleep(30 + (time.time() % 30))
    
    print(f"Session completed: {queries} queries in {duration_minutes} minutes")

if __name__ == "__main__":
    simulate_user_session("http://localhost:5000", duration_minutes=10)

2. Multi-User Load Test

Simulate concurrent users with locust:

# File: tests/load_test_multi_user.py

from locust import HttpUser, task, between
import base64
from PIL import Image
import io

class SmartGlassUser(HttpUser):
    wait_time = between(30, 60)  # Wait 30-60s between requests
    
    def on_start(self):
        """Called when user starts. Create session."""
        resp = self.client.post("/sessions")
        self.session_id = resp.json()["session_id"]
    
    @task(3)  # 3x more frequent than multimodal
    def text_query(self):
        """Send text-only query."""
        self.client.post(
            f"/sessions/{self.session_id}/query",
            json={"text": "What time is it?"}
        )
    
    @task(1)
    def multimodal_query(self):
        """Send multimodal query (text + image)."""
        # Create test image
        image = Image.new('RGB', (640, 480), color='red')
        buffer = io.BytesIO()
        image.save(buffer, format='JPEG', quality=85)
        image_b64 = base64.b64encode(buffer.getvalue()).decode()
        
        self.client.post(
            f"/sessions/{self.session_id}/multimodal",
            json={"text": "Describe this", "image": image_b64}
        )

# Run load test:
# locust -f tests/load_test_multi_user.py --host http://localhost:5000 --users 10 --spawn-rate 2

Run Load Test:

# Install locust
pip install locust

# Run with 10 concurrent users
locust -f tests/load_test_multi_user.py \
    --host http://localhost:5000 \
    --users 10 \
    --spawn-rate 2 \
    --run-time 10m \
    --headless

# Or with Web UI (open http://localhost:8089)
locust -f tests/load_test_multi_user.py --host http://localhost:5000

3. Stress Test

Find breaking point:

# Gradually increase load until failures occur
for users in 5 10 20 50 100; do
    echo "Testing with $users users..."
    locust -f tests/load_test_multi_user.py \
        --host http://localhost:5000 \
        --users $users \
        --spawn-rate 5 \
        --run-time 5m \
        --headless \
        --csv results_${users}users
    sleep 60  # Cool down between tests
done

# Analyze results
python analyze_load_test.py results_*.csv

Optimization Checklist

Mobile App (Android)

Backend Server (Python)

Infrastructure

Use GPU backend if available (2-3x speedup)
Enable HTTP/2 for multiplexed requests
Add Redis for session state caching
Configure CDN for static assets
Set up monitoring (Prometheus/Grafana)
Add alerting for latency > 5s
Implement graceful degradation (fallback to smaller models)
Test on target network conditions (4G/5G/Wi-Fi)

Monitoring Dashboard

Key Metrics to Track

Client-Side (App):

- Battery drain rate (%/hour)
- Memory usage (MB)
- Frame upload rate (fps)
- Network bandwidth (KB/min)
- Crash-free rate (%)

Server-Side (Backend):

- Request latency (p50, p95, p99)
- Active sessions
- Model inference times (CLIP, Whisper, SNN)
- Memory usage (GB)
- CPU/GPU utilization (%)
- Error rate (%)

End-to-End:

- Total query latency (ms)
- Success rate (%)
- User satisfaction score

Example Grafana Dashboard Query

# Average end-to-end latency (95th percentile)
histogram_quantile(0.95, 
  rate(smartglass_query_duration_seconds_bucket[5m])
)

# Battery drain rate
rate(smartglass_battery_level[1h]) * -100

# Active sessions
smartglass_active_sessions

Next Steps

Implement frame compression (Quick win: 50% bandwidth reduction)
Deploy faster-whisper (Quick win: 2x Whisper speedup)
Quantize SNN model (Medium effort: 2x inference speedup)
Add adaptive frame rate (Medium effort: 30% battery savings)
Profile with py-spy (Identify remaining bottlenecks)
Run load tests (Validate multi-user performance)
Monitor in production (Continuous optimization)

Additional Resources

Hardware Testing Guide: HARDWARE_TESTING_GUIDE.md
Implementation Progress: IMPLEMENTATION_PROGRESS.md
API Reference: API_REFERENCE.md
Benchmarking Scripts: /bench/audio_bench.py, /bench/image_bench.py
PyTorch Mobile: https://pytorch.org/mobile/android/
faster-whisper: https://github.com/guillaumekln/faster-whisper
Android Profiler: https://developer.android.com/studio/profile

Last Updated: December 2024 Version: 1.0 Target Deployment: Meta Ray-Ban + OPPO Reno 12

FilesExpand file tree

PERFORMANCE_OPTIMIZATION.md

Latest commit

History

PERFORMANCE_OPTIMIZATION.md

File metadata and controls

Performance Optimization Guide

Overview

Current vs Target Benchmarks

System Performance Metrics

Component Breakdown (Current)

Target Breakdown

Optimization Strategy 1: Frame Compression

Problem

Solution: JPEG Quality Adjustment

Implementation in Android App

Backend Frame Handling

Expected Impact

Optimization Strategy 2: SNN INT8 Quantization

Problem

Solution: INT8 Post-Training Quantization

Step 1: Export SNN Model for Mobile

Step 2: Update Android SNN Engine

Step 3: Build Script Updates

Expected Impact

Optimization Strategy 3: Adaptive Frame Rate

Problem

Solution: Dynamic Frame Rate Based on Context

Implementation

Configuration Matrix

Expected Impact

Optimization Strategy 4: Backend Model Optimization

Problem

Solution: Model Optimization Pipeline

4.1: CLIP Model Optimization

4.2: Whisper Model Optimization

4.3: Install Faster-Whisper

Expected Impact

Profiling Tools Setup

1. Android Profiler (Client-Side)

Setup in Android Studio:

Key Metrics to Monitor:

Record Performance Profile:

2. py-spy (Backend Profiling)

Installation:

Profile Backend Server:

Identify Bottlenecks:

3. Backend Metrics Endpoint

Enable Detailed Metrics:

Query Metrics:

Load Testing Methodology

1. Single-User Load Test

2. Multi-User Load Test

Run Load Test:

3. Stress Test

Optimization Checklist

Mobile App (Android)

Backend Server (Python)

Infrastructure

Monitoring Dashboard

Key Metrics to Track

Example Grafana Dashboard Query

Next Steps

Additional Resources