LightningMD ⚡

Ultra-fast Markdown to HTML converter built in Rust with WebAssembly support.

Features

🚀 Ultra-fast: Built with performance in mind using Rust and optimized algorithms
🌐 WASM Ready: First-class WebAssembly support for browser and Node.js
🔧 Extensible: Plugin system for custom processing and rendering
📊 Multiple Outputs: HTML, JSON AST, and custom formats
⚛️ MDX Support: React component integration with JSX syntax
🛡️ Secure: Built-in XSS protection and input sanitization
📱 Universal: CLI tool, library, and JavaScript bindings

Quick Start

CLI Usage

# Install (once Rust/Cargo is available)
cargo install lightning-md

# Convert Markdown to HTML
lmd html -i document.md -o output.html

# Convert to JSON AST
lmd json -i document.md -o ast.json --pretty

# Convert MDX to React component
lmd mdx -i component.mdx -o Component.jsx --format react

# Benchmark performance
lmd bench -i large-document.md -n 1000

Library Usage (Rust)

use lightning_md::{LightningMD, ParallelPerformanceReport};

fn main() {
    let lmd = LightningMD::new();
    
    // Basic usage
    let html = lmd.to_html("# Hello **World**!").unwrap();
    println!("{}", html); // <h1>Hello <strong>World</strong>!</h1>
    
    // Memory-optimized processing
    let html = lmd.to_html_optimized("# Memory efficient").unwrap();
    let memory_stats = lmd.get_memory_stats();
    println!("Memory efficiency: {:.1}%", memory_stats.overall_hit_rate() * 100.0);
    
    // Parallel batch processing
    let contents = vec!["# Doc 1".to_string(), "# Doc 2".to_string()];
    let result = lmd.batch_process_parallel(contents).unwrap();
    let report = ParallelPerformanceReport::new(result);
    println!("{}", report.format_report());
}

JavaScript/TypeScript Usage

import { markdownToHtml, mdxToReact, LightningMD } from '@lightning-md/core';

// Simple usage
const html = await markdownToHtml('# Hello **World**!');

// MDX to React
const reactCode = await mdxToReact(`
# Hello from MDX!

<MyComponent prop="value" />

export default function() {
  return <div>This is MDX!</div>;
}
`);

// Advanced usage
const lmd = new LightningMD();
await lmd.init();
const html = lmd.toHtml('# Hello **World**!');
const mdxResult = lmd.renderMdx(mdxContent, mdxOptions, renderOptions);

Architecture

Core Components

Parser: Based on pulldown-cmark with extensions for tables, footnotes, and more
AST: Structured, serializable Abstract Syntax Tree with serde support
Renderers: Multiple output formats (HTML, JSON) with customizable options
Plugin System: Extensible processing pipeline for custom transformations
WASM Interface: Browser and Node.js compatible WebAssembly bindings

Plugin System

use lightning_md::plugin::{BlockPlugin, PluginManager};

struct CustomPlugin;

impl BlockPlugin for CustomPlugin {
    fn name(&self) -> &str { "custom" }
    
    fn process(&self, node: &mut AstNode) -> Result<(), LMDError> {
        // Custom processing logic
        Ok(())
    }
}

let mut manager = PluginManager::new();
manager.add_block_plugin(Box::new(CustomPlugin));

MDX Support

LightningMD includes comprehensive MDX support, allowing you to embed JSX components in Markdown:

MDX Features

JSX Components: Embed React components directly in Markdown
Frontmatter: YAML, TOML, and JSON metadata support
Import/Export: ES6 module syntax with component imports
Multiple Runtimes: React, Preact, Vue, Solid.js support
TypeScript: Full TypeScript output generation
Built-in Components: CodeBlock, Alert, Tabs, Chart components

MDX CLI Usage

# Convert MDX to React component
lmd mdx -i blog-post.mdx -o BlogPost.jsx --format react

# Generate TypeScript component
lmd mdx -i docs.mdx -o Docs.tsx --format ts --typescript

# Multiple runtime support
lmd mdx -i component.mdx -o output.js --runtime preact

# Development mode with analysis
lmd mdx -i content.mdx --development --analyze

MDX Library Usage

use lightning_md::mdx::{MdxParser, MdxRenderer, MdxOptions, JsxRuntime};

let mut parser = MdxParser::new();
let options = MdxOptions {
    jsx_runtime: JsxRuntime::ReactAutomatic,
    allow_dangerous_html: false,
    component_naming: ComponentNaming::PascalCase,
    ..Default::default()
};
parser.set_options(options);

let document = parser.parse(mdx_content)?;
let renderer = MdxRenderer::new();
let result = renderer.render_mdx(&document)?;

println!("Generated code: {}", result.code);

MDX Example

---
title: "My Blog Post"
author: "John Doe"
tags: ["react", "mdx"]
---

import { MyButton } from './components/Button';
import Chart from './Chart';

# Welcome to MDX!

This is **Markdown** with JSX components:

<MyButton variant="primary">
  Click me!
</MyButton>

Here's some data visualization:

<Chart 
  data={[1, 2, 3, 4, 5]} 
  type="line"
  title="Sample Data"
/>

## Code Example

```javascript
const greeting = "Hello from MDX!";
console.log(greeting);

export default function Layout({ children }) { return

{children}

; }


## Performance

LightningMD is designed for high-performance Markdown processing with advanced optimizations:

- **Parsing**: >100MB/s on modern hardware
- **Rendering**: >200MB/s HTML output generation  
- **Memory**: Minimal allocations with efficient string handling and memory pooling
- **SIMD Optimization**: AVX2/NEON acceleration for text processing (2-3x faster)
- **Parallel Processing**: Work-stealing algorithm with dynamic load balancing
- **Memory Efficiency**: Object pooling reduces allocation overhead by 90%

### Benchmarking

Run comprehensive benchmarks with performance analysis:

```bash
# All benchmarks
./scripts/benchmark.sh

# Criterion benchmarks only
cargo bench

# CLI built-in benchmark with parallel processing
lmd bench -i tests/fixtures/sample.md -n 1000 --parallel

# SIMD performance comparison
lmd bench -i large-document.md --simd-compare

# Memory efficiency analysis
lmd bench -i document.md --memory-analysis

# Parallel processing analysis
lmd batch -i docs/ -o dist/ --parallel --workers 8 --performance-report

Development

Prerequisites

Rust 1.70+
wasm-pack for WebAssembly builds
hyperfine for benchmarking

Building

# Standard build
cargo build --release

# WASM build
wasm-pack build --target web --features wasm

# Run tests
cargo test

# Run benchmarks
cargo bench

Project Structure

src/
├── lib.rs          # Main library interface
├── ast.rs          # AST node definitions
├── parser.rs       # Markdown parser
├── renderer.rs     # HTML/JSON renderers
├── plugin.rs       # Plugin system
├── error.rs        # Error types
├── wasm.rs         # WebAssembly bindings
├── simd.rs         # SIMD text processing optimization
├── memory.rs       # Memory pooling and optimization
├── parallel.rs     # Work-stealing parallel processing
├── mdx/            # MDX support modules
│   ├── mod.rs      # Main MDX interface
│   ├── jsx.rs      # JSX parsing
│   ├── components.rs # Component registry
│   ├── transformer.rs # JSX transformation
│   ├── parser.rs   # Advanced parsing
│   └── renderer.rs # MDX rendering
└── bin/cli.rs      # CLI implementation

tests/
├── integration_test.rs  # Integration tests
├── golden_test.rs      # Golden file tests
└── fixtures/           # Test data

benches/
├── parser_bench.rs     # Parser benchmarks
├── renderer_bench.rs   # Renderer benchmarks
└── simd_bench.rs       # SIMD optimization benchmarks

js/                     # JavaScript bindings
scripts/               # Build and benchmark scripts

Testing

Comprehensive test suite including:

Unit Tests: Core functionality testing
Integration Tests: End-to-end conversion testing
Golden Tests: Reference output verification
Performance Tests: Benchmark validation
Edge Cases: Unicode, special characters, malformed input

# Run all tests
cargo test

# Run specific test suite
cargo test integration_test

# Run with performance assertions
cargo test golden_test -- --nocapture

Documentation

CLAUDE.md - Development guide for AI assistance
.claude/ - Detailed Japanese documentation
- Architecture decisions and patterns
- Performance optimization guidelines
- Plugin development examples

Contributing

Fork the repository
Create a feature branch
Run tests and benchmarks
Submit a pull request

Ensure all tests pass and performance benchmarks remain within acceptable ranges.

License

Licensed under either of:

Apache License, Version 2.0 (LICENSE-APACHE)
MIT License (LICENSE-MIT)

at your option.

Roadmap

MDX Support: React component integration with JSX syntax ✅
Runtime Environment: Node.js/Deno/Bun compatibility
CDN Distribution: Cloud-based content delivery
Math equation support (KaTeX integration)
Additional output formats (PDF, EPUB)
Real-time collaborative editing support
Advanced syntax highlighting
Plugin ecosystem and registry

Built with ⚡ by the LightningMD team

FilesExpand file tree

README.md

Latest commit

History