[Discussion] Notion To MD v4: Renderer Plugin System Technical Details #116

souvikinator · 2024-12-29T15:16:53Z

souvikinator
Dec 29, 2024
Maintainer

notion-to-md V4: Renderer Module Technical Documentation

With this notion-to-md is not more limited to Markdown but it can expand to any content format and can cater any usecase.

Overview

The Renderer module is the final transformation layer in the notion-to-md V4 pipeline. Following the Block Fetcher and Media Handler components, it transforms processed Notion blocks into the desired output format through a plugin-based architecture. The core architecture is making everything plugin-based, including our default Markdown/MDX renderer, while maintaining a simple and predictable API.

Plugin In Action

Using These Renderers

All these renderers can be used in the same way:

// Create the renderer
const renderer = new JSXRenderer();  // or HTMLRenderer, TerminalRenderer, etc.

// Use it with notion-to-md
const n2m = new NotionToMarkdown({
  notionClient,
  renderer
});

// Generate output
const output = await n2m.pageToMarkdown('page-id');

Plugin In definition

Let's explore how to create different types of renderer plugins for notion-to-md. We'll start with common use cases and move to more creative applications.

JSX Component Renderer

First, let's create a renderer that converts Notion content into a React component:

class JSXRenderer extends BaseRendererPlugin {
  template = `
    {{{imports}}}
    
    interface {{{componentName}}}Props {
      className?: string;
      customData?: any;
    }
    
    export function {{{componentName}}}({ className, customData }: {{{componentName}}}Props) {
      return (
        <div className={cn("notion-content", className)}>
          {{{content}}}
          
          {/* Add metadata for client-side usage */}
          <script
            type="application/json"
            id="notion-metadata"
            dangerouslySetInnerHTML={{ __html: JSON.stringify({{{metadata}}}) }}
          />
        </div>
      );
    }
  `;

  variables = {
    componentName: (context) => 
      // Convert filename to PascalCase for component name
      context.filename
        .split('-')
        .map(s => s.charAt(0).toUpperCase() + s.slice(1))
        .join('') || 'NotionContent',

    imports: (context) => `
      import { cn } from '@/lib/utils';
      ${Array.from(context.imports).join('\n')}
    `,

    metadata: (context) => ({
      pageId: context.pageId,
      properties: context.properties,
      lastUpdated: new Date().toISOString()
    })
  };

  // Block transformers optimized for React components
  blockTransformers = {
    table: {
      transform: async (block, context) => `
        <Table data={${JSON.stringify(block.table.rows)}}>
          {${block.table.rows.map(row => `
            <TableRow>
              ${row.map(cell => `<TableCell>${cell}</TableCell>`).join('\n')}
            </TableRow>
          `).join('\n')}}
        </Table>
      `,
      imports: [
        "import { Table, TableRow, TableCell } from '@/components/ui/table';"
      ]
    },

    callout: {
      transform: async (block, context) => `
        <Callout
          icon="${block.callout.icon?.emoji || '💡'}"
          intent="${block.callout.intent || 'info'}"
        >
          ${block.callout.content}
        </Callout>
      `,
      imports: ["import { Callout } from '@/components/ui/callout';"]
    },

    code: {
      transform: async (block, context) => `
        <CodeBlock
          language="${block.code.language}"
          filename="${block.code.filename || ''}"
          highlights={${JSON.stringify(block.code.highlights || [])}}
        >
          {${JSON.stringify(block.code.content)}}
        </CodeBlock>
      `,
      imports: ["import { CodeBlock } from '@/components/ui/code-block';"]
    }
  };
}

HTML Static Site Renderer

Now let's create a renderer for generating static HTML pages:

class HTMLRenderer extends BaseRendererPlugin {
  template = `
    <!DOCTYPE html>
    <html lang="en">
      <head>
        <meta charset="UTF-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <title>{{{title}}}</title>
        {{{seoMeta}}}
        {{{styles}}}
        {{{imports}}}
      </head>
      <body>
        <article class="notion-content">
          {{{content}}}
        </article>
        {{{scripts}}}
      </body>
    </html>
  `;

  variables = {
    title: (context) => 
      context.properties.title?.plain_text || 'Notion Page',

    seoMeta: (context) => `
      <meta name="description" content="${context.properties.description || ''}">
      <meta name="keywords" content="${(context.properties.tags || []).join(', ')}">
      <meta property="og:title" content="${context.properties.title || ''}">
      <meta property="og:description" content="${context.properties.description || ''}">
    `,

    styles: () => `
      <link rel="stylesheet" href="/styles/notion.css">
      <link rel="stylesheet" href="/styles/prism.css">
    `,

    scripts: (context) => `
      <script src="/scripts/prism.js"></script>
      <script>
        // Automatically initialize syntax highlighting
        Prism.highlightAll();
      </script>
    `,

    imports: (context) => Array.from(context.imports).join('\n')
  };

  blockTransformers = {
    table: {
      transform: async (block, context) => `
        <div class="table-container">
          <table>
            ${block.table.rows.map(row => `
              <tr>
                ${row.map(cell => `<td>${cell}</td>`).join('\n')}
              </tr>
            `).join('\n')}
          </table>
        </div>
      `
    },

    callout: {
      transform: async (block, context) => `
        <div class="callout callout-${block.callout.intent || 'info'}">
          ${block.callout.icon ? `
            <span class="callout-icon">${block.callout.icon.emoji}</span>
          ` : ''}
          <div class="callout-content">
            ${block.callout.content}
          </div>
        </div>
      `
    }
  };
}

Terminal Output Renderer

Let's create a renderer that formats content for terminal display:

class TerminalRenderer extends BaseRendererPlugin {
  template = `
    {{{header}}}
    
    {{{content}}}
    
    {{{footer}}}
  `;

  variables = {
    header: (context) => `
      ╔════════════════════════════════════════╗
      ║ ${context.properties.title.padEnd(38)} ║
      ╚════════════════════════════════════════╝
    `,

    footer: (context) => `
      Generated from Notion page: ${context.pageId}
      Last updated: ${new Date().toISOString()}
    `
  };

  blockTransformers = {
    heading_1: {
      transform: async (block) => `
        \x1b[1m\x1b[36m${block.heading_1.text}\x1b[0m
        ${'-'.repeat(block.heading_1.text.length)}
      `
    },

    code: {
      transform: async (block) => `
        \x1b[33m${block.code.language}\x1b[0m
        ┌${'─'.repeat(40)}┐
        │${block.code.content.split('\n').map(line => 
          line.padEnd(38) + '│'
        ).join('\n')}
        └${'─'.repeat(40)}┘
      `
    },

    callout: {
      transform: async (block) => `
        │ ${block.callout.icon?.emoji || '!'} ${block.callout.content
          .split('\n')
          .join('\n│ ')}
      `
    }
  };
}

LaTeX Document Renderer

Here's a renderer that converts Notion content to a LaTeX document:

class LaTeXRenderer extends BaseRendererPlugin {
  template = `
    \\documentclass{article}
    
    {{{packages}}}
    
    \\title{{{{title}}}}
    \\author{{{{author}}}}
    \\date{{{{date}}}}
    
    \\begin{document}
    
    \\maketitle
    
    {{{content}}}
    
    \\end{document}
  `;

  variables = {
    packages: () => `
      \\usepackage{amsmath}
      \\usepackage{graphicx}
      \\usepackage{listings}
      \\usepackage{xcolor}
      \\usepackage{hyperref}
    `,

    title: (context) => 
      this.escapeLatex(context.properties.title || 'Untitled'),
    
    author: (context) =>
      this.escapeLatex(context.properties.author || ''),
    
    date: () => '\\today'
  };

  blockTransformers = {
    heading_1: {
      transform: async (block) =>
        `\\section{${this.escapeLatex(block.heading_1.text)}}`
    },

    heading_2: {
      transform: async (block) =>
        `\\subsection{${this.escapeLatex(block.heading_2.text)}}`
    },

    code: {
      transform: async (block) => `
        \\begin{lstlisting}[language=${block.code.language}]
        ${block.code.content}
        \\end{lstlisting}
      `
    },

    callout: {
      transform: async (block) => `
        \\begin{tcolorbox}[title=${block.callout.icon?.emoji || ''}]
        ${this.escapeLatex(block.callout.content)}
        \\end{tcolorbox}
      `
    }
  };

  private escapeLatex(text: string): string {
    return text.replace(/[&%$#_{}~^\\]/g, '\\$&');
  }
}

Slide Deck Renderer

Finally, let's create a renderer that generates a RevealJS presentation:

class SlideRenderer extends BaseRendererPlugin {
  template = `
    <!DOCTYPE html>
    <html>
      <head>
        <link rel="stylesheet" href="reveal.js/dist/reveal.css">
        <link rel="stylesheet" href="reveal.js/dist/theme/white.css">
        {{{styles}}}
      </head>
      <body>
        <div class="reveal">
          <div class="slides">
            {{{titleSlide}}}
            {{{content}}}
          </div>
        </div>
        
        <script src="reveal.js/dist/reveal.js"></script>
        <script>
          Reveal.initialize({
            // Configuration options
            {{{revealConfig}}}
          });
        </script>
      </body>
    </html>
  `;

  variables = {
    styles: () => `
      <style>
        .reveal pre code {
          max-height: 400px;
          overflow-y: auto;
        }
        .reveal .callout {
          background: #f0f0f0;
          padding: 1em;
          border-left: 4px solid #007acc;
        }
      </style>
    `,

    titleSlide: (context) => `
      <section>
        <h1>${context.properties.title || 'Presentation'}</h1>
        <p>${context.properties.author || ''}</p>
        <p><small>${new Date().toLocaleDateString()}</small></p>
      </section>
    `,

    revealConfig: () => `
      hash: true,
      slideNumber: true,
      transition: 'slide'
    `
  };

  blockTransformers = {
    heading_1: {
      // Each h1 starts a new slide
      transform: async (block) => `
        <section>
          <h1>${block.heading_1.text}</h1>
        </section>
      `
    },

    bullet_list: {
      // Convert bullet lists to animated fragments
      transform: async (block) => `
        <ul>
          ${block.bullet_list.items.map(item => 
            `<li class="fragment">${item}</li>`
          ).join('\n')}
        </ul>
      `
    },

    code: {
      transform: async (block) => `
        <pre><code class="language-${block.code.language}">${
          block.code.content
        }</code></pre>
      `
    }
  };
}

Core Architecture

Base Renderer Plugin

The foundation of our system is the BaseRendererPlugin abstract class. Every renderer, including our default MDX renderer, inherits from this base class:

abstract class BaseRendererPlugin {
  // Required implementations for all renderers
  abstract template: string;
  abstract variables: Record<string, VariableResolver>;
  abstract blockTransformers: Record<string, BlockTransformer>;

  constructor(options: RendererOptions = {}) {
    if (options.template) {
      this.template = options.template;
    }
    
    if (options.variables) {
      this.variables = {
        ...this.variables,
        ...options.variables
      };
    }
    
    if (options.blockTransformers) {
      this.blockTransformers = {
        ...this.blockTransformers,
        ...options.blockTransformers
      };
    }
  }

  // Private render implementation - handles the core transformation process
  private async render(
    mediaHandlerOutput: MediaHandlerOutput, 
    options: RenderOptions
  ): Promise<string> {
    const context = await this.createContext(mediaHandlerOutput, options);
    const transformedContent = await this.transformBlocks(mediaHandlerOutput.blocks, context);
    context.content = transformedContent;
    return await this.resolveTemplate(this.template, context);
  }

  // Helper method to resolve template variables (can get into depth while handling)
}

Core Interfaces

The system is built around several key interfaces that define its behavior:

interface RendererOptions {
  template?: string;
  variables?: Record<string, VariableResolver>;
  blockTransformers?: Record<string, BlockTransformer>;
}

type VariableResolver = (context: Context) => string | Promise<string>;

interface BlockTransformer {
  transform: (block: Block, context: Context) => string | Promise<string>;
  imports?: string[];
}

interface Context {
  content: string;                 // Transformed content
  properties: NotionProperties;    // Page properties
  pageId: string;                 // Current page ID
  filename: string;               // Output filename
  variables: Record<string, any>; // User-provided variables
  imports: Set<string>;          // Collected imports
  blocks: Block[];               // Original blocks
}

Template System

The template system uses triple curly braces ({{{variableName}}}) to denote variables that will be replaced with computed values. This approach provides a simple yet powerful way to structure output.

Default MDX Implementation

Here's how our default MDX renderer implements these concepts:

class MDXRenderer extends BaseRendererPlugin {
  template = `
    {{{frontMatter}}}
    {{{imports}}}
    
    {{{content}}}
  `;

  variables = {
    frontMatter: (context) => {
      const { title, date, tags } = context.properties;
      return `---
title: ${title}
date: ${date}
tags: ${tags}
---`;
    },
    imports: (context) => Array.from(context.imports).join('\n'),
    content: (context) => context.content
  };

  blockTransformers = {
    table: {
      transform: async (block, context) => {
        const rows = block.table.rows;
        return `| ${rows.join(' | ')} |`;
      }
    },
    code: {
      transform: async (block, context) => {
        return `\`\`\`${block.code.language}\n${block.code.content}\n\`\`\``;
      }
    }
    // Other block transformers...
  };
}

Customization System

The system provides a unified approach to customization through the renderer's constructor options. Users can customize three aspects:

Template Structure
Variable Resolution
Block Transformation

Complete Customization Example

const renderer = new MDXRenderer({
  // Custom template with new sections
  template: `
    {{{customHeader}}}
    {{{frontMatter}}}
    {{{imports}}}
    
    {{{content}}}
    
    {{{footer}}}
  `,
  
  // Custom variables
  variables: {
    customHeader: (context) => `
      /**
       * Generated from: ${context.pageId}
       * Date: ${new Date().toISOString()}
       */
    `,
    footer: (context) => `
      <!-- End of ${context.filename} -->
    `
  },
  
  // Custom block transformers
  blockTransformers: {
    // Override default table implementation
    table: {
      transform: async (block, context) => {
        return `<DataTable 
          data={${JSON.stringify(block.table.rows)}}
          sortable
          searchable
        />`;
      },
      imports: ["import { DataTable } from '@/components/ui/data-table';"]
    },
    
    // Add custom block type
    visualization: {
      transform: async (block, context) => {
        return `<Chart 
          type="${block.visualization.type}"
          data={${JSON.stringify(block.visualization.data)}}
        />`;
      },
      imports: ["import { Chart } from '@/components/charts';"]
    }
  }
});

Future Considerations

While the current implementation provides a solid foundation, future versions might consider:

Template Validation
- Static analysis of templates
- Variable usage validation
- Type checking improvements
Developer Experience
- Better error messages
- Development tools
- Debug utilities

souvikinator · 2025-01-11T22:35:49Z

souvikinator
Jan 11, 2025
Maintainer Author

subject to change as per final conclusion at #112

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[Discussion] Notion To MD v4: Renderer Plugin System Technical Details #116

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

[Discussion] Notion To MD v4: Renderer Plugin System Technical Details #116

Uh oh!

souvikinator Dec 29, 2024 Maintainer

notion-to-md V4: Renderer Module Technical Documentation

Overview

Plugin In Action

Using These Renderers

Plugin In definition

JSX Component Renderer

HTML Static Site Renderer

Terminal Output Renderer

LaTeX Document Renderer

Slide Deck Renderer

Core Architecture

Base Renderer Plugin

Core Interfaces

Template System

Default MDX Implementation

Customization System

Complete Customization Example

Future Considerations

Replies: 1 comment

Uh oh!

souvikinator Jan 11, 2025 Maintainer Author

souvikinator
Dec 29, 2024
Maintainer

souvikinator
Jan 11, 2025
Maintainer Author