Merge pull request #9 from mbarbin/improve-intro-book

Improve intro book

Author: mbarbin

.vscode/settings.json

@@ -8,21 +8,32 @@
         "Barbin",
         "cmxs",
         "decr",
+        "dedented",
+        "dunolint",
+        "EDSL",
         "endline",
         "Fpath",
+        "iteri",
         "ltrim",
         "Mathieu",
+        "mathjax",
         "mbarbin",
         "mdbook",
         "mdexp",
         "noprompt",
+        "opam",
+        "prec",
         "rtrim",
         "runtest",
         "sexp",
+        "slipshow",
         "sprintf",
         "stdlib",
         "toplevel",
+        "Typst",
         "unflushed",
-        "Yojson"
+        "windtrap",
+        "Yojson",
+        "zstd"
     ]
 }

doc/book/introduction-to-mdexp/SUMMARY.md

@@ -6,3 +6,5 @@
 - [Snapshots](snapshots.md)
   - [ppx_expect](ppx_expect_snapshots.md)
   - [Expect tests without ppx](windtrap_snapshots.md)
+- [EDSL](edsl.md)
+- [Broadening the View](broadening_the_view.md)

doc/book/introduction-to-mdexp/book.toml

@@ -13,6 +13,7 @@ default-theme = "light"
 preferred-dark-theme = "navy"
 git-repository-url = "https://github.com/mbarbin/mdexp"
 theme = "../shared-theme"
+mathjax-support = true
 additional-js = ["../shared-theme/ansi-plugin.js"]
 
 [output.html.fold]

doc/book/introduction-to-mdexp/broadening_the_view.md

@@ -0,0 +1,104 @@
+# Broadening the View
+
+Now that you have a feel for how mdexp works, let's step back and
+consider where the approach leads and where you might take it.
+
+The previous chapters focused on mechanics: directives, snapshots,
+and the EDSL pattern. Here we broaden the view to explore
+additional use cases, discuss design choices, and invite you to
+think about how this pattern of *compilable, verified documentation*
+might apply to your own projects, whether or not you use mdexp
+itself.
+
+## Cram-style tests in OCaml
+
+OCaml's ecosystem uses `.t` cram files to test command-line tools:
+you write a shell session with expected output and the test runner
+checks it. mdexp offers a path to express the same tests in OCaml
+and we are experimenting with a `Mdexp_cram` library (wip, unpublished).
+The "background" part of the process becomes the evaluation environment
+(setting up state, running commands, capturing output) while the visible
+part reads as a tutorial showing the session step by step.
+
+This means the same file can serve as both a functional test and a
+user-facing walkthrough: the test runner verifies correctness, and
+mdexp extracts the narrative.
+
+## Client/server application testing
+
+The pattern extends naturally to networked applications. The
+background spins up a running server, and the visible part scripts a
+series of CLI client calls that exercise the service. Each call and
+its output become a snapshot in the document; the full session builds
+up an end-to-end functional test.
+
+The reader sees a coherent tutorial --- "first create a resource, then
+query it, then update it" --- while the build system verifies that
+every response matches expectations. The documentation becomes a live
+integration test.
+
+## Your own EDSLs
+
+The [EDSL chapter](edsl.md) showed a math expression language and a
+toy proof assistant. But there is nothing special about math --- the
+same pattern applies wherever you want to both **render** and **verify**
+structured content.
+
+The key point is that mdexp does not provide or prescribe any of
+these EDSLs. It provides the infrastructure --- directives, snapshot
+capture, the `pp` pipeline --- and you build whatever domain-specific
+layer makes sense for your project. It is just OCaml; there is no
+plugin architecture to learn. Your EDSL lives in your codebase, uses
+your types, and evolves with your project.
+
+## Host and output languages
+
+Neither the host language nor the output format is a hard constraint
+of the system. mdexp's directive syntax is designed to work with
+any language that has block comments, and we have prepared support
+for host-language expect-test frameworks written in **Zig** and
+**Rust** alongside OCaml. Similarly, the output is not limited to
+Markdown --- it could just as well be Typst, reStructuredText, or
+any other text format.
+
+That said, we have focused our effort on the **OCaml + Markdown**
+combination for practical reasons:
+
+- **OCaml** is well suited to the kind of work mdexp encourages.
+  The examples in this book --- symbolic computation, algebraic
+  rewriting, proof construction --- benefit from a language where
+  you can manipulate structured data at a high level without
+  worrying about memory management. Rust and Zig are equally
+  appealing here --- each handles memory in its own way without
+  burdening documentation-oriented code. Which language fits best
+  may depend on the libraries you need to pull into your document.
+  We would not be surprised if mdexp found a place in
+  multi-language codebases, with different chapters written in
+  different host languages.
+- **Markdown** integrates well with the rest of our tooling: static
+  site generators, mdbook, slipshow, and the broader ecosystem of
+  tools that consume Markdown as input.
+
+Other combinations are possible and welcome; the architecture does
+not privilege any particular pairing.
+
+## Closing thoughts
+
+Beyond the specific tool, the underlying pattern is worth
+reflecting on: write documentation as a compilable source file,
+use the type system and test framework to keep it honest, and let
+a thin preprocessor extract the readable output. This is not a new
+idea --- literate programming is decades old --- but the combination
+with snapshot testing and embedded DSLs gives it a practical edge
+that we believe makes it click in interesting ways.
+
+mdexp is one implementation of this pattern, intentionally minimal.
+It reads directives, extracts content, and produces output.
+Everything else --- the types, the tests, the rendering logic, the
+EDSLs --- is ordinary code in your project. The tool stays small
+and predictable; the power comes from what you build on top of it.
+
+We hope this introductory book has given you both a working
+understanding of mdexp and a sense of the broader possibilities.
+For deeper coverage --- reference material, how-to guides, and
+additional examples --- see the main documentation site.

doc/book/introduction-to-mdexp/broadening_the_view.ml

@@ -0,0 +1,112 @@
+(*********************************************************************************)
+(*  mdexp - Literate Programming with Embedded Snapshots                         *)
+(*  SPDX-FileCopyrightText: 2025-2026 Mathieu Barbin <mathieu.barbin@gmail.com>  *)
+(*  SPDX-License-Identifier: LGPL-3.0-or-later WITH LGPL-3.0-linking-exception   *)
+(*********************************************************************************)
+
+(* @mdexp
+
+# Broadening the View
+
+Now that you have a feel for how mdexp works, let's step back and
+consider where the approach leads and where you might take it.
+
+The previous chapters focused on mechanics: directives, snapshots,
+and the EDSL pattern. Here we broaden the view to explore
+additional use cases, discuss design choices, and invite you to
+think about how this pattern of *compilable, verified documentation*
+might apply to your own projects, whether or not you use mdexp
+itself.
+
+## Cram-style tests in OCaml
+
+OCaml's ecosystem uses `.t` cram files to test command-line tools:
+you write a shell session with expected output and the test runner
+checks it. mdexp offers a path to express the same tests in OCaml
+and we are experimenting with a `Mdexp_cram` library (wip, unpublished).
+The "background" part of the process becomes the evaluation environment
+(setting up state, running commands, capturing output) while the visible
+part reads as a tutorial showing the session step by step.
+
+This means the same file can serve as both a functional test and a
+user-facing walkthrough: the test runner verifies correctness, and
+mdexp extracts the narrative.
+
+## Client/server application testing
+
+The pattern extends naturally to networked applications. The
+background spins up a running server, and the visible part scripts a
+series of CLI client calls that exercise the service. Each call and
+its output become a snapshot in the document; the full session builds
+up an end-to-end functional test.
+
+The reader sees a coherent tutorial --- "first create a resource, then
+query it, then update it" --- while the build system verifies that
+every response matches expectations. The documentation becomes a live
+integration test.
+
+## Your own EDSLs
+
+The [EDSL chapter](edsl.md) showed a math expression language and a
+toy proof assistant. But there is nothing special about math --- the
+same pattern applies wherever you want to both **render** and **verify**
+structured content.
+
+The key point is that mdexp does not provide or prescribe any of
+these EDSLs. It provides the infrastructure --- directives, snapshot
+capture, the `pp` pipeline --- and you build whatever domain-specific
+layer makes sense for your project. It is just OCaml; there is no
+plugin architecture to learn. Your EDSL lives in your codebase, uses
+your types, and evolves with your project.
+
+## Host and output languages
+
+Neither the host language nor the output format is a hard constraint
+of the system. mdexp's directive syntax is designed to work with
+any language that has block comments, and we have prepared support
+for host-language expect-test frameworks written in **Zig** and
+**Rust** alongside OCaml. Similarly, the output is not limited to
+Markdown --- it could just as well be Typst, reStructuredText, or
+any other text format.
+
+That said, we have focused our effort on the **OCaml + Markdown**
+combination for practical reasons:
+
+- **OCaml** is well suited to the kind of work mdexp encourages.
+  The examples in this book --- symbolic computation, algebraic
+  rewriting, proof construction --- benefit from a language where
+  you can manipulate structured data at a high level without
+  worrying about memory management. Rust and Zig are equally
+  appealing here --- each handles memory in its own way without
+  burdening documentation-oriented code. Which language fits best
+  may depend on the libraries you need to pull into your document.
+  We would not be surprised if mdexp found a place in
+  multi-language codebases, with different chapters written in
+  different host languages.
+- **Markdown** integrates well with the rest of our tooling: static
+  site generators, mdbook, slipshow, and the broader ecosystem of
+  tools that consume Markdown as input.
+
+Other combinations are possible and welcome; the architecture does
+not privilege any particular pairing.
+
+## Closing thoughts
+
+Beyond the specific tool, the underlying pattern is worth
+reflecting on: write documentation as a compilable source file,
+use the type system and test framework to keep it honest, and let
+a thin preprocessor extract the readable output. This is not a new
+idea --- literate programming is decades old --- but the combination
+with snapshot testing and embedded DSLs gives it a practical edge
+that we believe makes it click in interesting ways.
+
+mdexp is one implementation of this pattern, intentionally minimal.
+It reads directives, extracts content, and produces output.
+Everything else --- the types, the tests, the rendering logic, the
+EDSLs --- is ordinary code in your project. The tool stays small
+and predictable; the power comes from what you build on top of it.
+
+We hope this introductory book has given you both a working
+understanding of mdexp and a sense of the broader possibilities.
+For deeper coverage --- reference material, how-to guides, and
+additional examples --- see the main documentation site. *)

doc/book/introduction-to-mdexp/broadening_the_view.mli

@@ -0,0 +1,5 @@
+(*_********************************************************************************)
+(*_  mdexp - Literate Programming with Embedded Snapshots                         *)
+(*_  SPDX-FileCopyrightText: 2025-2026 Mathieu Barbin <mathieu.barbin@gmail.com>  *)
+(*_  SPDX-License-Identifier: LGPL-3.0-or-later WITH LGPL-3.0-linking-exception   *)
+(*_********************************************************************************)

doc/book/introduction-to-mdexp/code_blocks.md

@@ -1,50 +1,61 @@
 # Code Blocks
 
-Use `@mdexp.code` to include code examples in your documentation.
-The extracted code is wrapped in a fenced code block, with `ocaml`
-as the default language.
+The `@mdexp.code` directive marks a region of code for inclusion in
+the documentation. The extracted code is wrapped in a fenced code
+block, tagged with the host language detected from the file
+extension (e.g. `.ml` → `ocaml`).
 
-## Code in Comments
+## Including compilable code
 
-Write code inside comment wrappers. mdexp strips the markers:
+Code between `@mdexp.code` and the next directive is real,
+compilable OCaml. The compiler type-checks it along with the
+rest of the file. Here is what the source looks like:
 
 ```ocaml
 (* @mdexp.code *)
-(* let greeting = "Hello, World!" *)
-(* let () = print_endline greeting *)
+let answer = 42
+let question_of_life () = Printf.printf "The answer is %d\n" answer
 (* @mdexp.end *)
 ```
 
-This produces:
+And here is the markdown that mdexp produces from it:
 
 ```ocaml
-let greeting = "Hello, World!"
-let () = print_endline greeting
+let answer = 42
+let question_of_life () = Printf.printf "The answer is %d\n" answer
 ```
 
-## Actual Code
+Code outside the directives is invisible to the document but still
+compiled and tested. For instance, the function above is
+exercised in an expect test that does not appear in the output.
 
-Code between the directives can also be real, compilable OCaml ---
-not wrapped in comments. This is useful when you want the compiler
-to type-check your documentation examples:
+## Prose to code transition
+
+A block comment can transition directly from prose into a code
+block by placing `@mdexp.code` before the closing comment marker.
+This avoids the need for a separate comment:
 
 ```ocaml
-(* @mdexp.code *)
-let answer = 42
-let question_of_life () = Printf.printf "The answer is %d\n" answer
+(* @mdexp
+Here is an example:
+@mdexp.code *)
+let greet name = Printf.printf "Hello, %s!\n" name
 (* @mdexp.end *)
 ```
 
-The code is included as-is:
+This produces the prose paragraph followed by the code block in a
+single, natural flow:
+
+Here is an example:
 
 ```ocaml
-let answer = 42
-let question_of_life () = Printf.printf "The answer is %d\n" answer
+let greet name = Printf.printf "Hello, %s!\n" name
 ```
 
-## Explicit Language
+## Explicit language
 
-Override the default language tag with `@mdexp.code { lang: "<lang>" }`:
+The language tag is inferred from the file extension. You can override it
+with `@mdexp.code { lang: "<lang>" }`:
 
 ```ocaml
 (* @mdexp.code { lang: "bash" } *)
@@ -58,22 +69,6 @@ This produces a `bash`-tagged code fence:
 opam install mdexp
 ```
 
-## Prose to Code Transition
-
-A block comment can transition directly from prose into a code block
-using `@mdexp.code` before the block comment closes:
-
-```ocaml
-(* @mdexp
-Here is an example:
-@mdexp.code *)
-let hello () = print_endline "Hello!"
-(* @mdexp.end *)
-```
-
-This outputs the prose followed by the code block, without needing
-separate comments:
-
-```ocaml
-let (_ : string) = "Hello!"
-```
+Note that the bash command is written inside OCaml comments. This is
+one case where commented code makes sense, since the content is not
+OCaml and cannot be compiled.