Implement zero-dependency Wasm binary parser (#754)

* update `DataDefn` and `ExternDefn`, fix type annotations

* add binary parser for numbers

* parse modules

* add integrations tests and compare legacy and new parser

* format

* pyupgrade

* organize imports

* fix `<exports>`: migrate to typed export indices

* migrate to the new parser

* add `#instrWithPos` support

* Set Version: 0.1.157

* fix `elem_init` after instrWithPos

* remove unused `BlockMetaData`

* fix `limits` address-type flag bytes for memory64

The i64 (memory64/table64) limit encodings are 0x04/0x05 per the
Wasm 3.0 grammar; 0x02/0x03 are the shared-memory (threads) flags for
i32 and were being misdecoded as i64. Address type is discarded
downstream regardless, so this only affects which flag byte is
recognized as which type.

Adds regression tests covering the i32/i64 limit cases and confirming
the shared-memory flags are now rejected instead of misdecoded.

* fix custom sections not being consumed

parse_custom_section skipped the section id byte but then read_bytes(n, s)
used n (the id, always 0) as the length instead of reading the section's
size:u32 first. This left the stream misaligned after any custom section,
silently dropping every section that followed. Also switch the
"no more custom sections" sentinel from falsy bytes to explicit None,
since an empty-payload custom section is valid and isn't the same as
"not a custom section".

Adds a regression test that builds raw bytes with custom sections
interleaved between real sections and confirms the type/function/export
sections still parse correctly.

* add memory.copy and memory.fill opcode parsing

0xFC 10 (memory.copy) and 0xFC 11 (memory.fill) map to the existing
aCopy/aFill K constructs. Their memory index operands are parsed and
discarded, matching the existing memory.size/memory.grow handling,
since the K semantics has a single implicit memory.

Adds parametrized unit tests (zero and multi-byte memidx values, plus
a trailing sentinel byte) proving the parser consumes exactly the
memidx operands rather than a fixed byte count, and exercises both
opcodes through the full K interpreter via instrs.wat.

* fix binary parser to reject trailing data after module end

parse_module never checked that the input stream was fully consumed, so
an unrecognized section id or garbage appended after a well-formed
module was silently ignored instead of raising a parse error.

* remove unused iterate combinator from binary parser

Dead code: nothing in the parser calls it, and its broad exception
swallow (WasmParseError | IndexError | ValueError) risked masking
real bugs if it were ever reused.

* raise WasmParseError instead of ValueError on func/code length mismatch

zip(function_section, code_section, strict=True) surfaced a bare
Python ValueError on mismatch, unlike every other malformed-input
path in the parser, which raises WasmParseError.

* fix peek_bytes to restore stream position on partial-EOF read

reset(pos, s) only ran on the success path, so a short read at EOF
(fewer than n bytes remaining but more than zero) left the stream
advanced past the bytes it consumed while trying, violating peek's
no-side-effects contract.

* fix parsing of if instructions without an else branch

'if bt in* end' is a valid encoding that omits the 0x05 else opcode;
the parser only handled the explicit-else form and raised
'Unsupported opcode: 0x0b' on the then-branch terminator.

* reject non-zero memory indices instead of silently using memory 0

The K semantics models a single memory; multi-memory support is future
work. Previously the memory index was parsed and discarded in memarg,
memory.size/grow/copy/fill, active data segments, and memory exports,
so a valid multi-memory module would silently execute against memory 0.
Now memidx raises WasmParseError on any index other than 0.

* improve comment clarity in binary parser unit tests

Explain the constructed bytecode in TestCustomSections and
TestFuncCodeLengthMismatch, state the sentinel-byte contract once at
class level instead of four times, explain why the shared-memory limit
flags 0x02/0x03 must be rejected, note the intent behind the LEB128
boundary values, and untangle the peek_bytes test comment.

* scope the recursion limit to run_module and clean up dead code

sys.setrecursionlimit(1500000000) was an import-time side effect
inherited from the 2020 py-wasm-era script, and the value effectively
disabled Python's recursion guard. The limit is only needed because pyk
serializes kore terms recursively (pyk.kore.syntax.Pattern.write) and
the depth grows with module size: basic-features.wat needs about 4000,
while the default 1000 is enough for everything else. Set 20000 inside
run_module, never lowering an externally raised limit.

Also remove an assert made dead by check=True and a try/except that
re-raised as a bare Exception, hiding the original exception type.

* reject unsupported 64-bit addressing and non-funcref table imports

The limits parser decoded the memory64/table64 flags (0x04/0x05) and
returned the i64 address type, which every caller discarded; a table
import's reference type was likewise parsed and dropped since the K
ImportDefn sort has no reftype slot. In both cases a valid module using
the feature would silently execute with wrong (32-bit / funcref)
semantics. The K AST cannot represent either feature, so raise a parse
error instead, keeping the spec-correct handling in place as
commented-out skeleton for future support.

* improve parse error reporting and exception hygiene

Magic and version checks no longer rely on assert (stripped under
python -O) and report expected vs. actual bytes; invalid UTF-8 in a
name raises WasmParseError instead of leaking UnicodeDecodeError; the
blocktype error names the unsupported type-index (i33) form; and the
0xFC handler catches WasmParseError instead of Exception so internal
errors are not disguised as parse errors. No accept/reject behavior
changes.

* turn version-bump CI job into a check instead of auto-commit

* update uv lock

* remove obsolete unit test file `test_wasm2kast.py`

---------

Co-authored-by: devops <devops@runtimeverification.com>

Author: bbyalcinkaya

.github/workflows/test-pr.yml

@@ -15,23 +15,19 @@ jobs:
       - name: 'Check out code'
         uses: actions/checkout@v3
         with:
-          token: ${{ secrets.JENKINS_GITHUB_PAT }}
           # fetch-depth 0 means deep clone the repo
           fetch-depth: 0
           ref: ${{ github.event.pull_request.head.sha }}
-      - name: 'Configure GitHub user'
-        run: |
-          git config user.name devops
-          git config user.email devops@runtimeverification.com
-      - name: 'Update version'
+      - name: 'Check version was bumped'
         run: |
           og_version=$(git show origin/${GITHUB_BASE_REF}:package/version)
           ./package/version.sh bump ${og_version}
           ./package/version.sh sub
-          new_version=$(cat package/version)
-          git add --update && git commit --message "Set Version: ${new_version}" || true
-      - name: 'Push updates'
-        run: git push origin HEAD:${GITHUB_HEAD_REF}
+          if ! git diff --quiet; then
+            echo "::error::Version was not bumped for this PR. Run './package/version.sh bump ${og_version} && ./package/version.sh sub' locally and commit the result."
+            git diff
+            exit 1
+          fi
 
   pykwasm-code-quality-checks:
     name: 'Code Quality Checks'

package/version

@@ -1 +1 @@
-0.1.156
+0.1.157

pykwasm/pyproject.toml

@@ -4,13 +4,12 @@ build-backend = "hatchling.build"
 
 [project]
 name = "pykwasm"
-version = "0.1.156"
+version = "0.1.157"
 description = ""
 readme = "README.md"
 requires-python = "~=3.10"
 dependencies = [
-    "kframework>=7.1.329",
-    "py-wasm@git+https://github.com/runtimeverification/py-wasm.git@0.3.1"
+    "kframework>=7.1.329"
 ]
 
 [[project.authors]]

pykwasm/src/pykwasm/binary/__init__.py

@@ -0,0 +1 @@
+from .module import parse_module

pykwasm/src/pykwasm/binary/combinators.py

@@ -0,0 +1,43 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+from .integers import u32
+from .utils import WasmParseError, reset
+
+if TYPE_CHECKING:
+    from .utils import A, InputStream, Parser
+
+
+def sized(p: Parser[A], s: InputStream) -> A:
+    size = u32(s)
+    start_pos = s.tell()
+    res = p(s)
+    end_pos = s.tell()
+    if end_pos - start_pos != size:
+        raise WasmParseError('Size mismatch')
+    return res
+
+
+def parse_n(p: Parser[A], n: int, s: InputStream) -> list[A]:
+    results = []
+    for _ in range(n):
+        x = p(s)
+        results.append(x)
+    return results
+
+
+def list_of(p: Parser[A], s: InputStream) -> list[A]:
+    n = u32(s)
+    return parse_n(p, n, s)
+
+
+def either(ps: list[Parser[A]], s: InputStream) -> A:
+    for p in ps:
+        pos = s.tell()
+        try:
+            return p(s)
+        except WasmParseError:
+            reset(pos, s)
+            continue
+    raise WasmParseError('None of the alternatives succeeded')

pykwasm/src/pykwasm/binary/floats.py

@@ -0,0 +1,21 @@
+from __future__ import annotations
+
+import struct
+from typing import TYPE_CHECKING
+
+from .utils import read_bytes
+
+if TYPE_CHECKING:
+    from .utils import InputStream
+
+
+def f32(s: InputStream) -> float:
+    bs = read_bytes(4, s)
+    f = struct.unpack('<f', bs)[0]
+    return f
+
+
+def f64(s: InputStream) -> float:
+    bs = read_bytes(8, s)
+    f = struct.unpack('<d', bs)[0]
+    return f

pykwasm/src/pykwasm/binary/indices.py

@@ -0,0 +1,74 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+import pykwasm.kwasm_ast as wast
+
+from .integers import u32
+from .utils import WasmParseError, read_byte
+
+if TYPE_CHECKING:
+    from pyk.kast.inner import KInner
+
+    from .utils import InputStream
+
+
+def typeidx(s: InputStream) -> int:
+    return u32(s)
+
+
+def funcidx(s: InputStream) -> int:
+    return u32(s)
+
+
+def tableidx(s: InputStream) -> int:
+    return u32(s)
+
+
+# TODO multi-memory support is future work; the K semantics models a single memory,
+# so any reference to a memory other than 0 is rejected instead of mis-executing.
+def memidx(s: InputStream) -> int:
+    x = u32(s)
+    if x != 0:
+        raise WasmParseError(f'Multi-memory is not supported. Expected memory index 0, got: {x}')
+    return x
+
+
+def globalidx(s: InputStream) -> int:
+    return u32(s)
+
+
+def tagidx(s: InputStream) -> int:
+    return u32(s)
+
+
+def elemidx(s: InputStream) -> int:
+    return u32(s)
+
+
+def dataidx(s: InputStream) -> int:
+    return u32(s)
+
+
+def localidx(s: InputStream) -> int:
+    return u32(s)
+
+
+def labelidx(s: InputStream) -> int:
+    return u32(s)
+
+
+def externidx(s: InputStream) -> KInner:
+    match read_byte(s):
+        case 0x00:
+            return wast.externidx_func(funcidx(s))
+        case 0x01:
+            return wast.externidx_table(tableidx(s))
+        case 0x02:
+            return wast.externidx_memory(memidx(s))
+        case 0x03:
+            return wast.externidx_global(globalidx(s))
+        case 0x04:
+            return wast.externidx_tag(tagidx(s))
+        case x:
+            raise WasmParseError(f'Invalid externidx descriptor: {x}')