CrossFit: Demystifying VM Callback Bugs in Interpreters

This is the artifact for the paper "CrossFit: Demystifying VM Callback Bugs in Interpreters" (to appear at FSE 2026). Paper: [link to be added upon publication]

CrossFit is a 2-tier approach combining static analysis and targeted fuzzing to systematically discover callback bugs in scripting language interpreters. It identifies 20 new bugs across PHP, Python, and Ruby, and provides a 150-PoC ground-truth dataset.

Badges claimed: Available, Functional, Reusable (see STATUS.md)

Getting Started (< 10 minutes)

Prerequisites

• Docker 20.10+ on x86_64 Linux (see REQUIREMENTS.md)

Setup

# Clone the repository
git clone https://github.com/HexHive/crossfit-artifact.git
cd crossfit-artifact

# Pull the image from Docker Hub
docker pull chibinz/crossfit-artifact

# Launch the container (image is prebuilt with all targets)
docker run -it --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
    -v $(pwd):/workdir -w /workdir \
    chibinz/crossfit-artifact fish

All interpreter targets and the LLVM analysis pass are prebuilt inside the Docker image at /targets/. No compilation is needed to start fuzzing. See INSTALL.md for rebuilding from source.

Reproducing Results

Run CrossFit on the PHP, CPython, and Ruby:

fish scripts/launch.fish              # 30-min campaigns, 1 repetition (default)
fish scripts/launch.fish 23h 8        # 23-hour campaigns, 8 repetitions

This launches 3 campaigns in parallel (one per language). After completion, results are printed in a summary table. Campaign results are saved to bench/.

Task 1: Bug Discovery (Table 4)

Verify that CrossFit discovers callback bugs on the latest interpreter versions.

What to check:

1. After the campaign finishes, look for crashing inputs in each campaign's crashes/ directory under bench/. Each crashing input triggers a memory safety violation (e.g., use-after-free, heap-buffer-overflow) caused by a callback bug.

2. To confirm a crash, replay it against the corresponding AddressSanitizer binary:

   ASAN_OPTIONS=detect_leaks=0 /targets/php-asan/bin/php bench/<campaign>/crashes/<poc>
   ASAN_OPTIONS=detect_leaks=0 /targets/ruby-3.5-april-asan/bin/ruby bench/<campaign>/crashes/<poc>

3. Compare the crashing PoCs against the triaged bugs in dataset/. The key pattern to look for is which magic method (e.g., __toString, __destruct, __getattr__, jsonSerialize) is used as the callback entrance, and which entrance operation (e.g., string coercion, object destruction, iterator advance, property access) triggers the callback. Crashes that share the same magic method and entrance operation as a triaged PoC in dataset/ correspond to the same underlying bug.

Task 2: Callsite Coverage (Table 3)

Verify the callsite coverage reported by CrossFit.

What to check:

1. Each campaign directory under bench/ is named using the convention:

   bench:<benchmark>,fuzzer:crossfit,target:<language>,date:<timestamp>
   

   Inside each campaign directory you will find:

   • corpus/ -- the accumulated test corpus
   • crashes/ -- inputs that triggered crashes
   • reached.csv -- callsites reached during fuzzing

2. Open reached.csv in any campaign directory. Each line records a single callsite that was exercised, in the format:

   <caller_function>,<line_number>
   

   where <caller_function> is the C function containing the callsite and <line_number> is the source line in the interpreter where a VM callback can be triggered.

3. To generate a summary of callsite coverage and crash counts across all campaigns:

python3 scripts/summarize.py

This prints, for each language: callsites reached vs. total, coverage percentage, crash count, and a breakdown of which callback-invoker functions (e.g., zval_ptr_dtor, PyObject_Repr, rb_funcall) were exercised.

Collected PoC Dataset (Optional)

• dataset/ — 150 proof-of-concept scripts (PHP .php/.phpt, Python .py, Ruby .rb) that trigger known callback bugs, used for validating and triaging crashes found by CrossFit.

Rebuilding from Source (Optional)

The Docker image ships prebuilt binaries. To rebuild targets from source (e.g., to modify the LLVM pass or test a different interpreter version), see scripts/build.fish:

scripts/build.fish callsite-pass      # Rebuild the LLVM analysis pass
scripts/build.fish callsite php       # Rebuild PHP with callsite instrumentation
scripts/build.fish asan php           # Rebuild PHP with AddressSanitizer

See INSTALL.md for the full list of build commands.

Bug Trophy

CrossFit discovered 20 new callback bugs across PHP, CPython, and Ruby. All bugs have been confirmed and fixed by the developers.

Language	Callback	Crash Site	Type	Report
PHP	__destruct	php_splice	UAF	php/php-src#16649
PHP	__toString	zend_std_compare_objects	UAF	php/php-src#16648
PHP	__destruct	spl_array_unset_dimension	UAF	php/php-src#16646
PHP	__serialize	zif_msg_send	null-deref	php/php-src#16592
PHP	__serialize	php_check_shm_data	assertion	php/php-src#16591
PHP	__serialize	ps_srlzr_encode_php	UAF	php/php-src#16590
PHP	__serialize	zim_SplDoublyLinkedList_serialize	UAF	php/php-src#16589
PHP	__serialize	zim_SplObjectStorage_serialize	UAF	php/php-src#16588
PHP	__destruct	zim_SplObjectStorage_setInfo	UAF	php/php-src#16479
PHP	__destruct	spl_fixedarray_object_unset_dimension	UAF	php/php-src#16478
PHP	__destruct	zim_SplDoublyLinkedList_offsetSet	UAF	php/php-src#16464
PHP	__toString	spl_ptr_heap_destroy	UAF	php/php-src#16337
Ruby	hash	rb_ary_difference_multi	OOB	Ruby #21303
Ruby	hash	rb_ary_hash_values	UAF	Ruby #21304
Ruby	hash	set_merge_enum_into	UAF	Ruby #21305
Ruby	block	set_i_initialize	UAF	Ruby #21306
Ruby	block	st_general_foreach	UAF	Ruby #21331
Ruby	eql?	set_general_foreach	UAF	Ruby #21332
Ruby	block	rb_st_update	UAF	Ruby #21333
CPython	__eq__	_odict_keys_equal	UAF	python/cpython#119004

License

This artifact is released under the MIT License.