forked from kozorizki/binaryen
9a174f0ce9
Previously the fuzzer constructed a new random valid wasm file from scratch. The new --initial-fuzz=FILENAME option makes it start from an existing wasm file, and then add random contents on top of that. It also randomly modifies the existing contents, for example tweaking a Const, replacing some nodes with other things of the same type, etc. It also has a chance to replace a drop with a logging (as some of our tests just drop a result, and we match the optimized output's wasm instead of the result; by logging, the fuzzer can check things). The goal is to find bugs by using existing hand-written testcases as a basis. This PR uses the test suite's testcases as initial fuzz contents. This can find issues as they often check for corner cases - they are designed to be "interesting", which random data may be less likely to find. This has found several bugs already, see recent fuzz fixes. I mentioned the first few on Twitter but past 4 I stopped counting... https://twitter.com/kripken/status/1314323318036602880 This required various changes to the fuzzer's generation to account for the fact that there can be existing functions and so forth before it starts to run, so it needs to avoid collisions and so forth.
36 lines
1.4 KiB
Python
36 lines
1.4 KiB
Python
import subprocess
|
|
from scripts.test import shared
|
|
from . import utils
|
|
|
|
|
|
class InitialFuzzTest(utils.BinaryenTestCase):
|
|
def test_empty_initial(self):
|
|
# generate fuzz from random data
|
|
data = self.input_path('random_data.txt')
|
|
a = shared.run_process(shared.WASM_OPT + ['-ttf', '--print', data],
|
|
stdout=subprocess.PIPE).stdout
|
|
|
|
# generate fuzz from random data with initial empty wasm
|
|
empty_wasm = self.input_path('empty.wasm')
|
|
b = shared.run_process(
|
|
shared.WASM_OPT + ['-ttf', '--print', data,
|
|
'--initial-fuzz=' + empty_wasm],
|
|
stdout=subprocess.PIPE).stdout
|
|
|
|
# an empty initial wasm causes no changes
|
|
self.assertEqual(a, b)
|
|
|
|
def test_small_initial(self):
|
|
data = self.input_path('random_data.txt')
|
|
hello_wat = self.input_path('hello_world.wat')
|
|
out = shared.run_process(shared.WASM_OPT + ['-ttf', '--print', data,
|
|
'--initial-fuzz=' + hello_wat],
|
|
stdout=subprocess.PIPE).stdout
|
|
|
|
# the function should be there (perhaps with modified contents - don't
|
|
# check that)
|
|
self.assertIn('(export "add" (func $add))', out)
|
|
|
|
# there should be other fuzz contents added as well
|
|
self.assertGreater(out.count('(export '), 1)
|