Create JUnit test cases to ensure transformers perform as expected

[GenericException]

You can probably prevent a lot of future issue reports by creating some test-cases and checking them regularly. Perhaps a format like this (pseudo-code) would work:

void stringsLight() {
    // create radon instance
    CLI cli = // ...
    cli.addTransformer(new StringsLight());
    cli.obfuscate("test-cases.jar", "test-cases-obf.jar");
    // run the obfuscated jar
    Process p = Runtime.exec("java -cp test-cases-obf.jar sample.SomeClass");
    Stream s = p.getOutputStream();
    String s = toString(s);
    // clean-up
    Files.delete("test-cases.obf");
    // print stack-trace if it exists
    if (s.contains("exception")) {
         print(s); 
    }
    // validate results with JUnit
    assertEquals(s, "Hello world, this is the expected output of System.out calls"));
}

In this single test case, the CLI obfuscated some sample jar file and then executes the output. If everything works well the output should be match the expected value. Otherwise the output should be a good'ol stacktrace. You can easily abstract away all of this and have each case method only specify the transformer settings & the expected output.

But what would test-cases.jar contain? It should be something that involves a little-bit of everything as long as it is all console-based and does not require user input. I made a jar that has a bunch of cases that can be tested this way (with the exception of one). I tested Radon 1.0.3 on it and went through each obfuscation feature one-by-one and I everything functioned as intended. This would just automate that process.

[Storyyeller]

I did this kind of testing for the Krakatau decompiler. A couple things to watch out for: JVMs may change behavior in different versions. I had a couple tests that stopped working after a JVM upgrade, so I had to disable them.

Another thing to watch out for is infinite loops. It is possible for a transformer to accidently turn your code into an infinite loop, so you need to add a timeout when executing the jvm. Lastly, it is useful to add caching, since invoking the jvm takes so long. For the Krakatau tests, it will cache jvm executions, so if the bytecode output doesn't change, it won't rerun it.