UI Test Kit

[ashleyj]

basic framework for UITestKit

[ntherning]

Interesting stuff! A few comments:

1. I think we should move the RX version bump to a separate PR. Merging that should be a no brainer.
2. Did you use the formatter.xml in the root of the project? I see that there's a switch block which has incorrect indentation in your changes. Also, the imports in TestClient uses wildcards which we don't do in other places in the codebase. Please fix up this formatting.
3. What is the use case for testkit? It's supposed to replace the surefire provider, right? Why is that a good thing? It's cool if the runner can be used with vanilla junit - we don't need any special IDE/gradle/maven support to run tests inside RoboVM. But how flexible is it? What if I want to run a test annotated with this runner in Oracle's JVM to compare with RoboVM? How do you run the same test in the simulator and on device?
4. System.exit() is dangerous. But it's only run inside the RoboVM compiled executable, right? The process being tested? We don't want the IDE JVM process to exit. :-)

Please split up the PR into separate PRs for RX bump, testkit, etc to make this easier for me to review and merge.

[ashleyj]

I'll close this PR and fix 1,2 and re-check 4 (you've got me worried).

3." It's cool if the runner can be used with vanilla junit - we don't need any special IDE/gradle/maven support to run tests inside RoboVM" -- that's exactly right. Anything that supports JUnit 4 runners will work - I've tested maven and IntelliJ and both have worked as expected.

"But how flexible is it? What if I want to run a test annotated with this runner in Oracle's JVM to compare with RoboVM? How do you run the same test in the simulator and on device?"
It's not that flexible yet. It's currently reading the robovm.xml file to decide sim or device. Implementing something like a TargetType() annotation will make this a lot easier -- I'll stick it on the todo list

[ashleyj]

Actually since this is marked as a milestone, i'll keep this PR open and fix the branch

[BlueRiverInteractive]

Yay, this will be very helpful! Especially for testing the new RoboPods!

[ntherning]

The RX version bump is already in a separate PR and I will merge that soon. I think we should split the runner and the UI testing into two different PRs. Maybe even create a separate repository or at least a separate top level Maven module for the UI testing framework. The UI testing is of course dependent on the cocoatouch bindings while the runner could be used regardless of cocoatouch being present, e.g. in Linux or an OS X app.

Having a junit runner is a neat idea! One concern I have though is that class initializers may cause issues. E.g. if the test class has a static final field that references something in the cocoatouch bindings. The static initializer will be run inside the Oracle JVM on the Mac which isn't going to work unless the caller of junit which creates the Class objects of the test classes uses Class.forName(name, false, loader) when loading the test classes. Second param has to be false. I don't think we can expect this to be the case for all IDEs/build tools we want to support.