Moving to HaxeFlixel forums

I realized that my random blog may not be the best place to find readers interested in HaxeFlixel development, when there is now a DevLog section at the HaxeFlixel forums.  So further posts will be found at my new development log there unless they are unrelated to HaxeFlixel development.  There's one there already about a FlxScrollableArea / FlxSpriteGroup / offscreen content gotcha that might save you some time.  Enjoy!

Mash Robot testing with HaxeFlixel!

I know I had said to stay tuned for certain coverage adventures.  I had an unexpected idea that took me on a wild tangent, on which I still am.  But the tangent involves HaxeFlixel and testing and scrollbars and all sorts of things.

So today's post is about creating a button-mashing robot to use for automated random stress testing, inspired by a post I saw on StackExchange.

I thought it would be fairly simple to do this in HaxeFlixel, since I had already delved into the VCR code to fix an issue with mouse event playback (the pull request hasn't been merged just yet, but I haven't had cause to change it yet, whenever I've used the updated version.)  It was.  Almost.  Well, it still was in the end, but took some more delving and debugging until I realized what was happening.

My first approach was to use the same approach I had taken with the VCR, and just go down a level into OpenFL and send input events there.  For various reasons, this didn't have the effect I had intended, but I won't go into them here.

So my next course of action was to leverage the VCR playback code itself (beyond just my sometimes-OpenFL-calling workaround), to keep things more integrated with HaxeFlixel's update flow.  This seemed to work on the keyboard end of things, but not with the mouse.

Where I ran into trouble is that I wanted my masher robot to work a bit differently than the VCR setup in flixel, but I hadn't acknowledged this difference enough when analyzing why it wasn't working.  The difference was that with the normal VCR, either you're recording, or you're playing back, never both.  My masher robot, OTOH, I wanted to pretend to be playing back, while still recording so that if the robot managed to crash my game, I had a way of reproducing it later.

Once I had this figured out, I saw that thankfully, HaxeFlixel makes it easy to override the spot where it would normally (when recording) update the mouse position using data from OpenFL (from the DisplayObject class, in the form of the FlxG.game instance.)  All I had to do was make a simple new class derived from FlxMouse, override its update() with everything except the top portion that calls setGlobalScreenPositionUnsafe(), and then replace it in my Main.hx like so:

FlxG.inputs.replace(FlxG.mouse, new MashMouse(FlxG.mouse.cursorContainer));

With that, everything worked great.  I now have a robot that sometimes clicks entirely randomly, usually clicks a random FlxObject, and presses random buttons, all the while recording everything it does.

If there's interest, I could package and release this later, as I think it can be a useful testing tool.

Note for Perl programmers using Haxe on compiler error "Float has no field MyVariableName"

You're missing a dot...instead of "for (i in 0..blah)" you want "for (i in 0...blah)" (three dots).  Also note that the latter number is excluded from the loop, unlike in Perl.

Code coverage coming along for FlxScrollableArea...which has a new release!

In fact, technically if we're just talking about FlxScrollableArea.hx, it's complete.  Thus, scrollable-area 0.0.2-alpha is out!  Just in time for the release of HaxeFlixel 4.0.0, which it's already compatible with.  (In fact, it wasn't compatible with any previous version, because I needed the dev branch for certain fixes.)

Unit testing paid off already, because I discovered two bugs while writing the tests.

What's incomplete is coverage and further testing of the scrollbars themselves, in particular a simulation of dragability.

Code Coverage Result: 81.29%

---

Missing Code Coverage: gimmicky.FlxScrollbar [61.45%]

 

Because FlxScrollbar has to update FlxScrollableArea as it goes, I've left it loosely coupled, but this means that to test the former to full coverage, I will need to mock and stub the latter.  Stay tuned for adventures with Mockatoo!  (Yeah, because blogs are tuned into.  Don't forget to jump up and down and hold your arms at funny angles to improve your reception.)

Buddy likes magic too! BDD FTW!

I was able to get tests going with a similar test.hxml for Buddy:

-main TestMain
-cp tests
-cp source
-cp C:/HaxeToolkit/haxe/lib/flixel/git
-D flixel=3.3.12
-cp C:/HaxeToolkit/haxe/lib/openfl/3,6,0
-D openfl=3.6.0
-cp C:/HaxeToolkit/haxe/lib/lime/2,9,0
-D lime=2.9.0
-cp C:/HaxeToolkit/haxe/lib/buddy/0,18,1
-D buddy=0.18.1
-cp C:/HaxeToolkit/haxe/lib/promhx/1,0,21/src/main
-D promhx=1.0.21
-cp C:\HaxeToolkit\haxe\lib\openfl/3,6,0/extern
--times
-D reporter=buddy.reporting.TraceReporter
-D native-trace
-D openfl-next
-D tools=2.9.0
-D flash-use-stage
-D no-compilation
-D openfl-flash
-D fdb-ci
-D web
--macro flixel.system.macros.FlxDefines.run()
-swf-lib export/flash/obj/assets.swf
-swf-version 11.8
-swf export/test/flash/bin/buddyhftest.swf
-swf-header 640:480:60:000000
-cp export/flash/haxe
-debug

All that was then needed was the same magic line, just in the new() of the Tests class.  And also a bit of trace redirection.  As ciscoheat says, enjoy your new Buddy!

Working coverage and unit testing, even autocompletion!

The key to getting munit and mcover working was in the test.hxml file.  This worked for me in the end:

## All targets

-main TestMain
-lib munit
-lib hamcrest
-cp C:/HaxeToolkit/haxe/lib/flixel/git
-D flixel=3.3.12
-cp C:/HaxeToolkit/haxe/lib/openfl/3,6,0
-D openfl=3.6.0
-cp C:/HaxeToolkit/haxe/lib/lime/2,9,0
-D lime=2.9.0
-cp C:/HaxeToolkit/haxe/lib/hscript/2,0,5
-D hscript=2.0.5
-cp C:\HaxeToolkit\haxe\lib\openfl/3,6,0/extern
-cp source
-cp test
-cp export/flash/haxe
--macro flixel.system.macros.FlxDefines.run()
--macro mcover.MCover.coverage([''],['source'])

--each

## Flash 9+

-swf-version 11.8
-swf export/test/flash/coveragetest.swf

#--next

## CPP

#-D HXCPP_M64
#-cpp export/test/cpp

A few notes:

1. "--each" is great.  DRY: it applies to all subsequent targets.
2. Despite "--each" being great, we're not really deriving full benefit from it at the moment, because CPP is commented out.  That's because I don't seem to be able to test on a CPP target.
3. The magic line (see previous post) creating a FlxGame object in TestMain.hx is still required if you want to test anything that refers to FlxG, otherwise it won't exist. 
4. The "-cp" lines I took from building a project that uses my library, then looking at the debug.hxml file generated in projectroot/export/flash/haxe.  Thanks to ciscoheat for inspiration in that regard.

(You may notice that ciscoheat is the developer behind Buddy, rather than munit.  I was also trying to set up Buddy to do BDD rather than TDD here.  Since I seem to have them both working now, I've decided that munit/mcover is a better fit for my library testing, while Buddy is a better fit for testing my actual game.  Although, code coverage would be nice there too, so maybe I'll use both, or something.)

That was great, but then I found out my earlier example of making a unit test:

haxelib run munit ct characterAIAimsAndFires -for Character

...was broken in a couple ways:

1. Tests need to be named ending in "Test" (well, "Test.hx") otherwise they aren't picked up by "munit ct".  This isn't done automatically if you supply the wrong name on the command line like I did.
2. They also need to start with an uppercase letter ("munit ct" also doesn't do this automatically.)

So, we instead just need this:

haxelib run munit ct CharacterAIAimsAndFiresTest -for Character

...and then everything's happy.  Almost.  The compiler is still barfing in the output pane whenever I type.  Turns out that this is just due to how autocomplete works, at least within FlashDevelop, because it looks for a Project.xml file to figure out which libraries and other source files to autocomplete from.  All I needed to do was create a (much simpler than normal) Project.xml in my project root:

<?xml version="1.0" encoding="utf-8"?>
<project>
 <!-- This file exists solely so that autocompletion works on FlashDevelop.  It's not intended to actually build the project. -->
 <!--Minimum without FLX_NO_GAMEPAD: 11.8, without FLX_NO_NATIVE_CURSOR: 11.2 -->
 <set name="SWF_VERSION" value="11.8" />
 <classpath name="source" />
 <haxelib name="flixel"/>
 <haxelib name="mcover"/>
</project>

...and then autocomplete worked like a champ for "Assert." and anything HaxeFlixel-related.

Now for the icing on the cake, I went to Project->Properties->Build and changed the pre-build command line to:

"$(CompilerPath)/haxelib" run munit t -coverage

And now I can run the test suite and get code coverage output simply by pressing F5.

So, the next challenges are:

• cpp testing
• android testing
• does the magic FlxGame line work with Buddy too?
• being able to check individual pixels in a cross-platform way would be a very solid way of testing scrollbars, IMHO

I hope this was helpful to somebody.  :)

A bit of progress

That issue with IBitmapDrawable...well, I didn't narrow down the exact line that was causing it, but an updated test.hxml file definitely helps (ignoring the cpp target problem, which this doesn't address, hence that target is commented out for now):

## All targets

-main TestMain
-lib munit
-lib hamcrest
-cp C:/HaxeToolkit/haxe/lib/flixel/git
-D flixel=3.3.12
-cp C:/HaxeToolkit/haxe/lib/openfl/3,6,0
-D openfl=3.6.0
-cp C:/HaxeToolkit/haxe/lib/lime/2,9,0
-D lime=2.9.0
-cp C:/HaxeToolkit/haxe/lib/hscript/2,0,5
-D hscript=2.0.5
-cp C:\HaxeToolkit\haxe\lib\openfl/3,6,0/extern
-cp source
-D native-trace
-cp test
-D openfl-next
-D tools=2.9.0
-D flash-use-stage
-D no-compilation
-D openfl-flash
-D fdb
-D web
-D noflxg
-cp export/flash/haxe
--macro flixel.system.macros.FlxDefines.run()
--macro mcover.MCover.coverage([''],['source'])

--each

## Flash 9+

-swf-version 11.8
-swf export/test/coveragetest.swf

#--next

## CPP

#-D HXCPP_M64
#-cpp export/cpp_test

With that, it actually compiles and runs, and the example test passes, with "haxelib run munit t."  Great!  Now, how about an actual test?  To be continued...