Tour of the “Assistive Programs” Code, Part 1: The Controller

Today I’d like to give you the first part of a brief tour of the “code base” of my suite of “assistive programs” for players and Judges of Strategic Primer.

This tour is primarily aimed at anyone who might like to contribute code or otherwise help in the development of these programs. But players may find it interesting or useful. You can follow along in the repository on BitBucket.

The “root” of the project largely contains various configuration files useful in development, but irrelevant to the code itself. We’ll come back to that later. All the code is under the src directory.

I’ve done my best to take the lessons of my early career in the CS department at Calvin, so the project is designed along the lines of the “model-view-controller” paradigm, as best I understand it (which probably isn’t all that well). So we have model, view, and controller packages into which almost everything is divided. There’s also a util package, which I’ll cover in a moment, and two directories containing data rather than code: “images” contains what you’d expect from that name, and “tables” contains the data files for what I called the “Encounter Model Mark II” (see, e.g., this post for more details about that).

The util package contains a number of “utility” classes that could conceivably be used by any of the parts of the application. Some of them really belong in one part or another (ImageLoader and FileLoader are really controller classes, for example) but are placed here to avoid circular dependencies between packages, which some of my static analysis tools object to. But others are clearly “utility” classes that don’t belong just to the model, the view, or the controller: an array-backed Set implementation, a Warning class, and so on.


Of the rest, we’ll turn first to the controller package, as it will probably contain the most that is most likely to be of interest to players.

The exploration subpackage contains only one class of significance: TableLoader. It loads the “encounter tables” from the tables directory I mentioned above, and is only used nowadays by two “drivers”—one to convert maps from “version one” to “version two” and put the kinds of ground, forest, and whatnot into the map, and the other to generate the list of fixtures to put in a tile—that are no longer used in the current campaign but would be useful in setting up a new map for a different campaign or when a player finds another world, and by the other classes in this package, which test that the loader works properly and helps debug a set of tables.

There’s a lot more code under the map subpackage, itself divided up into several packages.


The most interesting of these is the drivers package, which contains almost every “driver” in the project. (A “driver” is a program that a user can run directly, rather than having to call it from his or her own code.)

At this point the main driver—and the only one that the Windows or Mac “native” application can run—is the class ViewerStart, which starts the map viewer (with the specified map if one is specified on the command line, or prompting the user with a “open file” dialog if not).

The AdvancementStart driver similarly starts the worker-advancement application I mentioned in my last development report, which may be useful for players to keep track of their workers until I develop a more suitable application.

The rest are probably less interesting for players. In my role as “Judge” of the current campaign, I routinely both of the drivers I just mentioned, but also the ExplorationCLI to move players’ units around the map and determine what they saw; the MapChecker to check that no syntax or format errors have crept into any of the map files I’m dealing with; the QueryCLI to run hunting, fishing, and food gathering, and also to get the basic information a player automatically knows about a tile his or her fortress is on; and the SubsetDriver to make sure that players’ maps are subsets of the main map (i.e. that changes I’ve made to the main map affecting them have propagated down, and changes I’ve made to their maps got made to the main map too). I also sometimes use the EchoDriver, which reads a map and writes it back out (mainly to test the reading and writing code, and make sure that the map file will be parsed properly), so that when I do make changes later only those changes will go into my version-control system.

In the past I have also used the ConverterDriver to reduce the resolution of the map (halving it after I had quadrupled it), DuplicateFixtureRemover to find and remove duplicate fixtures (multiple forests of the same kind, for example) on a single tile from a map, the MapUpdater to update players’ maps from the main map, and the GenerateTileContents driver I mentioned in passing above to generate the contents of tiles.

In my role as developer of this suite of assistive programs, I’ve used the DrawHelperComparator to test which drawing backend for the viewer is more efficient and the ReaderComparator to make sure that each new map-reader implementation produces the same results as the old and to test implementations’ performance.

I/O interfaces

I always keep two complete implementations of the map reading and writing code in the repository, so as to be able to use them to check each others’ correctness. (I’ll get to each of them in turn later.) But most of the code should neither know nor care about those implementation details; instead, it uses the interfaces in the iointerfaces subpackage, knowing only that the object it is dealing with is either a map reader, an “SP reader” (which can read any object, so long as the XML represents the kind of object the caller is expecting), or an “SP writer” (which can write any object that should be writable to my XML format).

The package also contains a “factory” class that my unit tests use to get instances of both the current and the “deprecated” (old) implementation.

Format Exceptions

When you’re dealing with user input—including files that should be in a specified format—it’s likely you’ll run into some problems. Java lets you hand those problems back up the chain until someone knows how to deal with them; this is called “throwing an exception.” In the formatexceptions subpackage, I have several custom exception classes representing various common problems with map files. Some of them are usually raised as warnings instead; I’ll talk about my Warning class when we get to the top-level “util” package later.

SPFormatException is the root of the hierarchy; all the methods declare only that they may throw an “SP format exception,” not what kind. But it’s not possible to create one of these directly, only one of the subclasses.

The first two kinds I’d like to talk about have to do with the “include” tag. Even though I’ve abandoned the idea of reading a map from, and writing it back to, multiple files (each tile being in its own file, for example) because files kept getting corrupted, I’ve kept the ability to “include” data into one file from another using the include tag, because this is (once I did the hard work once) easy enough to do, doesn’t add much complexity, and could be useful. However, an include tag might reference a file that’s not there or that doesn’t contain well-formed XML. In those cases, since the code can only throw an “SP format exception” (and not the exceptions that indicate a missing file or an XML format problem), I throw a MissingIncludeException or a BadIncludeException, respectively.

If the map file is a version this implementation of the reading code doesn’t know how to handle, it will throw a MapVersionException. And if it encounters an XML tag it doesn’t know how to handle (yet), the result is an UnsupportedTagException.

If the XML indicates an object should have a member it can’t have—a forest inside an ogre, for example—the reader throws an UnwantedChildException. Conversely, if an object should have a member but the XML doesn’t list one, this produces a MissingChildException.

The remaining kinds of exceptions have to do with “properties” or “parameters” on tags in the XML, like the name or ID number of a unit. If a parameter is required but missing, the code throws a MissingPropertyException. If a parameter is explicitly not wanted, an UnsupportedPropertyException is thrown (or at least warned about). And if which property we use has changed (for example, we recently standardized on kind for all sorts of things, but stone deposits used to use the stone property for the kind of stone, and so on), the old form continues to work, but a DeprecatedPropertyException is warned about.

ReaderNG and CompactXML

Now we come to the two map-reading frameworks. The first is called ReaderNG, following a somewhat common pattern of calling implementations of something well after the first “such-and-such: the next generation” or “yet another such-and-such.” It replaced SimpleXML, which itself supplanted the original (I think unnamed) implementation.

All my reader implementations, except the very first, have been based on Java’s STaX API. (The first iteration was based on the SAX API, and very quickly got impossibly complicated even though it only had to read a much simpler format than we have now.) So I don’t have to deal with reading lines from the disk; in every case, the framework hands me a stream of objects representing XML tags, which I can then transform into the object graph I want.

The first iteration, as I said, was built on SAX; my reader class kept track of the current state, and the framework called one of my methods every time it hit a tag. There were two problems with this: first, keeping track of the current state required vastly complicated and (for me) unnatural code, including idioms that my static-analysis tools screamed about; and second, it meant handing off control to code I knew very little about.

My second attempt, SimpleXML, my first attempt using STaX, made my life much easier. I increased the version number and, for the only time in the history of the project, didn’t bother keeping “feature parity” between the two readers, because SimpleXML made my life simple enough that I could start making maps contain more (and thus more complicated) information. SimpleXML worked by creating a “Node” to represent each XML tag, telling that “Node” object about all the properties the tag had, and adding the Nodes representing all the tag’s children as children of the Node—and then checking the “SP format,” and then producing the “model” objects the rest of the application cared about.

That worked fairly well, until maps started getting bigger, and all this started getting really, really slow, and (when I tried increasing the map’s resolution) even running out of memory, because at the last stage I had the Node tree and the “model object” tree representing the same data both in memory at once. Something needed to change.

The one thing shared between the original implementation and SimpleXML was that XML output was handled by the objects themselves: each model object had a “toXML” method that returned an XML representation of it. While this worked (sort of) in practice, it had significant problems.

The next attempt was ReaderNG. There was still one class in the XML I/O subsystem for every model class, but instead of creating a new Node for each XML tag, which then stayed around until the whole map was parsed, an instance of the relevant “reader” class was created or retrieved (from a cache), which had a method that simply produced the model object the tag represented. And for output, another method method was called and passed the model object, producing a lightweight “SP Intermediate Representation” object that then produced the XML as text. (That was because I was trying to read from and write to multiple files per map, as I mentioned above, but it also simplified the handling of indentation, a perennial problem with the previous implementation.)

To let me get rid of SimpleXML (and, more importantly, the toXML method specified in the model object interface) without losing the benefit of having two different implementations to check each other’s correctness against, I wrote the CompactXML framework to supplant ReaderNG. Like ReaderNG, there are “reader” classes with methods that produce the model objects; unlike ReaderNG, one “reader” class often handles several similar but distinct model object classes. And the “write” methods simply produce text, not an intermediate representation. This reduction in the number of classes needed is a significant efficiency gain in both disk space (though the whole application including images is less than five megabytes, or only a little more than twice that once I build the Javadocs, which isn’t large at all) and in memory usage (and probably speed) at runtime.


In the converters subpackage of the controller package, there are three classes that convert a map from one version to another.

The first, ZeroToOneConverter, converts a map in the original format (which only the old SAX-based reader knew how to read reliably) to “format version one.” This is the one converter that works on the stream of XML events, as if it were a map reader, rather than on the model objects. The major difference between “version zero” and “version one” is that “version zero” had at most one discoverable thing—what it called an “event”—per tile, while “version one” added “tile fixtures”; the converter has a list of tile fixtures equivalent to the various “event” numbers.

The second, OneToTwoConverter, does three things: it converts a map from “format version one” to “format version two,” it quadruples the resolution, and it does some populating of the map with “tile fixtures.” First, it converts the map: while “version one,” like the original map, had “mountain,” “temperate forest,” and “boreal forest” as available terrain types, “version two” doesn’t: instead, those are things that are on a tile of some type. Second, it quadruples the resolution, distributing the fixtures on a tile in the old map among the sixteen corresponding tiles on the new map in a supposedly deterministic way (so that, ideally, the players’ maps would be subsets of the new world map after conversion if they were subsets of the old world map before conversion—though that broke down a bit in places). And third, it added villages every so often, forests in some places near water, and some other fixtures so that, even before my quasi-manual pass through the map adding fixtures, the map would be less monotonous.

And the third, ResolutionDecreaseConverter, undid some of the previous converter’s work, cutting the resolution of the (by now way-too-large) map in half in each dimension. The main difficulty there was to make sure that rivers ended up sensibly. (And I’m still not sure that’s quite right, but I hope never to have to use this converter ever again!)

Miscellaneous and Utility Controller Classes

The misc subpackage of the controller package contains a variety of miscellaneous utility classes used by other “controller” classes.

There’s a MapReaderAdapter which “fits over” whichever map reader (and now writer) class is now current, so that everything in an application that has to read from or write to file can do so without having to be changed every time the XML I/O framework changes.

There’s a MapHelper class, which helps “drivers” other than the viewer—the “exploration CLI” and the advancement application, for example—interact with the map, by giving a list of the players in the map and of a player’s units, handling I/O, and so on. (There’s a TODO item in its header asking me to rename it, so its name will probably change at some point.) And there’s a test class testing that its methods (except those interacting with the disk or with the user) are correct.

There’s the IncludingIterator class, which handles the include tags for the map-reading frameworks. (This is why I haven’t bothered dropping support for that even though it’s now not possible to write such a tag anymore—all I have to do is wrap the stream of XML events into one of these before handing it to the framework.)

The IncludingIterator class needs the ComparableIterator class to keep track of the relationship between files and Iterators; I use a Pair class that requires the things that it’s a pair of to implement the Comparable interface, but an Iterator by default doesn’t meet that, so I had to write the (simple) ComparableIterator class.

Another complication of the include tag feature that is neatly encapsulated here is that, to make automated testing of it possible, an include tag with a “file name” beginning string: turns the remainder of the “file name” into XML that’s handed off to the parser instead of going off to the disk. That’s all handled by the FileOpener class.

There’s an IOHandler class, which handles file I/O for all the graphical applications (the viewer and the advancement application, so far); I pulled that code out of the viewer and moved it here to avoid circular dependencies between packages.

Most objects in the model have a supposedly-unique ID; the IDFactory class handles the task of generating new unused ones when the XML doesn’t specify the ID number for an object that should have one.

I had intended to finish the tour in this post, but just the controller has made it quite long, so we’ll stop here and start up again in a few weeks with a look at the model.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s