Chasing the Bugs!

A lot of bugs have been squashed - but a lot of them have also been discovered...

A lot of bugs have been squashed – but a lot of them have also been discovered…

As mentioned last week, my primary aim is to get the system into a relatively bug-free state prior to adding new functionality. The graph above shows that I fixed quite a few this week, but I have also discovered quite a few too.

Although no closer to starting the column movement code, it is still a good position to be in as I now have a system that has 13 fewer issues in it. Alas, not all coding can be glorious and exciting new functionality!

Most of the work that took place this week occurred in the Armies module – the module that models all aspects of units and armies.

The first ‘issue’ was this one…

Issues orders to cavalry units…

Starting positions for the default cavalry units.

Starting positions for the default cavalry units.

Runs a 30 second turn and ends up with:

Errr.... What happened?

Errr…. What happened?

The units didn’t exactly move very far…

These kind of bugs can be quite hard to find as the system keeps track of a multitude of variables for each unit. These variables can be hard to follow because the system runs multi-threaded.

Luckily, one of the pieces of work that I also did this week was to update the formation classification system to bring it up to date with the current modelling.

The classification system puts a symbol on a unit to tell the user whether it is Open Order (OO), Order (O), Close Order (CO) or Dense (D). The idea being that a player can instantly see how tightly packed a formation is – something that would be impossible to tell just by looking at the unit.

This subsystem was very old and didn’t reflect what was really happening. However, this was rectified, resulting in units being properly classified.

That said, I should probably point out that the game does not use this classification – it knows a unit’s precise measurements – the classification is solely a player aid only.

In my case, the classification helped track the issue above…

Ah... That would be it...

Ah… That would be it…

When I zoomed in onto one of the cavalry units I saw that its density classification is ‘D’ or dense. That would explain why the units were having problems moving!

These units were created very quickly using default unit sizes, formations, frontages and densities. However, it would appear that these defaults don’t give you units that function well…

This defined my next task:

‘Update the defaults to generate units that function correctly when created!’

The result:

Starting positions with the new unit default sizes...

Starting positions with the new unit default sizes…

Note the defaults have made the cavalry units a lot larger. Ok, lets run a turn and see how they perform…

That's more like it - there was actually some movement!

That’s more like it – there was actually some movement!

Bingo! All sorted….

Well almost….

The problem with the new defaults is that they highlight just how large the unit size variations can be. Alas, the Army Editor’s display window just couldn’t cope with some of these new sizes:

Oh - that doesn't look right...

Oh – that doesn’t look right…

Damn! I’m missing a fair bit of the unit. Creating a Chariot unit results in the whole screen being filled by part of that unit! Not good at all.

Conversely, small units like a Roman Maniple became very small and difficult to see.

To fix this, I altered the code in that pane to automatically zoom the camera to properly show off the chosen unit. This code was not trivial as one has to work out apparent sizes in metres based on the camera distance.

Nevertheless it got fixed:

Much Better!

Much Better!

Perfectly zoomed!

Even small units are now zoomed correctly:

Even the tiny Maniple is now scaled appropriately!

Even the tiny Maniple is now scaled appropriately!

The eagle eyed amongst you will have noticed that the window also gets a new scale in the bottom left so that users can more easily determine the size of a unit.

Fixes to tooling might seem a waste of time, but these tools are used to create the game assets – they will also be distributed with the game to allow players to create their own maps, units, armies and scenarios too….

Going back to the loaded scenario, I noticed another potential issue:

Classification 'O' (Order) - Really? A Macedonian Phalangite?

Classification ‘O’ (Order) – Really? A Macedonian Phalangite? You are kidding?

I thought ‘Oh No – the new classification system is broken already.’

Based on general wargamer bias and expectation, the Macedonian unit seemed to have an incorrect classification. Such a unit would at least be close order, if not dense!

However, after doing my research, I have discovered that this is in fact correct!

This is the default phalanx formation at 16 deep. The Macedonians didn’t even give it a name, because it was the ‘standard’ formation. However, some texts do refer to this formation as the Open formation!!! Woot! My system seems to have classified correctly!

If the above unit closes down to the 8 deep Pyknosis formation, as used prior to contact, the result is:

Pyknosis - classified as CO (Close Order) - Breaths a sigh of relief!

Pyknosis – classified as CO (Close Order) – Breaths a sigh of relief!

CO!!! Perfect. Closing the formation down further to the 4 deep Synaspismos results in the system correctly classifying it as ‘D’ or dense.

Of course, when I did my first formation change, I noticed a bug where the visible model’s density classification was not being updated. But this was a trivial issue and easily fixed.

So what’s planned for next week?

Well, I now have two big issues I need to look at.

First up is the text on the units. These don’t seem to be created at the correct sizes anymore. The font is either too small on large units, or too large on small units. I suspect that this is due to Windows 10 DPI scaling – most machines seem to be set to around 150%.

Fixing this issue will be difficult, but it is one that will need addressing as the unit text is the system’s primary way of communicating unit information to the player.

The next issue is related to the potential ‘bug’ described at the top of this blog entry….

It seems that Ancient Armies has got to the point where there are many subsystems acting on each unit. These affect a multitude of hidden variables with results that sometimes seem counter-intuitive – like in our example above.

However, debugging these issues is difficult as one really needs to see these variables changing over time to fully appreciate what is happening and why. This is made all the harder by the game being multi-threaded.

Luckily, Ancient Armies already records everything, all thanks to Chronos that was fitted last year. So the information is available, I just need a way to display it.

So one of my next tasks will be to create a debug monitor application. One that I can run on a separate computer that will monitor the game running and display graphs and debug information for each unit.

This will be a non-trivial task that will take a fair bit of time to complete. At first, this might seem to be a frivolous thing to do, but it will save a lot of time in the long run. This is because with such a tool I will be able to see what’s happening inside the units in realtime so that I can assess whether what I’m seeing is intended behaviour or not.

Going forward, the system will only get more complex, so a monitoring tool like this one will become critical to speeding up development.

So that’s my tasks fixed for the next few weeks! Sorry column movement, I will visit you eventually! Honest!





Back on the Horse!

The new workplace!

The new workplace!

I’m now back coding and designing Ancient Armies!

To aid with the ongoing development I bought myself a new development machine – A Microsoft Surface Book. This replaces my now deceased Apple Mac Pro.

I chose this machine because I was very impressed with them at my workplace. Plus having the ability to draw directly on the screen is essential for visualising the many geometrical problems that I have to overcome.

The fact that the screen also detaches from the keyboard to form a tablet is a bonus! All in all a very flexible machine!

So what have I been up to so far?

Firstly I spent 2 days designing how I was going to implement column based movement – both on road and off road. I also took additional time to design how movement orders will work for indirect orders – ie the orders you send by herald as opposed to direct command.

This is now all documented and safely stashed away in my Confluence system.

During the code research for the design work, I noticed that a few classes had got a little too big for their boots – yes Game I’m looking at you! As a result I invested some time to refactoring these so as to maintain a high level of code cohesion.

Refactoring is a perfectly normal activity and generally happens as a system evolves and takes shape. You can ignore the need to refactor, but you do so at your peril! The result can be an unmaintainable mess of code.

Although the design work for column based movement is now done, I will not start this new functionality until I have cleared down the backlog of bugs and issues that are still outstanding.

One issue I wanted to sort out straight away, is that I didn’t like the way the system automatically let the user side-slip a formation left or right. So I have modified the system so that the player must now make a conscious decision to side slip. This is achieved by simply holding down the side-slip key (currently mapped to left-shift), whilst issuing the orders.

If the side-slip key is not held down, the system will prevent the unit from side slipping.

In terms of bugs, the first one I ran into was this one:

Where did the overlay go?

Loads Desert Map…. Where did the overlay go?

Believe it or not this is the desert test map, which should normally look something like this:

A unit side slipping to the right...

Desert overlay in place!

It turned out that my overall approach to map overlays was way too simplistic. It failed to adequately accommodate the movement of files from system to system.

This bug only got discovered because Windows 10 on my new development machine gave me a different username from my other machines.

On the plus side, the system did fail gracefully – it loaded the map, but sans overlay. It was close, but no cigar!

The new overlay system is now much more robust.

When overlays are added to a map they are now imported into the system as opposed to merely loaded. Of course this in itself is complex as you need to be able to differentiate between images that are new and those that are already imported. Plus the system also needs to deal with the possibility that two different overlays may well have the same filename…

However, don’t fear! This has now all been taken into account and the system has been thoroughly tested.

So what’s next?

As mentioned above, I want to clear down the backlog before starting on the column movement functionality. There is a fair bit of work in there and I can see it potentially occupying a fair bit of my time.

If the bugs I’m fixing are relatively interesting, I will put up blog posts up about them. If not, you will probably hear from me in a few weeks time when I will declare that the backlog is empty!




Spot the Difference!

Things have been a little quiet around here.

The following photos may provide a clue as to why….


The now…

Notice anything odd in the photo above?


Then check out the one below:


The long gone past…

If you haven’t got it yet, I’ll spill the beans.

Basically, my development machine has finally stopped working. It was a 2009 Apple Mac Pro. This is an expensive machine and one that’s completely uneconomical to repair.

I had a love-hate relationship with that machine. It suffered from many issues including overheating and a display that could best be described as temperamental. But I did enjoy the earlier editions of OSX, that is until Apple dropped the ball with the most recent releases.

It gave around 7 years of sterling service before finally succumbing to its fate.

In terms of Ancient Armies, no source code was lost as this is stored on a separate RAID NAS drive system. In retrospect this was probably one of the smartest moves I ever made. If I hadn’t moved the source code, we would be in a bit of a pickle right now.

The only real issue is that my JIRA, Confluence and Fisheye servers were located on the Apple Mac. These are the primary management tools that help keep track of the project and its documentation.

My original intent was to wait until I got a replacement machine as I didn’t want to develop on my games machine. However, I have now decided that the games machine will just have to do!

The hard part of getting JIRA, Confluence and Fisheye transferred from the old hard-drives to the games machine took a little imagination and a not inconsiderable amount of time. However, I can now declare that these systems are all safely running on the games machine with no loss of data. 😎

So where does this leave the project?

With Christmas coming up, I have decided to schedule a restart in early January. So have a Merry Christmas and a Happy New Year everyone. I’ll see you all, bright and bushy tailed in the New Year!




Slipping Away…

A unit side slipping to the right...

A unit side slipping to the right…

A five minute job it said…

Only five minutes!

Why not add side slip whilst fixing the other issues?

Come on it won’t take long….

What’s 5 minutes between you and me?

That ‘five minute’ job took around a day to code. That’s the last time I listen to my brain! 😛

But we now have the ability for units to side slip.

What is side slipping?

It’s the ability for a unit to move or charge forward whilst slipping the whole formation either left or right.

Ancient units used side slipping where it could replace wheeling as it didn’t take as long and didn’t affect cohesion as much. But unlike wheeling, side slips are limited by unit type, training, formation density and unit speed.

So not all units will be able to side slip very well.

Why did side slipping take so long to implement?

In the end, even the trivial case of simply side slipping forward dropped me in deeper than I thought! The green line is the route the unit would take if it followed the order arrow. However, following this line would result in the unit overshooting its endpoint. Instead, the red line is the route the unit must take to arrive at the shadow marked unit. This is because the arrow the player sees marks the front of the unit, but all unit's positions are calculated from their centres and not their fronts. Hence the additional, and rather unexpected calculations.... (Click for full sized image)

Even the trivial case of simply side slipping forward dropped me in deeper than I thought! The green line is the route the unit would take if it followed the order arrow. However, following this line would result in the unit overshooting its endpoint! Instead, the red line is the route the unit must take to arrive at the shadow marked unit. This is because the arrow the player sees marks the front of the unit, but all units’ positions are calculated from their centres and not their fronts. Hence the additional, and rather unexpected calculations…. (Click for full sized image)

The image above shows one reason why there were a few technical hiccups. Who would have thought that a side-slipping unit’s actual course is actually slightly different from the order arrow? Not me! 🙄

Of course (pardon the pun), that’s just a simple ‘trivial’ case.

Once you start adding lots of other orders, such as wheeling, the calculations get exceedingly hairy indeed!

Lots of testing was required!

Lots of testing was required!

But after a day of solid work it all seems to work across multiple orders and formation shapes. 😎

Although side slipping has accidentally stolen the limelight this week, the real bulk of the work was fixing the issues introduced from the formation changes that were added last week.

Still behind the curve. You can see where the new formations change functionality got introduced on the 19th.

Still behind the curve. You can see where the new formations change functionality got introduced on the 19th.

I have a policy of not starting any new functionality (side slipping excused), until I get the outstanding issues under control. As can be seen from the graph, I still have a little ways to go before I have a clean system ready for adding road travel functionality.

So what were the ‘outstanding’ issues that I had to fix?

Many of them came about from required functionality that I had simply not thought about for formation changes….

Units that change formation now block line of sight properly across their more complex shapes.

Such as the ability for units that are changing formation to block line of sight properly across their more complex shapes….

Units that are changing formation can now be selected over their whole complex shape. The system now also detects the unit when the mouse cursor is waved over any part of the complex shape.

Or maybe the ability to select and detect units that are changing formation over their whole complex shape.

There were other issues too. Such as removing the ability for a player to delete a change formation order if a unit is already in the middle of carrying it out.

All these smallish issues were issues that were not necessarily thought about up front, but are needed to make the change formation functionality usable.

In addition, as a bonus, I fixed the graphical anomalies on unformed units!

Regular readers will be thinking, ‘But you already fixed that?’ Well I thought I had, mainly because I couldn’t see the issue on my development machine.

Further investigation revealed that the corruption was down to the way I was dynamically calculating index buffers for the units. On AMD hardware (my dev machine) it worked perfectly, but on my games machine which has nVidia hardware it caused the occasional graphical issue.

I have since re-coded how the index buffers are calculated using a much safer algorithm and now, finally, everything is back and working again! Woot! 😀

I even managed to spend a little time on cleaning up the wheeling orders graphic…

On the left is the old wheeling symbology, on the right is the new. Can you spot the difference?

On the left is the old wheeling symbology, on the right is the new. Can you spot the difference?

The older symbology used to have a solid line that was drawn perpendicular to the unit that connected the pivot point to the endpoint of the unit. However, these days, the order graphic shows a shadow of the unit, so there is no need for this line to be displayed anymore!

That’s it for this week. For the next week I will carry on stabilising the system, and once it is ready I will start on the new road travel functionality.

Some people might be thinking that this is a slow way of going about software development, but in my view if you don’t stay on top of an application’s quality it will soon get on top of you!

Anyways, I will leave you with a short video demonstrating what unit side slipping is. Best viewed in HD.




All Formed Up!

Notice anything new? :)

Notice anything new?

The last week or so has kept me extremely busy with the implementation of formation changes in the actual game.

Formation modelling has always been part of Ancient Armies, but until now, the only way to get at this modelling was via the Army Editor. Not any more! 😛

I originally estimated 7 hours for the job, but in the end it took a tad over 26 hours – a lot of time and effort.

Part of the reason for this can be seen in the screenshot above. The orders symbology had to be updated to cope with the fact that a unit’s formation can change at any time. Any subsequent orders would need to take the last formation change into account!

In addition, I wanted the leading edge of a unit to remain in the same place regardless of which formation it transitioned to. This is a requirement to allow units to keep their place in line. Of course, this wasn’t entirely simple because different formation types have different centre of masses.

So what is modelled in an Ancient Armies formation?

Three characteristics are modelled:

  1. Formation: This defines the formation’s actual shape (Ancient Armies supports many historical shapes)
  2. Frontage: This defines the number of people in each row and column. Of course it’s a little more complex than this as certain shapes don’t have rows and columns 🙂
  3. Density: This defines the gap surrounding each person in the formation. Depth and width are both independently modelled.

These characteristics allow the user to model practically any historical formation!

Density menu filtering in action!

Density menu filtering in action!

The above screenshot shows a unit that has a formation shape of ‘Hipparch’, a frontage of ‘Standard’ and a density of ‘Standard’. As you can see, the upper menu is quite long….

I decided to add menu filtering to the system that would allow the menu options for formations to be intelligently pruned to keep them manageable. This can be seen in the screenshot above where the bottom menu has received pruning.

This pruning makes it much easier for the user to interact with formations and removes a number of what are essentially unnecessary choices.

This filtering also extends to frontages:

Frontage menu filtering in action!

Frontage menu filtering in action!

And it will even remove the entire ‘Change Formation’ entry if a unit only has one formation that it can be:

Formation menu filtering in action!

Formation menu filtering in action!

Formation changes take time to accomplish. These changes are defined per formation and frontage within the Army Editor. However, these timings are not fixed, they are merely a base timing.

In terms of gameplay the timings will always vary slightly, plus there are chances for units to take additional time for the formation change. This is based on the unit’s training and discipline.

Of course, changing the formation of a unit reduces its cohesion somewhat (again based on training and discipline). This reduction is such, that players should try to allow enough time for the formation change before enemy contact!

If any enemy contacts a unit in the middle of a formation change it could be game-over for that unit!

So how have I handled the transitions?

In an ideal world I’d precisely model the position of every man and show them moving to their allocated positions. But alas, this is not really practical due to the many different drills adopted by the various nationalities.

So in the end, I decided to take a functional approach. I wanted the graphics to be able to show a player when a unit was changing formation and I also wanted them to give that player a rough idea as to how far through the transition the unit is.

A time lapse composite showing a Macedonian Phalanx transitioning from Synaspismos to Open.

A time lapse composite showing a Macedonian Phalanx transitioning from Synaspismos to Open.

The above screenshot shows the compromise I arrived at. Here a Macedonian unit is expanding to open order. One can see how the graphics show the transitioning of the formation.

The individual dots don’t necessarily represent the positions of actual men, they are purely indicative of the formation change in general.

I’m rather excited by all this and the fidelity that Ancient Armies brings to the table.

For the first time that I know of, we have a simulation that explains why a Macedonian Phalanx had three formations and demonstrates to the player how these formations work.

Just by playing the game, you will soon get to understand why the Macedonian Phalanx operated 16 deep, then switched to the 8 deep Pyknosis formation prior to contact. If you don’t do this, you might run into the same difficulties that the real life commanders faced!

Of course, I have just singled out the Macedonian Phalanx here, but the formation modelling is so detailed that it applies equally well to other formation types.

The simulation has now got to the point where it is providing me with genuine insight into the workings of Ancient formations and is also enabling me to probe the authenticity of many Ancient historical texts!

I’m not aware of any other game system that has this level of fidelity – hence the excitement! 😎

To show off the formations change system and to demonstrate the subtleties of operating a  Macedonian Phalanx, I have put together the following video – enjoy (Best viewed in HD):

As with any large update there are a few more additional bugs that I will need to sweep up. My intent is to switch back to maintenance mode and address these as a priority.

Once this is done I will start working on the next subsystem for which architecturally, the formation change system has provided me with a bit of a leg-up…

So be ready for road and track movement – Ancient Armies style!



Bug Hunt!

This week has been one of fixing bugs and generally stabilising the system so as to put me in a good position for the next phase of development.

Getting on top of things!

Getting on top of things! (Click for larger image)

The above graph shows raised issues (red) vs fixed issues (green). Over the last few weeks a lot of testing has taken place resulting in many red issues being raised. The volume of these issues was such, that I was finding them faster than I was fixing them!

However, over the last week or so, the tables have turned, with my fix rate exceeding the detection rate. I’m now in a happy position where there are only 2 minor issues left and both of these are enhancements rather than genuine issues. This means that I am now in a good solid position for the next phase of development!

Although many things got fixed over the last week, I’ll just go over the two main issues that were addressed:

Chronos replays were too slow!

This was an interesting bug and one that can be seen in the Chronos video.

The original replay functionality would wait for the time of one frame or larger to pass and then display the next frame. This is all well and good, but if the machine is doing other stuff, the next frame request could be several frames in the future. However, because of the way the system was coded, it would still retrieve the next frame in sequence – thus resulting in a slow playback.

To fix this, I simply changed the system to always retrieve the frame for the currently calculated time, regardless of how far in the future. This ensures, that even under heavy system load, the correct frame is always displayed for the elapsed time!

Irregular formation graphical bugs!

This issue has been a thorn in my side for a while. It was intermittent in nature, thus making it very difficult to track down. An obvious case of this bug can be seen in the Chronos video, where one can see the malformed irregular units.

Whilst fixing the code for this, I discovered that many mathematical ‘bodges’ had been put into the irregular unit calculation code. I decided to remove these, as I was fairly certain that it was these ‘bodges’ that were causing the graphical anomalies.

Once removed, I spent a bit of time trying to figure out why they were added in the first place. The result was rather surprising…

It seems that Ancient Armies was calculating unit widths incorrectly!!!! 😯

Before I go into the how and why, here is a question for you:

If I have a row of 5 men, each with 1 metre of frontage, how wide would the formation be?

Got the answer yet?

If you guessed 5 mtrs, you guessed wrong! 😛

The correct answer is that the resultant formation should be 4 metres wide.

However, Ancient Armies was originally and incorrectly calculating 5 metres of width. This meant that the code for generating the irregular unit shapes had many bodges in it to try and mitigate the incorrectly calculated widths.

Once I knew that this was the core issue, it was relatively simple to fix and meant that the code for calculating the irregular unit shapes could now work with no bodges in it! 😎

For those that are still unconvinced, lets consider our hypothetical man:

Consider a man with a defined formation with setting of 1 metre...

Our example man!

As you can see he has a frontage of 1 metre centred around himself.

Now if I pop him into a 5 man formation, this is what I get:

A unit of five of the above men (Click for larger image)

A unit of five of the above hypothetical men (Click for larger image)

As you can see from the above diagram, the correct unit width is 4 metres!

This is because Ancient Armies unit markers represent where the men actually are, it does not represent their raw frontages. Frontages are only used to calculate the spacing between each individual man. For the men on the ends, this spacing is irrelevant – hence the 4 metres width! 🙂

That’s it for this week. Next week should be more interesting as I can now start work on formation changes…



Zed Said ‘No!’….

Whilst putting the new contour system through its paces I noticed a particularly annoying bug:

See the bug? (click for a larger image)

See the bug? (click for a larger image)

If you look carefully you can see that the contour is drawn over the unit and not under it as expected.

Fixing this bug resulted in a major re-write of the contour system.


Basically, Direct-X has an issue where it treats lines and polygons quite differently with regard to Z calculations (depth). Especially if one is using any kind of bias.

Of course, I didn’t realise this straight away and spent a number of hours banging my head against the wall trying to work out what was going on! After all, my contour lines Z values were calculated to draw them under the units and not over them.

Once I made this discovery I realised that the fix for this issue would be quite major:

The fix was changing from drawing lines to drawing triangles... (click for a larger image)

The fix was changing from drawing lines to drawing triangles… (click for a larger image)

This might seem like a simple change, but calculating the triangles’ corner points to fit the requested bezier curve is not an easy problem. Luckily, I had already solved this issue with my linear terrain features, so I could reuse many of the algorithms from that section of code.

With most of the contour system re-written to work with triangles rather than lines, I then set about fixing other issues…

First up was contour clipping. When one draws a hill contour it should be clipped so that it fits the map. Alas, I had forgotten to test this aspect. So I was a little surprised when I saw this:

Whoops! That's not supposed to happen! (click for a larger image)

Whoops! That’s not supposed to happen! (click for a larger image)

The hill was being clipped, but not the contour! Had I not re-written the system, I would not have been wise to this issue. Anyways, it is now fixed:

Contour lines are now clipped to the map boundary. (click for a larger image)

Contour lines are now clipped to the map boundary. (click for a larger image)

Whilst in the contour system, I decided to update the code that deals with the elevation numbers to add a small enhancement:

Large contour elevations now have commas put in them for that more 'authentic map look'. (click for a larger image)

Large contour elevations now have commas put in them for that more ‘authentic map look’. (click for a larger image)

Although the re-write sucked up a large amount of time, the one positive thing to come from it, is that we can now set the thickness of our contours! From very thin to very thick…

On the plus side, I can now manipulate contour thickness.... (click for a larger image)

On the plus side, I can now manipulate contour thickness…. (click for a larger image)

I’m not sure why anyone would want contours this thick, but hey, the system can now do them! 🙂

Back in the main game the new contours perform flawlessly:

Contours - now in their 'All-Fixed' Edition! (click for a larger image)

Contours – now in their ‘All-Fixed’ Edition! (click for a larger image)

Notice how the units now appear over the contours? – Perfect! Just wish that it didn’t require a re-write to fix…

That’s it for this week’s unexpected tour of the contour system.