Character Averages and Enemy Samples through Ensembles

[Balcerzak]

So I was reading through the "Theory of Debating" thread, when the following portion of one of the posts, in particular, really caught my attention.

When looking at averages, you may discover that they are often pretty simple to calculate: simply take a unit's base stat, multiply their decimal growth by the amount of levels they gained, and voilá. For example, Lyn's Strength average at L7 is 4 + 0.4 * 6 + 6.4. However, when stat caps come into play, this formula is flawed. Lyn's Spd at her very last levels appears to be growing extremely slowly (28.5 to 28.9 is only 0.4, while her growth is 0.6). This is because when calculating and giving weight to possibilities of a unit's stats, the game gives equal weight to a stat that turns out above average to one that turns out below average, also called RNG blessed versus RNG screwed. However, at this point, Lyn's Spd is very likely to be already capped at 30, so RNG blessing makes almost no difference. That leaves nothing to outweigh the Spd stat turning out below average.

Now, I'd already had a pretty clumsy spreadsheet implementation of naive averaging in place (with rudimentary protections against stat-capping), but it was by no means elegant or robust. I've always prided myself on being able to implement, understand, and do things myself, rather than simply relying on the work of others (for the most part, that is, there are times when even I recognize the futility of re-inventing the wheel). So I sort of took this as a bit of a personal challenge to figure out how best to go about getting a better solution than what I already had.

I'm sure there's probably some rather simple algorithmic approach to generate a formula, but I didn't spend a lot of time considering that, as it seemed to be more work than the other method I had in mind, ensemble testing. Now, I'm a frequentist at heart, so buy into ensemble testing. I believe the distributions obtained from large numbers of simulated distributions will tend to approach the true distributions. So it was only a matter of figure out the best way to code together the necessary pieces.

Since I work mostly in C++, I simply stuck with that, and started designing a few classes. When I was happy with what I had I did some test runs.

Generating 1000000 Eliwood characters, levelling them to 20, promoting them, and levelling them further to 20.

10%-20%-30%-40%-50%-60%-70%-80%-90%-100%-done

A random example of such a character would be:

Current Status of Unit named Eliwood:

Unit is Class: Knight Lord

Unit is Level: 20

Unit was promoted at Level: 20

Unit is mounted

Unit is male

Unit is Affinity: Anima

Unit has HP: 52

Unit has Power (S/M): 22

Unit has Skill: 25

Unit has Speed: 20

Unit has Luck: 25

Unit has Defense: 14

Unit has Resistance: 13

Unit has Constitution: 9

Unit has Aid: 16

Unit has Move: 7

Generating and levelling 1M characters took about 25 minutes, which is not particularly feasible for comparing many characters, or many level points. I then tried 10k, which took under a minute, and the accuracy error from the smaller sample size should still be negligible (as I understand it for 10k the uncertainty is at 1%, rather than the .1% for 1M).

Generating 10000 Lyn characters, levelling them to 20, promoting them, and levelling them further to 20.

10%-20%-30%-40%-50%-60%-70%-80%-90%-100%-done

A random example of such a character would be:

Current Status of Unit named Lyn:

Unit is Class: Blade Lord

Unit is Level: 20

Unit was promoted at Level: 20

Unit is female

Unit is Affinity: Wind

Unit has HP: 51

Unit has Power (S/M): 22

Unit has Skill: 29

Unit has Speed: 30

Unit has Luck: 22

Unit has Defense: 15

Unit has Resistance: 20

Unit has Constitution: 6

Unit has Aid: 5

Unit has Move: 6

For perhaps the most interesting part of the topic, some visual histograms to allow quick and simple determination at a glance. (Explanation each bin of the histogram is a different value of the statistic in question, and its height is equal to the number of generated characters with that value of that stat.) Number of entries, as well as mean and RMS are provided in a small stats box. For the record, I'm curious whether Mean and RMS are the best choices really, rather than perhaps picking the median of the distribution to be the central value, and then constructing positive and negative errors so as to generate a central coverage area of 68%. I plan on pursuing this a bit further in the future if I remember to do so. (Consider the meaning of 29.4 ±1.2 when the stat cap is 30. Clearly the upwards error is nonsensical; asymmetric errors are necessary.) Images are thumbnailed for your convenience. Click to enlarge (and actually manage to read the statistics boxes). So far all results have been in good agreement with the numbers listed at feawiki, which is quite promising. I did notice that feawiki only includes uncertainties on FE7 stats, and not any other, so by deciding to get off my ass and actually try something out, I've already made some progress beyond what's publicly available (as soon as I extend my code to cover FE6).

Witness 20/20 Eliwood's Skill, and Resistance stats (Resistance stat will show a distribution unaffected by cap ramming)

And 20/20 Lyn's Power and Speed (for vivid illustration of cap ramming)

One issue I came across while doing the basic building and testing in this current phase was that I honestly don't know what the promotion bonus for the transporter class is, so I fudged it. All I know is that Merlinus gains move, but I'm pretty sure he gets more than that as well. I couldn't find it listed in the SF database though. Does anybody know?

As the method is quite general, it could easily be extended to the other GBA FEs with little to no modification needed. A little more work and I could probably do similar things for other FEs that don't share the same basic vital stats, or their relations. (e.g. FE9 has no Con, has Strength and Magic, so definite alterations would be necessary).

That's all I really have for now, but figured I'd throw it out there, and if there were any comments, suggestions, discussion, etc. let people have a chance to do so. I'm sure I forgot to include something, but I've been constructing the topic for long enough, and if I remember, I can always include it later as an update.

Edit: Old code examples removed, as things have evolved significantly since then.

Edited June 14, 2009 by Balcerzak

[VincentASM]

Transporter doesn't gain any stats besides Move.

[Mekkah]

Interesting approach. I've been trying to program an fe-wiki like averager for a while now, but I can't just piece it together. This is a cool way to do it...I didn't like the idea of just using samples first, but with a huge ass number as 10k the margin of error is pretty slow.

I'd be interested in running this so I can use it offline. If I can boldly make a suggestion, maybe make room for using stat boosters at some point? Or even more complicated, FE10 BEXP? Come to think of it, FE10 also has that "must have 1 stat minimum in level ups" thing that could be a challenge to put in.

And of course, it would be totally marvelous if you could create such a sample maker for generic enemies.

[Lord Raven]

I did all of FE10's averages through the FEA style by hand on excel. I'll email the zip to anyone who wants to convert the excel files to HTML. : D

[Balcerzak]

Time warp killed my previous updates, but whatever, here's a new one for you all.

Okay, so I was trying to use my freehostia account to create the webpages necessary to display the progress and results, but quite frankly their user-interface sucks balls. Their web-based file manager is crap, and their ftp control is little better. After wrestling for hours trying to get the files moved into nice neat subdirectories, I'm giving up. The base directory where (some) FE6 enemy samples are stored is here. At the moment there are no web-pages, per say, only hordes and hordes of .pngs to look at. I plan of fixing that eventually, but I hope to be able to write a slick python script to make these webpages for me like I did for the PC averages.

Speaking of the PC averages, I gave up on freehostia, and am simply using my UM account to take care of the site. I really had hoped it didn't come to this, but it was so much easier to just tar up the file structure from my laptop, ssh it across, untar it, and voila suddenly and instantly a working webpage! Please, feel free to take a gander at it here. It is understandably bare bones. All I care about is hosting the results. Prettifying them is not a priority for me.

Yes, the enemies samles link is broken, for the aforementioned reasons. After I can get the webpages to write themselves, I'll ssh the htmls and the pngs across and have something functional and informative. IF, however, you stumble across broken links, images, or something that is just plain wrong in the working half of the site, though... Please, please, let me know.

There is progress people, believe it or not.

Yes, it is slow. Such is life.

Edit: Found the first mistake. I fucked up and had the FE7 flag toggled for my HM bonus calculator, so for all characters with HM bonuses their displayed luck is lower than it ought to be. This will be fixed. Eventually.

Edited June 14, 2009 by Balcerzak

[Jaybee]

Now, I'm confused. Should we use the FEPlanet averages or the SF averages?

[Balcerzak]

I didn't know FEPlanet had averages. I went and looked and don't see any here, but I guess that doesn't really matter. I know FEAwiki has some, and they are actually proper averages, so to within statistical error they should agree perfectly. However, they don't seem to include HM bonuses, unless I just don't know where to find them, and perhaps more importantly they only list the mean value and don't display any information about the spread, as far as I can tell. Clicking the "details" to see the actual histograms documenting the spread of generated values for the statistics is a very straightforward, and visual way to get a good idea about this, and the ± is a decent rough quantitative way.

I mean, I guess ultimately its up to any individual debater as to which averages they choose to use, and as long as they're choosing something that properly implements the stat cap effects through proper averaging, rather than simple averaging, I'm fine with it. Originally this was more just an exercise for myself, but then it grew when the seed for the idea of enemy sample generation was posed, and I figured that I might as well publish the PCs information as well as the enemy information, just for the sake of completeness, and in case anybody else would possibly find it useful, especially as some level of interest had been expressed.

Edited June 15, 2009 by Balcerzak

[VincentASM]

FEPlanet has averages for some of the games. I know they have them for FE11, possibly FE10. Not sure about the others.

Personally I'm not really bothered about which averages to use. Of course, if they're available, the "real" averages that FEPlanet and FE Averages are probably more preferable.

However, are the differences really that great (I'm not talking about the radical behaviour nearing stat caps, but in general)? At the end of the day, both versions of the averages are just statistical approximations of some real phenomenon.

That's just my thoughts in general anyway. It's nice to know you're still working on this ^^

[Jaybee]

Yeah, I meant FE10. Sorry if I was unclear.