Fates RN Regression Project

[Crimean Archivist]

I haven't been, but I can start. I can also say that the overwhelming majority of my trials at 75 Hit have been player-side primary strikes (not Dual Strikes), if we want to try separating testing for first strikes and Dual Strikes.

Do keep in mind however that enemy hit rates tend to be lower, so the two data sets are probably already going to be somewhat distinct from each other.

Edited February 29, 2016 by BlenD

[Dark Holy Elf]

I've been toying with the math that might lead to turn 75 displyed hit into 82-83 true hit.

FE6-12 use the average of 2 RNs, in other words:

Let A and B each be random numbers from 0 to 99
If (A + B) / 2 < displayed Hit, attack hits

This results in 75 becoming roughly 87.5%... actually 87.75%, because we rounded down. (If A and B were actually random real numbers chosen continuously between 0 and 100, then the more mathematically pleasant 87.5% would be the true probability.) I'm sure a number of people reading this post already know this, but restating it for those who don't.


For Fates, one possibility that jumps out at me right now is the following:

Let A and B each be random numbers from 0 to 99
If (3*A + B) / 4 < displayed Hit, attack hits

What does this do? Well, with 75 displayed Hit, we will see the attack hit if:
A = 99 and B = 0
A = 66 and B = 99
Or in general, for any other points which lie below the line from (99, 0) to (66, 99) on the A-B plane (roughly). This results in an 83.3333% chance of true hit if A and B are continuous variables, which they aren't.

The exact value using discrete variables varies depending on rounding and how the values are chosen, but if we just assume the exact method posited above and rounding down, it's 83.83%. This is slightly higher than the mean of the tested data so far, but well within the last posted confidence interval. Of course, it's possible that some rounding is used besides just rounding down, which would change the numbers slightly... or that the random numbers chosen aren't from 0 to 99, but from 0 to 65535 or something else of that ilk. It doesn't matter too much, there's probably no reasonable amount of testing which can determine the subtle effects of the specifics here. (For that matter, do we even know the rounding specifics of the GBA games via looking at the code, or is at all just from testing?)

One thing that I find attractive about this formula is that Awakening/Fates' programmers have already shown an affinity for using (3*A+B) in their formulas... i.e. the calculations for Hit and Avo. It's also a logical extension of the formula used previously, simply taking the 2RN system and weighting one of the random numbers more strongly in order to create a "compromise" between 1RN and 2RN.

It could very obviously easily be wrong. One other thing I find attractive about the formula is that it's "fair", having a roughly symmetrical effect on 25 Hit as it does on 75 Hit. But there's already suggestion in the data that this might not be the case. If it isn't, we are indeed looking at something completely different. We won't know until we test some low-Hit values extensively.

Edited February 29, 2016 by Dark Holy Elf

[XeKr]

All the numbers I’ve provided are player-side primary attacks as well.

I was about to post something similar to Dark Holy Elf, some kind of weighted averaging. Re: the gba games, I think there are lua scripts that display the RNG, so it should be known. Not sure about 9-12.

I’ll poke around in the fixed stat dlc for any useful hit rates. Swords against Axes like in Anna's level with the many berserkers might work for fast collection.

Edited February 29, 2016 by XeKr

[Crimean Archivist]

One thing that I find attractive about this formula is that Awakening/Fates' programmers have already shown an affinity for using (3*A+B) in their formulas... i.e. the calculations for Hit and Avo. It's also a logical extension of the formula used previously, simply taking the 2RN system and weighting one of the random numbers more strongly in order to create a "compromise" between 1RN and 2RN.

It could very obviously easily be wrong. One other thing I find attractive about the formula is that it's "fair", having a roughly symmetrical effect on 25 Hit as it does on 75 Hit. But there's already suggestion in the data that this might not be the case. If it isn't, we are indeed looking at something completely different. We won't know until we test some low-Hit values extensively.

This is an attractive possibility. There are still 10000 combinations of A and B, but the equation is shifted considerably:

@ Hit = 1:

(0,0), (0,1), (0,2), (0,3), (1,0) --> 5 combinations, net rate of 0.05% as opposed to 0.03%

@ Hit = 2:

(0,0), (0,1), (0,2), (0,3), (0,4), (0,5), (0,6), (0,7), (1,0), (1,1), (1,2), (1,3), (1,4), (2,0), (2,1) --> 15 combinations, net rate of 0.15% as opposed to 0.1%

I tried to come up with a summation but my model was a little off. I'll keep working on it.

Edited February 29, 2016 by BlenD

[XeKr]

I just brute force calculated all the values (floor((3A+B)/4) < disphit) with a meshgrid. Math check later. edit: numbers seem okay, 2nd column matches with true hit chart on SF, and I got the same as Dark Holy Elf for at 75% disp. Obviously need to note averaging where applicable.

         A   (A+B)/2  (3A+B)/4
         0         0         0
    0.0100    0.0003    0.0005
    0.0200    0.0010    0.0015
    0.0300    0.0021    0.0030
    0.0400    0.0036    0.0051
    0.0500    0.0055    0.0077
    0.0600    0.0078    0.0108
    0.0700    0.0105    0.0145
    0.0800    0.0136    0.0187
    0.0900    0.0171    0.0234
    0.1000    0.0210    0.0287
    0.1100    0.0253    0.0345
    0.1200    0.0300    0.0408
    0.1300    0.0351    0.0477
    0.1400    0.0406    0.0551
    0.1500    0.0465    0.0630
    0.1600    0.0528    0.0715
    0.1700    0.0595    0.0805
    0.1800    0.0666    0.0900
    0.1900    0.0741    0.1001
    0.2000    0.0820    0.1107
    0.2100    0.0903    0.1218
    0.2200    0.0990    0.1335
    0.2300    0.1081    0.1457
    0.2400    0.1176    0.1584
    0.2500    0.1275    0.1717
    0.2600    0.1378    0.1850
    0.2700    0.1485    0.1983
    0.2800    0.1596    0.2117
    0.2900    0.1711    0.2250
    0.3000    0.1830    0.2383
    0.3100    0.1953    0.2517
    0.3200    0.2080    0.2650
    0.3300    0.2211    0.2783
    0.3400    0.2346    0.2917
    0.3500    0.2485    0.3050
    0.3600    0.2628    0.3183
    0.3700    0.2775    0.3317
    0.3800    0.2926    0.3450
    0.3900    0.3081    0.3583
    0.4000    0.3240    0.3717
    0.4100    0.3403    0.3850
    0.4200    0.3570    0.3983
    0.4300    0.3741    0.4117
    0.4400    0.3916    0.4250
    0.4500    0.4095    0.4383
    0.4600    0.4278    0.4517
    0.4700    0.4465    0.4650
    0.4800    0.4656    0.4783
    0.4900    0.4851    0.4917
    0.5000    0.5050    0.5050
    0.5100    0.5247    0.5183
    0.5200    0.5440    0.5317
    0.5300    0.5629    0.5450
    0.5400    0.5814    0.5583
    0.5500    0.5995    0.5717
    0.5600    0.6172    0.5850
    0.5700    0.6345    0.5983
    0.5800    0.6514    0.6117
    0.5900    0.6679    0.6250
    0.6000    0.6840    0.6383
    0.6100    0.6997    0.6517
    0.6200    0.7150    0.6650
    0.6300    0.7299    0.6783
    0.6400    0.7444    0.6917
    0.6500    0.7585    0.7050
    0.6600    0.7722    0.7183
    0.6700    0.7855    0.7317
    0.6800    0.7984    0.7450
    0.6900    0.8109    0.7583
    0.7000    0.8230    0.7717
    0.7100    0.8347    0.7850
    0.7200    0.8460    0.7983
    0.7300    0.8569    0.8117
    0.7400    0.8674    0.8250
    0.7500    0.8775    0.8383
    0.7600    0.8872    0.8512
    0.7700    0.8965    0.8635
    0.7800    0.9054    0.8753
    0.7900    0.9139    0.8866
    0.8000    0.9220    0.8973
    0.8100    0.9297    0.9075
    0.8200    0.9370    0.9172
    0.8300    0.9439    0.9263
    0.8400    0.9504    0.9349
    0.8500    0.9565    0.9430
    0.8600    0.9622    0.9505
    0.8700    0.9675    0.9575
    0.8800    0.9724    0.9640
    0.8900    0.9769    0.9699
    0.9000    0.9810    0.9753
    0.9100    0.9847    0.9802
    0.9200    0.9880    0.9845
    0.9300    0.9909    0.9883
    0.9400    0.9934    0.9916
    0.9500    0.9955    0.9943
    0.9600    0.9972    0.9965
    0.9700    0.9985    0.9982
    0.9800    0.9994    0.9993
    0.9900    0.9999    0.9999
    1.0000    1.0000    1.0000

Edited February 29, 2016 by XeKr

[Kanethedragon]

lol still a high school student here but damn, all this math is truly addling my brain XD. Anyways, I'm not as great with numbers as many of the others here are, so I'll just be cheering you on from the sidelines til you crack the formula so that we all know how to use the RNG in our favour for this title. Only real information I can provide is that I've been noticing that all my level ups have been practically exactly the same regardless of restarting chapters or going to different ones. But I don't think the RNG for levels aren't necessarily set in stone...at least, up until the route split as my Hoshido playthrough stems from the same initial save and has a slightly different stat distribution and levels for my MU.

Nvm, it appears that they are about the same. Depends on if I nab 2 luck for the next two levels in my Hoshido playthrough as I did use a goddess icon on myself. It'd be kinda funny though to get luck on consecutive levels for my MU despite it being my flaw -_-

Edited February 29, 2016 by Kanethedragon

[Crimean Archivist]

Looks promising. Importantly, it aligns (+/- 1 sigma) with the values we have the most points for, which are currently 75, 76, 85, and 87.

[MrPopo]

Anecdotally, I was noticing that my Conquest playthrough was feeling similar to my recent time with XCOM 2 (which uses a single number) while my Birthright playthrough I feel like it was coming through much more on my middling hit rates hitting and the enemy's lower hit rates missing. Which got me thinking: what if the two story paths use different calculation methods? It's conceivable that Birthright uses 2 RN while Conquest goes back to old school 1 RN.

Edited February 29, 2016 by MrPopo

[Crimean Archivist]

Anecdotally, I was noticing that my Conquest playthrough was feeling similar to my recent time with XCOM 2 (which uses a single number) while my Birthright playthrough I feel like it was coming through much more on my middling hit rates hitting and the enemy's lower hit rates missing. Which got me thinking: what if the two story paths use different calculation methods? It's conceivable that Birthright uses 2 RN while Conquest goes back to old school 1 RN.

Possible. All of my numbers so far have been from Hoshido, and I've collected the bulk of the data, so I'm not really worried about anything confounding the values. That would beg the question, though, which system is used in the initial six chapters?

The thing about the data we've collected so far is that it pushes both 1-RN values and 2-RN values outside of the confidence range, so we're pursuing all possibilities. At any rate, I'll keep my personal data from Conquest and Birthright separate and make a third sheet for the aggregation of the two and we'll see what changes, if anything.

This is quickly becoming quite the project.

[Crimean Archivist]

With more than 4,000 points, we now actually have several values over 100 trials, so I figured those might be worth sharing.

@ 84: 106 hits, 6 misses -- measured success rate: 94.64%

@ 85: 119 hits, 4 misses -- success rate: 96.74%

@ 87: 117 hits, 9 misses -- success rate: 92.86%

@ 93: 108 hits, 4 misses -- success rate: 96.43%

And now the (3A+B)/4 model is almost universally underestimating values, although it is in some cases still comparing to sample sizes from n=4 to n=50.

Edited March 2, 2016 by BlenD

[XeKr]

Is there a way to get error bars on those or is the graph just too messy?

tbh the 60-100 region looks pretty good.

Edited March 2, 2016 by XeKr

[Crimean Archivist]

Super pretty graph. All grey points are currently within the error margin for our outcomes. Should shrink with time, and given that we know that all values are bounded above and below by other values, things could narrow very quickly.

[RoyOurBoy]

I just so happen to have bought a new fire emblem ninty 3ds. From what the nice VincentASM says any new 3ds will do? I've never used a hacked rom so I don't know what to do or if I need to buy anything, but I don't mind helping. Idk what Vincent is even talking about though. I am still on my first conquest play through, as soon as I've beaten it I intend to start a conquest lunatic run and Hoshido hard run (to vent my conquest frustrations :) ). If I can help then please let me know.

[Crimean Archivist]

I just so happen to have bought a new fire emblem ninty 3ds. From what the nice VincentASM says any new 3ds will do? I've never used a hacked rom so I don't know what to do or if I need to buy anything, but I don't mind helping. Idk what Vincent is even talking about though. I am still on my first conquest play through, as soon as I've beaten it I intend to start a conquest lunatic run and Hoshido hard run (to vent my conquest frustrations :) ). If I can help then please let me know.

That's a question for VincentASM or someone who frequents/contributes to the hacking threads like shadowofchaos725 (he's the only one I know by name). Message them, find a tutorial, something along those lines.

As for helping, you could use whatever hacked version you find to fix the stats of a specific player/enemy unit pair so that they have a certain fixed chance to hit each other and deal no damage. That way, you have a risk-free way of testing the same hit chance over and over again. Repeat for 50 turns and you have at least 200 data points. Do that at 20, 30, 40, 50, and 60, and this project is basically finished.

Even without hacking, if you simply record values and outcomes through your playthroughs you can get a lot of data without trying very hard or getting bored. I picked up almost 3000 points in one run of Birthright, even without grinding.

Edited March 2, 2016 by BlenD

[Crimean Archivist]

At 5100 points now, but the data is still heavily concentrated at the top. All hit values with more than 100 data points are within +/-5 of their expected values under the (3A+B)/4 model. Some values are spot on, and the value for 75 has slowly approached our expected value, currently at 82.5. It still looks like we're undershooting most middling values (from about 30 to 60) but in general the relation holds pretty well. Would really like to find somewhere I can farm hit values 40 and below.

[eclipse]

What level are your units? See if you can find a really high level castle, and try that.

Or, you can give it a shot on the DLC maps. I think the formula would be similar.

Currently doing Nohr, and it feels more like the RN system I've known for a while.

[Crimean Archivist]

Just started Conquest yesterday, but I still have my Birthright file that I can poke around castles in (when my wifi comes back). I'm marking up my Conquest data but to me it seems largely the same. We'll find out in time, I'm sure.

[Crimean Archivist]

6000 data points, and I've enacted relative upper and lower bounds on the data, so that a value's upper bound can't be higher than the upper bound of a value to the right of it and its lower bound can't be lower than the lower bound of a value to the left of it. There are glitches below values of 40 because of the dearth of data points there, but above that the data is sound. I've highlighted that region.

This relative bounding produces a few rectangles where data is scarce, but we can rest assured that the rightmost point is accurate above the curve and the leftmost point is accurate below the curve. We can confidently wrap the error bounds with a function if we can find one that fits all of the upper bound maxima and one that fits all the lower bound minima. That would be under a logistic model L / (1 + e^-k(x-x0)) with L = 100 (or 99.99) and x0 = 50. Starting that calculation now because there's only one unknown (k) for any point.

Edit: Steep approximation (high k) is almost useless, but shallow approximation (low k) enforces hard caps on certain values which will allow us to extrapolate behaviors for low values from high values. It narrows the range of likely values at the mid-extremes (20-40 and 60-80) to a range of +/- 10%. Better still, both approximations will improve in accuracy no matter what values records come from.

Edited March 7, 2016 by BlenD

[Verile]

Some people have set up their castles so they're Multiplayer friendly e.g., all units on hold, easy to seize. Should be easy to farm lower hit rates from there.

I set my Corrinne to 80 Avoid and Hana to 92 Avoid with 1 range weapons within Lilith (64 avoid) heal range, all units set to hold. Einherjar Shop +20 Avoid. If you think that might be useful or want anything more specific, let me know.

Castle Address: 09413-68970-35568-96895

Though, in retrospect, you don't even need other people's castles to do this. Just setup your defenders however you want and then "Check Defenses" under Streetpass settings.

Edited March 8, 2016 by Verile

[Minternets]

You look like you're way more into this than I am. I'll post my stuff here. Do what you can with it. All data is gathered by hand in online MyCastle battles. Low stat units with a collection of weapons with varying hit rates and Gamble/Certain Blow allows me to get a wide variety of hit rates while dealing zero damage.

Some of the top ranked (BP) maps have castles perfect for testing, including the one titled "Grind", and the one that has all 24 resource points.

All tests done from player side, NOT enemy side.

--

Charlotte + Frying Pan + Certain Blow

94% hit rate displayed. 99.34% with Average 2RN (True Hit).

272/274 hit. 99.27%.

One eighth of the misses expected from 1RN

Silas + Bottle

82% hit rate displayed. 93.7% with Average 2RN (True Hit).

184/200 hit. 92%

Half the misses expected from 1RN. (expected 36, actual 16)

About what was expected for True Hit. (expected 13, actual 16)

67% hit rate displayed. 78.55% with Average 2RN (True Hit).

171/200 hit. 85.5%.

44% of the misses expected from 1RN (expected: 66, actual 29)

67% the misses expected from True Hit (expected: 43, actual 29)

**Anomaly/Luck? Test a 60%, 50% later.

Kana + Dragonstone

23% hit rate displayed. 10.81% with Average 2RN (True Hit).

36/170 hit. 21.17% (Go figure, dragon fang procs on a unit with 11 skill during 3 of the hits)

92% of the hits expected from 1RN

Twice as many hits as expected for True Hit.

Silas + Raider Naginata

9% hit rate displayed. 1.71% with Average 2RN (True Hit).

15/200 hit. 7.5%

83% of the hits expected from 1RN.

Five times as many hits as expected from True Hit

--

new

--

Beruka with Frying Pan, Jakob support

49% hit rate displayed. 48.51% with Average 2RN (True Hit)

104/200 hit. 52%

107% of the expected hits from 2RN.

106% of the expected hits from 1RN.

Right in line with both 1RN and 2RN. This should help rule out some new system that either benefits the player, enemy, or simply increases hit rate overall.

--

Jakob with Steel Dagger, Beruka support (Instantly regret this, due to debuff animation appearing every combat)

63% hit rate displayed. 72.99% with Average 2 RN (True Hit)

141/200 hit. 70.5%

109% of the expected misses from 2RN.

79% of the expected misses from 1RN.

--

Beruka with Hand Axe + Jakob Support

36% hit rate displayed. 26.28% with average 2RN (True Hit)

70/200 hit. 35%

97% of the expected hits from 1RN.

133% of the expected hits from 2RN

--

All performed by player. (based on previous assumption that it was either Average 2RN or 1RN. Not taking 3A+B into account)

94% - Definitely True Hit

82% - Very likely True Hit

67% - Anomaly. Higher than True Hit. Much higher than 1RN

63% - Likely True Hit

49% - In line with both 1RN and True Hit

35% - Very likely 1RN

23% - Very likely 1RN

9% - Definitely 1RN

Needs testing:

-Does the enemy get this same treatment?

<b>Theory:

2RN is used for all hit rates over 50%.

1RN is used for all hit rates under 50%.</b>

Edited March 8, 2016 by Minternets

[ruadath]

<b>Theory:

2RN is used for all hit rates over 50%.

1RN is used for all hit rates under 50%.</b>

Oh whoa, this would be pretty cool. So having a high hit rate is still rewarded with near guaranteed hits, but low hit rates aren't as nerfed as they used to be. Could be an interesting system to play.

[Verile]

Followed my own advice and took 400 samples from the lower ranges. Goes pretty quick if you only have 1 unit (auto end) and nobody's moving.

[Hit Rate], [Outcome H/M]

36, 34/66
31, 23/77
22, 24/76
17, 17/83

[Crimean Archivist]

First off, thanks to you guys for brute-forcing all of those data values. That brought our total point count up to 8600 and greatly improved confidence for lower values. I revised the graph to show both the (3A+B)/4 model and the split model and killed the error bars as they were cluttering the graph too much with both models on there.

The low end definitely favors a 1-RN model over a raw or weighted 2-RN model. The upper half of the graph is too close to call at present. The high-confidence values established by testing (63, 67, 75, 82, 85, and 94) are all very close to both values, as the models don't differ by much. However, the reason we pursued something other than raw 2-RN to begin with was the oddity of extensive testing at a value of 75. Of the high-confidence values, 75 is the only point whose 99% confidence interval excludes its raw 2-RN probability.

Aggregate stats at 82 are at 294/26 H/M ratio, or 91.875. This is extremely close to the (3A+B)/4 model, which predicts 91.72. The rest are all toss-ups, save 75, which predicts something other than a raw 2-RN system. (Of note: a lower-confidence but still fairly confident value of 74 is almost exactly equal to its raw 2-RN expected value.)

In the limit, if things are too close to call, I would lean towards keeping a raw 2-RN over a weighted one because it seems more likely to not change a system than to change it in such a way that barely affects the outcome for most values. For now, let's keep testing.

Edited March 9, 2016 by BlenD

[XeKr]

Interesting developments. Margin of error for 99% confidence, ~200 points is still around 10%, which seems to be too much to distinguish the weighted 2RN from either system. The tricky part is how close it is to both 1RN and traditional 2RN (pretty much just in between for large ranges, which I guess is the point).

imo the 75% data is pretty compelling, and the most elegant explanation is a symmetric system. Still, I think focusing efforts around 25%, like we did for 75%, could tell us a lot.

[Crimean Archivist]

To generalize, at any measured success rate P, the number of trials N to get the one-way margin of error of less than a value K is

N = 6.6049 * P * (1-P) / K^2

The current proportion for 82 is 294 successes out of 320 trials, or 91.875. If we assume the success rate doesn't change, the positive margin of error has to be less than 0.01825 in order to invalidate an unweighted 2-RN system. That's 1480 trials. Ew.

Unfortunately the number of trials we have to perform until we hit the effective limit of a 2-RN system for any given value is...10000 and counting (you know, exactitude). If we assume one model or another is correct, we can take a 400- or 500-trial count for most values as definitively exclusive of at least one of the possibilities. The sample size for 75 is close to 400 already and I still have my Ch 2 save, so I'll go ahead and take that to 500 and see if anything changes. The Castle battling method has certainly proven effective, so I'll also take one of my files and play with skill/weapon combinations until I get something close to 25 and start hammering away there.

I can't decide if this is as absurd as trying to figure out FE9's Forge calculations or not.