I was going to give it to Thracia, but after two pages worth of thought, I'm going to give it to Binding Blade.
Thracia is no doubt hard, but it always felt more cheap than truly difficult. It needs a strategy guide because if you go it blind with all of those mechanics, you'll probably hit a wall pretty soon. In the same way I don't consider certain modes in otherwise easy games to qualify (because they're specifically made to be obtuse pains), I don't think a game literally designed to be unplayable without outside help counts, either. It's not hard with a guide, as intended, and it's certainly doable without one if you have patience.
That left me with Genealogy and Binding Blade. For me, 4 was a culture shock. Gone were the succinct battles in favor of gruelling, huge maps where even the prologue took longer than three Sacred Stones missions. There was castle management and baby-making to think about (okay, that part I could handle thanks to Awakening). All in all, it felt like a FE game that was designed to rework the series and add a depth of strategy without putting it into insane territories. Hard? Yes, but maybe more because it's such different mechanics.
The title goes to Binding Blade because it's exactly what you expect of a FE game (at least, GBA and GC/Wii offerings) but it's difficult. Bigger maps, RNG that hates you, and (for me) a cast and story that didn't hook me, so the entire reason to continue with the game was just overshadowed by its difficulty level.
tl;dr everything can be hard if you make it hard but binding blade is pretty hard by itself thanks for your time