Well the problem with the second fight wasn't his offense, even Lon'qu could take 3 hits from Walhart, and in my successful run he had vantage+. The problem is his defense, since you have to kill him in one PP, or his lackeys will heal him. You could kill the healers, but they are four, and they heal Walhart over attacking you( I think). I had to rely on an Aether Proc, even when everyone but Panne attacked him. And since he has auto Pavise+ and Aegis+.
Chapter 21