While It's true that Arvis has no control over his own empire, the soldiers of said empire thinks that the orders they receive are from Arvis himself and no one else: this is visible in chapter 10, where Fat bishop tells That Guy that if he doesn't follow the orders he will be betray Emperor Arvis and That Guy immediately starts moving. Another example is a female paladin in Thracia 776, who says that she became a knight to serve Arvis and she doesn't understand why he does things like the child hunt.
Perhaps I'm just reading too much into rhe quotes of the various soldiers from the empire, but if Arvis rebelled against Julius, Manfroy and the Lopt Sect, deciding to stop the Child Hunt, the soldiers of the empire would have probably followed him without any question: this just shows that Arvis could have stopped this whole situtation by himself if he wasn't such a coward, in my opinion.
I could be forgetting some imoortant details tough, so...yeah