Presumably Griel, Mist, Ike and Titania didn't leave Gallia automatically, as Cain spoke to Griel before he left, so Sephiran managed to erase memories before the events Ike would've remembered happened.
According to this site's characters ages page, Ike is 17 during PoR according to the official Path of Radiance site.