There are arguably two reasons why he wears the Keaton mask and hides. The first reason why he chose to disguise himself is because he had been turned into a child, which was the doing of the Skull Kid. There's a second reason in which Kafei explains that he's not concerned about his appearance, but he had lost the Sun Mask he had promised to bring to Anju. The second reason is way more valid since through the quest that took up most of the game, you're helping Kafei obtain the Sun Mask once again.
Kafei seems to be friends with many of the townspeople since he's the mayor's son. The fact that he can show off his Sun Mask, which is basically like a wedding ring, is a slight indicator of that. Anju's mother doesn't seem to like him though. Kafei is also friends with Cremia, who runs Romani Ranch with her sister Romani.
We are never shown Kafei's face as an adult, but in the end, it's indicated that he returned to his adult form.