S1E8 : Beloved Infidel

When Frasier and Niles see their father dining with an old family friend, they conclude that Martin once had an affair with the woman.