The children are 1 year old baby and  6 year old twin.
The man asking more information after the mathematician’s first reply means he could not identify the ages of the mathematician’s three children at that moment; This situation happens when the combination of the ages are (1, 6, 6) or (2, 2, 9), both  add up 13. The mathematician’s second reply means the youngest child is single, excluding possibility of the later combination.