I always thought kids could start showing signs of magic at any time. Doesn't it say in the Philosopher's Stone that odd things had always seemed to happen around Harry? And if he could, at the age of one, use a toy broomstick, I would assume that to mean that he already had some visible magic in him. Neville first showed signs of magic when he was 8 (the time he bounced), and that was apparently late. (Quote can be found on this page)
Depending on what type of magic, they'd probably begin showing signs at birth. Teddy Lupin's hair started changing color about five minutes after his birth, if we go by what Remus said when he burst into Shell Cottage in DH. Harry used a toy brromstick at age one.