Had a great holiday there last August - and yes I would definitely take my bay with me ... our experience was that Sri Lankan's love kids. It IS hot and very humid though, other than in the hill country (and I'm South African, so that's saying something) .... so be prepared for that. I guess you already know about the rather unusual monsoon season they have. I'd probably stick to the better known hotel chains if you want more familiar western style food for your baby, but then again every hotel we stayed in was very accommodating for our rather fussy 7 year year old. I'd definitely hire a driver, the roads are tricky in places and as lousmith mentioned, the driving rather erratic! It's not pricey and all hotels offer accommodation for drivers.
We booked our holiday through [www.srilankandreams.com
] and they were brilliant. They offered options across the price range and very flexible in terms of the itinerary - and do make it clear if you just want a fly and flop rather than a cultural sight seeing trip. Some distances are long, and bit be difficult with a baby, although you can fly from the East Coast back to Columbo easily and it's a fantastic way to see the country if you do it in a small plane. Worth the splurge!