I'm glad you decided to go check out the meetinghouse! A few pieces of advice that I would give is to try to call over to confirm the meeting times- sometimes wards share buildings so the times are different. Dress "conservative Sunday best", so no tank tops or short skirts, or flip-flops! Guys usually wear slacks, a collared shirt, and a tie. Women usually wear dresses or long skirts. Babies aren't a problem in meetings, if they cry, you just take them out to the lobby until they calm down. If you're nervous about that just sit near a door or at the end of a row so you don't have to climb over your new friends to leave if you need to. If you plan to stay for all the meetings, introduce yourself to the bishop (he should be sitting up at the front if the meetinghouse is like mine) and ask him to point you in the right direction for the next meeting, or you can ask someone around you.
One last piece of advice that seems to be helpful to first-timers- you don't applaud after anything, even if it's a musical performance. It's not rude, it's just keeping the reverence and the Spirit. I learned to give this advice after we brought a friend to church and she clapped one single clap before realizing nobody else was doing it, and she was mortified!
If you have any other questions don't hesitate to ask, and good luck! Let us know how it goes!