My little Kismet was the runt of the litter. Kismet is the fluffy shih tzu on my siggy. I originally had her sister, Karma, for a week and she died of a birth defect. Now, Kismet, is 6 lbs at 4mths old! She's a healthy girl.
I say go w/ your gut. If when you saw the pup it looked healthy and playful I wouldn't worry much.