Many people are unaware that even smoking away from babies or pregnant women presents a risk, according to US research.
Poisons in cigarette smoke can linger on fabrics or hair, but a survey of 1,500 households found that fewer than half of smokers knew this.
Only a quarter had strict rules about not smoking in the house, according to the report in the journal Pediatrics.
UK baby charity Tommy's said it was vital that pregnant women were alerted.
There is plenty of evidence that "second-hand" smoke - breathed when you are in the same room as someone smoking - can be harmful, particularly to children, and some parents adopt a strategy of never smoking in their child's presence.
However, Professor Jonathan Winickoff, from Massachusetts General Hospital, said this would not offer complete protection.
Toxic particles in cigarette smoke can remain on nearby surfaces, as well as the hair and clothing of the smoker, long after the cigarette has been put out, and small children are susceptible because they are likely to breathe in close proximity, or even lick and suck them.
Other studies have linked this exposure to learning problems in children.
Breastfeeding mothers who smoke also pass toxins on to their baby in their milk, he said.
His team surveyed more than 1,500 households, asking smokers and non-smokers about their attitudes.
They found that while 95% of non-smokers and 85% of smokers agreed that direct inhalation of second-hand smoke was harmful to children, just 65% of non-smokers, and 43% of smokers believed the same for "third-hand" smoke.
Just 26.7% of households which included a smoker had strict rules about not smoking in the home.
Professor Winickoff said: "The dangers of third-hand smoke are very real - when you smoke - any place - toxic particulate matter from tobacco smoke gets into your hair and clothing."
Professor Andrew Shennan, from Tommy's, said the results had "significant implications", particularly for pregnant women, who may have stopped smoking themselves, but are still in contact with others that do.
"It is vital that women are made aware of the possible risks associated with third hand smoke, and alert those around them of the impact it could potentially have on the health of their unborn baby.
"The chemicals in cigarettes are known to significantly increase the risk of serious pregnancy complications."