A baby is to be monitored by a network of microphones and video cameras for 14 hours a day, 365 days a year, in an effort to unravel the seemingly miraculous process by which children acquire language.

Deb Roy at MIT's Media Lab, US, devised the unusual project and even volunteered his own family as its guinea pigs. Since his newborn son left hospital nine months ago, Roy's whole family has been monitored by 14 microphones and 11 one-megapixel "fish-eye" video cameras, attached to the ceilings of each room in their house.

By capturing a continuous stream of data about his son's experiences, Roy hopes to better understand the early development of language.

How babies go from gurgling at birth to fluent speech by the age of three is hotly debated. Most psycholinguists agree that simply listening to speech is not enough for a child to piece together the basic rules of a language. Yet they still argue about the importance of specific "language genes" and other non-verbal environmental stimuli.

Deleted scenes

In an effort to identify these extra environmental cues researchers have previously recorded mothers and babies playing, both at home and in the laboratory. But following babies in such an unnatural environment provides a distorted picture of their experiences, says Roy, and makes it difficult to tell whether changes in a child's speech are sudden or merely appear that way because of gaps between recordings.

To provide a more complete picture Roy has developed a surveillance network at his own home in a project dubbed "speechome". The cameras are switched on between 8am and 10pm each day and will capture 85 per cent of the baby’s waking hours up to his third birthday. For occasional privacy, Roy and his wife can switch off cameras or delete recordings using wall-mounted touch displays. A selection of video clips of the surveillance set up can be viewed here.

Footage recorded by the cameras is automatically transmitted to MIT for analysis. Computer algorithms identify activity in specific sections of each room and collect these into so-called "behaviour fragments". Human analysts then classify specific acts, such as making coffee or doing the dishes.

Unparalleled scale

Comparing footage with early sounds made by Roy's son should help his team better understand the stimuli crucial to language development. Algorithms are also being developed to automatically transcribe speech and recognise people and objects in each room.

"It allows us to put a microscope on the day-by-day and hour-by-hour changes that go into learning a language," says Steven Pinker, a psycholinguist at Harvard University, who is an adviser to the project. "Nothing remotely on this scale has ever been done."

If successful, Roy says the project could lead to better strategies for diagnosing and treating language disorders. It could even spawn computer programs that can learn to how to speak for themselves, he adds.

Roy is aware that the project raises ethical issues. But ultimately he thinks he may be providing his son with an incredible gift. "He might be the first person to have a memory that goes back to birth," he says.