Leading explorer David Hempleman-Adams, whose children attended Prior Park College, told a packed audience health and safety is crippling children's sense of adventure.
The adventurer, who lives with his family in Box, near Bath, is the first person in history to reach the geographic and magnetic North and South Poles. He said rules and regulations are having a big impact on society.
"It's madness," he said. "We have surrounded our kids in cotton wool. I have taken all three of my kids away to the North and South Pole and places because they learn."
His daughter Alicia undertook an Arctic trek in Baffin Island aged 15, and her sister Amelia was the youngest person to do a cross-country ski trek to the South Pole aged 16.
His dother aughter Camila was the youngest person to do a skiing expedition to the North Pole, while a pupil at Prior Park College.
"In the big wide world if you fall out of a tree you're going to hurt yourself, but we have got to let youngsters do it.
"It's a shame that teachers who want to take youngsters away are scared because of legal reasons and it's crazy.
"We have got to change, otherwise we are going to have a society where kids are going to get to adulthood never having pushed themselves and never grazed anything.
"I think that is one of the problems with obesity – we have to let kids out into the wild."
Mr Hempleman-Adams was speaking at the Cheltenham Literature Festival.
Promoting his new book, No Such Thing As Failure, he told a packed town hall audience how youngsters are scared of failing.
"They are frightened to go out and try something," he said.
"People do not push themselves because they are afraid of failing, but you should follow your dreams.
"What annoys me is people who I say I wish I could achieve something. If you really want something and have the drive for it, go for it."
Afterwards he said he felt youngsters were held back by a lack of preparation.
"I think youngsters are scared partly because they haven't been through basic training, a Duke of Edinburgh award or scouts," he said.
"They've got this idea of going to the Alps or Everest, but they won't go because they don't want to fail.
