Mr Lower said the hospital was "not conceding defeat, it's recognising the reality of the situation. We have got signs, we have staff come out to ask smokers not to smoke or move off our premises - it doesn't work".
"We have thousands of visitors to the hospital, often in times of stress, and many of them have a need to smoke - they are addicted to nicotine."
But Patrick Thompson, from the patient group Norfolk Link, said rebuilding smoking shelters on hospital grounds sent out the wrong message.
"We'd like the site to be a non-smoking site, but 'the site' is only covered by the building and not the outside areas.
I think it's pretty telling that these shelters that were removed in 2005 are going to be rebuilt.
A shelter not fit for pigs is progress
I think if they are truly rebuilt then it sends a message that smoking can't be stopped,not by force or threats or by busybodies with nothing better to do than chastise others.