Cornwall and the Isles of Scilly have the highest proportion of second homes in England, according to a league table published today to counter government attempts to downplay the problem.
Yvette Cooper, the housing minister, angered housing campaigners earlier this week by claiming the growth in second home ownership was not pricing people out of the property market.
In an interview in the Financial Times on Monday, she said: "In most parts of the country the number of second homes is extremely small ... it's not a significant fact in affecting affordability."
The Liberal Democrats have accused Ms Cooper of displaying "jaw-dropping naivety".
Using the government's own figures, they have produced a top 10 table to show where the proportion of second home ownership is highest.
Top of the list is Cornwall and Isles of Scilly, where there are 13,458 second homes, comprising 5.6% of the housing stock.
Andrew George, Liberal Democrat MP for St Ives, said: "Second home purchase in areas such as west Cornwall entirely dictate market price and conditions.
"The minister has demonstrated a level of jaw-dropping naivety, which merely demonstrates to constituents like mine just why the so-called metropolitan elite is so out of touch with the reality of their lives."
In May, the government's affordable rural housing commission urged the government to consider allowing councils to impose a second home impact tax in areas where there is a disproportionate number of second homes.
It also said all councils should exercise the option of reducing council tax discounts on second homes from 50% to 10%.
The commission, chaired by the former Channel 4 News political editor, Elinor Goodman, heard that more than half the properties in some villages were second homes.
However, it found the impact of second homes across rural England as whole was only modest and, in some places, was beneficial to the local economy.
Dan Rogerson, MP for North Cornwall, said: "The government should stop setting up commissions it then ignores. These new figures show the problem is urgent. The days of timid initiatives and more inquiries have to come to an end. It is time for bold action."
The Liberal Democrat proposed going further than the Goodman commission's recommendations by requiring owners to seek planning permission before their property can become a second home.
They also want to give local authorities the power to charge business rates on second homes.
Areas of England with the most second homes (number of properties and proportion of housing stock)
· Cornwall and Isles of Scilly 13,458 (5.6%)
· Cumbria 7,906 (3.4%)
· Dorset 10,540 (3.2%)
· Norfolk 11,857 (3.1%)
· Devon 14,813 (3%)
· East Sussex 7,583 (2.1%)
· Northumberland 2,805 (2%)
· North Yorkshire 7,074 (1.9%)
· West Sussex 6,266 (1.8%)
· Suffolk 5,414 (1.8%)