Wreckers were poised to begin demolishing the house on the day in October that Mr Whitlam, Australia's 21st prime minister, died aged 98.

Following articles published by The Age on the house, Mr Guy wrote to the Heritage Council asking that an interim protection order be placed on the property.

The Heritage Council, which decides if buildings get protection, made the order immediately. It expires next month.

However, in his recommendation to the Heritage Council, executive director of Heritage Victoria Tim Smith has said the house in Kew's Rowland Street should not be added to the heritage register.

The house, named Ngara, was built in 1915 by the late prime minister's grandfather. Mr Whitlam was born in the house.

In 2013 it was bought for $3.3 million by overseas buyer Youquing Liang, who intends to raze the property and rebuild.

Mr Smith has deemed the existing house not worth protecting. In his recommendation to the Heritage Council, he wrote that while Ngara was Mr Whitlam's home from birth to the age of about two, his association with the property was not meaningful enough to save the house.

Mr Whitlam's link to the house "was brief and there has been no ongoing association", he wrote. "Whitlam had no association with the place during his term as prime minister and did not spend any of his politically formative years at the place.

"Although this place is one of the few physical links in Victoria to Gough Whitlam's life, the association of the place with Gough Whitlam is not strong enough to meet the threshold for state significance."

Mr Smith compared the Kew house to others on the Victorian Heritage Register with clear associations to prominent individuals. They included Longacres at Olinda, where artist Arthur Streeton lived, and Ballara at Point Lonsdale, with its links to Alfred Deakin.

"Gough Whitlam left Victoria around the age of two and did not return to live in the state," his recommendation said. By contrast, six other Australian prime ministers born in Victoria had long associations with the state.

Boroondara Council wants the house given heritage protection, and will make submissions to the Heritage Council on why it should not be razed.

One neighbour said he was "very disappointed" to learn of the decision, but would not pursue the matter any further. "It appears that a fairly comprehensive and forensic piece of work has been completed by Heritage Victoria, so I will leave it with others to take further," he said.

Since October, Ngara has fallen into further disrepair in the wake of initial demolition work. A number windows have been smashed and parts of the house have been vandalised.

"I think some kids have been in there wrecking the place," one neighbour said on Wednesday.

At the time of Mr Whitlam's death, Premier Daniel Andrews said the former prime minister would not have wanted any money spent on a house he lived in only briefly as a baby.

"I would imagine Gough Whitlam, being the humble man he was, would much prefer the taxpayers' money to be invested in health and education," he said. A spokesperson for Mr Andrews said the premier's view was unchanged.

The public can make submissions over the fate of Mr Whitlam's birthplace to the Heritage Council until February 16.