The firm failed to agree terms with client Medway council at the end of last month, saying extra work needed on land decontamination and flood defences affected the land price.
The move has opened the door for Wimpey to take over. It was beaten to the project by Berkeley in March, and is in now in negotiation with Medway to complete the deal.
Berkeley said it was still keen to complete the brownfield scheme, which would have included 900 homes, hotels, offices and a riverside walk.
Berkeley hired neoclassical architect Demetri Porphyrios to masterplan the scheme, which was hailed by urban taskforce chairman Lord Rogers as a flagship regeneration project.
Berkeley project director John Anderson said that having examined the 30 ha site in detail, the firm discovered more problems than it had anticipated.
Anderson said: "The land value was coming out at less than we originally offered. I am sure when Wimpey look into it they will find the same issues that we did. It's a very complicated site."
A source close to the Berkeley team said that there was strong hope that the project could be restarted.
The source said: "It was very exciting. The idea was to create a compact European city. Because of the nature of the site, you can create a highly defined place."
A Wimpey spokesperson said the firm was hoping to sign an exclusivity agreement with Medway council by the end of the year.
The spokesperson declined to name the project team working on its proposal. She said: "We are not far enough down that road as yet."
Berkeley's withdrawal follows delays to the scheme in recent months. The deal was originally due to be finalised in the summer, but was then put back to October.
Colin Lovell, Medway council's regeneration manager, said the council would have to speed up finalising details of the scheme.
The council, which owns two-thirds of the site, has until the end of 2001 to complete the acquisition of the final third.
Lovell denied there were any problems with flooding on the site, one of the most significant urban riverside areas in the UK still available for redevelopment.
He said: "We are not in an area prone to flooding. It's the estuarial part of the river rather than one prone to tidal flooding."
