In total, 378 MPs voted against the bill, while 296 supported it.
A majority of German lawmakers rejected a bill on Thursday that would have required all residents over the age of 60 to be vaccinated against the coronavirus.
The bill, which had been pushed through by Chancellor Olaf Scholz and his health minister, Karl Lauterbach, was already considered a compromise solution after lawmakers from the governing coalition and opposition parties had balked at the idea of a vaccine mandate that would apply to all adults in the country.
However, the bill failed to receive the backing of a majority of parliament, with opposition parties arguing that a vaccine mandate was not necessary given that case numbers were going down across the country.
In total, 378 MP voted against the mandate, while 296 supported it.
Germany has managed the pandemic reasonably well compared to some of its European neighbours, with fewer deaths per capita than Italy, France, Britain or Sweden. However, the country is still experiencing high daily rates of transmission.
Official figures show that confirmed coronavirus cases in Germany have declined in recent weeks, to a little over 200,000 new daily cases, down from about 300,000 in mid-May. There were 328 new COVID-related deaths, according to Germany's disease control agency.
According to the European Centre for Disease Prevention and Control, 90% of Germans over the age of 60 have been vaccinated.
In December, German MPs passed a law requiring everyone working in a hospital, as well as retirement and nursing homes staff, to get a COVID-19 vaccine. At the time, a poll showed that 68% of Germans were in favour of vaccination requirement for all adults. A more recent opinion poll showed that some 60% of Germans supported a vaccine mandate.
The government had hoped that the compromise measures, put forward by a cross-party group of lawmakers, would receive a parliamentary majority, following months of back and forth discussions.
The bill would have also established compulsory counselling for all adults, to help them weigh up the advantages and risks of getting a vaccination against COVID-19.
Before today's vote in parliament, Social Democratic lawmaker Dagmar Schmidt, who presented the bill, said it was necessary to prepare for a new rise in cases and a possible new variant later in the year.
"Today is not about what's happening now, but what will very likely happen in the fall,” she said.
It was a message that failed to resonate with enough of her colleagues.
In late March, Germany began scaling back its enforced COVID-19 protection measures, with the responsibility shifting to federal states to impose stricter measures if and when required. Germany has also dropped mask requirement at shops.
However, on Wednesday Health Minister Lauterbach reversed course on plans to end mandatory quarantine for those who tested positive for coronavirus, after announcing that obligatory self-isolation, usually for 10 days, would be scrapped from May 1 and replaced with a strong recommendation to isolate for five days.
“I have withdrawn the proposal because the completely wrong impression would have arisen that either the pandemic is over or the virus has become significantly more harmless than was assumed in the past,” he told reporters in Berlin.