He had already resigned from his first stint in office in 2007 due to his health.
Shinzo Abe, Japan's longest-serving prime minister- is stepping down to fight against a "chronic disease" which he can no longer keep "under control" while in office.
Abe said he had managed to treat the disease with drugs until his condition suddenly deteriorated in August, leaving him "no choice but to resign".
He regretted leaving his post without "achieving" all his "targets", but stressed the country needs to have a leadership which is fully capable to fight the current coronavirus emergency.
Abe said the decision caused him "anxiety" but was supported by his fellow Liberal Democratic Party members.
Concerns about a chronic health issue, simmering since earlier this summer, intensified this month when he visited a Tokyo hospital two weeks in a row for unspecified health checkups.
Top officials had said publicly that he was overworked and badly needed rest, while members of the press asserted that the Japanese prime minister looked tired and was moving slower than usual.
Abe, whose term was supposed to end in September 2021, said he will remain in charge until his party chooses a new leader.