Kei Saito failed an out-of-competition doping test the day he arrived in Pyeongchang.
Saito, a reserve athlete who had not yet competed in South Korea, was provisionally suspended after testing positive for masking agent acetalozamide and a final ruling would be issued after the Games, the Court of Arbitration for Sport said.
Masking agents are often used to cover up banned performance-enhancing substances.
This is the first doping case during the testing period of the Olympics in South Korea and comes as Tokyo prepares to host the Summer Games in 2020.
Team Japan officials said Saito was tested hours after arriving in the athletes' village on Feb 4.
The athlete said he was stunned by the finding.
"I am shocked by this as I have never tried to commit doping," Saito said in a statement.
"I have been trained in anti-doping. I have never taken a steroid so there is no need for me to mask something, and I never had to drop weight by using this medicine.
"So I have no motivation to use this. I want to prove my innocence but I do not want to burden my team so I accepted this decision," he said.
Team chief Yasuo Saito said the athlete had also been tested at a training camp on Jan. 29 on the order of the International Skating Union (ISU) and his test was negative.
"We respect the athlete and his will and intentions, and in conjunction with the Japanese skating federation we will continue to look into this matter with all our means," he added.
The team chief said it was "inexplicable" how the athlete could have tested positive but in order not to burden his team mates he agreed with the International Olympic Committee proposal to have him removed from the Games.
He added that while the suspension was only provisional it was better for the athlete to leave as there was no time to clear his name.
"There was no opportunity to prove his innocence during the Olympic Games. It was a difficult decision, one he did not want to take, but he accepted to leave the Olympic village so as not to cause problems for his fellow team members," the team chief said.