Yoshitaka Sakurada made the admission to a committee of lawmakers.
"Since I was 25 years old and independent I have instructed my staff and secretaries. I have never used a computer in my life," he said, according to a translation by the Kyodo news agency .
The 68-year-old was appointed to his post last month.
His duties include overseeing cyber-defence preparations for the 2020 Olympic Games in Tokyo.
A politician from the opposition Democratic Party, Masato Imai, whose question had prompted the admission, expressed surprise.
"I find it unbelievable that someone who is responsible for cyber-security measures has never used a computer," he said.
But Mr Sakurada responded that other officials had the necessary experience and he was confident there would not be a problem.
However, his struggle to answer a follow-up question about whether USB drives were in use at the country's nuclear power stations caused further concern.
The disclosure has been much discussed on social media where the reaction has been a mix of astonishment and hilarity, with some noting that at least it should mean Mr Sakurada would be hard to hack.