Why this man was deemed the "most dangerous American" in N. Korea

In 2014, American Kenneth Bae came home after two years in a North Korean prison. He is detailing his time inside the secretive country in the new book, "Not Forgotten."

Bae made 18 trips to North Korea. As a foreigner, each one was as dangerous as the one before. But in 2012, he was arrested and accused of trying to overthrow the government, and became the first American sentenced to hard labor and the longest-held since the Korean War, reports CBS News correspondent Margaret Brennan.

"It's still stuck in my head. And I feel like I'm carrying this badge of 103 in my chest forever," Bae said.

Before he was "prisoner 103," Korean-born Kenneth Bae was a preacher. He grew up in a tight-knit family in California and went on to start a tourism business, bringing Christian groups into North Korea.

But he made a fateful mistake. In 2012, he brought in a computer hard drive loaded with prayers and pictures of starving North Korean children.

Any criticism of the regime is forbidden. Supreme leader Kim Jong Un and his family consider themselves gods. He was arrested, charged with espionage and sentenced to 15 years of hard labor.

"One of the prosecutors told me that I was the worst, most dangerous American criminal they ever apprehended since the Korean War. And I said, 'Why?'" Bae recalled. "And they said, 'Because not only you came to do mission work on your own, you asked others to join.'"

Bae's fate was now in the hands of a young and brutal dictator scorned by the U.S. for carrying out a series of underground nuclear tests. Tensions with the U.S. spiked.

Bae believes he was a political pawn. "All of America really was on trial with me," Bae wrote in his book.

"I believe that they blamed everything wrong with their country [on] America. They said the reason for poverty, the reason for their suffering is all caused by U.S. foreign policy against them," Bae said. "And therefore, by indicting me, they were indicting the U.S."

Bae spent nearly two years under 24-hour watch by 30 North Korean guards. The conditions were dire - he shoveled coal and worked the fields. He lost 50 pounds and was briefly hospitalized.

"I am looking in the mirror in the bathroom every day, and say, 'remember, you are a missionary. This is what you are here for,'" Bae said. "I took it more as a blessing, rather than a curse or suffering... Well it is very hard for me to even say that right now, but no one likes suffering, no one will embrace suffering but when suffering comes to you, you have to face it."

Play Video Inside the release of American captives from North Korea

Kim Jong Un finally issued a pardon in 2014, after the White House sent U.S. Intelligence Chief James Clapper to pick up Bae and another prisoner.

Bae said he'd never been so proud to be American.

"I was just overwhelmed that-- that after being there for 735 days, I was finally going home," Bae said.

Bae said he's not angry about his imprisonment. He believes it was an opportunity to share his faith and teach his guards what life is like outside of North Korea.

"I was just there to love the people, let people know that God cares about them, and the rest of the world care about them," Bae said. "I hope that this book become a reminder to people to not forget the people of North Korea, have more compassion for the people who are living as a prisoner in their life."

Twenty-one-year-old student Otto Warmbier is currently facing a similar 15-year hard labor sentence in North Korea.

Bae said he has reached out and advises Warmbier's family to speak out, and believes press attention helps.