The country star opens up about how he’s making amends following a difficult past

Jelly Roll for PEOPLE. Photo: Peter Yang

Jelly Roll knows his journey to the top wasn’t without its setbacks — and while he understands that not everyone from his past will be thrilled by his accomplishments, he’s working on making amends.

The country star, 39, opens up to PEOPLE in this week’s issue about reconciling with his early days, during which he was in and out of jail some 40 times over 10 years, starting when he was just 14.

“I’m rounding third on my amends list, and I think when I get there, I’ll feel a little better,” he says. “I was hitting some stone walls with people that wouldn’t forgive me and I was like, ‘God, maybe this has something to do with the fact [that] I haven’t forgiven myself either.”

Jelly Roll for PEOPLE. Peter Yang

Jelly says that while talking about his past in interviews, he’s often forced to “brush over 10, 12, 15 years of living an extremely crazy, narcissistic, selfish lifestyle” to condense it into a quick soundbite. But he knows the reality goes much deeper than that.

“The truth is, there was a lot of pain in there. I hurt a lot of people,” he says. “No matter how much I’ve changed, they still watch and are mad that I’m successful. I understand that. But I’m doing better at letting go of the past and realizing that I don’t owe that part of me anything.”

Jelly has previously been open about being “embarrassed” by his behavior during his younger years, saying on a recent podcast that he’d had an inflated sense of “entitlement” in his youth, and hoped that he could make amends with the victim of his robbery.

Jelly Roll for PEOPLE. Peter Yang

Since rising to fame, Jelly has dedicated his success to paying it forward, raising money for charity and visiting juvenile detention centers and jails. The Grammy nominee recently donated pre-orders of his latest album Beautifully Broken to four different charities, raising more than $1 million.

“Philanthropy is the legacy I hope to leave with the music,” he tells PEOPLE. “The music will live on its own… I quit looking at myself as an entertainer. I realized we’re here to serve. That’s the thought when I write, when I perform, when we go feed the homeless, when we go talk to kids at juvenile [detention centers] or jail — we just want to be a good steward with what we got.”