Daniel Sturridge's 82nd-minute header earned Hodgson's side a 1-0 win over Denmark at Wembley on Wednesday, but afterwards Olsen seemed unimpressed by England's performance.
As the 64-year-old left his post-match press conference he said: "Good luck in the World Cup... you'll need it."
Olsen also suggested that his side would have won if Tottenham playmaker Christian Eriksen had been available.
Hodgson, however, was not worried about what Olsen had to say, and felt his team had put in a positive showing.
"I don't care about that (Olsen's remarks)," Hodgson said. "Every team will need good luck. We didn't need it tonight, that's for sure.
"This was an important game. It was important we put in a good performance.
"I wanted a level of energy, players running back when they lost the ball, making tackles from the wrong side, not being afraid to make mistakes knowing, with their pace, they can recover.
"That's what I did see and we put the icing on the cake with the goal to win it."
The victory on Wednesday was England's last game before Hodgson names his provisional 30-man squad for the finals in Brazil on May 13.
‘Maybe I’ll be signed by Wrexham, who knows? English football would suit me – if an offer came, I wouldn’t think twice, I’d go immediately’ Euro 2024 cult hero reveals his UK dream
‘Managing Leeds? It was an option that appeared, but it wasn’t the right timing. I decided it wasn’t a good idea to leave the club I was at mid-season’: Premier League boss admits to turning down opportunity to replace Jesse Marsch in 2023
