The 19-year-old, who scored both goals, had been regularly jeered by Scotland fans unhappy at what they considered his feigning of injury in the early stages.
He said he did not see the banana being thrown, but saw it on the ground next to him as he was near the touchline.
"It's sad that these things happen but I'm not going to worry about it too much," Neymar told Reuters.
"It was a marvellous day and certainly one to remember."
Brazil coach Mano Menezes said he had not seen the incident but that if it had happened it was "lamentable."
Midfielder Lucas Leiva, who plays in England for Liverpool, told Brazil's Globo TV Sports channel: "There is no room for racism nowadays.
"Europe, which is considered the first world, is the place where this thing happens more. Today, colour and race shouldn't mean anything."
Last week former Brazil defender Roberto Carlos had a banana thrust towards him as he prepared to play for Russian club Anzhi Makhachkala against Zenit St Petersburg.
Get the best features, fun and footballing frolics straight to your inbox every week.
Thank you for signing up to Four Four Two. You will receive a verification email shortly.
There was a problem. Please refresh the page and try again.