Despite taking the lead at White Hart Lane on Monday, Sunderland slumped to a 5-1 defeat against Tottenham to further dent their hopes of avoiding relegation.
Brown has admitted being at fault for Tottenham's first two goals and knows Sunderland, who are seven points adrift of safety at the foot of the table, are making life far too easy for opposing teams.
"Their player has got behind me for the first and second goals. We can talk all we want to about having to cut out the mistakes, but we have to," Brown told the Sunderland Echo.
"We have to do better when we're going forwards too. We're not helping ourselves.
"Unless we can produce 90 minutes where we're 100 per cent, we can only blame ourselves."
Brown insists Sunderland cannot give up hope of pulling off a great escape ahead of their final seven games this season.
"The table doesn't lie, it's bad," he added. "But we've got to keep going. That's all you can do.
"Of course it’s difficult. You can try not to look at how bad it is, but you do.
"We've got to show a bit of pride in ourselves and keep on grinding it out."
Get the best features, fun and footballing frolics straight to your inbox every week.
Thank you for signing up to Four Four Two. You will receive a verification email shortly.
There was a problem. Please refresh the page and try again.