Raheem Sterling put the visitors in front in the eighth minute at White Hart Lane after Mario Balotelli had almost headed in a debut goal even before that, and Steven Gerrard struck from the penalty four minutes into the second period of the Premier League encounter.
Eric Dier pulled back Joe Allen for that spot-kick and Andros Townsend was then guilty of losing the ball to Alberto Moreno soon after coming off the bench to allow the Spaniard to race away and score a sensational third.
"It's a very disappointing result and two key moments at the start of each half cost us," bemoaned Pochettino.
"After we went 1-0 down, we stayed in the game and created chances, but in football you need luck at key moments and [it] was difficult."
Tottenham should have had a penalty themselves in the closing stages when Emmanuel Adebayor had his shirt pulled by Dejan Lovren, but it would have only given them the chance to net a consolation score.
Having struggled against the top teams last term, including 4-0 and 5-0 losses to Liverpool, this result burst their early-season bubble following league wins over West Ham and QPR.
However, Pochettino insists it is too early to tell if it will be the same story throughout this campaign, adding: "We need to improve and look forward. It's the beginning of the season and only three games in."
Striker Adebayor echoed his manager's thoughts and expressed a desire to respond.
"We have to find a way to correct the mistakes and move forward. It's very important to bounce back from this," he added.
"We are all disappointed but the season has a long way to go and we have to learn from our mistakes."
