The move, approved by the executive committee of world football's governing body on Friday, is potentially controversial as the competition is not part of the international match calendar and clubs already believe they are asked to release their players too often.
However, the executive committee softened the blow by agreeing to proposals from European clubs to reduce the number of international matches and set up an insurance scheme to compensate clubs when their players are injured playing for their countries.
"The executive decided that the release of under-23 players by the clubs shall be compulsory," FIFA said in a statement.
The 2008 Olympic tournament was marked by a tug-of-war between Barcelona and Argentina over the release of Lionel Messi, which went to the Court of Arbitration for Sport (CAS).
CAS ruled in Barcelona's favour but the Catalan club then decided to release him in any case and Argentina went on to win the tournament.
The Olympic tournament, which clashes with the European pre-season, is restricted to under-23 players although each team can field up to three over-age players.
The tournament at London 2012 kicks off the competitive action at the Olympics on July 25.
FIFA President Sepp Blatter said the committee had given its thumbs-up to the insurance scheme, which will now go before the annual Congress in May.
"For the first time, all of these players will be insured, as of the September 1 from this current year," Blatter told reporters.
"This is a novelty, it is to the benefit of everyone, from the smallest such as Djibouti, to large federations such as Italy. Everyone stands to benefit from this."
FIFA said the provisional budget for the project was $100 million and would cover all players involved in full internationals played on officially approved dates.
The European Club Association (ECA) has angrily criticised FIFA over the issue, citing the case of Bayern Munich forward Arjen Robben who aggravated a muscular injury at the 2010 World Cup and was sidelined for several months of the following season.
The new match calendar, if approved by Congress, would limit the number of full internationals to 18 per two-year cycle from 2015. This compares to 24 international dates in the 2011/12 cycle.
These 18 games would be divided into so-called "double dates" each featuring two international matches in the period of five days.
This compares to the 24 international dates scheduled for the 2011/12 cycle and 22 for 2013/14.
The controversial February and August friendly dates, the latter having been described as "nonsense" games by ECA president Karl-Heinz Rummenigge, would be scrapped.
