The Young Boys Berne defender was forced to come off just nine minutes into his country's 5-2 Group E thrashing at Arena Fonte Nova after being accidentally kicked in the face by France striker Olivier Giroud.
Von Bergen, who suffered a nasty cut just under his left eye, was taken to hospital in Salvador and it was confirmed by the Swiss Football Federation on Saturday that his tournament is over due to a fractured eye socket.
The 31-year-old will fly home from Brazil and must watch from afar as Switzerland attempt to put their hammering at the hands of France behind them and qualify for the last 16.
Ottmar Hitzfeld's side face Honduras in their final group game next Wednesday, with second-placed Ecuador - level on points with Switzerland, but with a superior goal difference - up against group leaders France.
Get the best features, fun and footballing frolics straight to your inbox every week.
Thank you for signing up to Four Four Two. You will receive a verification email shortly.
There was a problem. Please refresh the page and try again.