Viacom Inc. and the original owners of the studio that makes its Rock Band video games are enmeshed in a legal battle, wrangling over hundreds of millions of dollars at the same time that sales of the games are plummeting.

Former shareholders in Harmonix Music Systems have filed a lawsuit that has triggered a high-stakes arbitration with Viacom just as the media giant is attempting to sell the video game maker after years of losses. The legal and financial morass could likely complicate that effort.

The two sides have gone to an independent arbitrator over additional payments based on Rock Band sales that the Harmonix shareholders believe are owed them under the terms of the 2006 acquisition of the studio by Viacom. The amount in dispute totals hundreds of millions of dollars, according to people familiar with the situation.

At the same time, a proxy for the original Harmonix owners bought out by Viacom is suing the New York-based owner of MTV Networks and Paramount Pictures in Delaware Chancery Court for more than $13 million for alleged unfair business practices.

The legal conflicts come amid a rapid downturn in sales for music video games in the last two years, which led Viacom to decide to exit the business.

MTV acquired Harmonix, which created the hit Guitar Hero games for Activision, for $175 million plus potential future payments based on performance.

After initially estimating in a regulatory filing with the Securities and Exchange Commission that the 2007 performance payments would be $208.7 million, Viacom ended up paying only $150 million after the media company concluded that Harmonix's performance was not as strong as it originally thought. Nonetheless, two months later, according to a November 2008 SEC filing, Viacom said it expected the 2008 performance payment to be more than the $150 million it paid the previous year.

That estimate also proved overly optimistic.

Despite more than $1 billion in retail sales to date, Viacom has been unable to make a profit on Rock Band games. The company has said that is partly due to the costs of producing and shipping the guitar- and drum-shaped controllers. The trend worsened in 2009 as sales of music video game began falling.

By early 2010, Viacom had radically changed its view on what it owed Harmonix's shareholders. In an SEC filing, it made the claim that it was owed a "refund of a substantial portion of amounts previously paid."

But Viacom's legal gambit is a high bar to meet, experts said.

Ehud Kamar, a professor at the USC School of Law, said such a move is very unusual. "Earn-outs are quite typical when buying a private company," he noted, "but it’s very difficult to get money back in that context."

People close to the matter said the "substantial portion" in Viacom's claims represents nearly all of the $150 million. They added that Viacom believes it does not owe any performance payment for 2008.

Walter Winshall, an early investor in Harmonix who was designated proxy for more than 150 shareholders in the purchase agreement, disputed Viacom's calculation, according to the lawsuit.

Although the suit does not specify how much Winshall believes shareholders are owed, people close to the matter said he has claimed it is hundreds of millions of dollars more than the $150 million Viacom has already paid out.

In his lawsuit, Winshall alleges that Viacom breached a covenant of good faith and fair dealing by failing to renegotiate an agreement with Electronic Arts Inc., which distributed and marketed Rock Band, on more favorable terms after the the game's launch. The media company allowed the deal to run its course through 2009, the suit states, because an earlier renegotiation would have increased the performance payment due Harmonix shareholders.

The Delaware lawsuit also alleges that Viacom has improperly held back $12 million from the initial $175- million payment against potential losses stemming from four patent infringement lawsuits filed by companies including Activision and Gibson Guitar Corp. Winshall claims that Harmonix shareholders are indemnified against any such lawsuits under the purchase agreement.

A Viacom spokesman said in a statement, "We have been more than fair with Mr. Winshall, who is trying to rewrite the terms of our contract.  We intend to defend this vigorously."

The lawsuit comes at a time when retaining Harmonix employees, including Chief Executive Alex Rigopulos and Chief Technical Officer Eran Ergozy, will be important to Viacom in order to successfully sell the studio. One person close to Viacom said the dispute is primarily with outside investors and that the company believes it has retained good relationships with Harmonix staff.

-- Ben Fritz

Related:

Viacom looks to sell Rock band developer after years of losses

Yeah, yeah, yeah: Viacom wants back bonus paid to Rock Band game maker

Beatles: Rock Band sales slow over holidays as music video game genre bombs

Viacom hopes the Beatles: Rock Band game sets stage for rebound

[Update: An earlier headline on this post said: "Viacom and Rock Band creator fighting over hundreds of millions of dollars in high-stakes legal battle." In fact, Viacom is fighting with former shareholders of the creator of Rock Band.]

Photo: Harmonix employees demonstrate Rock Band 3 at the E3 industry trade show in June. Credit: Lawrence K. Ho / Los Angeles Times