By Rina Chandran

MUMBAI (Reuters) - A deal between India's Reliance Communications and South Africa's MTN Group looked unlikely on Monday, a day before exclusive talks were due to end, as a dispute with Reliance Industries remained unresolved.

Reliance Communications began talks about a tie-up with MTN, sub-Saharan Africa's top mobile operator, in May. A deal would create a global top-10 telecoms firm with operations in about two dozen countries.

The two sides have said they are discussing a business combination and media reports and a source with knowledge of the talks have said this could take the form of a reverse takeover of Reliance Communications with its chairman, Anil Ambani, becoming the biggest shareholder in MTN.

Reliance Industries, India's most valuable firm, has staked a right of first refusal on Reliance Communications shares, but India's number two mobile firm maintains that the claim is untenable.

Reliance Industries and Reliance Communications are run by Mukesh and Anil Ambani respectively, estranged brothers ranked the world's fifth and sixth richest people by Forbes magazine.

Reliance Communications said on Monday it was prepared to meet Reliance Industries next week "to clarify any doubts" adding the meeting was not part of "conciliation" or "dispute resolution".

Reliance Industries said representatives from Reliance Communications had not turned up to a meeting it had set up for Monday to begin a process of mutual conciliation, and so said it "is left with no alternative but to adopt appropriate proceedings against Reliance Communications, as advised."

"Anil may be seeking more time to convince Mukesh," said Emeka Obiodu, senior telecoms analyst at Global Insight in London. He was speaking before Reliance Industries issued its statement on Monday evening.

"But anything short of an unequivocal surrender of the purported first right of refusal by Mukesh will not be enough for MTN. The South Africans will not give up control of MTN to an uncertain future," he said.

Shares in Reliance Communications ended Monday down 4.2 percent at 419.80 rupees in a Mumbai market that rose 0.5 percent.

Shares in MTN closed up 6.6 percent at 129 rand, outperforming a 0.1 percent rise in the JSE Top-40 index.

For more on the talks, see