(Reuters) - A unit of Royal Bank of Scotland Group Plc has agreed to pay more than $44 million and enter a non-prosecution agreement to settle a U.S. criminal probe into traders accused of defrauding customers on bond prices.