A small bank and two conservative advocacy groups said on Thursday that they would file a lawsuit against the new Consumer Financial Protection Bureau, challenging what they call the agency's "unlimited regulatory power."

It is unclear how much of a threat the suit is to the agency created by the 2010 Dodd Frank financial reform act to police consumer products such as mortgages and credit cards.

The challenge, brought by the State National Bank of Big Spring, Texas; the Competitive Enterprise Institute; and the 60 Plus Association, will ask the court to invalidate key parts of Dodd-Frank relating to the CFPB's oversight.

The plaintiffs say parts of the law are unconstitutional because Congress does not have direct control over the CFPB's budget, while the president and the courts lack serious oversight of the agency.

"There is no statute anywhere that so combines the power of all three powers in one bureaucrat," said C. Boyden Gray, who was White House counsel under former President George H.W. Bush and is representing the plaintiffs.