(CNN) -- Facing questions on his position about federal disaster aid Tuesday, Sen. Tom Coburn of Oklahoma confirmed he would want to offset any relief money with cuts to other parts of the budget but quickly added now was not the time do be discussing such matters.
"It's insensitive to even talk about it now," Coburn said in an exclusive interview with CNN's Chief Congressional Correspondent Dana Bash on Capitol Hill. "It just shows the crassness of Washington versus the sensitivity that we need to have."
His comments came as he was walking out of a meeting with the Oklahoma congressional delegation, who were on their way to Joint Base Andrews to get on a military plane for Oklahoma to survey the damage and assess and help with the federal response.
Earlier this year Coburn and fellow Republican Sen. James Inhofe were two of 36 senators who voted against a large chunk of the massive federal relief bill for Superstorm Sandy, citing concerns about the hefty price tag.
Coburn spokesman John Hart later told CNN his boss' concern is the same now as it always is when a disaster hits -- he wants to make sure federal aid goes where it is needed -- and is not wasted.
"Many times he has seen aid not targeted towards disaster. There is waste and inefficiency. He is not against federal relief, he just wants to make sure it's spent right. And yes, he has always said anything borrowed should be offset somewhere else in the budget," said Hart.
While the delegation was racing to a van to get to the airport -- Inhofe told CNN he thinks it's possible Congress may not even have to pass a new funding bill, because FEMA may have enough funds in its existing budget to take care of relief and recovery in his state.
"We wouldn't need an additional appropriations bill. Everything's in place right now," Inhofe told CNN, though he quickly added his assessment could change after he gets to Oklahoma.
"We do have both individual and public assistance already approved. [Oklahoma Governor] Mary Fallin called in yesterday and obviously we, we qualify for everything, any type of emergency funding, in two categories -- public, that's individuals, helping to rebuild houses and that type of thing. And individual. So that is there and we're doing well," said Inhofe.
And Coburn said in a statement earlier in the day that as "the ranking member of Senate committee that oversees FEMA, I can assure Oklahomans that any and all available aid will be delivered without delay."
Inhofe and Coburn voted against the $50 billion portion of Sandy relief, as it was not offset by other cuts, but they did not object to $9.7 billion that was passed separately.
President Barack Obama signed a disaster declaration for tornado relief late Monday night, making available federal funds for the area.
As first responders still sift through the rubble of storm damage in Oklahoma, House Speaker John Boehner also faced questions in Washington Tuesday over how Congress will respond in support of federal aid efforts, given that some in the GOP resisted previous relief legislation without spending cuts elsewhere.
Asked three times about disaster relief, Boehner reiterated that Capitol Hill will "work with the administration on making sure they have the resources they need to help the people of Oklahoma."
Inhofe earlier in the day pointed to the devastating 1999 tornado that struck the Oklahoma City area, which still ranks as one of the 10 costliest tornadoes in the U.S. since 1950 and caused $1.3 billion in damage. The May EF5 twister killed 36 people and smashed some of the same communities as Monday's tornado.
"But there are little things you can could do," the fiscal conservative said on CNN's "Starting Point," regarding this week's storm. "The Salvation Army and the Red Cross have a number where you can make small contributions. It's going to be necessary to raise a lot of money. We had to do this in 1999."
Republican Rep. Tom Cole, whose district includes the area hardest hit Monday, supported the Sandy aid bill and said the people in his home state are going to need as much assistance as they can get.
"This is just like Katrina or Sandy - these people are going to need help longer term," he said Tuesday morning on NPR.
Asked if they'll need federal funds, Cole said yes, adding that these situations are what disaster relief is for and said he was proud he had voted for both parts of Sandy relief.
Cole added that the president expressed his condolences when the two spoke on the phone Monday night.
Republican Rep. Frank Lucas of Oklahoma also voted in full for the Sandy bill--voting against the vast majority of their caucus--while Rep. James Lankford supported only the $9.7 billion.
The others in the Ohio delegation--Rep. Jim Bridenstine and Rep. Markwayne Mullin--voted in opposition to both parts of the funding.
Coburn told CNN in December that 64% of the Sandy relief money would not be spent before 2015. "This is a stimulus bill, not an emergency bill," he said.
Coburn said $20 billion to $22 billion probably was enough money to meet the immediate needs of Sandy victims, and more money could be approved later.
On the question of offsetting the total Sandy cost by making equivalent spending cuts elsewhere, Coburn said that despite the fiscal crisis facing the country right now, he doubted Republicans would succeed in getting cuts elsewhere.
CNN's Ashley Killough, Rachel Streitfeld, Lisa Desjardins, Deirdre Walsh and Ted Barrett contributed to this report.