(CNN) -- It was clear from House Speaker John Boehner's weekly address on Saturday that the fiscal cliff negotiations with President Barack Obama weren't going anywhere soon.
Both men left Washington on Friday, leaving the looming spending cuts and tax increases for after Christmas.
They left the capital with an exchange of pointed words that appeared to show there had been no significant breakthrough in the final hours before vacation.
"Unfortunately, the president and Senate Democrats have vowed to reject and veto all of our proposals while failing to offer a responsible solution of their own," Boehner said in his recorded weekly address Saturday morning.
Obama told reporters shortly before taking off for Hawaii: "The challenge that we've got right now is that the American people are a lot more sensible and a lot more thoughtful and much more willing to compromise, and give, and sacrifice, and act responsibly than their elected representatives are. And that's a problem."
Boehner used his address to put the burden for a breakthrough on Obama and Democrats, who he said were not making the tough choices he had put forward.
"The House has done its part to avert this entire fiscal cliff," he said. "On the 10th of May and again on Thursday, we passed legislation that would replace the 'sequester' with responsible spending cuts. We also passed a bill to stop all of the January 1 tax hikes."
But the measure that narrowly passed the House by six votes late Thursday wasn't Boehner's Plan B -- which would have prevented the tax increases at year's end for some income levels -- but a measure to avert spending cuts for military programs that are part of the fiscal cliff bundle.
Boehner called off the vote on Plan B, saying in a statement, "it did not have sufficient support from our members to pass."
The speaker has said Obama's plans include far more in tax increases than spending cuts, and on Saturday he said Obama "refuses to challenge the members of his party to deal honestly with entitlement reform and the big issues facing our nation."
"The American people re-elected President Obama on Election Day. They also re-elected a Republican majority in the House," he said. "In doing so, they gave us all a mandate. It was not a mandate to raise tax rates on families and small businesses. It was a mandate for us to work together to begin solving the massive debt that threatens our country's future."
After calling off the Thursday vote, Boehner said Obama needs "to work with Senator (Harry) Reid on legislation to avert the fiscal cliff."
Obama has called for a "balanced" approach that includes both the tax increases and spending cuts. The across-the-board cuts that would go into effect without action otherwise are considered undesirable by both sides.
Congress designed the sequester as an incentive for the two sides to reach a deal.
Obama and Boehner spoke on Friday, Obama said, their first conversation since Monday, when the two sides exchanged offers that put them closer to a solution. They had each offered concessions on taxes -- Boehner offering an extension of lowered tax rates on income under $1 million, and Obama on income under $400,000 -- and on other aspects of a deal, such as the mechanism for computing entitlement benefits, the Consumer Price Index.
The House speaker sounded a hopeful note in closing.
"Of course, hope springs eternal, and I know we have it in us to come together and do the right thing," he said. "We will continue to work with our colleagues in the Congress and the White House on a plan that protects families and small businesses."