While Rep. Charles Rangel of New York celebrates a major primary victory this week - his toughest challenge in 42 years - the state is stepping in on a number of ballot complaints that could wind up changing the results.

The campaign of Rangel's opponent, New York state Sen. Adriano Espaillat, filed a lawsuit that contends too many ballots were unaccounted for in Tuesday's election.

The state Supreme Court decided Friday to hold a hearing, which is on the docket for Monday, according to state court system.

Espaillat conceded defeat to Rangel Tuesday night, as tallies showed the 21-term Democratic congressman had netted about 45% of the vote, compared to Espaillat at 40%, in a three-candidate race. However, the margin since has narrowed as more votes come in.

"Three days after a winner was declared in this election, there are still votes to be counted," the Espaillat campaign said in a statement. "Our campaign has not been allowed to adequately monitor the Board of Elections' proceedings, as required by law."

Rangel's new district--the state's 13th Congressional District--stretches from East Harlem to the northwest Bronx.

A spokeswoman from the Board of Elections said they hope to have by the end of Saturday an official count of the votes cast Tuesday on voting machines.

However, those will not include an additional 2,494 affidavits that remain outstanding, the spokeswoman, Valerie Vazquez said.

These consist of paper ballots from those whose names were not on voter register lists at respective polling sites and therefore ineligible to vote on a machine. The board has to first verify that those names are in fact registered before their votes can be added to the final count.

Also remaining unaccounted for were 776 absentee ballots, Vazquez said.

She could not yet confirm whether the outstanding number of votes was larger than the margin between Rangel and Espaillat.

"Our policy and procedures require a manual hand count if the margin of victory is less than one half of a percentage point," she told CNN.

Espaillat, a 57-year-old Dominican-American, has used his background to court Latino votes and take aim at the 82-year-old Rangel over his long tenure in Washington and his ethics abuses. If he had won Tuesday, he would have been the first Dominican-American elected to Congress.

Rangel campaign spokeswoman Ronnie Sykes had no comment on the upcoming hearing.

CNN's Ashley Killough, Dan Verello, Jason Kessler and Gabriella Schwarz contributed to this report.