The jury had remained deadlocked for days with the seven men and five women unable to reach a unanimous decision after some 53 hours of deliberations in Norristown, Pennsylvania.
Mr Cosby, 79, was tried for drugging and sexually assaulting Andrea Constand in 2004. His lawyers had argued the sex was consensual.
The US comedian could now face new proceedings.
The prosecution said they were pursuing a fresh trial.
Dozens of women accuse Cosby of assaulting them, but statutes of limitation rules mean he was allowed to be tried for Ms Constand’s allegation only.
Had he been found guilty of three counts of aggravated indecent assault, which allegedly took place at his Philadelphia home 13 years ago, he could have faced up to a decade in prison.
Announcing the decision, Pennsylvania judge Steven O’Neill reminded Mr Cosby that he remains charged and on bail, despite the mistrial.
One of Mr Cosby’s lawyers, Brian McMonagle, applauded the decision.
“The judge is right: justice is real,” the counsel said
“We came here looking for an acquittal. But like that Rolling Stone song says ‘you don’t always get what you want’. Sometimes you get what you need.”
But a lawyer representing many of his accusers, Gloria Allred, said she was hoping the prosecution would push for a retrial.
“We can never underestimate the blinding power of celebrity but justice will come. I hope that the prosecution will try this case again.”
The jury had been instructed by the judge to work into the weekend to reach a verdict, after they first revealed that they were deadlocked on the case on Thursday.
But the panel returned yesterday to tell the judge they were still deadlocked on all three counts.
The mistrial will be seen as a blow to the dozens of women who have accused Mr Cosby of sexual assault, a number of whom were present in court last week awaiting the verdict.
The accuser, Constand Andrea, took the stand during the trial, telling the court the assault had left her feeling “humiliated” by someone she considered a friend and mentor.
Cosby, who faces at least four separate civil lawsuits, refused to testify at the trial.
Logging you in...
