Clinton Morrison grabbed two goals in the last half hour - the second in the fourth minute of injury time - to give Birmingham a famous 2-2 draw at Liverpool.
Match reaction
For Liverpool it was a case of another two-goal lead thrown away after Newcastle had clawed their way back from two behind at Anfield last week.
Morrison was vilified here a couple of years ago when he was at Crystal Palace and had criticised Michael Owen's goalscoring ability.
But he returned to take revenge while Owen at the other end was seeing chance after chance go begging.
Danny Murphy and Steven Gerrard had Liverpool on cruise control before Morrison's double strike. For Birmingham, having beaten Leeds in their previous game, it underlined that their return to the top flight may not be as fleeting as many at first feared.
Birmingham spent the opening minutes chasing shadows as they were hit by a tidal wave of attacks, sweeping from side to side at lightning pace.
Michael Owen was robbed of a great scoring chance after he had charged down a Nico Vaesen clearance, only to be judged by a linesman some 30 yards away to have handled.
But in all Liverpool's swirling action and the dominance of Gerrard and Didi Hamann in midfield, chances were rare thanks to Kenny Cunningham and Purse's diligence at the back for the visitors.
In fact, Birmingham managed to grab their breath and get Morrison into the game thanks to much vigorous running and pressing from Robbie Savage in particular.
But having survived the initial barrage, Birmingham found themselves a goal down on 25 minutes thanks to a clumsy foul by Liverpool-born Bryan Hughes on the all action El-Hadji Diouf.
Up stepped Danny Murphy to send a curling free kick over the wall and in off the post.
It didn't take long for Liverpool to put right some of the wastefulness of the first half when they grabbed their second with a sweeping, punishing attack on the break early in the second period.
Diouf surged from his own half and with the defence backing away, he slipped the ball into Gerrard's for the England midfield to drill home his shot off Vaesen's despairing dive.
Birmingham's response was creditable. On 58 minutes Hughes launched a stunning hooked volley that Dudek matched with a wonderful mid-air block.
That inspired Birmingham even more and on 61 minutes the Premiership's new boys did score.
Stern John created the chance on the right after Traore's mistake, and Morrison hooked the ball past Dudek in the six-yard box.
Another sweeping Liverpool move saw Hamann set up Murphy for a shot against the foot of a post.
Liverpool put on Patrik Berger for Diouf, and virtually his first touch was a deflected shot that Vaesen needed to save smartly, as he did at Owen's feet a minute later when Berger put him through.
But Morrison was still to strike again, sending a lopping header into the top corner from Lazaridis' cross.
After the match Steve Bruce praised the 'bottle, courage and honesty' of his battling Birmingham side.
City manager Bruce said: 'We have had a tough start at places like Arsenal
and here, and against Leeds.
'But we have come through it. We just didn't give up and if we keep going
like that we will be okay this season.
'We were two down, we'd been hit on the break for the second, but we came
back and deserved what we got.
'This has given the side belief and that is the hardest thing for a new team
in this division to achieve.
'Six months ago when we came here in the FA Cup soon after I'd taken over, we
just didn't play at all. But we have made lots of positive strides since.'
The Reds also threw away a two-goal lead in their previous home game against
Newcastle, and Houllier said: 'We should have finished the game off from two
up.
'But we have let another two-goal lead slip, and this is our third 2-2 draw
on the trot.
'We should be top of the league by now with at least another four points. But
we have let games slip in the last minutes on three occasions, six points have
been squandered in about nine minutes of play.
'But I don't believe you achieve anything without going through some
adversity, and we have a young side that will learn from this.'