The free-scoring striker equalised on 53 minutes to cancel out a first-half goal from Kidderminster front runner Blair Sturrock.
Agogo also bagged a brace when Bristol Rovers beat Kidderminster 2-0 at the Memorial Stadium on October 23.
Harriers called the tune for most of the first half and were good value for their interval lead.
Their welcome breakthrough came in the 29th minute when determined defender Johnny Mullins won possession halfway in Rovers territory before pushing the ball through to the alert Sturrock.
He confidently charged forward before firing past keeper Kevin Miller for his eighth goal in five starts since arriving at Aggborough in December from Plymouth Argyle.
Harriers other first-half efforts came from Bertrand Cozic, Dean Keates and Chris Beardsley.
Rovers were restricted to just one main first-half effort when Agogo slid the ball wide after a shot from James Hunt had been blocked by Simon Weaver.
Ian Atkins' men stepped up a gear in the second half and were soon rewarded when Steve Elliott and Richard Walker combined to enable Agogo to shoot into the net through keeper John Danby's legs.
Rovers continued to unsettle Harriers and were unlucky in the 70th minute when danger man Agogo hammered a fierce 18-yard shot against the post.
Kidderminster huffed and puffed in a bid to get their noses back in front, but in the end had to be content with their first score draw since October 2 when they were deadlocked 1-1 at home to fellow relegation strugglers Cambridge United.
Harriers breathed a sigh of relief in injury time when Rovers had penalty appeals turned down when Wayne Hatswell appeared to push substitute Lewis Haldane.