It should be the handle is loose. You will need the proper size allen key to first loosen the handle, pull it off and reset it with the set screw facing the flat section of the stem sticking out of of the faucet. The set screw should be at the bottom of the handle facing you.
Sounds lie it's time to buy a new faucet. Between the cost to repair the old one and the aggravation, it's probably not worth the bother.