Try running cgminer --benchmark and see what u get, it should be a good score. Mining BTC seems to work fine under OSX but scrypt is really bad for some reason.
If you run the same cgminer --benchmark --scrypt you will get ~10kh/s or less, tweaking gets it upto a max of 100 or so on the 5770 which is still bad, I get about 150mh/s on my debian server using a 5770.
I am trying with a 7850 on OSX (Yes hackintosh). On linux I can get over 300 kh/s just adjusting the -I setting.
On OSX I get massive HW error setting the I over 11. At best I can get about 60kh/s
It seems like something is really broken with the scrypt implementation on OSX.