Ok something really weird happened. I was messing around with turning on and off different collision objects, and suddenly it started working. When I reloaded the project again and did the same type of things, it was back to it's normal weirdness though. Can't for my life figure out what made it suddenly work.