Nvidia control panel on os x lion slowing down system performance

I’m using a MacBook pro with Apple OS X Lion with NVIDIA video drivers. For some reason, I have a feeling that NVIDIA control panel on OS x lion is slowing down system performance. Sometimes, it comes to worst that it even crashes the system and I have no other choice but to force close the applications with my unsaved work and restart all over again. I am wondering if I am the only experiencing this issue with Apple OS X Lion?
