Cudatel Voice Delay 3-4 seconds :(

Well looks like the issue is much improved but we are still getting a second to few second delay from time to time causing users to talk over each other.  This doesn't happen every call but is happening.  I've done some digging around and see other Freeswitch users complaining of voice delay.  Makes me wonder if its a freeswitch bug that's carried over to the cudatel.  Or if there is something we are using that is overloading the cudatel.  Phones maybe?  I'm almost starting to feel we need to look at another product.