I don't use crits or fumbles. I think they are fun and neat, but I don't use them for these reasons.
1. If you base crits and fumbles off the attack roll (the most common system being a 20 is a crit, and a 1 is a fumble), then the less likely you are to hit, the more likely you are to get a crit - so if you actually need a 20 to hit, you always crit. The reverse is true for fumbles.
To 'fix' this, you would have to roll a second time for each hit or miss, to see if it was a crit or fumble.. and then roll a third time, if it was a crit or fumble to see what happened.
Combat takes a long time in my game, since it's so high level and the groups tend to be large, so I don't want to so the game down any more.
2. A combat system like this rewards those who fight with weapons, and thus, punishes those who are primarily spell casters. Why shouldn't you have a chance to crit with a Fireball? Again, the game slows down.