I'm not sure what you are driving, but it could very well be the IAC (idle air control) actuator. This pesky thing gets dirty and and stops actuating properly. It is designed to allow air into the intake or to block the extra air. This gives the computer an extra tool to control idle speed. There are other inputs to the computer that may take a part in affecting the control of the IAC such as brake switch, speed sensor, throttle position sensor, air conditioning, and coolant temperature sensor. Most of those things will trigger a fault code. If you are not getting an engine light, then try disconnecting the IAC to see if the engine idle changes. If it doesn't change, it is likely sticking. Some people are successful in cleaning them and getting them to work. Most just replace the IAC. Also, a vacuum leak could be causing the same issue. Any vacuum leaks could be bypassing air and the computer is trying to compensate for the air leak.