The total momentum* of a system (rocket and exhaust) is 0 (initially): the rocket and exhaust are not moving. When the rocket takes off, its body moves forward, gaining positive momentum, while the exhaust is gaining negative momentum.
* Momentum is a vector quantity that describes how hard it is to stop an object. Or, in other words, it is the quantity of motion that an object has.