The Center of Gravity or Cg is the point that behaves as if all the mass was concentrated in that one spot. Simply put it is the point at which you could balance the rocket on your finger. The Cg is also the point around which the rocket rotates. This becomes very important for the rocket's stability as you will soon see.
Center of Pressure
The Center of Pressure or Cp is similar to the Cg. The Cp is the point at which the forces from the air pushing on the vechicle seem to be focused.
Why Does This Matter ?
When a rocket is in flight, the force of the air pushing on it keeps the Cp behind the Cg. This is important because the rocket is pushed by the force of the exhaust coming out the back. If the Cp was in front of the Cg, the rocket would want to spin around (so that its back was pointing forwards).. However, because the exhaust is always pushing the rocket forward, it spins around again and again... and you get the picture from there.
Fins
So now that you know all about Cgs and Cps, it should make perfect sense what fins are for.
Still a little confused? The fins are extra surface area near the back of the rocket. This brings the Cp further back on the rocket without significantly moving the Cg..
Calculating Cp
Calculating a Cp is rather involved. The easiest way to do it is by using Barrowman's Equations, although these are still fairly complicated. I was going to explain them, but I found a site that does a better job than I could ever do.