In my experience, hole saws usually make a hole that is close enough to their stated size that it typically works. For greater precision, pick up a set of forstner bits. They are great for drilling flat bottom holes in wood at a known diameter.
For drilling insert holes, use the same sized forstner bit as the diameter of the insert, and drill down to the appropriate depth (typically just short of the height of the insert so it can be sanded flush later) and then use the dimple in the center created from the pointed tip to drill through the rest of the playfield with a forstner bit one size smaller. That will leave a small lip for the insert to sit on.