Accurate distance measurement in 3D confocal microscopy is important for quantitative analysis, volume visualization and image restoration. However, axial distances can be distorted by both the point spread function (PSF) and by a refractive-index mismatch between the sample and immersion liquid, which are difficult to separate. Additionally, accurate calibration of the axial distances in confocal microscopy remains cumbersome, although several high-end methods exist. In this paper we present two methods to calibrate axial distances in 3D confocal microscopy that are both accurate and easily implemented. With these methods, we measured axial scaling factors as a function of refractive-index mismatch for high-aperture confocal microscopy imaging. We found that our scaling factors are almost completely linearly dependent on refractive index and that they were in good agreement with theoretical predictions that take the full vectorial properties of light into account. There was however a strong deviation with the theoretical predictions using (high-angle) geometrical optics, which predict much lower scaling factors. As an illustration, we measured the PSF of a correctly calibrated point-scanning confocal microscope and showed that a nearly index-matched, micron-sized spherical object is still significantly elongated due to this PSF, which signifies that care has to be taken when determining axial calibration or axial scaling using such particles.