In general, video see-through based augmented reality (AR) cannot change the magnification of camera zooming parameter due to the difficulty of dealing with changes in intrinsic camera parameters. To realize the usage of camera zooming in AR, we propose a novel simultaneous intrinsic and extrinsic camera parameter estimation method based on an energy minimization framework. Our method is composed of the online and offline stages. An intrinsic camera parameter change depending on the zoom values is calibrated in the offline stage. Intrinsic and extrinsic camera parameters are then estimated based on the energy minimization framework in the online stage. In our method, two energy terms are added to the conventional marker-based camera parameter estimation method. One is reprojection errors based on the epipolar constraint. The other is the constraint of continuity of zoom values. By using a novel energy function, our method can estimate accurate intrinsic and extrinsic camera parameters. In an experiment, we confirmed that the proposed method can achieve accurate camera parameter estimation during camera zooming.