Sparse PCA is one of the most popular tools for the dimensional reduction of high-dimensional data. Although many computational methods have been proposed for sparse PCA, Bayesian methods are still very few. In particular, there is a lack of fast and efficient algorithms for Bayesian sparse PCA. To fill this gap, we propose two efficient algorithms based on the expectation–maximization (EM) algorithm and the coordinate ascent variational inference (CAVI) algorithm—the double parameter expansion-EM (dPX-EM) and the PX-coordinate ascent variation inference (PX-CAVI) algorithms. By using a new spike-and-slab prior and applying the parameter expansion approach, we are able to avoid directly dealing with the orthogonal constraint between eigenvectors, and thus making it easier to compute the posterior. Simulation studies showed that the PX-CAVI outperforms the dPX-EM algorithm as well as other two existing methods. The corresponding R code is available on the website https://github.com/Bo-Ning/Bayesian-sparse-PCA.