Multi-task Bayesian optimization (MTBO) is an effective method for optimization involving multiple tasks. In this work, we propose competitive multi-task Bayesian optimization (CMTBO) where the primary task is not pre-defined. Instead, it is chosen as the task whose optimal solution is the best among all tasks. We adopt the multi-task Gaussian process to model the multiple tasks and design two criteria to select the query point as well as the task to query in each iteration. The two criteria differ in whether they select the task and the query point jointly or separately. We theoretically analyze the regret bounds for the two criteria and test their performances on several synthetic and real-world problems.