It is shown that the Mean Integrated Square Error (MISE) of a binary classifier is a weighted average of its probabilities of type I (α) and type II errors (β). This provides a foundation for minimizing a linear cost function consisting of a weighted average of α and β to design an optimal classifier. Such a cost function is shown to have the interpretation of a MISE of the classifier under a subjective probability distribution. We derive the closed-form expression of the optimal α for the mean test, provide an equation that can be solved numerically to find the optimal cutoff of the Probit classifier, and illustrate the relevance of the results by simulation. In general, the optimal α for a significance test is different from the conventional 0.05 or 0.01 and the optimal cut-off for probabilistic classifiers deviates from 0.5.