We study from a finite-sample viewpoint the problem of building tests and simultaneous confidence bands for cumulative distribution functions (CDFs), continuous or discrete. We emphasize procedures based on reweighted empirical distribution function (EDF) with shrinking bandwidths in the tails of the distribution. Since weighted statistics may have a problematic behavior when scaling factors are small (or zero), we propose to use regularized statistics. We consider a wide class set of modified EDF-type statistics, and give general characterizations of their distributions in the case of i.i.d. observations, so that the relevant distributions can be simulated in finite samples. We show that test criteria in the class studied can be implemented through the technique of Monte Carlo tests (MCT), so that the level is fully controlled in finite samples, irrespective of whether the tested distribution is continuous or discrete, without the need to establish an asymptotic distribution. Standard criteria such as the Kolmogorov-Smirnov (KS), Anderson-Darling (AD), Eicker (E), and Berk– Jones (BJ) statistics are covered as special cases. Confidence bands are built by inverting sup-type goodness-of-fit test statistics. We show that the bands based on regularized AD-type and E-type statistics have closed forms which are especially easy to compute, without nonlinear optimization. For continuous variables, the null distributions of the statistics do not depend on the CDF tested. For noncontinuous distributions, we show that the MCT approach transparently controls test levels irrespective of the distribution tested. We also establish monotonicity properties (based on nesting image sets) and a general dominance result, so continuous critical values are valid (conservative) critical points. We show in Monte Carlo simulations that the proposed regularized goodness-of-fit tests and confidence bands are numerically tractable, reliable and yield power and precision improvements over standard procedures. The proposed methods are applied to the distribution of households’ consumption in Kenya.