A core problem in compressive sensing (CS) concerns how to stably recover sparse signals from a small number of measurements in the presence of noise.  It is known that if the noise can be bounded in norm by $\epsilon$, then standard algorithms recover sparse signals with error at most $C \epsilon$.  However, these algorithms perform suboptimally when the measurement noise is structured.  We address a hierarchy of structured noise models.  We first consider the scenario where a CS system acquires a signal of interest contaminated by a corrupting signal. Under the assumption that the corrupting signal is sparse with known support and orthogonal to the signal of interest, we demonstrate that it is possible to efficiently filter out the corrupting signal from the compressive measurements and recover the signal of interest exactly in the absence of additional sources of noise.  We then consider the case where the measurements are corrupted by sparse noise with unknown support.  We propose a simple algorithm that can accurately identify the corrupted measurements and recover the underlying signal, which we dub Justice Pursuit.  Justice Pursuit handles unbounded errors, has no input parameters, and is easily implemented via standard recovery techniques.  We conclude by observing that the main results concerning corruption and justice can be combined to demonstrate that random matrices are democratic, by which we mean that when using random measurement matrices CS is robust to the loss of a small number of arbitrary measurements.