A time-series model in which the signal is buried in noise that is non-Gaussian may throw up observations that, when judged by the Gaussian yardstick, are outliers. We describe an observation-driven model, based on an exponential generalized beta distribution of the second kind (EGB2), in which the signal is a linear function of past values of the score of the conditional distribution. This specification produces a model that is not only easy to implement but which also facilitates the development of a comprehensive and relatively straightforward theory for the asymptotic distribution of the maximum-likelihood (ML) estimator. Score-driven models of this kind can also be based on conditional t distributions, but whereas these models carry out what, in the robustness literature, is called a soft form of trimming, the EGB2 distribution leads to a soft form of Winsorizing. An exponential general autoregressive conditional heteroscedastic (EGARCH) model based on the EGB2 distribution is also developed. This model complements the score-driven EGARCH model with a conditional t distribution. Finally, dynamic location and scale models are combined and applied to data on the UK rate of inflation.