We present radio, optical, ultraviolet, and X-ray observations of the isolated, thermally emitting neutron star RX J0720.4-3125 using the Parkes radio telescope, the Very Large Array, the Hubble Space Telescope, and the Chandra X-Ray Observatory. From these data we show that the optical/UV spectrum of RX J0720.4-3125 is not well fitted by a Rayleigh-Jeans tail as previously thought but is instead best fitted by either a single nonthermal power law or a combination of a Rayleigh-Jeans tail and a nonthermal power law. Taken together with the X-ray spectrum, we find the best model for RX J0720.4-3125 to be two blackbodies plus a power law, with the cool blackbody implying a radius of 11-13 km at an assumed distance of 300 pc. This is similar to many middle-aged (105-106 yr) radio pulsars such as PSR B0656+14, evidence supporting the hypothesis that RX J0720.4-3125 is likely to be an off-beam radio pulsar. The radio data limit the flux at 1.4 GHz to be less than 0.24 mJy, or a luminosity limit of 4πd2F < 3 × 1025dEquation or symbol description not available ergs s-1, and we see no sign of extended nebulosity, consistent with expectations for a pulsar like RX J0720.4-3125.