We study the dynamical excitation that large planetesimals, scattered either by Neptune or Jupiter, could have provided to the primordial Edgeworth-Kuiper belt and the asteroid belt. Using both a refined Monte Carlo approach and direct numerical integrations, we show that the Monte Carlo method is useful only to give a qualitative insight on the resulting excitation, but cannot be trusted from a quantitative view point. According to our direct integrations, Neptune-scattered planetesimals of mass from few tenths to one Earth mass could have ejected most of the bodies from the primordial Edgeworth-Kuiper belt, thus explaining the large mass deficiency of the present belt up to about 50~AU. The remaining bodies are left on orbits with eccentricity and inclination comparable to those observed. This dynamical excitation is not restricted to the inner part of the belt but may extend to 100~AU. We also show that Pluto has too small a mass to destabilize the motion of other bodies in the 2:3 mean motion resonance with Neptune. The same mechanism involving Jupiter scattered planetesimals of about one Earth mass can excite the outer asteroid belt, hence depleting it of most of its primordial mass. However this fails to excite the inner belt. In the case where the planetesimals are isolated by mutual gravitational perturbations on long lived main belt-like orbits, safe from Jupiter's encounters, the resulting asteroid belt is very similar to the presently observed one, in terms of mass deficiency, excitation in eccentricity and inclination, and radial mixing. Pallas-like bodies are also obtained. However the decoupling of planetesimals from Jupiter on well behaved orbits is rather improbable (2\% of our simulations), and the resulting asteroid belt is very critically dependent on the mass of the scattered planetesimals and their residence time in the belt.