We study the behavior of a moving wall in contact with a particle gas and subjected to an external force. We compare the fluctuations of the system observed in the microcanonical and canonical ensembles, by varying the number of particles. Static and dynamic correlations signal significant differences between the two ensembles. Furthermore, velocity-velocity correlations of the moving wall present a complex two-time relaxation that cannot be reproduced by a standard Langevin-like description. Quite remarkably, increasing the number of gas particles in an elongated geometry, we find a typical time scale, related to the interaction between the partitioning wall and the particles, which grows macroscopically.