They do but not in the same way as adults as they don't have the space or the knowledge to draw on.
Studies have shown that for babies dreaming is part of their development and provides a pathway to their learning for language and other vital activities they need as they grow. They can't dream about superheros or similar because they have not yet learned about them but they definitely enter that dreaming stage of REM as we can witness by their twitching and eye rolling.