In his 1936 paper, Turing explains that humans compute by manipulating symbols that are external to the human brain (humans compute with pen and paper). Electronic digital computers do the same thing - the symbols are external to the machine: printed on or by attachments: on keys, displayed on screens, printed by printers. Why wasn't the human idea of computing with external shapes simply applied to the machine? Why did Turing say that the machines also internally manipulate internal symbols? Why apply the concept of external manipulation to what happens inside?
I think perhaps the issue at hand is the word symbol. A symbol is not a picture. A symbol is a representation of some other concept. We might talk about the queen of England being a symbol. We might talk about a gift or action being a symbolic gesture of remorse. Even when pictures are used they can represent different things: usually 5 represents the number of toes on a typical human foot but in a paint by numbers it could represent a pleasant sort of green.
It is in this broader understanding that computers, whether modern physical machines or theoretical ones like the turing machine, manipulate symbols. Those high and low voltages in the circuits of a RAM chip represent other things. What they represent could be anything from the colour of a pixel on screen to the health of a player in a game.
Sadly I still am not yet allowed to comment, so I'll do so by an answer to @Roddus who says
The question is still there: why apply the idea of manipulation of external interpretable shapes to the uninterpretable things on the inside?
Any programmer who has had to analyse program dumps will know that at least in theory it is all interpretable.
The internal state of the human is harder to be certain about, which is maybe why he [Turing] treats them differently.
Philosophically it is informative to speculate about both kinds of data manipulation, to try to understand what goes on beneath the surface. But to do that you need to have a full set of theories covering existence, time, metaphysics, consciousness, AI, logic, and set theory probably. Maybe that was beyond the scope of his paper.