What were the benefits of arithmetizing syntax for Gödel? What did the arithmetization of syntax allow for Gödel that was otherwise not possible?
Arithmetization of syntax allows Gödel to show that statements about number theory are also statements in number theory. This allows him to construct self-referential statements about number theory in a simple way. This in turn allows him to show that Self-reference is inevitable, and that it is impossible to avoid self-reference when trying to construct complete formal axiomatic systems rich enough to describe arithmetic (And thus refuting Russell's logicist project).
There are many resource, but a good informal one which accessible to mathematicians and non-mathematicians alike is Douglas Hofstadter's "I am a Strange Loop" Chapter 10 - Gödel's Quintessential Strange Loop.
Self reference. When a formula can be encoded by a number and also take numbers as arguments, it can also take its own number as an argument, allowing to formulate something like "this sentence is not provable".