There are two topics to be distinguished here for discussion: (1) the object language vs metalanguage distinction, and (2) the use of metavariables (which may be distinguished by using a different font, different color, Fraktur!, etc) in logic. I will limit myself to a few brief remarks about (1) and suggest that you look at the works cited at the bottom, and talk about some uses of metavariables in more detail.
§1. Object-language vs Metalanguage
Whenever you have two languages Lo and Lm such that the expressions of Lm are used to talk about the expressions of Lo, the two languages are respectively called 'the object language' and the 'metalanguage'. For example, so far I've been using English both as an object language and a metalanguage. You can notice the spots where I've gone beyond the object level to the meta level by finding the quoted expressions ' 'metalanguage' ' and ' 'the object language' '. Notice that I used double quotes there, because now I'm using English to talk about expressions (e.g. ' 'metalanguage' ') that are used to talk about expressions (e.g. 'metalanguage') of English.
§2. Two uses of Metavariables
Teller's boldface capitals are used as metavariables for arbitrary sentences of his object language. Suppose the object language is that of propositional logic:
Definition 1. (Lo) Given a propositional letter 'p', the language of propositional logic is generated by the following grammar: φ := p | φ′ | ¬φ | (φ ∧ φ).
This tells us that the object language we're working with has the following sort of primitive expressions: "p", "p′", "p′′", "p′′′", etc., as well as compound expressions formed by logically connecting the primitive expressions, e.g.: "p ∧ p′", "¬p′′′", "(p ∧ p′′)", and so on. Now suppose that we want to introduce a helpful connective such as "∨" (inclusive disjunction) into our discussion. I didn't make the alphabet explicit, but Lo does not include the symbol '∨' among its symbols, so we are denied the option of simply positing the following equivalence for all substitutions of 'p', 'p′':
(2)* (p ∨ p′) ↔ ¬(¬p ∧ ¬p′),
because while "¬(¬p ∧ ¬p′)" belongs to Lo, "(p + p′)" is not part of the object language, which is defined explicitly by (Definition 1) as the smallest set containing 'p' and closed under 'tallying', negation and conjunction. But '∨' is available as a symbol to us at the meta level, so following Teller's convention about using boldface letters for metavariables, we could express (2)* correctly as follows:
(3) (p + p′) ≡ ¬(¬p ∧ ¬p′),
where I'm using '+' instead of boldface '∨' (because bold and or don't seem to work well together here), and we're supposing that (3) is closed under uniform substitution. The '≡' symbol is a syntactical equivalence relation that says that whenever you have an expression of form 'φ ≡ ψ' with φ possibly containing metavariables in it and ψ in Lo, you can replace φ with ψ. For examples let's start with a demonstration of the closure under uniform substitution aspect. This sentence:
(4) (p′′ + ¬p′′),
while not identical to the left side of (3), is a substitution instance of it, because it is the left side of (3) but with 'p′′' substituted for 'p', and '¬p′′' substituted for 'p′'. Therefore, (4), which again, is not in Lo, is syntactically equivalent to an expression, specified by (3), that is in Lo, namely:
(4′) ¬(¬p′′ ∧ ¬¬p′′).
This talk of 'closure under uniform substitution' is a cannon for shooting birds, so we want to simplify things. The natural way of doing it, as Teller suggests, is to use boldface letters (we'll use Greek instead) to denote metavariables that stand for arbitrary expressions in Lo. For example, we could reformulate (3) in a way that the condition about closure is no longer needed as follows:
(3′) (φ + ψ′) ≡ ¬(¬φ ∧ ¬ψ′).
Now (3) defines a syntactical equivalence between or-expressions in the metalanguage and their De Morgan equivalent and-expressions in Lo without using any 'p's and closure conditions.
Carnap, R. (1958) Introduction to Symbolic Logic and Its Applications; look at p. 5, §12, §15.
Goldfarb, W. (2003) Deductive Logic; look at §12.
Quine, W.V. (1940) Mathematical Logic; look at §§4–6.