Instrumentalism is a pragmatic school of thought which asserts what is real (ontology) and what is true (epistemology) are ideas that aren't answerable and thus is an antirealist position. This is contradistinction to scientific realism.
From WP's entry on instrumentalism:
According to instrumentalists, a successful scientific theory reveals nothing known either true or false about nature's unobservable objects, properties or processes.
When you hear instrumentalism, what you should understand in lay terms is the fundamental assumption that what the universe is "really like" is a meaningless question, and it's better to focus on tools or instruments that enlarge our understanding of science generally. One example of those tools would be model theory which might provide answers about natural language theories in science.
Operationalization is a different sort of beast, and it exists to track down particularly abstractions. Most people can agree on what a horse is, because a horse is a natural kind which any 3-year old could identify. But ask that same child to define the Genus Equidae and the conversation will go nowhere, because how does one define a genus? In biology, for instance, there is a theory known as cladistics which purports to define the extension of Equidae. How does one build this superset? By building trees regarding measures of anatomical and physiological interest. Is this the only way to measure things and build supersets of animals? Nope. One can use phylogenetics which measures different aspects. So, how you measure horses and animals that are horse-like (zebra? deer?) determines the category. The procedures of measurement are given by operational definitions and the objects of their processes are any phenomena that can be measured. In essence, operationalization is a way to show something is real by having procedures to measure it.
A more abstract example is the psychological example of the Five-Factor Model which claims there are 5 properties of personality: openness, conscientiousness, extroversion, agreeableness, neuroticism (OCEAN). Since you can't see or measure these directly (they are abstractions), in order to prove they exist means you have to have reliable measurements of related phenomena; this is the basis of the operational definition.
In research design, especially in psychology, social sciences, life sciences, and physics, operationalization is a process of defining the measurement of a phenomenon that is not directly measurable, though its existence is inferred by other phenomena.
So, your concepts aren't entirely unrelated. Both are about things, but the the first concept is a metaphysical position in science which rejects the proposition that things can be real and known (usually interpreted as true). The second concept says that we can define real things by specifying rigorously methods of indirect measurements allowing others to replicate and provides a way to demonstrate that abstractions are things too; hence it presumes things exist even if they are unobservable or abstract and is a realist method.
