In the maximally formal sense logics are defined with only one connective and negation. Most often the connective is either "and" or "implies". The reason this is allowed is because all of the other connectives can be constructed out of a series of negations and the chosen connective, you can think of the other connectives as short hand.
A v B is shorthand for ¬(¬A ∧ ¬B) in a logic defined with negation and conjunction.
It is entirely similar to functions, predicates, and variables. In the maximally formal sense variables are not called "x", "y", and "z". They are all called x with subscripts running from 1 to however many variables exist. Function letters and predicate letters are the same, they are not truly written "F", "G", or "H", but instead are written with a superscript which labels how many arguments it takes and a subscript also running from 1 to the total number of functions or predicates. The subscripts differentiate them in the same way "x" is differentiated from "y".
In any advanced textbook on logic you will find that initially the authors present the idea in the most formal way and then relax the constraints so that we use different letters instead of numbered letters for the sake of convenience. It is similarly done for connectives as well. As an example, from Saul Kripke's Elementary Recursion Theory and Its Application To Formal Systems:
We have described our official notation; however, we shall often use an unofficial notation. For example, we shall often use 'x', 'y', 'z', etc. for variables, while officially we should use 'x1', 'x2', etc. A similar remark applies to predicates, constants, and function letters. We shall also adopt the following unofficial abbreviations:
(A ∨ B) for (~A ⊃ B);
(A ∧ B) for ~(A ⊃ ~B);
(A ≡ B) for ((A ⊃ B) ∧ (B ⊃ A));
(∃xi) A for ~(xi) ~A.
In this excerpt "⊃" is the symbol for implication which is the connective he chose to define the logic with, "≡" is the symbol for biconditional, "~" is the symbol for negation and (x) is the universal quantifier. The numbers next to the variables in the beginning as well as the i next to the universal quantifier are supposed to be subscripts, however the philosophy.SE sadly does not allow for mathjax typesetting.
So in the most formal sense logics only have either "ors" or "ands" or another connective. However, that might seem confusing since colloquially we do have both. The answer as to why we do have the concept of both of these comes from the logical study of natural language. Aristotle's logic is really the progenitor to modern logic, although modern logic is quite different. From the SEP article on the same subject:
In On Interpretation, Aristotle argues that a single assertion must always either affirm or deny a single predicate of a single subject. Thus, he does not recognize sentential compounds, such as conjunctions and disjunctions, as single assertions. This appears to be a deliberate choice on his part: he argues, for instance, that a conjunction is simply a collection of assertions, with no more intrinsic unity than the sequence of sentences in a lengthy account (e.g. the entire Iliad, to take Aristotle’s own example). Since he also treats denials as one of the two basic species of assertion, he does not view negations as sentential compounds. His treatment of conditional sentences and disjunctions is more difficult to appraise, but it is at any rate clear that Aristotle made no efforts to develop a sentential logic. Some of the consequences of this for his theory of demonstration are important.
Aristotle agrees with modern logic that conjunctions, for example, are not atomic formulae. They are compound formulas and as such they consist of subformulas. Really, the introduction of connectives comes from our natural use of words such as "and" and "or". We defined "and" as trying to represent what we logically mean when we say "It is raining AND it is cloudy". What does the "and" mean in that sentence? Well it means that both of those atomic formula, the assertions "it is raining" and "it is cloudy" are true at the same time. Therefore we structured the logical "and" to reflect a common connective we use when dealing with propositions, or assertions, in natural language. Due to the fact that we can construct all of our connectives out of just a negation and one other connective, the choice of which connectives we use is arbitrary.