Hope this guides you, I'm not an academic philosopher, your question has a lot of common elements which sometimes require just some deeper philosophical approach. Not trying to change your perspective, but moreover to provide you some ideas and foundations for the discussion.
there was a debate on only believing in what can be proved with science
Science seeks for empirical truth, that is, the truths that correspond to our five-senses experience; you can say science is the way of formalizing experience. It is impossible to believe something that is not experienced: say for example I'm an atheist, I think your religious book has contradictions, and according to the principle of explosion, a single contradiction allows me to prove that God wants evil. What I should believe? in what you say? Why, if the book tells that "if your arm causes you to sin, so cut it off", and you haven't cut your arm off (perhaps you are from another religion, sorry, but this is just an example)? So, we need science. That, on one hand. On the other, science should and MUST not be incompatible with religion. If you remove the contradictions from the book (ALL religious people does that, otherwise, all religious people would have not arms), and if you remove the contradictions with your common experience, science should be 100% compatible with such religion. You can see that the problem is the book, not a consistent/logical belief in God. You can be a scientist and believe in God, if you do it consistently with logic, the laws of nature and the laws of your self.
for example, we should only have laws (juridical ones) that are based
on science (and, therefore, that can be falsified).
Hoo, not so fast. To start, science has nothing to do with laws, except that it is also based on logic. Science tells empirical truths like the 2nd Law of Thermodynamics and that has nothing to do with juridical laws. The problem with juridical law is that it is not necessarily based on physical facts (experience, objectivity, five-senses, science, empirical truth, etc.) but on its thinking counterpart: metaphysics. This is important.
Metaphysics covers all such concepts and rules that refer to ideals, to what is not physical, scientific, measurable, objective (yes, metaphysics is moreover about subjectivity) like beliefs, justice, loyalty, aesthetics, politics, etc. And the metaphysical rules that govern our social life (ho can I live in peace with you) are called morals.
Law is based on morals, not on science. Morals are a set of rules that determine how we can live in peace (e.g. your grandma telling you to be nice and say thank you). Ethics is more or less the formalization of that (e.g. your teacher telling you the same thing in a different way: that it is correct to have respect for the others). Formal law is the same thing, but from a pragmatic and mostly punitive perspective (e.g. the judge telling you the same rules with different words: given that I have proof (that is not necessarily scientific proof) that you've been disrespectful, you should pay this fine).
Being honest, I do not like this, since I am religious, but that alone isn't a good point.
Correct. Never base an argument in your authority (fallacy: ad authoritatem) or in your beliefs (fallacy: biased sample). To discuss, you are forced to use logic. You will always lose an argumentation that is not logical.
I'd like to comprehend if only believing on what can be proved has any
sense and can actually lead somewhere beneficial for society.
It has no physical/objective/empirical sense, but it has an absolute logical/metaphysical/subjective sense. Don't try to prove God with science, by definition, God exceeds the world of things. Even proving God metaphysically is difficult. God is a metaphysical concept, so you can't prove and don't need to prove it scientifically. You need to live with that. And even more, you should consider (not necessarily accept), consider that God might not exist in order to challenge your own logic and have the proper sustain to debate.
A solid believer must not be who has not logically challenged the existence of God (that is just a credulous, a gullible person), a solid believer is who has challenged logically the existence of God and found, logically, a possibility. That's my case.
if you can't prove that something exists, then it doesn't
Correct, but that's like climbing a ladder over the wrong wall. But to start, you need to know what does "to exist" means. In my writings (I wrote a book called Theory of Interaction), I use this concept: existence is defined by interaction. This means that if I-subject (observer) can interact with myself as an I-object (observeD), then I exist (that is precisely the sense of Decartes' cogito ergo sum). If I can interact with black matter, then it exists (sadly, we can't, but we can interact with some unclear facts that signal its existence). More complex: how can I say that Einstein existed? Because I interact with his knowledge, his books, with people that knew him, etc.
Once the concept existence is clear, there is another concept that must be crystal clear: what does "God" means? which super-powers does it has? What is its size? Can it occupy space, physical or metaphysical? Has it a body? Can it think? Using logic? Can it speak your language? If you don't have a clear concept that YOU-YOURSELF can challenge, you are going to lose all discussions.
Now, how do you interact with such a God, so you can metaphysically prove its existence to yourself? That's a very difficult one. But like dark matter, there are facts that tend to show that it might exist.
Think on this: considering the physical and metaphysical facts of our experience, it is as unbelievable for God to exist as is is for it not to exist (can't remember who said that). The more you believe in science, the stronger the argument becomes*, and that leads always to metaphysical considerations, where science is useless. There you have an strong argument.
* ...given that science is an attempt to find empirical truth, and it tends to fail in border conditions, like the case of quantum physics vs. relativity, where logical/mathematical calculations do not match, leading to... (sorry for the pop expression) a possible fault in the matrix! Wait, is it really so? It seems. Check this, which is really amazing: for scientific knowledge to be valid, it should be logically consistent, and in this point it is not, so it leads to a principle of explosion. So, A) either some, or both theories are incomplete or wrong, or B) all science is essentially a biased belief! You want more on B, because that's what philosophers are inclined to? Get to the depths of metaphysics with Immanuel Kant: Critique of Pure Reason. Warning: it can get hugely complex, but will worth the effort.