It's a fair question to explore in political philosophy the connection between the term and advocacy of "freedom", and the outcomes. But, ultimately, "freedom" is a complex, highly disputed abstraction, and is often involved in oversimplification. Thus, there is no answer to your question, which is a tad oversimplified.
Well, what exactly is freedom? From WP:
Freedom is understood as either having the ability to act or change without constraint or to possess the power and resources to fulfill one's purposes. Freedom is often associated with liberty and autonomy in the sense of "giving oneself their own laws", and with having rights and the civil liberties with which to exercise them without undue interference by the state. Frequently discussed kinds of political freedom include freedom of assembly, freedom of association, freedom of choice, and freedom of speech.
To paint with a broad brush this or that political system or philosophy as being for or against freedom is a difficult sell. In fact, the notion of the political spectrum is a bit complicated and is wont to be abused by political propaganda. Political philosophy and political science offer some insight into reality, but the one irrefutable claim is that politics, like people is a complicated affair. Political scientists attempt to create certainty by using scientific methods to sort out what is ideological and what is praxis.
In this scenario: how is freedom justified if it makes the country worse?
If you're looking for acknowledgement that Western notions of freedom in political philosophy don't always correlate to a high standard of living, then you are right. The nation of Singapore. It has excellent economics, but its human rights are questioned by the West. From the WP article on the topic:
The government has restricted freedom of speech and freedom of the press and has limited other civil and political rights.
Certainly both China and Cuba saw systemic improvements after their political revolutions to "free the people", and certainly banana republics that profess to be bastions of economic freedom can also violate the rights of its people. Despite the US being a constitutional republic that professes to be freer than most other countries, the constraints of government in the Nordic model seem to lead to better outcomes as far as happiness and fairness are concerned. What is a philosopher to make of all of this?
It's that the word "freedom" and those that use it must be carefully considered, because it is a word prone to abuse. Some philosophers call it an essentially contested concept, and I for one am in agreement. And as such, there is no simple answer to your simple question, because as they say, the devil lies in the details.