The makeshift facilities are illegal but this huge community faces no other option. Athens, a metropolis on the edge of the Muslim world, is the only EU capital without a mosque.
Since Greece gained independence from the Ottoman Empire in 1832, no government has allowed a mosque to be built in the city. It was seen by many as "un-Greek" - out of place in a country in which much more than 90% of the population are Orthodox Christians.
But as Greece has become the main entry point for migrants to the EU, its Muslim population has swelled.
Some estimates place the number of Muslims in Athens alone at around 300,000, in a city with a population of around five million, and the clamour for an official place of prayer is growing.
"It is a very big tragedy for us Muslims that there is no mosque here," says Syed Mohammad Jamil from the Pakistan-Hellenic Society.
"Greece produced democracy and civilisation and the respect of religion - but they don't respect our Muslims to provide us with a regular, legal mosque."
One of the Friday worshippers, Ashifaq Ahmad, says: "I feel somehow cut off from society.
"When we have a celebration, there is nowhere proper for us to get together. Society is not accepting us."
Pressure on the government to provide a secure, protected mosque has grown as the neo-fascist Golden Dawn party continues to rise.
Its members stand accused of beating immigrants and vandalising some of the underground prayer rooms.
The party's deputy, Ilias Panagiotaros, told me earlier in the year that landmines should be placed on Greece's border with Turkey, saying: "If immigrants die trying to jump into our country, that's their problem."
Now perhaps the call for a place of worship may be answered.
A disused army barracks near the city centre has been chosen as a site for the capital's first mosque.
Behind heavy gates lie old buildings, broken glass and rubble strewn across the floors. The crumbling shells currently there would be torn down, making space for a mosque that could accommodate 500 people.
If it is built, Muslims entering would catch sight of a small church next door, the two religions finally operating officially shoulder to shoulder.
The government insists the project will go ahead, but similar plans have been promised in the past - only to fall foul of political infighting.
And the financial crisis could still blow the idea off course. A government struggling to afford schoolbooks or healthcare may find it hard to announce 1m euros (£814,000; $1.3m) for a state-funded mosque.
"In the past, there was a fear in some segments of Greek society about constructing a mosque but we must overcome that fear," says Stratos Simopoulos, the secretary general of the ministry for development.
"The financial crisis is a problem. The government has other priorities for now, but this mosque must be constructed and we may be in a position to start the process in a few months."
I ask whether he is committed to the plan.
"Of course", he replies, "because it's not my commitment - it's a commitment of the Greek state."
And yet there is still resistance within the country.'Islamic tyranny'
The Greek Church has warmed to the mosque idea but some senior ecclesiastical figures remain opposed.
"Greece suffered five centuries of Islamic tyranny under Turkish rule and building a mosque would offend the martyrs who freed us”
In a packed service in St Nicolas's Church in Piraeus, just outside Athens, the strength of religious devotion is clear.
Members of the congregation kiss the icons and repeatedly cross themselves. Orthodox Christianity goes to the heart of what it means to be Greek and the Bishop here, Seraphim, says his nation must preserve its identity.
"Greece suffered five centuries of Islamic tyranny under Turkish rule and building a mosque would offend the martyrs who freed us," he says.
Greece, he adds, "does not hate anyone" but he believes that "most Muslims have come here illegally" to, as he puts it, "Islamise Europe".
I put it to him that his position appears Islamophobic, out of touch with a multicultural European Union, and his response may betray other prejudices too.
"We are not a multicultural country," the Greek bishop says. "We are one Greek nation and everything else is an invention of the 'new order' and of Zionism. They are trying to corrupt our character."'Hypocrisy'
On the streets of Athens, opinions are mixed.
"Muslims should have their temple," says Kali Patounia, a banker.
"Greek immigrants in other countries build their own churches and perform their own religion, so it's hypocritical."
Marios, a student, disagrees. "We must not have a mosque here," he tells me.
"This is a Christian country and if they want a mosque, they can go back to their own countries and have one."
Religion is intrinsic to national identity here and Church and state are closely linked. The mosque issue has become a symbol of what sort of state today's Greece is willing to become.
The financial crisis has made this nation more inward-looking, more fearful.
However for Greece the decision is whether to extend its hand fully to Islam - and whether its capital will no longer stand alone in Europe.