For a speaker to be used as a mic, it must be UNpowered, I believe.
Doesn't matter, the phone (obviously) has a mic and they can turn that on from the base station, without any indication on the phone that that has been done.
For those saying, "They can't pinpoint where it is because the phones don't have GPS", sorry, wrong. a) A lot of phones are now shipping with GPS, but regardless of that b) the phone is constantly saying "hello" to towers nearby. They can easily track the phone to the nearest tower, which gives a fairly small radius. Furthermore, they can triangulate the phone using multiple towers, by seeing what the signal strength is to the different towers. As a first approximation, if a phone is "visible" to towers A, B and C, pick the midpoint of those three towers and that's a more accurate location of the phone. If they use signal strength to each tower as a weight in the triangulation, it's even more accurate, accurate enough that GPS is irrelevant. GPS is a satellite technology (obviously) and it only works when the receiver has a clear view of a good chunk of the sky. Cell phone technology is more powerful and it works indoors and out. My point is that GPS is not relevant to phone location tracking because it would only have a chance of working outdoors in places with an unobstructed view of the sky.
All this stuff is standard in all the modern base station equipment.
So yes, they can use your phone as a listening device as long as it's on. Nothing on the phone would indicate that that is happening, except that it would burn its battery faster than expected. And they can get quite an accurate position on the phone, too, as long as it's on. They can't do anything if the phone is turned off, because then there is no power to the radio unit of the phone.
Whether you believe it or not, that's how it is.