No you do not need to request each stream separately.
You can still do
https://api.twitch.tv/helix/streams
?user_login=streamer1
&user_login=streamer2
&user_login=streamer3
&user_login=streamer4
However if streamer2 is offline, and the rest are online, you will get 3 objects.
// This is not the exact response
[
{
"name": "streamer1"
},
{
"name": "streamer3"
},
{
"name": "streamer4"
},
]
So you need to loop through the results to see who is online and who isn’t.