Getting proxy from file Error cURL - (5) Unsupported proxy syntax

Today for a project, was trying to hit an endpoint URL with post request via a lit of proxy. 1st was trying with Burp Repeater , but then realise, there is not direct function exists to add 1000s of upstream proxy in a shot. So, thought of settling with good ol' cURL.

So, the syntax to use proxy with curl request is like

-x, --proxy [protocol://]host[:port] 

curl "URL" --proxy PROXY_IP:PORT
or 
curl "URL" --socks5 socks5_ip:port

So, to use this easiest option created a loop to take proxy from a text file and execute command per line

for line in $(cat proxy.txt); do curl COMMAND --proxy "$line"; done

But - WTF , started getting error like

curl: (5) Unsupported proxy syntax in '$line'
curl: (5) Unsupported proxy syntax in '$line'
curl: (5) Unsupported proxy syntax in '$line'
curl: (5) Unsupported proxy syntax in '$line'
curl: (5) Unsupported proxy syntax in '$line'
curl: (5) Unsupported proxy syntax in '$line'

Tried with single line manally and worked well. But, when fetching from proxy file, it is not working.

After a bit of research, found, the culprit , it is \r in txt file, the input file has carriage return characters at the end of each line. So, when the command taking one line from the txt file , it is returning 127.0.0.1:7070\r rather than 127.0.0.1:7070 . What a loose of time !!!

So, formatted the command line in different way. Used tr to remove '\r' from the end of each line and after that it went fine.


while read line; do curl -p "URL" --proxy "$line" --connect-timeout 2; done < <( tr -d '\r' < proxy.txt )