2

I work with a team to develop a web service and client, each of us, of course, working on our own machines. To keep things simple and more consistent, we use similar domains in the code and update /etc/hosts to resolve those domains to localhost.

  1. works fine for my web browser; response times are snappy
  2. curl and wget take ~4 seconds to resolve DNS before successfully completing the request

I did find the --resolve flag for curl which resolves the delay, but I could just as well use 127.0.0.1 and define the headers necessary to get the same effect.

with flag (and updated /etc/hosts file)

# /etc/hosts
...
127.0.0.1 mp-api.example.local

command

curl -k -o /dev/null -s -w %{time_connect}:%{time_starttransfer}:%{time_total}\\n \
  --resolve mp-api.example.local:8094:127.0.0.1 \
  -H 'Host: mp-api.example.local:8094' \
  'https://mp-api.example.local:8094/api/categories/tree.json'

with 127.0.0.1

curl -k -o /dev/null -s -w %{time_connect}:%{time_starttransfer}:%{time_total}\\n \
  -H 'Host: mp-api.example.local:8094' \
  'https://127.0.0.1:8094/api/categories/tree.json'

What are some of the reasons for the delay in DNS resolution for tools like curl and wget? And what are some effective ways of troubleshooting that delay?

Community
  • 1
David Vezzani
  • 131
  • 1
  • 4

2 Answers2

1

Per suggestion that there might be settings in the Network tool, I found the values in "Bypass proxy settings..." interesting.

enter image description here

It would appear that I have been having problems with DNS resolution for curl and wget because I have been using a domain ending with .local. Perhaps it's essentially a reserved value.

So I changed my settings so I now use a domain ending with .loc and all is well again -- no more 4 seconds off in la-la land waiting for DNS resolution.

David Vezzani
  • 131
  • 1
  • 4
0

Check /etc/nsswitch.conf file

There configuration how names to id converted. One of lines for hostnames:

hosts:      files mdns4_minimal [NOTFOUND=return] dns myhostname mymachines

Ensure, "files" on first place.

UPD.

Curl local host names on Mac OS X Yosemite

look like curl take responsibility for name resolving and prefer ipv6

try add --ipv4 key or add ipv6 to hosts

Community
  • 1
Mikhail Moskalev
  • 2,008
  • 16
  • 13
  • Does a service need to be recycled to pick up changes? `/etc/nsswitch.conf` doesn't exist on my computer. I added it, supplied with the values you suggested, but I'm still getting a 4 sec delay on curl and wget requests. Browser requests do not experience the same delay. – David Vezzani Jul 23 '16 at 18:16
  • strange it not exists. AFAIK nss is part of glib. Try strace curl ... What distro you use? – Mikhail Moskalev Jul 23 '16 at 19:04
  • I'm on Mac OS (El Capitan); I don't have `strace` in my path, so I can't run that command. ``` curl --version curl 7.43.0 (x86_64-apple-darwin15.0) libcurl/7.43.0 SecureTransport zlib/1.2.5 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp Features: AsynchDNS IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz UnixSockets ``` I installed with `brew`. – David Vezzani Jul 23 '16 at 19:12
  • Sorry. I implied it Linux based. I added tag and note to question. – Mikhail Moskalev Jul 23 '16 at 19:17