It's a little weird to me that getaddrinfo() is considered a "low-level legacy API". Maybe things are drastically different on macOS, but getaddrinfo() is the way to resolve names on Linux and I suspect the *BSDs.
Sure, I expect most macOS apps will use something in Foundation or some other NetworkKit-type framework to do DNS queries, but it's odd to me that the code there wouldn't then call down to getaddrinfo() or the like to do the dirty work. I guess GAI is blocking, so presumably there's some other low-level non-blocking call?
>so presumably there's some other low-level non-blocking call
Correct, CFNetwork is open source so you can check implementation but last I remember it used some variant like `getaddrinfo_async`. But Apple really doesn't want you (the end-user) to use getaddrinfo (or the async variant CF exposes) to resolve an IP and then directly connect() via that ip, everything is geared towards connect-by-hostname since then Apple's can internally handle the implementation of happy-eyeballs.
That’s not the current documentation, as evidenced by the “archive” in the URL.
If you want to stay at a lower level the recommendation these days is to use Network.framework. If you want something higher level then use CFNetwork (probably through the classes exported by Foundation like NSURLSession).
It is not best practice to use `getaddrinfo` for DNS resolution, for sure. But it is best practice to use it before connecting to an IP address directly because that address may need to be translated.
Applications should not use getaddrinfo(). Because for the connect by name, the OS or app SDK can parallelize the entire multi-step lookup and connection process, not just step by step:
“Now, I’m not saying that all implementations of these APIs [Java, Apple Foundation, etc., doing connect by name] necessarily do the right thing today, but if applications are using these APIs, then the implementations can be improved over time.”
“The difference with getaddrinfo() and similar APIs is that they fundamentally can’t be improved over time. The API definition is that they return you a full list of addresses, so they have to wait until they have that full list to give you. There’s no way getaddrinfo can return you a partial list and then later give you some more.”
The deck's position on implementation of happy-eyeballs (which could sound dismissive here but is treated as "you had one job" important by the deck), is finding a way to avoid waiting 5 seconds for either side of IPv4 vs. IPv5 stack to timeout before finishing connection setup and serving the user a web page.
Yeah, I went in thinking that it was going to be some case of Apple wanting to bend the Unix philosophy to their will for their own desires and steer implementations in their direction, but no - they are simply pointing out a clear flaw in the design of the function in question for a usecase that does not apply only to Apple. Basically all OS vendors need to be doing something like this usecase to support IPv6 adoption.
I often find this is the case with Apple on a technical level.
For instance, their recent Spatial (stereographic) Video features uses a format that has basically zero current support outside of Apple—which is in fact just standard MV-HEVC [0] (with some extra optional metadata [1]), which is just the H.265 evolution of the standard H.264 MVC that 3D Blu-rays have used for a long time. (AFAIK no 4K 3D Blu-rays have been released, presumably due to space constraints, explaining the lack of usage of MV-HEVC outside Apple).
In piracy world, most re-encoded 3D movies just use objectively inferior composited 2D formats like half-side-by-side or over/under. And without diving in you’d just assume Apple was using some bespoke format to be evil, when in fact they are popularizing what should be the canonical, standardized format for 3D video.
> getaddrinfo() is the way to resolve names on Linux and I suspect the *BSDs.
At least on OpenBSD, all classical/standard DNS functions (getaddrinfo/gethostbyname/...) are wrappers around OpenBSD's libc asr implementation, written by Eric Faurot.
For quite some versions that modern networking APIs on macOS using Objective-C frameworks, starting in 2018.
See WWDC 2018's "Introducing Network.framework, A modern alternative to sockets".
NeXTSTEP might have been a UNIX, and macOS derives from it, but the whole UNIX story has always been to bring UNIX software into the platform, not to make it easier to move elsewhere.
Regardless, even if they renew the certification, they aren't obliged to expose in the classical POSIX APIs more than what the certification requires in features, or happens to be optional, implementation defined.
As anyone that has painfully tried to write POSIX portable code across big iron UNIX is aware of.
> Regardless, even if they renew the certification, they aren't obliged to expose in the classical POSIX APIs more than what the certification requires in features, or happens to be optional, implementation defined.
getaddrinfo() is part of POSIX, so it would be necessary to expose it:
And it is exposed, the code still compiles if one uses it, it just doesn't get additional nice macOS networking features that aren't explicilty required for UNIX certification.
Everything in the UNIX compatibility layer is low-level in macOS. Not necessarily "legacy" though.
But this is no different than saying that, for example, calling out platform-specific native OS APIs from Java is "low-level." Which it is, from the perspective of compile-once, run-anywhere Java applets. macOS is a NeXT-compatible non-UNIX API, and you are supposed to use the macOS frameworks for everything. Calling down to BSD or even mach is definitely not what Apple wants you to do.
Seems to be badly phrased and meant something else, since macOS is certified to be UNIX - https://www.opengroup.org/openbrand/register/ - contrary to Linux which is not UNIX-certified.
No, mach is a microkernel, like L5. It was developed for the purpose of replacing the BSD kernel, by having a small amount of functionality in the kernel itself, and the rest of the BSD-compatibility layer implemented in user space. macOS' frameworks are then a layer on top of that.
The UNIX libcompat (a compatibility library for older UNIX functions) was integrated into Mac OS (specifically, macOS) rather than directly into Mach OS.
Here’s the breakdown:
• Mach OS refers to the Mach microkernel, which primarily focuses on low-level system functions such as task scheduling and memory management. It is not a full-fledged operating system, and thus, libraries like libcompat, which are higher-level UNIX compatibility libraries, would not be integrated directly into the Mach kernel itself.
• Mac OS (particularly macOS, formerly OS X) is a complete operating system that includes the Mach microkernel, the BSD layer, and various other components. macOS has a strong Unix heritage, and libcompat is part of the broader Unix-like environment included in macOS to support legacy Unix APIs and applications.
Thus, libcompat was integrated into macOS (or its predecessor, NeXTSTEP) as part of its Unix compatibility layer, rather than into the Mach kernel directly
> Maybe things are drastically different on macOS, but getaddrinfo() is the way to resolve names on Linux and I suspect the *BSDs.
I'm not sure if this is the case in this case, but it might be worth noting that some system functions with the same name have drastically different internal/implementation differences between Linux/*BSD/MacOS. With there being differences between the *BSDs too.
So on some systems one function call is "the way", because its been maintained over the years, but then on another it might actually be old and not useful.
> getaddrinfo() is the way to resolve names on Linux
Not at all. That's just a glibc function, it's got nothing to do with Linux. People just assume that glibc is how things are done in Linux user space but it doesn't have to be that way. For example, systemd came up with its own resolved mechanism which turned out to be much better than the glibc stuff. I will probably end up inventing my own at some point as well since I'm working on freestanding software targeting Linux.
getaddrinfo is defined by POSIX and UNIX. Where the implementation is doesn’t matter. It’s portable, which is why it’s used. The slide deck referenced above talks about better implementations for various platforms, but they are all platform specific.
So OP might not be completely accurate, but getaddrinfo is _the_ way to resolve names if you are writing portable POSIX and/or UNIX code.
Linux and the popular Linux distributions are not POSIX compliant to begin with. Only GNU tries to be, and even GNU adds on a ludicrous amount of extensions because the truth is POSIX isn't good enough.
Not nitpicking at all. I just don't like how people see Linux as a "POSIX implementation". It's much more than that.
Practical compatibility is not compliance, the manuals document many subtle differences and incompatibilities. The practicality of it mostly comes from glibc which everyone uses and which does strive to be compliant. Even then it's a hit and miss, the so called "Linuxisms" crop up in the most unexpected of places. The executable path that people write in the shebang lines of their shell scripts, for example. It's gotten to the point some BSDs have started emulating Linux system calls instead of porting software. Even Windows did this once upon a time.
My point is glibc is not even guaranteed to exist on the system. POSIX is not at all mandatory on Linux. The POSIX interfaces are just one of the ways to interface with the kernel. It's also possible to bypass all the POSIX stuff and interface with it directly. Linux is the only operating system to offer this ability via the stable kernel-userspace binary interface. It's even defined at the instruction set level which makes it programming language agnostic. On Linux you actually can trash all that POSIX stuff and reinvent it all in Rust if you want.
> Good luck trying to compile and run any kind of software without providing getaddrinfo(), socket(), connect(), etc
I'm working on a freestanding lisp language with built in Linux system call support at least in part because I want to to prove that this sort of thing is possible. No legacy interfaces will be provided and yet I have no doubt in my mind that one day it will be able to everything you mentioned and much more.
Your comment above was that it is a glibc function, which is true, but it’s there for reason. It’s also a libc, musl, uClibc, and Windows Sockets 2 function: because it’s defined in POSIX 1.1 and extended in RFC 3493.
I have no opinion on whether it’s good enough (it seems like not if every platform has a connect-by-name implementation), just that calling it a glibc function overly simplifies it’s origin.
It’s also false to say only GNU tries to be POSIX compliant. There are 8 commercial UNIXes that meet some POSIX standard, another 8 that are discontinued (at least one of which was a Linux distro), and dozens that are mostly compatible. POSIX doesn’t care if that compatibility comes from the kernel or user space libraries.
POSIX isn’t good enough at what? Maybe you don’t understand what it’s goal is/was. POSIX exists for portability. It’s a minimal set of functions developers can target to get things done on any UNIX. Any OS will always have something beyond POSIX to differentiate it.
The point I tried to make is the getaddrinfo function is not sacred. It's not the way to do anything at all on Linux. It's just the function that glibc has, and most people use it. Whether it came from POSIX or something else seems like a minor detail to me. POSIX and glibc are not sacred either.
> Any OS will always have something beyond POSIX to differentiate it.
Linux is no exception. We should all be enjoying those exclusive features to their fullest extent. Not restricting ourselves to the lowest common denominator between them. Portability is a trap.
The Apple deck linked elsewhere in this thread suggests the developer's goal generally isn't "DNS resolution", the dev's goal is usually establishing a connection to a host/server/endpoint to start doing something.
So, usually devs should use the Java or Apple or whatever higher level OS API gets you connected the fastest, and that API is free to implement the connection however most quickly gets to the point of able to return data to the user (app or end user).
The API that returns a list of addresses is stuck doing that, instead of being able to parallize the entire "get connected" data flow.
Not at all actually, passing hostnames means they can fully handle happy eyeballs for you and all other performance optimizations that you can do if you resolve and connect in one call.
It also means if you do it the 'Apple' way they might choose to intercept or modify responses. That seem in line with Apple's practices as a company even if they are not doing it yet. I feel anything they might do like that might be less likely to extend to what the article refers to as a legacy API.
If you were writing a Java application, would you do your own DNS resolution, or would you make a new socket address object and give it a hostname and let the api resolve the hostname for you? If you don’t hand roll your own dns protocol lookups, how do you know the OS, or Java or your socket library aren’t intercepting and modifying request out from under you? Heck, even if you use getaddrinfo directly how do you know your libc implementation isn’t intercepting and modifying the lookups on you? If the threat model you’re coding for is “Apple is a hostile actor intercepting and modifying dns queries” then you really can’t trust their provided posix calls either.
> If you were writing a Java application, would you do your own DNS resolution,
Java isn't known to nanny the users of apps developed in it's language. It's never even tried IMO.
> If the threat model you’re coding for is “Apple is a hostile actor intercepting and modifying dns queries” then you really can’t trust their provided posix calls either.
Sure, but that isn't the threat model. I described the threat model above, which is closer to "I don't trust a company famous for trying to nanny not to try to nanny if using their preferred developer frameworks, while I kind of trust they won't for a legacy API they barely pay attention to".
> Java isn't known to nanny the users of apps developed in it's language.
> It's never even tried IMO.
I think anyone who's run into java keystore / cert problems would beg to differ.
> Sure, but that isn't the threat model. I described the threat model above,
> which is closer to "I don't trust a company famous for trying to nanny not to
> try to nanny if using their preferred developer frameworks, while I kind of
> trust they won't for a legacy API they barely pay attention to".
Sorry, this just makes no sense to me. You have a threat model that thinks Apple is a malicious actor and will interfere with the implementation of DNS resolution in higher level APIs because they "nanny" their users and developers. But you equally think they won't bother to do such nannying at a the lower levels because they "barely pay attention to [those legacy APIs].
But we're talking about the same apple that implemented SIP which locks out even root (and by extension sudo) from being able to make changes to "system" directories like `/bin`, and necessitated things like homebrew having to migrate their entire deployment hierarchy from `/usr/local/` to `/opt/homebrew/`. The same Apple who replaced init scripts in BSD with their own proprietary `launchd`. The same Apple who swapped `bash` for `zsh` to avoid GPL3 licensing stuff. The same Apple that last OS update changed what signals were sent when doing illegal memory access, breaking a mess of java/docker things in the process. This is the Apple you think would impose hidden DNS interception and replacement at the swift library level but wouldn't bother to either change libc too or point getaddrinfo() at their own internal interceptor.
You are of course free to have whatever threat model you like, but surely you can see why other people find it a little inconsistent?
I would feel far more concerned that an arbitrary application that decided to do its own DNS resolution would be doing so for nefarious reasons, or might mess up the process (such as not supporting encrypted DNS in this case).
If you genuinely cannot trust the OS vendor, you don't try to tinker around in user space but you stay off their platform. Personally, this is why I don't have any machines with a Microsoft OS, and why I don't have a Playstation.
The concern isn't that Apple controls the OS and so could do nefarious rootkit type stuff, but rather they may try to nanny through the framework they prefer and push for all apps for their platform to be developed in.
But you are already on their OS, so they would always be able to do that. They make the kernel, the hardware and it's firmware, so it's a moot point and needless paranoia. Might as well use the API that gives a better user experience.
I'm not talking about Apple being able to patch the OS and control everything at that level - of course they can, but it seems unlikely.
I'm talking about a developer framework, a high level abstraction, where the method of resolving would be more likely to be intercepted - consider for example something like that on an iPhone with the justification being safety or 'for the children' or whatever.
That doesn't seem unlikely or improbably at all, and certainly not moot or any kind of paranoia.
Assuming arguendo that apple did want to do that kind of messing with DNS though - what's there to stop them from changing getaddrinfo() in the same way? As someone pointed out upthread, if you don't trust your OS vendor to do DNS lookups correctly, your only option is to not usre your OS vendor for DNS lookups, which is in the realm of Byzantine faults.
(And further, assuming arguendo that there was DNS meddling happening but somehow getaddrinfo() was exempt - now the user has one app that behaves differently to all their others, which is worse in every practical sense.)
> Assuming arguendo that apple did want to do that kind of messing with DNS though - what's there to stop them from changing getaddrinfo() in the same way?
Nothing, I already acknowledge they have the power to do rootkity things if they wanted to, but I don't consider that likely.
I do consider it likely they might do that kind of a thing at a framework level and try to push most developers to use it.
> As someone pointed out upthread, if you don't trust your OS vendor to do DNS lookups correctly, your only option is to not usre your OS vendor for DNS lookups, which is in the realm of Byzantine faults.
I responded to that as I'm responding here, by pointing out that isn't relevant to the threat model that I've described.
> now the user has one app that behaves differently to all their others, which is worse in every practical sense.
Not if that app actually gets the user to where they actually wanted to go.
It's not necessarily EEE. Maybe it's just that the old wheel sucks. They want a better wheel and so they reinvented it, hopefully better this time.
The corporations making proprietary software are not the only ones who have that attitude. I've resolved to make all my free software Linux-exclusive so that I can use Linux to the fullest. The Linux kernel is packed full of exclusive non-portable features that very few people take advantage of because they're obsessed with portability, POSIX compliance or whatever. I think that's a waste.
Portable software is usually sucky lowest common denominator software. We should not limit ourselves to whatever glibc offers.
You actually think that a Swift developer, developing against Cocoa APIs, targeting Mac and iOS devices cares about a portable API.
Because not sure if you know this but the entire software industry is built on high level libraries on top of largely portable code. For example this Swift library wraps c-ares a portable API.
getaddrinfo isn't its predecessors, there's nothing error-prone about it. The only thing that's nontrivial is falling back if the first server is unresponsive, and even there the obvious calling code is fine for almost all apps.
Yes, this! I even wonder how else you would do this. By the way I worked with many IoT devices that do not use your dhcp dns but just hardcode quad 8 or similar
We recently had a developer join our team and he got stuck setting up his dev environment.
We use a .dev domain as a localhost alias, and turns out his ISP’s DNS wouldn’t resolve 127.0.0.1 (or whatever it is) for the .dev domain. Changing his resolver at the network level to 1.1.1.1 fixed it.
I imagine there are lots of difficult support tickets for app devs, and at a certain point they just hardcode the DNS to remove one variable from the equation when debugging bug reports.
Wayyyy back in 1995 or '96 I was working for a non-profit called "Next Generation Magazine" and our goal was to have young people write content for web sites to get their names out there. Back then it was all local ISPs, so we went to our ISP and asked for ngm.org and were stoked when we got it! We built out the site (Thanks to Building Killer Websites of course) and it looked awesome!
Only problem was that nobody in my family out of state could see it. It took awhile to realize realize that we never bought that domain. Our local ISP just added it to their DNS records, and since we all hooked into them we thought we were live across the 'net.
I remember one of the first times I used the Internet and opened my local radio station’s website from several states away. It was incredible to me that it worked and I also wondered why anyone across the country would care. The early internet was amazing.
Not resolving 127.0.0.1 or RFC1918 addresses or even ULA for IPv6 is done to avoid DNS rebinding attacks. For most end users that is probably the correct move.
My home router even seems to inspect any UDP/53 traffic and redact any responses containing local/private A entries, so not even switching to a public resolver bypasses the protection.
In case you want to look into it further: My router actually allows adding exemptions to this policy on a per-hostname basis!
Sometimes I wish it would allow wildcards, but honestly that's probably just another way for users to shoot themselves in the foot (e.g. by adding '*').
pfSense for example uses unbound, and while it doesn't have a switch for disabling rebind protection, it does allow injecting arbitrary unbound config, which can disable rebind protection for any depth of a DNS zone or IP space. E.g.:
Totally, but most IoT customers are not homelab hobbyists, so I think its defensible for IoT vendors to just hard code known good DNS in their devices instead of relying on broken ISP resolvers.
Related story, there was a period of time where my ISP's resolver that would replace hostnames with no DNS record with their own ad filled garbage page.
So you mistype google.com to foofle.com or something and instead of getting "host not found" you get... ads.
Yeah, this report seems a little spun. The essence is basically that the encrypted DNS needs to go through the proxy, and there's resolver code elsewhere in the OS that doesn't use the proxy. It's a bug, sure. It could plausibly have interesting exploits, though none are shown. But it's not a very interesting bug.
>UPDATE: Spoke too soon…
The problem discussed here turned out to be specific to Little Snitch 6.1 and not a general issue in macOS. It will be fixed in an update of Little Snitch later today.
> After further investigation, we found that this bug has already existed at least since macOS 14.5 Sonoma (maybe even earlier, but we currently don’t have access to an older 14.x system for testing).
Did they test it ever worked with getaddrinfo? Or did they just see it worked once with CFNetwork and called it a day and then later publish a blog post saying it’s broken?
It's ridiculous us developers still have to jump through hoops to save around older versions of the OS for testing. There is 0 technical reason why Apple can't let us downgrade.
Can someone fill me in on this? What hoops have to be jumped through? The last time I used macs, there were no issues downloading and installing older OS versions, but I have not used them recently.
You can do a fresh install of an older macOS version whenever you like (you need to enable that option in the rescue system tho).
You can also run older macOS in a VM (the hypervisor framework keeps getting new features that make guest macOS more fully supported).
Name an OS (ok maybe NixOS) that allows you to do clean downgrades out of the box. Also wonder what's gonna happen to your data in e.g. Postgres if you blindly downgrade.
Before Sequoia when using OpenDNS for VPN, could be on VPN and iMessage and other apps still work, but since Sequoia, when on VPN iMessage (text messages) etc no longer work. Once I disconnect to VPN all goes through. Is this related at all? Do have macOS firewall enabled. But not block all incoming connections.
After upgrading to Sequoia, I could not browse with Safari or Mozilla. What fixed it for me was to go to the DNS settings for my Wi-Fi connection, and add Google's DNS servers (8.8.8.8. and 8.8.4.4). They replaced the autofilled DNS servers that were there.
Were the autofilled DNS servers in RFC1918 private space (10.0.0.0/8, 192.168.0.0/16, etc.)? I had issues after the upgrade with Google Chrome being unable to access hosts in these ranges, and fixed it by going to System Settings -> Privacy & Security -> Local Network and toggling Google Chrome off and on again.
No, they weren't local. I have no idea where they came from. I couldn't even delete them, but when I added the Google servers, they autofilled ones were automatically deleted.
It worked before I upgraded to Sequoia. But I don't know enough to point fingers. Just mentioning that turning off the firewall long enough for Firefox to update fixes the problem.
There is no such thing as a remotely cross-platform DNS resolution API that has the system do the lookup and does not utterly suck for asynchronous use.
I suspect "cross-platform" is doing a lot of heavy lifting for your claim. Browser engines and application frameworks built on top of them have no trouble using platform-specific APIs under the hood.
I have one browser setup to do DNS differently than another. I don't want to have to set it at a system level and then need multiple systems just to run 2 browsers with different DNS lookup
I remember 20+ years ago when one of the most commonly seen attacks was malware configuring a proxy server in Internet Explorer which by design overrode the operating system's configuration.
What a lot of software does today by ignoring the operating system in lieu of their own shit is just like the above. If your program doesn't (or can't) respect the operating system, your shit is malware and you should reconsider who you write code for.
If you consider the source of income of what's most likely a considerable portion of the HN community, I think this makes more sense. Apple is one of the only companies interested in preventing tracking, and it hurts, in the billions sort of way [1][2].
At some point in my copious spare time, I plan on writing software to allowlist in my firewall outbound connections only to IPs resolved using my DNS servers.
You could also configure your router to intercept rogue plaintext DNS lookups on your network with responses from a resolver you trust (for example a Pihole, Cloudflare or Google Public DNS, Quad9, PCH, etc). Adding something like Pihole would give you comprehensive blocking and custom internal DNS entries too.
I already redirect DNS queries to my own DNS servers running unbound, and block UDP and TCP ports 25 from machines other than these from going out on the Internet.
This will force machines misconfigured with 8.8.8.8 as default resolver (cough, systemd) from leaking my browsing history to Google, thank you very much, but won't stop DNS-over-HTTPS like Firefox, or more insidious devices like fallback IPs hardcoded in SmartTVs and other IoT devices (they are on their own VLAN with all traffic logged, but it's not as if I have time to inspect their traffic for suspicious behavior. There are blocklists of DoH, but at that point it becomes a whack-a-mole game, and it makes more sense to block anything that is not the result of a legitimate DNS query instead.
This would only be enforced on untrusted machines like Macs, iPhones, Android devices, IoT devices and Ubuntu machines, as opposed to trustworthy OpenBSD and Alpine Linux servers.
How would that work? Do you only access a really small/known set of IPs? Or would you program the firewall to only allow connections to an arbitrary IP if it had seen a DNS query to your preferred servers go out and return that IP within a few seconds prior?
In the latter case, would you have to aggressively disable local DNS caching on devices to make the behavior work (is that even possible on some devices)? How would encrypted DNS fit into this scheme?
The second. The only way untrusted devices connect is to IP addresses that were resolved by my DNS servers so I know what traffic is happening on my network. My DNS servers handle their own encryption via WireGuard to bypass ISP snooping, so I don't need Mozilla's DoH and Apple's and CloudFlare's I do not trust at all.
To avoid race conditions, the trusted DNS servers would add the result IP to the firewall allowlist table before returning it to the client, so either implement it as a Caddy proxy module (I already wrote a DynDNS module for Caddy so I know how to make that work). Or alternatively use unbound's dnstap support. I just need to implement some reliable and secure protocol to send those requests from the DNS server to my OpenBSD firewall running pf.
I am fine with the only Java application I have used in the lease decade not working. I did not even bother putting a JVM on any of the OS I installed in the last 5 years. So yeah, I’d rather have fewer security holes.
Yep, I wish they would go the full way and block socket access entirely so your own outgoing traffic is always introspectable even with cert pinning. It would make it blatantly obvious when apps try shady shit.
I had a great Windows firewall like this about 20 years ago. It would pop up a dialog for every network request from an app. You could block or allow based on port or destination, or "block all". It was amazing, because as you say, it made it very obvious when an app was trying shady shit.
I would love to have that back, but I was never able to find a firewall so hostile to the user experience of the general population.
It sounds similar in spirit to Little Snitch, mentioned in the article (on macOS, but which inspired OpenSnitch, which runs on Linux). It is awesome indeed, if a bit overwhelming at first. Most regular users would just uninstall it to avoid the constant barrage of requests initially, and then every time a new piece of software tries to connect to anything.
Shady shit? Not every network request is a call to an HTTP REST API.
Blocking socket APIs would break every app that supports other protocols. Goodbye file transfer apps, VPN apps, file sync apps, database tools, SSH clients, remote desktop clients, audio and video conferencing apps, etc.
Devil's advocate would say: They could do this and make it look like a bug that never gets fixed in order to avoid backlash. How it gets achieved is flexible if the goal is met.
Why would they be afraid of backlash on such an obscure, technical feature? They never were in the past and are expected to take controversial technical decisions by now. And by “now”, I mean in the last 30-odd years.
I maybe imagining but I feel like deja vu that there will be a problem with DNS that would affect Little snitch., Mullvad and others with new releases of iOS and Mac. If true I would really question what apple is doing during their months long developer and beta testing.
I was confused at the Little Snitch mention, and then reading further it just seems like a LS bug, that it only works in certain cases.
Well, seems this is the LS blog, so only confusion is why this is portrayed as a macOS bug? I'm not saying it's wrong, it's their domain not mine after all, it just doesn't seem to be justified in TFA?
The /etc/resolv.conf system is woefully inadequate. It doesn't have a concept of per-interface customization so you can't customize according to the currently active network interface. It doesn't distinguish between DNS configuration delivered by the network administrator (which can and should be changed remotely) versus set by the computer administrator. It doesn't work very well with VPNs where a specific DNS server is used for resolving addresses on that VPN.
If I recall, Apple deprecated use of certain network apis for third party developers. But Apple’s own apps (App Store) do not have these same restrictions. Thus, when trying to filter network traffic via app firewall via new APIs. It would fail since App Store uses legacy APIs.
Maybe part of this old bug (that I thought was fixed)
There have been POSIX-certified Linux variants. But the open source projects you use don't bother (for obvious reasons) and commercial derivatives like Android and ChromeOS don't need it. Similarly Window NT was POSIX-certified way back in the day yet its descendants aren't, even though they implement the same API set (via very different technology).
I use routedns [0] as my local stub resolver so that I can pick and choose which requests go to where and also what transport they use. It can also blocklist, re-write, cache, load balance, and/or handle fall back requests; so it give you lots of control.
I use a stub listener on localhost:53 for local requests and then forward them via UDP QUIC (TLS 0-RTT) requests to Cloudflare (1.1.1.1) with caching for most requests. Fast and reasonably secure.
> After further investigation, we found that this bug has already existed at least since macOS 14.5 Sonoma
Isn't this an inherent risk when attempting to do network stuff in userspace? You're at a very high level so hoping that lower level things comply seems risky if DNS encryption is critical to your use case.
Apple removed support for kernel extensions, and instead added a bunch of APIs that allow to do network filtering etc in user space. Unfortunately, some of their networking code just bypasses those network filter extensions (probably because of bugs) -- this is not the first time the developers of Little Snitch publicized a bug like this.
The battle of DNS resolving ownership rages on: who has the rights to set the DNS nameservers/resolver.
As a long-time DNS security researcher, the ultimate and final end means would be to mirror the root servers, but I assert, for now, popping in your own `resolv.conf` should suffice, … again, for now.
I wonder how little snitch sets the dns encryption up. In macOS, you need to setup encrypted dns via a profile System (Settings => General => VPN, DNS & Device Management) and then in the browser. However, I think terminal and appstore still use whatever server is obtained via DHCP and is not encrypted.
> "To protect (DNS lookups) from prying eyes, Little Snitch 6 offers a new feature: DNS encryption."
Browsers such as Firefox have offered this directly for a while. Of course, that only covers DNS lookups made from the web browser, but it doesn't rely on OS-level hooks that (at least in Apple's case) can break.
They're using Little Snitch as an OS-level DNS proxy, which should intercept all DNS requests from any app and encrypt them. But, depending on what API the app uses for its DNS lookups, some DNS requests do not go via the proxy. Presumably Firefox, in its default configuration with DNS encryption set to OFF ("Use your default DNS resolver"), uses one the affected APIs.
Plugging https://www.supernetworks.org/ -- when on wifi/vpn all DNS will go up over DNS over HTTPS as plaintext DNS is DNAT'd to CoreDNS which is by default configured to use DoH.
Deploying DNS encryption on macOS is in general really tedious. Applying it as a system or user profile has different results. Switching between providers or temporarily disabling DNS encryption is painful.
I also still haven't figured out how to get SSID-based switching to work, does it even?
My read of this is that it shouldn't affect pi.hole given the system's default nameserver would still received by DDNS and thus be the pi.hole? Or do these requests go somewhere that's hard-coded?
No this appears to be if an application registers a DNS resolver proxy on the local system, getaddrinfo doesn't use the proxy, and presumably just hits whatever the network interface's configured DNS server is.
I’ve had issues using the Resolv library in Ruby when I’m connecting to the internet via a tethered iPhone. Never ran into that until Sequoia. I wonder if that’s related?
TBH I’m too lazy to dig in and find out. Has anybody else run into this issue?
I would believe so. I have a custom DNS profile setup that redirects a few domains to a server I run. The server has custom SSL certs issued by a private CA. I the certificate installed on iOS as a trusted root certificate.
Everytime I'm connected to my home WiFi I would randomly get `peer closed connection in SSL handshake (104: Connection reset by peer)`. I have absolutely no clue why it does this and this issue goes away when I'm connected on mobile data.
Now I'm guessing that it is bypassing the DNS profile and resolving it using my ISPs DNS or some other way.
It won't, it was specifically a bug in Little Snitch (which doesn't currently run on iOS, I believe.)
"The problem discussed here turned out to be specific to Little Snitch 6.1 and not a general issue in macOS. It has already been fixed in Little Snitch 6.1.1."
It's a little weird to me that getaddrinfo() is considered a "low-level legacy API". Maybe things are drastically different on macOS, but getaddrinfo() is the way to resolve names on Linux and I suspect the *BSDs.
Sure, I expect most macOS apps will use something in Foundation or some other NetworkKit-type framework to do DNS queries, but it's odd to me that the code there wouldn't then call down to getaddrinfo() or the like to do the dirty work. I guess GAI is blocking, so presumably there's some other low-level non-blocking call?
>so presumably there's some other low-level non-blocking call
Correct, CFNetwork is open source so you can check implementation but last I remember it used some variant like `getaddrinfo_async`. But Apple really doesn't want you (the end-user) to use getaddrinfo (or the async variant CF exposes) to resolve an IP and then directly connect() via that ip, everything is geared towards connect-by-hostname since then Apple's can internally handle the implementation of happy-eyeballs.
Edit: You can read https://www.ietf.org/proceedings/72/slides/plenaryw-6.pdf for their thoughts on why they don't like the getaddrinfo() model [there are speaker notes at the bottom of each slide]
If you do need the lower-level control, Apple does still recommend `getaddrinfo`. It handles NAT64 translation for IPv6-only carrier networks:
https://developer.apple.com/library/archive/documentation/Ne...
That’s not the current documentation, as evidenced by the “archive” in the URL.
If you want to stay at a lower level the recommendation these days is to use Network.framework. If you want something higher level then use CFNetwork (probably through the classes exported by Foundation like NSURLSession).
I actually found it linked from here, which seems current: https://developer.apple.com/support/ipv6/
It is not best practice to use `getaddrinfo` for DNS resolution, for sure. But it is best practice to use it before connecting to an IP address directly because that address may need to be translated.
I'm actually surprised they suggest getaddrinfo() directly over the async CF wrapper.
TL;DR:
Applications should not use getaddrinfo(). Because for the connect by name, the OS or app SDK can parallelize the entire multi-step lookup and connection process, not just step by step:
“Now, I’m not saying that all implementations of these APIs [Java, Apple Foundation, etc., doing connect by name] necessarily do the right thing today, but if applications are using these APIs, then the implementations can be improved over time.”
“The difference with getaddrinfo() and similar APIs is that they fundamentally can’t be improved over time. The API definition is that they return you a full list of addresses, so they have to wait until they have that full list to give you. There’s no way getaddrinfo can return you a partial list and then later give you some more.”
The deck's position on implementation of happy-eyeballs (which could sound dismissive here but is treated as "you had one job" important by the deck), is finding a way to avoid waiting 5 seconds for either side of IPv4 vs. IPv5 stack to timeout before finishing connection setup and serving the user a web page.
indeed: https://opensource.apple.com/source/Libinfo/Libinfo-222.1/lo...
Thanks for that link, it's a very convincing presentation that very clearly explains the shortcomings of getaddrinfo.
Yeah, I went in thinking that it was going to be some case of Apple wanting to bend the Unix philosophy to their will for their own desires and steer implementations in their direction, but no - they are simply pointing out a clear flaw in the design of the function in question for a usecase that does not apply only to Apple. Basically all OS vendors need to be doing something like this usecase to support IPv6 adoption.
I often find this is the case with Apple on a technical level.
For instance, their recent Spatial (stereographic) Video features uses a format that has basically zero current support outside of Apple—which is in fact just standard MV-HEVC [0] (with some extra optional metadata [1]), which is just the H.265 evolution of the standard H.264 MVC that 3D Blu-rays have used for a long time. (AFAIK no 4K 3D Blu-rays have been released, presumably due to space constraints, explaining the lack of usage of MV-HEVC outside Apple).
In piracy world, most re-encoded 3D movies just use objectively inferior composited 2D formats like half-side-by-side or over/under. And without diving in you’d just assume Apple was using some bespoke format to be evil, when in fact they are popularizing what should be the canonical, standardized format for 3D video.
[0] http://hevc.info/mvhevc [1] https://developer.apple.com/av-foundation/HEVC-Stereo-Video-...
> It's a little weird to me that getaddrinfo() is considered a "low-level legacy API"
I don't think it is considered legacy. The blog post gets that wrong.
(Whether it's "low-level" or not just depends on your perspective.)
> getaddrinfo() is the way to resolve names on Linux and I suspect the *BSDs.
At least on OpenBSD, all classical/standard DNS functions (getaddrinfo/gethostbyname/...) are wrappers around OpenBSD's libc asr implementation, written by Eric Faurot.
https://man.openbsd.org/man3/asr_run.3
https://github.com/openbsd/src/tree/master/lib/libc/asr
For quite some versions that modern networking APIs on macOS using Objective-C frameworks, starting in 2018.
See WWDC 2018's "Introducing Network.framework, A modern alternative to sockets".
NeXTSTEP might have been a UNIX, and macOS derives from it, but the whole UNIX story has always been to bring UNIX software into the platform, not to make it easier to move elsewhere.
macOS is still certified POSIX UNIX
EDIT: maybe not anymore, Sequoia isn't listed yet, https://www.opengroup.org/openbrand/register/xy.htm
Regardless, even if they renew the certification, they aren't obliged to expose in the classical POSIX APIs more than what the certification requires in features, or happens to be optional, implementation defined.
As anyone that has painfully tried to write POSIX portable code across big iron UNIX is aware of.
> Regardless, even if they renew the certification, they aren't obliged to expose in the classical POSIX APIs more than what the certification requires in features, or happens to be optional, implementation defined.
getaddrinfo() is part of POSIX, so it would be necessary to expose it:
* https://pubs.opengroup.org/onlinepubs/9699969599/functions/g... (2004)
* https://pubs.opengroup.org/onlinepubs/9699919799/functions/g... (2017)
* https://pubs.opengroup.org/onlinepubs/9799919799/functions/g... (2024)
And it is exposed, the code still compiles if one uses it, it just doesn't get additional nice macOS networking features that aren't explicilty required for UNIX certification.
Where does it say DNS encryption is required?
Everything in the UNIX compatibility layer is low-level in macOS. Not necessarily "legacy" though.
But this is no different than saying that, for example, calling out platform-specific native OS APIs from Java is "low-level." Which it is, from the perspective of compile-once, run-anywhere Java applets. macOS is a NeXT-compatible non-UNIX API, and you are supposed to use the macOS frameworks for everything. Calling down to BSD or even mach is definitely not what Apple wants you to do.
> macOS is a ... non-UNIX
Seems to be badly phrased and meant something else, since macOS is certified to be UNIX - https://www.opengroup.org/openbrand/register/ - contrary to Linux which is not UNIX-certified.
HN posted about this at least once - https://news.ycombinator.com/item?id=29984016
UNIX is/whas a commercial product from AT&T. macOS is UNIX-certified. Not the same thing.
Curious about this
Isn't the Mach kernel based on BSD?
How much of getaddrinfo is in the kernel, how much of it is pure "libc"?
No, mach is a microkernel, like L5. It was developed for the purpose of replacing the BSD kernel, by having a small amount of functionality in the kernel itself, and the rest of the BSD-compatibility layer implemented in user space. macOS' frameworks are then a layer on top of that.
IIRC most of the BSD compat was moved to kernel space for performance reasons (either just on macOS, or the version of Mach they built on top of)
The UNIX libcompat (a compatibility library for older UNIX functions) was integrated into Mac OS (specifically, macOS) rather than directly into Mach OS.
Here’s the breakdown:
• Mach OS refers to the Mach microkernel, which primarily focuses on low-level system functions such as task scheduling and memory management. It is not a full-fledged operating system, and thus, libraries like libcompat, which are higher-level UNIX compatibility libraries, would not be integrated directly into the Mach kernel itself. • Mac OS (particularly macOS, formerly OS X) is a complete operating system that includes the Mach microkernel, the BSD layer, and various other components. macOS has a strong Unix heritage, and libcompat is part of the broader Unix-like environment included in macOS to support legacy Unix APIs and applications.
Thus, libcompat was integrated into macOS (or its predecessor, NeXTSTEP) as part of its Unix compatibility layer, rather than into the Mach kernel directly
> Maybe things are drastically different on macOS, but getaddrinfo() is the way to resolve names on Linux and I suspect the *BSDs.
I'm not sure if this is the case in this case, but it might be worth noting that some system functions with the same name have drastically different internal/implementation differences between Linux/*BSD/MacOS. With there being differences between the *BSDs too.
So on some systems one function call is "the way", because its been maintained over the years, but then on another it might actually be old and not useful.
Of course, Apple does not want their app to call `getaddrinfo()` directly, because it would interfere with their internal XDR/NDS/IPS mechanism.
I can’t blame them but I personally would still have my apps use them, even knowingly that it would be made off-limit to iOS/iPadOS apps … soon.
What's XDR/NDS/IPS?
https://en.wikipedia.org/wiki/Intrusion_detection_system
No mention of apple, xdr or nds.
https://en.wikipedia.org/wiki/Extended_detection_and_respons... Not sure what nds stands for, but I assume it's something related to network intrusion, possibly the I in NIDS (network intrusion detection system) is missing
> getaddrinfo() is the way to resolve names on Linux
Not at all. That's just a glibc function, it's got nothing to do with Linux. People just assume that glibc is how things are done in Linux user space but it doesn't have to be that way. For example, systemd came up with its own resolved mechanism which turned out to be much better than the glibc stuff. I will probably end up inventing my own at some point as well since I'm working on freestanding software targeting Linux.
getaddrinfo is defined by POSIX and UNIX. Where the implementation is doesn’t matter. It’s portable, which is why it’s used. The slide deck referenced above talks about better implementations for various platforms, but they are all platform specific.
So OP might not be completely accurate, but getaddrinfo is _the_ way to resolve names if you are writing portable POSIX and/or UNIX code.
Linux and the popular Linux distributions are not POSIX compliant to begin with. Only GNU tries to be, and even GNU adds on a ludicrous amount of extensions because the truth is POSIX isn't good enough.
Why are you nitpicking? Linux demonstrates a high degree of practical compatibility to run software written against POSIX standards.
Not nitpicking at all. I just don't like how people see Linux as a "POSIX implementation". It's much more than that.
Practical compatibility is not compliance, the manuals document many subtle differences and incompatibilities. The practicality of it mostly comes from glibc which everyone uses and which does strive to be compliant. Even then it's a hit and miss, the so called "Linuxisms" crop up in the most unexpected of places. The executable path that people write in the shebang lines of their shell scripts, for example. It's gotten to the point some BSDs have started emulating Linux system calls instead of porting software. Even Windows did this once upon a time.
My point is glibc is not even guaranteed to exist on the system. POSIX is not at all mandatory on Linux. The POSIX interfaces are just one of the ways to interface with the kernel. It's also possible to bypass all the POSIX stuff and interface with it directly. Linux is the only operating system to offer this ability via the stable kernel-userspace binary interface. It's even defined at the instruction set level which makes it programming language agnostic. On Linux you actually can trash all that POSIX stuff and reinvent it all in Rust if you want.
> Linux and the popular Linux distributions are not POSIX compliant to begin with.
While not (entirely) wrong, not entirely correct either.
Good luck trying to compile and run any kind of software without providing getaddrinfo(), socket(), connect(), etc to userland:
* https://pubs.opengroup.org/onlinepubs/9699969599/functions/g...
> Good luck trying to compile and run any kind of software without providing getaddrinfo(), socket(), connect(), etc
I'm working on a freestanding lisp language with built in Linux system call support at least in part because I want to to prove that this sort of thing is possible. No legacy interfaces will be provided and yet I have no doubt in my mind that one day it will be able to everything you mentioned and much more.
Your comment above was that it is a glibc function, which is true, but it’s there for reason. It’s also a libc, musl, uClibc, and Windows Sockets 2 function: because it’s defined in POSIX 1.1 and extended in RFC 3493.
I have no opinion on whether it’s good enough (it seems like not if every platform has a connect-by-name implementation), just that calling it a glibc function overly simplifies it’s origin.
It’s also false to say only GNU tries to be POSIX compliant. There are 8 commercial UNIXes that meet some POSIX standard, another 8 that are discontinued (at least one of which was a Linux distro), and dozens that are mostly compatible. POSIX doesn’t care if that compatibility comes from the kernel or user space libraries.
POSIX isn’t good enough at what? Maybe you don’t understand what it’s goal is/was. POSIX exists for portability. It’s a minimal set of functions developers can target to get things done on any UNIX. Any OS will always have something beyond POSIX to differentiate it.
The point I tried to make is the getaddrinfo function is not sacred. It's not the way to do anything at all on Linux. It's just the function that glibc has, and most people use it. Whether it came from POSIX or something else seems like a minor detail to me. POSIX and glibc are not sacred either.
> Any OS will always have something beyond POSIX to differentiate it.
Linux is no exception. We should all be enjoying those exclusive features to their fullest extent. Not restricting ourselves to the lowest common denominator between them. Portability is a trap.
You're probably aware of c-ares, if not then check it out unless you really want to write your own.
(As an administrator I'm getting a bit tired of working around the differing bugs and behaviour of different resolver implementations).
glibc also has an async getaddrinfo_a function for asynchronous name resolution, with completion notification.
I will look into it even if I end up writing my own. Reading source code is always helpful. Thank you.
The correct term is "heirloom".
I'll pile on, as someone who has never developed for Apple systems: What APIs are supposed to be used for DNS resolution?
What "API" is there? Why doesn't an app doing system-wide DNS modifictions just modify the settings for default resolver?The Apple deck linked elsewhere in this thread suggests the developer's goal generally isn't "DNS resolution", the dev's goal is usually establishing a connection to a host/server/endpoint to start doing something.
So, usually devs should use the Java or Apple or whatever higher level OS API gets you connected the fastest, and that API is free to implement the connection however most quickly gets to the point of able to return data to the user (app or end user).
The API that returns a list of addresses is stuck doing that, instead of being able to parallize the entire "get connected" data flow.
> This library wraps around the dnssd framework and the c-ares C library with Swift-friendly APIs and data structures.
https://github.com/apple/swift-async-dns-resolver
It feels like Embrace, Extend, Extinguish to claim that a portable API is "legacy" and that its replacement is Apple-only.
As near as I can tell, Apple doesn't call it a legacy API. The article does, but the article wasn't written by nor does it appear to be quoting Apple.
That's what you use to create a utility like nslookup in swift, Apple does not want you to do any resolving yourself, just pass a hostname instead:
https://developer.apple.com/documentation/network/nwendpoint...
> Apple does not want you to do any resolving yourself
Which honestly sounds like a good reason to make sure you do do it yourself.
Not at all actually, passing hostnames means they can fully handle happy eyeballs for you and all other performance optimizations that you can do if you resolve and connect in one call.
It also means if you do it the 'Apple' way they might choose to intercept or modify responses. That seem in line with Apple's practices as a company even if they are not doing it yet. I feel anything they might do like that might be less likely to extend to what the article refers to as a legacy API.
If you were writing a Java application, would you do your own DNS resolution, or would you make a new socket address object and give it a hostname and let the api resolve the hostname for you? If you don’t hand roll your own dns protocol lookups, how do you know the OS, or Java or your socket library aren’t intercepting and modifying request out from under you? Heck, even if you use getaddrinfo directly how do you know your libc implementation isn’t intercepting and modifying the lookups on you? If the threat model you’re coding for is “Apple is a hostile actor intercepting and modifying dns queries” then you really can’t trust their provided posix calls either.
> If you were writing a Java application, would you do your own DNS resolution,
Java isn't known to nanny the users of apps developed in it's language. It's never even tried IMO.
> If the threat model you’re coding for is “Apple is a hostile actor intercepting and modifying dns queries” then you really can’t trust their provided posix calls either.
Sure, but that isn't the threat model. I described the threat model above, which is closer to "I don't trust a company famous for trying to nanny not to try to nanny if using their preferred developer frameworks, while I kind of trust they won't for a legacy API they barely pay attention to".
> Java isn't known to nanny the users of apps developed in it's language. > It's never even tried IMO.
I think anyone who's run into java keystore / cert problems would beg to differ.
> Sure, but that isn't the threat model. I described the threat model above, > which is closer to "I don't trust a company famous for trying to nanny not to > try to nanny if using their preferred developer frameworks, while I kind of > trust they won't for a legacy API they barely pay attention to".
Sorry, this just makes no sense to me. You have a threat model that thinks Apple is a malicious actor and will interfere with the implementation of DNS resolution in higher level APIs because they "nanny" their users and developers. But you equally think they won't bother to do such nannying at a the lower levels because they "barely pay attention to [those legacy APIs].
But we're talking about the same apple that implemented SIP which locks out even root (and by extension sudo) from being able to make changes to "system" directories like `/bin`, and necessitated things like homebrew having to migrate their entire deployment hierarchy from `/usr/local/` to `/opt/homebrew/`. The same Apple who replaced init scripts in BSD with their own proprietary `launchd`. The same Apple who swapped `bash` for `zsh` to avoid GPL3 licensing stuff. The same Apple that last OS update changed what signals were sent when doing illegal memory access, breaking a mess of java/docker things in the process. This is the Apple you think would impose hidden DNS interception and replacement at the swift library level but wouldn't bother to either change libc too or point getaddrinfo() at their own internal interceptor.
You are of course free to have whatever threat model you like, but surely you can see why other people find it a little inconsistent?
I would feel far more concerned that an arbitrary application that decided to do its own DNS resolution would be doing so for nefarious reasons, or might mess up the process (such as not supporting encrypted DNS in this case).
If you genuinely cannot trust the OS vendor, you don't try to tinker around in user space but you stay off their platform. Personally, this is why I don't have any machines with a Microsoft OS, and why I don't have a Playstation.
The concern isn't that Apple controls the OS and so could do nefarious rootkit type stuff, but rather they may try to nanny through the framework they prefer and push for all apps for their platform to be developed in.
But you are already on their OS, so they would always be able to do that. They make the kernel, the hardware and it's firmware, so it's a moot point and needless paranoia. Might as well use the API that gives a better user experience.
> so it's a moot point and needless paranoia.
No it's not. You are misunderstanding my point.
I'm not talking about Apple being able to patch the OS and control everything at that level - of course they can, but it seems unlikely.
I'm talking about a developer framework, a high level abstraction, where the method of resolving would be more likely to be intercepted - consider for example something like that on an iPhone with the justification being safety or 'for the children' or whatever.
That doesn't seem unlikely or improbably at all, and certainly not moot or any kind of paranoia.
Assuming arguendo that apple did want to do that kind of messing with DNS though - what's there to stop them from changing getaddrinfo() in the same way? As someone pointed out upthread, if you don't trust your OS vendor to do DNS lookups correctly, your only option is to not usre your OS vendor for DNS lookups, which is in the realm of Byzantine faults.
(And further, assuming arguendo that there was DNS meddling happening but somehow getaddrinfo() was exempt - now the user has one app that behaves differently to all their others, which is worse in every practical sense.)
> Assuming arguendo that apple did want to do that kind of messing with DNS though - what's there to stop them from changing getaddrinfo() in the same way?
Nothing, I already acknowledge they have the power to do rootkity things if they wanted to, but I don't consider that likely.
I do consider it likely they might do that kind of a thing at a framework level and try to push most developers to use it.
> As someone pointed out upthread, if you don't trust your OS vendor to do DNS lookups correctly, your only option is to not usre your OS vendor for DNS lookups, which is in the realm of Byzantine faults.
I responded to that as I'm responding here, by pointing out that isn't relevant to the threat model that I've described.
> now the user has one app that behaves differently to all their others, which is worse in every practical sense.
Not if that app actually gets the user to where they actually wanted to go.
It's not necessarily EEE. Maybe it's just that the old wheel sucks. They want a better wheel and so they reinvented it, hopefully better this time.
The corporations making proprietary software are not the only ones who have that attitude. I've resolved to make all my free software Linux-exclusive so that I can use Linux to the fullest. The Linux kernel is packed full of exclusive non-portable features that very few people take advantage of because they're obsessed with portability, POSIX compliance or whatever. I think that's a waste.
Portable software is usually sucky lowest common denominator software. We should not limit ourselves to whatever glibc offers.
You actually think that a Swift developer, developing against Cocoa APIs, targeting Mac and iOS devices cares about a portable API.
Because not sure if you know this but the entire software industry is built on high level libraries on top of largely portable code. For example this Swift library wraps c-ares a portable API.
no no, it's just "ours is better", "think different", and "it just works". /s
It kinda is yea: https://www.ietf.org/proceedings/72/slides/plenaryw-6.pdf
It's the same thing they did with Metal. Portable OpenGL is now "legacy." Straight from the 90s Microsoft playbook.
OpenGL is legacy in general, its been on 4.6 for almost a decade.
It is also legacy from Khronos point of view.
Well but the portable API is too low-level and error prone. What is the last time you used getaddrinfo? How often do you actually need to use it?
One can make a good technical argument based on the merit of the portable API without immediately resorting to the EEE argument.
getaddrinfo isn't its predecessors, there's nothing error-prone about it. The only thing that's nontrivial is falling back if the first server is unresponsive, and even there the obvious calling code is fine for almost all apps.
If you are using getaddrinfo directly, you likely wouldn't bother to implement Happy Eyeballs, for example.
And for almost all programs, that doesn't matter in $CURRENTYEAR.
Yes, this! I even wonder how else you would do this. By the way I worked with many IoT devices that do not use your dhcp dns but just hardcode quad 8 or similar
We recently had a developer join our team and he got stuck setting up his dev environment.
We use a .dev domain as a localhost alias, and turns out his ISP’s DNS wouldn’t resolve 127.0.0.1 (or whatever it is) for the .dev domain. Changing his resolver at the network level to 1.1.1.1 fixed it.
I imagine there are lots of difficult support tickets for app devs, and at a certain point they just hardcode the DNS to remove one variable from the equation when debugging bug reports.
Wayyyy back in 1995 or '96 I was working for a non-profit called "Next Generation Magazine" and our goal was to have young people write content for web sites to get their names out there. Back then it was all local ISPs, so we went to our ISP and asked for ngm.org and were stoked when we got it! We built out the site (Thanks to Building Killer Websites of course) and it looked awesome!
Only problem was that nobody in my family out of state could see it. It took awhile to realize realize that we never bought that domain. Our local ISP just added it to their DNS records, and since we all hooked into them we thought we were live across the 'net.
That’s incredible!
I remember one of the first times I used the Internet and opened my local radio station’s website from several states away. It was incredible to me that it worked and I also wondered why anyone across the country would care. The early internet was amazing.
Not resolving 127.0.0.1 or RFC1918 addresses or even ULA for IPv6 is done to avoid DNS rebinding attacks. For most end users that is probably the correct move.
My home router even seems to inspect any UDP/53 traffic and redact any responses containing local/private A entries, so not even switching to a public resolver bypasses the protection.
I agree that it’s usually the right behavior.
Interesting. I hadn’t considered it might be a security feature of his router!
In case you want to look into it further: My router actually allows adding exemptions to this policy on a per-hostname basis!
Sometimes I wish it would allow wildcards, but honestly that's probably just another way for users to shoot themselves in the foot (e.g. by adding '*').
> Sometimes I wish it would allow wildcards
pfSense for example uses unbound, and while it doesn't have a switch for disabling rebind protection, it does allow injecting arbitrary unbound config, which can disable rebind protection for any depth of a DNS zone or IP space. E.g.:
Most isp resolvers are shit and broken
That’s why it is imperative (at least, for a homelab hobbyist) to host your own DNS servers in your own VSP.
Totally, but most IoT customers are not homelab hobbyists, so I think its defensible for IoT vendors to just hard code known good DNS in their devices instead of relying on broken ISP resolvers.
Related story, there was a period of time where my ISP's resolver that would replace hostnames with no DNS record with their own ad filled garbage page.
So you mistype google.com to foofle.com or something and instead of getting "host not found" you get... ads.
Disgusting behaviour IMO.
Yeah, this report seems a little spun. The essence is basically that the encrypted DNS needs to go through the proxy, and there's resolver code elsewhere in the OS that doesn't use the proxy. It's a bug, sure. It could plausibly have interesting exploits, though none are shown. But it's not a very interesting bug.
Wait until you try to get the mac address on an iphone.
Yes, things are drastically different on MacOS. There's like a whole nother level of APIs ip there
/s
>UPDATE: Spoke too soon… The problem discussed here turned out to be specific to Little Snitch 6.1 and not a general issue in macOS. It will be fixed in an update of Little Snitch later today.
Dang can we get an update to the title to reflect this?
I put "[fixed]" in there temporarily but if that's not accurate we can change it again.
I would say the new source title is more accurate
> Warning: DNS encryption in Little Snitch 6.1 may occasionally fail
I see people tagging him as @dang, not sure if there's some backend logic to notify him but here goes nothing.
New title from source: Warning: DNS encryption in Little Snitch 6.1 may occasionally fail
There isn't.
> @dang is a no-op. The only way to get reliable message delivery is to email hn@ycombinator.com
https://news.ycombinator.com/item?id=36526450
> Update 2024-09-17, 7:10 p.m.
> After further investigation, we found that this bug has already existed at least since macOS 14.5 Sonoma (maybe even earlier, but we currently don’t have access to an older 14.x system for testing).
Did they test it ever worked with getaddrinfo? Or did they just see it worked once with CFNetwork and called it a day and then later publish a blog post saying it’s broken?
It's ridiculous us developers still have to jump through hoops to save around older versions of the OS for testing. There is 0 technical reason why Apple can't let us downgrade.
Can someone fill me in on this? What hoops have to be jumped through? The last time I used macs, there were no issues downloading and installing older OS versions, but I have not used them recently.
What?
You can do a fresh install of an older macOS version whenever you like (you need to enable that option in the rescue system tho).
You can also run older macOS in a VM (the hypervisor framework keeps getting new features that make guest macOS more fully supported).
Name an OS (ok maybe NixOS) that allows you to do clean downgrades out of the box. Also wonder what's gonna happen to your data in e.g. Postgres if you blindly downgrade.
yeah it feels like they decided bank accounts flush with cash were a better investment than legacy system support
Newsflash, they've always thought this way.
Which is pretty wild too, considering they're selling the product and the new OS came out yesterday.
Sequoia also breaks an application's ability to use DNS (or presumably anything UDP-based) if the macOS firewall is enabled, and an app is listed as "Block incoming connections". https://waclaw.blog/macos-firewall-blocking-web-browsing-aft...
I can't reproduce this. Some people say it has to do with ESET: https://www.reddit.com/r/MacOS/comments/1fievr5/updating_mad...
Confirmed, it is from an old ESET network filter: https://support.eset.com/en/alert8723-network-connection-los...
It's easily reproducible with a fresh macOS install. Yes, ESET has its own issue. But this is a problem in and of itself. https://imgur.com/a/Nr7Gk6c
I was talking about the "Can't change" part:
> Going to the firewall settings screen, there can be no way to toggle access for the browser.
Before Sequoia when using OpenDNS for VPN, could be on VPN and iMessage and other apps still work, but since Sequoia, when on VPN iMessage (text messages) etc no longer work. Once I disconnect to VPN all goes through. Is this related at all? Do have macOS firewall enabled. But not block all incoming connections.
Disabling the firewall for testing is simple enough. If things work after turning off the firewall, then this is your problem.
After upgrading to Sequoia, I could not browse with Safari or Mozilla. What fixed it for me was to go to the DNS settings for my Wi-Fi connection, and add Google's DNS servers (8.8.8.8. and 8.8.4.4). They replaced the autofilled DNS servers that were there.
Were the autofilled DNS servers in RFC1918 private space (10.0.0.0/8, 192.168.0.0/16, etc.)? I had issues after the upgrade with Google Chrome being unable to access hosts in these ranges, and fixed it by going to System Settings -> Privacy & Security -> Local Network and toggling Google Chrome off and on again.
No, they weren't local. I have no idea where they came from. I couldn't even delete them, but when I added the Google servers, they autofilled ones were automatically deleted.
They came from DHCP or RDNSS in RA.
> could not browse with Safari or Mozilla
FYI, it looks like Firefox fixed this.
So not macOS error?
> not macOS error?
It worked before I upgraded to Sequoia. But I don't know enough to point fingers. Just mentioning that turning off the firewall long enough for Firefox to update fixes the problem.
Honestly, I'm fine with that. Applications themselves should not be resolving DNS outside of what I set in settings.
The reasons applications do this is to prevent users from blocking telemetry etc. It's my computer, I should have final say on what goes out.
There is no such thing as a remotely cross-platform DNS resolution API that has the system do the lookup and does not utterly suck for asynchronous use.
I suspect "cross-platform" is doing a lot of heavy lifting for your claim. Browser engines and application frameworks built on top of them have no trouble using platform-specific APIs under the hood.
Yeah, but frameworks are yet another level of abstraction and dependency that just kills momentum.
All major browsers now implement the ability to use a browser-defined resolver.
Yes, and some of them even make it the default under some circumstances.
I agree with GP that this is generally not a great trend.
I have one browser setup to do DNS differently than another. I don't want to have to set it at a system level and then need multiple systems just to run 2 browsers with different DNS lookup
Seeing this getting downvoted is fucking wild.
I remember 20+ years ago when one of the most commonly seen attacks was malware configuring a proxy server in Internet Explorer which by design overrode the operating system's configuration.
What a lot of software does today by ignoring the operating system in lieu of their own shit is just like the above. If your program doesn't (or can't) respect the operating system, your shit is malware and you should reconsider who you write code for.
> Seeing this getting downvoted is fucking wild.
If you consider the source of income of what's most likely a considerable portion of the HN community, I think this makes more sense. Apple is one of the only companies interested in preventing tracking, and it hurts, in the billions sort of way [1][2].
[1] https://www.forbes.com/sites/kateoflahertyuk/2022/10/08/appl...
[2] https://www.forbes.com/sites/timbajarin/2022/07/26/apples-do...
Those ideas are not isomorphic.
One malicious overrides universal network communication while the other just conducts DNS queries limited to a single application domain.
You are describing something that violates system setting for it's own benefit instead of the end user.
You are describing malware. Benign malware is still malicious, even if it does no active harm. Intent (of how the software operates) matters.
>just conducts DNS queries
Queries that will ignore configurations you set.
If I see something ignoring/evading my configured DNS server, that shit is fucking malware.
At some point in my copious spare time, I plan on writing software to allowlist in my firewall outbound connections only to IPs resolved using my DNS servers.
You could also configure your router to intercept rogue plaintext DNS lookups on your network with responses from a resolver you trust (for example a Pihole, Cloudflare or Google Public DNS, Quad9, PCH, etc). Adding something like Pihole would give you comprehensive blocking and custom internal DNS entries too.
I already redirect DNS queries to my own DNS servers running unbound, and block UDP and TCP ports 25 from machines other than these from going out on the Internet.
This will force machines misconfigured with 8.8.8.8 as default resolver (cough, systemd) from leaking my browsing history to Google, thank you very much, but won't stop DNS-over-HTTPS like Firefox, or more insidious devices like fallback IPs hardcoded in SmartTVs and other IoT devices (they are on their own VLAN with all traffic logged, but it's not as if I have time to inspect their traffic for suspicious behavior. There are blocklists of DoH, but at that point it becomes a whack-a-mole game, and it makes more sense to block anything that is not the result of a legitimate DNS query instead.
This would only be enforced on untrusted machines like Macs, iPhones, Android devices, IoT devices and Ubuntu machines, as opposed to trustworthy OpenBSD and Alpine Linux servers.
How would that work? Do you only access a really small/known set of IPs? Or would you program the firewall to only allow connections to an arbitrary IP if it had seen a DNS query to your preferred servers go out and return that IP within a few seconds prior?
In the latter case, would you have to aggressively disable local DNS caching on devices to make the behavior work (is that even possible on some devices)? How would encrypted DNS fit into this scheme?
The second. The only way untrusted devices connect is to IP addresses that were resolved by my DNS servers so I know what traffic is happening on my network. My DNS servers handle their own encryption via WireGuard to bypass ISP snooping, so I don't need Mozilla's DoH and Apple's and CloudFlare's I do not trust at all.
To avoid race conditions, the trusted DNS servers would add the result IP to the firewall allowlist table before returning it to the client, so either implement it as a Caddy proxy module (I already wrote a DynDNS module for Caddy so I know how to make that work). Or alternatively use unbound's dnstap support. I just need to implement some reliable and secure protocol to send those requests from the DNS server to my OpenBSD firewall running pf.
Have fun troubleshooting Java apps w/their own cert stores...
I am fine with the only Java application I have used in the lease decade not working. I did not even bother putting a JVM on any of the OS I installed in the last 5 years. So yeah, I’d rather have fewer security holes.
Yep, I wish they would go the full way and block socket access entirely so your own outgoing traffic is always introspectable even with cert pinning. It would make it blatantly obvious when apps try shady shit.
I had a great Windows firewall like this about 20 years ago. It would pop up a dialog for every network request from an app. You could block or allow based on port or destination, or "block all". It was amazing, because as you say, it made it very obvious when an app was trying shady shit.
I would love to have that back, but I was never able to find a firewall so hostile to the user experience of the general population.
It sounds similar in spirit to Little Snitch, mentioned in the article (on macOS, but which inspired OpenSnitch, which runs on Linux). It is awesome indeed, if a bit overwhelming at first. Most regular users would just uninstall it to avoid the constant barrage of requests initially, and then every time a new piece of software tries to connect to anything.
Zonealarm?
Yes! That was it. I'll try it again.
Shady shit? Not every network request is a call to an HTTP REST API.
Blocking socket APIs would break every app that supports other protocols. Goodbye file transfer apps, VPN apps, file sync apps, database tools, SSH clients, remote desktop clients, audio and video conferencing apps, etc.
As long as I can add exceptions for those apps to my firewall, I’m kind of… okay with that?
Shady shit meaning really obvious when you're making http calls with encrypted opaque blobs.
The title sort of implies this is intentional or privileged to Apple, while it rather seems more like just a bug.
I also wish people would post the FB numbers and the details of their report when they say they've reported things like this.
Devil's advocate would say: They could do this and make it look like a bug that never gets fixed in order to avoid backlash. How it gets achieved is flexible if the goal is met.
Why would they be afraid of backlash on such an obscure, technical feature? They never were in the past and are expected to take controversial technical decisions by now. And by “now”, I mean in the last 30-odd years.
Yeah, if it was intentional, it would probably be a hard-coded, encrypted URL. Some devices are starting to do that to get around ad blocking.
Good thing you can still see the domain over the network if you control the network.
You can’t control anything if they do DNS over HTTPS to a hardcoded IP they control and cert pin so you can’t MITM the connection, can you?
That's what a firewall is for.
If the pinned cert is stored on some kind of ROM chip you could probably rewrite it to replace it with your own cert.
You can at the very least block traffic to the hardcoded IP.
I maybe imagining but I feel like deja vu that there will be a problem with DNS that would affect Little snitch., Mullvad and others with new releases of iOS and Mac. If true I would really question what apple is doing during their months long developer and beta testing.
I was confused at the Little Snitch mention, and then reading further it just seems like a LS bug, that it only works in certain cases.
Well, seems this is the LS blog, so only confusion is why this is portrayed as a macOS bug? I'm not saying it's wrong, it's their domain not mine after all, it just doesn't seem to be justified in TFA?
If the OS allows the registration of a DNS proxy, and some calls bypass the proxy, it's squarely an OS bug.
Doesn't getaddrinfo respect /etc/resolv.conf? So LittleSnitch should install itself there if it wants to be used by getaddrinfo.
Besides, apps can always make direct lookups to a resolver of their choice, bypassing any resolver hints of the operating system.
The /etc/resolv.conf system is woefully inadequate. It doesn't have a concept of per-interface customization so you can't customize according to the currently active network interface. It doesn't distinguish between DNS configuration delivered by the network administrator (which can and should be changed remotely) versus set by the computer administrator. It doesn't work very well with VPNs where a specific DNS server is used for resolving addresses on that VPN.
It wasn't.
If I recall, Apple deprecated use of certain network apis for third party developers. But Apple’s own apps (App Store) do not have these same restrictions. Thus, when trying to filter network traffic via app firewall via new APIs. It would fail since App Store uses legacy APIs.
Maybe part of this old bug (that I thought was fixed)
getaddrinfo() is not a legacy API, it's a standard cross platform API for doing DNS lookups.
Specifically defined by POSIX: https://pubs.opengroup.org/onlinepubs/9699919799/functions/g...
Funny how that goes: macOS is POSIX certified but no other desktop BSD or Linux is.
According to https://www.opengroup.org/openbrand/register/ macOS 15 is not certified.
Because it's an expensive and lengthy process. And every version would need to be re-certified.
There have been POSIX-certified Linux variants. But the open source projects you use don't bother (for obvious reasons) and commercial derivatives like Android and ChromeOS don't need it. Similarly Window NT was POSIX-certified way back in the day yet its descendants aren't, even though they implement the same API set (via very different technology).
It's POSIX which Apple generally abides by except for timer_create. macOS is historically officially a UNIX, which would require getaddrinfo.
it is apparently on Mac and arguably with good reason. See this comment https://news.ycombinator.com/item?id=41572770
I always love the announcements of, "Bug found in new OS release! EDIT: Actually it's been there for a while!"
I use routedns [0] as my local stub resolver so that I can pick and choose which requests go to where and also what transport they use. It can also blocklist, re-write, cache, load balance, and/or handle fall back requests; so it give you lots of control.
I use a stub listener on localhost:53 for local requests and then forward them via UDP QUIC (TLS 0-RTT) requests to Cloudflare (1.1.1.1) with caching for most requests. Fast and reasonably secure.
[0] https://github.com/folbricht/routedns
> After further investigation, we found that this bug has already existed at least since macOS 14.5 Sonoma
Isn't this an inherent risk when attempting to do network stuff in userspace? You're at a very high level so hoping that lower level things comply seems risky if DNS encryption is critical to your use case.
Apple removed support for kernel extensions, and instead added a bunch of APIs that allow to do network filtering etc in user space. Unfortunately, some of their networking code just bypasses those network filter extensions (probably because of bugs) -- this is not the first time the developers of Little Snitch publicized a bug like this.
Dupe: https://news.ycombinator.com/item?id=41568128
The battle of DNS resolving ownership rages on: who has the rights to set the DNS nameservers/resolver.
As a long-time DNS security researcher, the ultimate and final end means would be to mirror the root servers, but I assert, for now, popping in your own `resolv.conf` should suffice, … again, for now.
https://tailscale.com/blog/sisyphean-dns-client-linux
"Mac OSX has complex paths into name-to-address translation and a single entrypoint is not well enforced."
this is not "bypass encryption" this is "uses a range of ABI/API bindings in code which don't expose well into a single control point"
I wonder how little snitch sets the dns encryption up. In macOS, you need to setup encrypted dns via a profile System (Settings => General => VPN, DNS & Device Management) and then in the browser. However, I think terminal and appstore still use whatever server is obtained via DHCP and is not encrypted.
This is why I firewall egress port 53 at the router level.
> "To protect (DNS lookups) from prying eyes, Little Snitch 6 offers a new feature: DNS encryption."
Browsers such as Firefox have offered this directly for a while. Of course, that only covers DNS lookups made from the web browser, but it doesn't rely on OS-level hooks that (at least in Apple's case) can break.
What am I missing here? Reading the article, it appears that Firefox is the browser that seems to be bypassing.
They're using Little Snitch as an OS-level DNS proxy, which should intercept all DNS requests from any app and encrypt them. But, depending on what API the app uses for its DNS lookups, some DNS requests do not go via the proxy. Presumably Firefox, in its default configuration with DNS encryption set to OFF ("Use your default DNS resolver"), uses one the affected APIs.
Ah, that makes sense. Ty.
Plugging https://www.supernetworks.org/ -- when on wifi/vpn all DNS will go up over DNS over HTTPS as plaintext DNS is DNAT'd to CoreDNS which is by default configured to use DoH.
Deploying DNS encryption on macOS is in general really tedious. Applying it as a system or user profile has different results. Switching between providers or temporarily disabling DNS encryption is painful.
I also still haven't figured out how to get SSID-based switching to work, does it even?
My read of this is that it shouldn't affect pi.hole given the system's default nameserver would still received by DDNS and thus be the pi.hole? Or do these requests go somewhere that's hard-coded?
No this appears to be if an application registers a DNS resolver proxy on the local system, getaddrinfo doesn't use the proxy, and presumably just hits whatever the network interface's configured DNS server is.
I’ve had issues using the Resolv library in Ruby when I’m connecting to the internet via a tethered iPhone. Never ran into that until Sequoia. I wonder if that’s related?
TBH I’m too lazy to dig in and find out. Has anybody else run into this issue?
See: https://waclaw.blog/macos-firewall-blocking-web-browsing-aft...
If disabling the firewall (for testing) solves this problem, this is likely your issue.
Am I susceptible to this if I redirect all DNS traffic on my network to a pihole, which is the only device I let make external DNS requests?
If pihole is using DNS encryption for upstream lookups, the this would only affect you on your local network.
In other words, it would be unencrypted to the pihole but encrypted when going out to the internet.
Can some ELI5 why you'd use a proxy rather than reset the server name?
the (complicated) rules:
also try with a domain that existsApple ignoring standards again.
> UPDATE: Spoke too soon… The problem discussed here turned out to be specific to Little Snitch 6.1 and not a general issue in macOS.
Not really.
I wonder if this affects iOS, too
I would believe so. I have a custom DNS profile setup that redirects a few domains to a server I run. The server has custom SSL certs issued by a private CA. I the certificate installed on iOS as a trusted root certificate.
Everytime I'm connected to my home WiFi I would randomly get `peer closed connection in SSL handshake (104: Connection reset by peer)`. I have absolutely no clue why it does this and this issue goes away when I'm connected on mobile data.
Now I'm guessing that it is bypassing the DNS profile and resolving it using my ISPs DNS or some other way.
> I would believe so.
It won't, it was specifically a bug in Little Snitch (which doesn't currently run on iOS, I believe.)
"The problem discussed here turned out to be specific to Little Snitch 6.1 and not a general issue in macOS. It has already been fixed in Little Snitch 6.1.1."
Oh this is not good. I hope the vpn providers chime in, the MullVads and ExpressVPNs etc
The standard way to use dnscrypt-proxy is to set the resolver to 127.0.0.1.
Does Little Snitch do things differently?
Hmm. I use NextDNS for this feature. I think. May have to do some testing to see whether or not it is operational at all.
macos may bypass LITTLE SNITCH'S encrypted dns proxy, more like it.
Why is a DNS proxy needed? My assumption is that you configure DoT or DoH (which I interpret as DNS encryption) somewhere in the settings of the OS.