Amen to that. It's funny how virtues like accessibility and good API design matter just as much as ever in this new era. Good ideas don't go out of style.
Hi Saagar! One could indeed use this method to export this information en-masse, however that would require the kind of automated access that the site ToS explicitly forbids.
For the Swift standard library and other open-source frameworks, you could probably extract this from documentation comments using DocC, Jazzy, or the like.
I think HTML -> Markdown is a bit of a red herring.
In many cases, a Markdown distillation of HTML can improve the signal-to-noise ratio — especially for sites mired in <div> tag soup (intentionally or not). But that's an optimization for token efficiency; LLMs can usually figure things out.
The motivation behind Sosumi is better understood as a matter of accessibility. The way AI assistants typically fetch content from the web precluded them from getting any useful information from developer.apple.com.
You could start to solve the generalized problem with an MCP that 1) used a headless browser to access content for sites that require JS, and 2) used sampling (i.e. having a tool use the host LLM) to summarize / distill HTML.
This is only possible for some of Apple‘s open source Swift code, including the Swift standard library. This is not the case for hundreds of other SDK frameworks, such as SwiftUI.
Even for those open source projects, there is still some value added in the generated documentation that isn’t directly available from documentation comments, such as type members and protocol conformances (though a LLM could certainly suss that out with the right context).
Following up — I just pushed a fix for this. This latest version significantly improves how references like protocol conformances and default implementations are rendered.
Once upon a time, I built a project called `swift-doc`, which eventually got Sherlocked. I think what I was most upset about was their decision to call their thing "DocC". Like, adding redundant consonants to avoid name collisions is my shtick.
In my experience, coding agents seem to do a normal fetch when provided links. Which makes sense — headless browser automation is expensive, and only really necessary for interacting with a webpage.
But with these RLHF'd AIs, being confident and helpful as they are, it took me a while to realize that they couldn't actually read the Apple developer links I was giving them. Like a kid who can't read the chalkboard, but doesn't realize they need glasses.
It's expensive for sure, but probably a drop in the water compared to the cost of renting H100. Plus, it would be a massive boost in terms of data quality/quantity for them. But maybe you are right, I'm just surprised it's not the case
I agree, it'd be great if Apple provided accessible documentation in the first place. Time was, Apple published self-contained docsets that you could download and read offline.
Apple's ToS pretty explicitly forbid the kind of automation required to download everything. But even if someone did that, it'd only be a snapshot in time. And a lot can can change between OS releases.
As for the hosted web app, I wanted to provide this as a public service. I plan to open source it, so anyone can self-host instead, if they're inclined.
Yeah, pre Swift documentation was very sufficient, dense and locally explorable, since then documentation started to resemble .h copy pasta without comments.
reply