I'm creating an infinite canvas that has all your organization's code and documentation on it. If you zoom in, you can see the code, if you zoom out you see the big picture. By giving everything a place on the map, it becomes easier to figure out your way through the landscape and understand the systems. Different modes can you show you different things: code age, authorship (bus-factor, is the person still with the company etc), languages used, security issues. There's time-travel, think Gource for all software in your company, and maybe the most fun: a GeoGuessr for code. Select the repos for your team (or if you feel confident, of the entire org), you get a snippet and have to guess where it is. The plan is for LLMs + tree-sitter to analyze all the code and show relations to other systems, databases etc.
I had the idea 2 years ago, but starting building in earnest 2 months ago. Spending all my time on it now, minus 3 or 4 days per week of earning money. Currently looking for a GTM/sales-oriented cofounder in NL.
Man...If you built this for large mainframe codebases, I think every outsourcing provider would use it. many of these apps have >1 yr parallel runs even when rewritten because there is so much dormant and seasonal code that it is very hard to be confident in the apples:apples functional comparison over any shorter timeline.
Thanks, that seems useful! Too bad my entire backend is in Rust and works directly on git repos instead of checked out code.
I want to build a local company in my city of Utrecht, primarily on-site. That gives me the most energy and fun and is something that I want to optimize for.
On a smaller scale it reminds me of the original concept of Light Table, which let go of the abstraction of individual files in favor of editing your code in a tree like structure. It's a shame this concept seems to have died out, I'd be curious about alternatives to plain file based UX.
Cool. My long-term vision for software development would be to make a new programming environment that structures code in ASTs and stores it in a database rather than file based. However, that will have to come later. First this :)
You should check Unison language. Each Unison definition is identified by a hash of its syntax tree. So if two unrelated functions have identical AST this means they that from language perspective it's the same function.
Sometimes I daydream that we moved beyond text files and had a format with a UUID per line.
Version control could reliably track movement between files, we could identify copy and paste (and prompt when refactoring), and code coverage changes would be accurate.
Intriguing - what’s the default visualisation? You mentioned treemaps, so a treemap of each directory with the files inside that you can zoom into to see the code? How is tree-sitter helping you here?
Treemaps are the default, and are great for finding outliers with heatmaps, but terrible in terms of explainability. So now I'm heading into architecture diagram territory, and tree-sitter is helping with extracting the names of classes, functions, variables etc. LLMs can make decent diagrams out of this.
We use IcePanel for a similar functionality but like all diagramming solutions it suffers if you don’t constantly feed it. If you can solve that problem you’re definitely on to something.
That’s awesome! I’ve always wanted something similar like a Prezi presentation where you could navigate through different layers of the architecture down to the code.
Edit: It would be great if you could set the context and AI would generate it. It would make as an amazing addition to a standard Readme.
Further, at one level it could show endpoints and function signatures with parameters and how the argument usually looks as a value.
Which brings up another point, why doesn't Cursor or others allow me to say, "I'm in debug mode, show me if a value is dissimilar the values you normally get."
I took inspiration from GitDiagram.com but pass a "richer" description of the codebase into the LLMs based on info from tree-sitter. It generates explanatory diagrams quite OK, but this is only a first step.
This sounds lovely. I am a spatial thinker so this is right up my alley.
How do you deal with different kinds of groupings and connections? For example, some things could be connected because they are “integrations”, or because they deal with notifications, or because they’re available only in the enterprise plan. Not all related things are related in the same way.
Still a lot of thinking to be done here to be honest. I've built a very fast canvas with zoom-to-code, parsing the git history, code age overlays etc, but understanding the architecture and connections is the next big thing I have to figure out. Plenty of ideas though!
I've wanted to work with code thus, but it becomes a problem in that things are only really readable/relatable in single-screen chunks, and when one tries to show more than is accurately related on a single screen it becomes an unreadable, confusing, blurry mess.
It's interesting how everyone reading my description must have a different picture in their mind :) Right now I'm doing super simple treemaps, but I'm close to starting work on diagram overlays. No screengrabs yet.
Thanks for the support! Haystack is definitely interesting. I'm following a bunch of companies in this space, like Haystack, Greptile, Territory.dev, IcePanel, Spectral, Codeviz.ai, eraser.io.
What’s the general algorithms or patterns for these infinite canvas type things? I’ve always wondered. How do you handle interactivity also? Seems all very complex with a html canvas…
There are engines for this. I started out with Fabric.JS but it turned very slow with hundreds of repos. Then I moved to PixiJS (a game engine) which is super fast. I feel like I'll need to move to WASM / OffScreenCanvas and implement a custom engine, like Figma is doing.
Went through your profile searching for a demo and discovered fractional CTO. Could you share a bit about how you evaluate new gigs and figure out how much time each one would need?
People ask if I'm available and if I find the work interesting and they can pay me I say yes. I have never looked for work since starting and have slightly more requests than I can fulfill (almost everything through my network). For efficiency I do full days of work + ad-hoc meetings when necessary and no more than 2 days per week per client.
Note that I'm not always a CTO in the strictest sense of the word, I like doing complex technical challenges with software companies and sometimes just lead a complex project like implementing ISO 27001 or re-packaging a software suite for on-prem deployment.
Sorry to off topic further but I also have a question - how do you deal with recruiters? As a consultant I get grilled and rejected if I ever have overlapping projects or anything that even remotely looks like more than 40h a week. They're very intimidated with implications of being "overemployed"
This isn't a problem for me. I mostly stopped getting interest from recruiters when I became CTO. Now I just get calls from CEOs or CPOs etc and they understand what I offer them. I have a rule for myself to never charge for hours when I'm not productive and never charge overlapping hours though.
I would pay for it as an individual. Just things like going into code and spending 3 weeks reverse engineering everything is not ideal. Especially if it's something where the entire team has quit or been laid off. And lately the solution for that is just rewriting the damn thing lol.
How much we're willing to pay is a whole other question. I feel like this is the kind of thing that Cursor already does by itself but it's just not releasing a user-readable output of it.
It won't likely be a subscription thing, but one off payments per repo makes sense, and there should be some kind of satisfaction guarantee or say, charge to have the output in a human readable format.
Monorepos are also a pain. On the front end, they sometimes share design. On BE they may share databases. It would be cool to break it down into DDD-style domains if applicable or propose things like anti corruption layers. More often it's like a "pacific ocean meets atlantic ocean" kind of thing, where you can tell there's a difference in the way things are done, but it's not entirely clear where the border is. This would probably be worth a lot more.
To a much lesser extent, an architectural copilot would also make sense. On the front end, we have a lot of redundant components. Say a button might be PrimaryButton, but the same thing is GreenButton or FilledNoOutlineButton by other devs. We tried documenting this which just ended up being a waste of 1 week because nobody read the doc. It's worse with complex components like TwoButtonModal vs TwoButtonModalWithClose. And what happens is code is always built in parallel; people don't realize that the designer's new style applies to both teams so you get two people building the same components at the same time. Not a major problem, but I think this is worth a few cents every PR.
Ultimately it's hard to gauge. Like Copilot underdelivered, Cursor overdelivers, and yet both essentially do the same thing. I guess the amount we're willing to pay is just vibe-based.
I think you should check out the Light Table kickstarter[0] which originally had a similar premise to yours, and raised several hundred thousand dollars. I personally put in $50 to be in the beta, and that was almost 15 years ago.
I think you would get a lot of takers if you could make a convincing demo.
Demo yes, feel free to reach out if you'd be willing to pay as I'm slowly starting to look for beta customers. You can also ask me to be put on the waiting list. Downloadable: I think in about two months.
Only to prospective buyers/partners/team members for now, as I need to carefully manage my time. If that's you, please reach out :) In any case, based on the interest in this thread I'll do a separate Show HN thread in a couple of weeks/months!
mapping the software items (and whatever related)... has been a conquest ever since. Although what you may infer without knowing deeper essential details (engines and what data drives them how), would be static. Still, "communication diagrams" are very useful meta-view.
some suggestions:
* have ways for multiple views and/or start-points and/or both up/down directions. e.g. hierachy vs reason/effect vs dependence vs whatever-else. Then think about animating those in time
* heat-map over the views, as e.g. churn(changes)-in-time, or usage(number of dependents), etc
* requirements engineering kind-of-view ~ may overlap with dependency (both directions!) but with explicit requirements/assumptions tied to respective stakeholders. Though this may need links to/from JIRA and similar issue-trackers etc
* check Wardley maps - yet another view, starting from customer/stakeholder/vendor-points. Also may move in time. It may need user decision on which things are big/separate-enough to surface on that view - sometimes a single script is on-par with whole subsystem
* future maybe - growing above into zoomable per-project thing (more proj.mngmnt than just code, incl. related e-mails etc) - described here: https://news.ycombinator.com/item?id=43060108
* probably more.. will add if something else dawns on me
Lots of good ideas in your comments, thanks. Like you say, you want call-graphs, hot-spots from your monitoring systems, seeing user interaction diagrams, links to other components, amount of engineering time spent on features, mapping security issues, tech debt, etc. The challenge is going to be to keep a simple product :)
These are not meant to distract you from current goals/ways - pick those which ~80% match what you already have, postpone/abandon the others. Like, devOps view is very different from all else, with somewhat weak links across to the product-related or build-related views (heh, new view: costs-related. Be them people-hours or compute/storage). No need to compete with focused tools in that space, (e.g. see [0]). Though some linkage/hints might be doable and valuable. (also see some github-repo visualizers below.)
3+ years ago i landed as architect onto a working system having 300Kloc js codebase across 20+ repos, plus other ~100 unused ones with unknown Mlocs in them, running over 25+ containers in 2 datacenters.. without neither good architectural diagram nor deployment diagram nor runtime flow diagram... took me quite some months to build some mental image of which is what and where and why. It was event-sourcing engine so event-flows mapping from parsing the source was doable (few weeks, python+esprima+graphviz), but all else... nope.
I interviewed ~10 CTOs and the problem of codebases being too difficult to understand, people outside of the team having no clue what's happening and documentation being outdated is obvious to everyone. But when I asked how this idea could help, my take-away was that they couldn't imagine how it would work from my description. So now I'm building it and I hope that seeing it in action will convince people. I have some non-paying pilot customers and over the next couple of months we'll continue the validation :)
I had the idea 2 years ago, but starting building in earnest 2 months ago. Spending all my time on it now, minus 3 or 4 days per week of earning money. Currently looking for a GTM/sales-oriented cofounder in NL.