I loved the TinaCMS blog post on how they still use flat files as their source of truth, but have a level.js database that powers a bunch of their functionality. https://tina.io/blog/Tina-CMS--Leveljs/
You can blow up the db & regenerate it very quickly. All the data is still flat files. But the advanced functions harvest & are derived from the source of truth. Seems like a perfect compromise.
I haven't done it yet but I have a git-auto-cimmit project I intend to use with logseq, that'll help me better keep a record of changes over time. That ability to have a flat file source of truth feels hugely hugely empowering to me, and I have a hard time imagining having anything else.
You can blow one database (level), but not the second one (git) which you also need to sync, and this doesn't cover the issues of changing the data layout, git isn't always great in tracking those (doesn't store renames). Then also all your data is duplicated in a git db and a flat file db
I'm not sure what you are talking about vis a vie "changing the data layout". You can do whatever you want, it's just flat files. Git is great at tracking file renames.
Trying to say data is duplicated is basically wrong, as far as I'm concerned. There's a materialized view, which very few people I think would argue breaks the idea of data-nornamization. There is only one data source, the other is a generated view which can be regarded as ephemeral data.
I sympathize a lot with the parents perspective that Joplin switching to database backed world is a pain. I think the appeal of well managed flat files, controlled by git, is much more directly manageable & has much broader tooling. I appreciate attempts to raise dissent against this system, but none of your points carry much water for me personally. It strikes me as pretty weird having these litany of basically wrong non-issues to swat away?
But you don't need two, one in git (which stores the full copy) and another one in flat files when you can have only one in the database and then store just the diffs (and have an easier way to discard history when needed vs git, which would be especially painful with many clients)
Git uses heuristics to identify renames, and not a great one (and also limited in number of files), which is a known limitation of this "great" tracking
And data structure means renaming/moving your flat assets, even if git catches them, your note app will not. Or breaking cross-references due to that. Ability to regenerate broken wouldn't help you here
What "broader tooling" do you mean for notes when you don't even have a real system, just something all you could imagine as the best?
Also wild to ignore smartphones as yet another non-issue
Feels exceeding weird to me to draw a distinction between git and the flat files it has.
Git does come with some challenges (deleting a file from history is a good one; the filter-branch command for doing so can be a beast!), but overall there's so many git tools available. Gitlense, GitHub, magit, gitk, and hundreds more ("broader tooling"). And is imminently google-able. Love it or hate it, it's a well known system, and TinaCMS using git seems like a huge advantage to me (with my reasonably competent modern developer hat on) in any circumstance.
Tools like logseq and TinaCMS have ways to rename pages, without breaking references in their markdown. Seems solved.
Smartphones are pretty good at using web apps I've found so not sure why you once again are leaping to attack again.
"you can have only one in the database and then store just the diffs" has big "draw the rest of the owl" vibes to me. Maybe it is really easy. But I have to learn that system & figure out how to work with that history. I see very little benefit in a simple system, that is custom tailored to one particular use, that will have a range of limitations I can't assess or judge well ahead of time. We have a great tool for handling control of different versions of text over time. I'm here for it. It seems wild to me to imagine DIY'ing for your specific app.
But I don't care about "overall", I've asked you about note-specific workflow, and "hundreds" of bad/irrelevant tools hurts googlability, not helps
> "It seems solved"
Care to share specifics? You've changed file names/folder structure/image file name/format etc. in your favorite flat file tools
How does a system reconcile those edits happening outside its garden?
> Good at using web apps
A single good example for this domain would be nice. Say, you have your flat files in Dropbox with .git? Which web apps could I use?
> "draw the rest of owl" vibes
More like "imagine someone can draw it" just like you had an image of the owl in this meme in your mind.
By the way, if you plan to use autocommit, so basically have just a linear series of diffs added automatically, what does git buy you?
> but I have to learn
Well, yeah, doing something better requires learning how to do better
You can blow up the db & regenerate it very quickly. All the data is still flat files. But the advanced functions harvest & are derived from the source of truth. Seems like a perfect compromise.
I haven't done it yet but I have a git-auto-cimmit project I intend to use with logseq, that'll help me better keep a record of changes over time. That ability to have a flat file source of truth feels hugely hugely empowering to me, and I have a hard time imagining having anything else.