Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

  > Your domain is weird/rare, so LLMs are terrible because their training data is very limited.
And this is how knowledge collapse [1] shows its' head.

[1] https://arxiv.org/abs/2404.03502

I am not a big fan of LLMs so I try them once in a while, asking to "implement blocked clause decomposition in Haskell." They can recite a paper (and several other papers, with references) almost verbatim, but they do not posess enough comprehension of what is going on in these papers. As time passes by, with each new LLM, the level of comprehension drops. Last time LLM tried to persuade me to write code instead of providing me with code.



I gave Claude 3.5 Sonnet your prompt and it generated this: https://claude.site/artifacts/7aa41881-937e-4863-a407-c999ea...

With this example usage:

    -- Example usage:
    let clause1 = Set.fromList [1, 2]  -- represents (x1 ∨ x2)
    let clause2 = Set.fromList [-1, 3] -- represents (¬x1 ∨ x3)
    let formula = Set.fromList [clause1, clause2]
    
    -- Decompose the formula
    let (nonBlocked, blocked) = decompose formula
How did it do?


This is very good.

Did it do just from the prompt or you had to nudge it? Can you share full chat history?


No nudging. This was the whole interaction: https://gist.github.com/jcheng5/c6f15f4c3dc31bf15ab44683ad6a...


This is a gist, not an interaction with Claude AI.

Can you share actual interaction on their site?


No, ChatGPT has that feature but not Claude (that I could find) so I pasted my input and its response into a gist. It’s verbatim, I promise.





Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: