Funny idea, some days ago i was really annoyed again by the idea that these AI crawlers still ignore all code licenses and train their models against any github repo no matter what so i quickly hammerd down this
basically a github action that extends your README.md with a "polymorphic" prompt injection. I run some "llm"s against it and most cases they just produced garbage.
Thought about also creating a JS variant that you can add to your website that will (not visible for the user) also inject such prompt injections to stop web crwaling like you described
Ok, i kinda get the idea, and with some modification it might be quite handy - but i wonder why its deemed like an "unsolvable" issue right now.
It may sound naive, but packages which include data like said session related or any other that should not persist (until the next Global GC) - why don't you just scramble their value before ending your current action?
And dont get me wrong - yes that implies extra computation yada yada - but until a a solution is practical and builtin - i'd just recommend to scramble such variables with new data so no matter how long it will persist, a dump would just return your "random" scramble and nothing actually relevant.
It is fundamentally not possible to be in complete control of where the data you are working with is stored in go. The compiler is free to put things on the heap or on the stack however it wants. Relatedly it may make whatever copies it likes in between actions defined in the memory model which could leak arbitrary temporaries.
Yeah, .NET tried to provide a specific type related to this concept (SecureString) in the past and AFAIK there were were two main problems that have caused it to fall into disuse;
First one being, it was -very- tricky to use properly for most cases, APIs to the outside world typically would give a byte[] or string or char[] and then you fall into the problem space you mention. That is, if you used a byte[] or char[] array, and GC does a relocation of the data, it may still be present in the old spot.
(Worth noting, the type itself doesn't do that, whatever you pass in gets copied to a non-gc buffer.)
The second issue is that there's not a unified unix memory protection system like in windows; The windows implementation is able to use Crypt32 such that only the current process can read the memory it used for the buffeer.
Hard to understand what you’re asking. This is the solution that will practical and built-in. This is a summary of a new feature coming to Go’s runtime in 1.26.
without language level support, it makes code look like a mess.
Imagine, 3 level nesting calls where each calls another 3 methods, we are talking about 28 functions each with couple of variables, of course you can still clean them up, but imagine how clean code will look if you don't have to.
Just like garbage collection, you can free up memory yourself, but someone forgot something and we have either memory leak or security issues.
There are two main reasons why this approach isn't sufficient at a technical level, which are brought up by comments on the original proposal: https://github.com/golang/go/issues/21865
1) You are almost certainly going to be passing that key material to some other functions, and those functions may allocate and copy your data around; while core crypto operations could probably be identified and given special protection in their own right, this still creates a hole for "helper" functions that sit in the middle
2) The compiler can always keep some data in registers, and most Go code can be interrupted at any time, with the registers of the running goroutine copied to somewhere in memory temporarily; this is beyond your control and cannot be patched up after the fact by you even once control returns to your goroutine
So, even with your approach, (2) is a pretty serious and fundamental issue, and (1) is a pretty serious but mostly ergonomic issue. The two APIs also illustrate a basic difference in posture: secret.Do wipes everything except what you intentionally preserve beyond its scope, while scramble wipes only what you think it is important to wipe.
While in my case i had a program in which i created an instance of such a secret , "used it" and than scrambled the variable it never left so it worked.
Tho i didn't think of (2) which is especially problematic.
Prolly still would scramble on places its viable to implement, trying to reduce the surface even if i cannot fully remove it.
As I understand it, this is too brittle. I think this is trivially defeated if someone adds an append to your code:
func do_another_important_thing(key []byte) []byte {
newKey := append(key, 0x0, 0x1) // this might make a copy!
return newKey
}
key := make([]byte, 32)
defer scramble(&key)
do_another_important_thing(key)
// do all the secret stuff
Because of the copy that append might do, you now have 2 copies of the key in data, but you only scramble one. There are many functions that might make a copy of the data given that you don't manually manage memory in Go. And if you are writing an open source library that might have dozens of authors, it's better for the language to provide a guarantee, rather than hope that a developer that probably isn't born yet will remember not to call an "insecure" function.
This proposal is worse because all the valuable regions of code will be clearly annotated for static analysis, either explicitly via a library/function call, or heuristically using the same boilerplate or fences.
Makes sense basically creating an easy to point out pattern for static analysis to find everything security related.
As another response pointed out, its also possible that said secret data is still in the register, which no matter what we do to the curr value could exist.
> Makes sense basically creating an easy to point out pattern for static analysis to find everything security related.
This is essentially already the case whenever you use encryption, because there are tell-tale signs you can detect (e.g., RSA S-Box). But this will make it even easier and also tip you off to critical sections that are sensitive yet don't involve encryption (e.g., secure strings).
yes, you now have to deal in pointers, but that's not too ugly, and everything is stored in secretStash so can iterate over all the types it supports and thrash them to make them unusable, even without the gc running.
I'm now wondering with a bit of unsafe, reflection and generics magic one could make it work with any struct as well (use reflection to instantiate a generic type and use unsafe to just overwrite the bytes)
This is, in a rather short period of time, the second time that an ovh scheduled database upgrade leaded to hanging instances, this time paired with the information that database backups are not reachable.
While the page claims that the incident is resolved, this is far from true and there are multiple users also experiencing still stuck/non accessible database instances.
Can maybe anyone else confirm they also experience this ?
Cuz of exactly this reason i couldnt find any solution like that for me yet, so i started building my own one. I think the only way something like this is practical is in an on-premise scenario
Since it runs locally on my machine not really much :) im working always on the same machine therefor i don't need any online hosting for my work notes.
Reading your comment made me think of an ad i saw several years ago.
It was an ad that started off like a typical quality coffee ad, nice pictures of beans, people harvesting, some roasting etc. But than it switched to the topic if making sure that "fair trade" was really applied - switching to IBM Blockchain and claiming that through the use of Blockchain there is safety that everything went fair trade.....
And i just thought.... sure... your blockchain approves that those workers harvesting got paid fair... or does it? All it actually does is proof that someone inserted the information that they got paid fair. If it really happend? Noone knows. Therefor Blockchain is a representation of what you feed into it nothing more and nothing less. At the moment it touches real life - as in the fair trade/payment for coffee bean harvesting - especially in countries where alot of payment is still for dayjob/cash - there's no way the Blockchain can assure that everything went the way it is stored as.
Yes. The oracle problem. Because of it, the only blockchain use case is basically crypto (and other purely on-chain stuff, such as NFTs (well, they typically link to off-chain sources by URL, haha) and crypto kittens).
I have not seen this scenario, but I assume that a blockchain could more reliably and believably be made public, recording all the way down to the farmer, then could other technologies.
Blockchain enables transparency from the farmer to the consumer. It does not ensure that the farmer was not coerced.
Right. That's a forgery - a lie. No blockchain could prevent that type of lie. What a blockchain could enable is honest transparency.
An attempt to pass off a lie as truth would be much more damaging to a company's reputation than would be just not addressing an issue. By addressing the issue and introducing a channel for transparency, the company is demonstrating that they are willing to put their reputation on the line.
I would agree that as a regulatory measure, this would not solve any problem. But as a self-imposed measure, it builds consumer trust.
THanks for sharing this. Partly because i forgot about this great website :D also because i would never thought of giving this as an LLM task because its so simple that i prolly just had hacked it down myself :D
I recently experimented alot with agentic coding (mostly with gemini+ intellij plugin, copilot intellij plugin and intellij's own junie) and also condsidered to give it a try and feed images to the AI, but than all tasks i tried so far were pure backend-ish so it never came to the point.
Im really curious how especially junie will act and i will give it a try with the very same task you gave it. We gonne see how it ends :D
When i first read the papers for titans for me it was a "this will be a big step forward".
While i have no "AI" title or work in the respective AI industry, ive spend many years thinking about AI concepts, even long before the whole NN/LLM hype started.
Maybe because of that i was always really annoyed that LLM are called AI because in my years of thinking about how an actual "human like" thinking AI might work, the things an LLM does was far below what my minimum definition was.
But when i stumbled accross the Titans paper, while it still is not an "AI" as i would call it, from my POV its a massive step towarsd the right direction.
Sometimes i consider to write all my ideas/thoughts about AI down in my blog, but than i think nobody would care anyway since im not a known figure shrug - so if not to say "look i wrote it years ago!" theres no actual point in doing so i guess.
However - im looking forward to see titans in action, and i guess it will impress us all.
A lot of LLM/AI writing these days can feel lost in the weeds – the specifics of very detailed techniques are interesting undoubtedly, but writing that steps back and looks at the big picture, informed by those details, could be very useful for people who want to think about where this all may be going.
Thanks, and i gonne think about going for a writeup. As i mentioned in another comment, reading my previous comment back from yesterday i dont even know why i mentioned it - probably because i think so much about the topic but than i think "well your just a guy in a shed" type of thing and decide that prolly noone would care about what i would write. At all - if its just something i can look back onto im some years, prolly worth it.
Tbh, if i read back my comment from yesterday i don't even know exactly why i did mention that part. Sounds even to me like a "look at my blog" thingy which it definitely should not. Maybe some day ill give it a try and write something about my 'ideas' and drop it here. Tho not today (w0rk w0rk) ^
So, since this seems to be relevant im a CISO myself.
And i would definitely not agree with everything in this letter.
Personally, i think the worst part about it is handling a low probability as something that's not gonne happen. Thats, especially in IT-Sec, one of the worst practices.
To take on point as example - the "never scan public QR codes".
Apart from the fact that there have been enaugh exploits in the past (The USSD "Remote Wipe", iOS 11 Camera Notification Spoofing (iOS, 2018), ZBar Buffer Overflow (CVE-2023-40889), etc) even without an 0day exploit qr codes can pose a relevant risk.
As a simple example, not to long ago i was in a restaurant which only had their menu in form of a qr code to scan. Behind the QR code was the link to an PDF showing the menu. This PDF was hosted on a free to use webservice that allowed to upload files and get a QR code link to them. There was no account managed control about the pdf that they linked to, it could be replaced at any time opening a whole different world of possible exploitations via whatever file is being returned.
Sure you could argue "this is not a QR code vulnerability just bad practice by the restaurant owner" - but that's the point. For the user there is literally no difference if the QR code itself has a malicious payload or if the URL behind it has (etc etc).
While we in the tech world might understand the difference, for the John and Jane Doe this is the same thing. And for them its still a possible danger.
Apart from that, recently a coworker linked me a "hacker" video on youtube showing a guy in an interview talking about the O.MG cable. Sure, you might say this is also an absolutely non standard attack vector, yet it still exists. And people should be aware it does.
My point is - by telling people that all those attack vectors are basically "urban myths" you just desensitize the already not well enough informed public from the dangers the "digital" poses to them. And from my personal view, we should rather educate more than tell them "don't worry it will be fine".
It's funny your warning about QR codes goes onto warn about PDF exploits. Yet you clicked the link to this article, by your own definition opening you up to "a whole different world of possible exploitations via whatever file is being returned". It's the nature of the internet to follow links, but our updated browsers keep us safe from exploits.
When was the last time you saw an un-targeted mass 0-day exploit campaign? There haven't been any for modern browsers. If we're talking about 0-days, you likely known there have been zero-click iMessage/WhatsApp vulnerabilities in the past. There's no protecting against those, but you're not here warning users to disable iMessage and WhatsApp. What's more realistic is making sure users keep their software updated, and trust that QR codes and links aren't going to waste a 0-day worth a million dollars on you.
First of all, the problem here is more a point of trust.
Ill try explain based on your example with "any link".
If you type amazon.com you trust that there will be amazon.com returned and not any maleware. On a QR code, the target url isn't as obvious so the user should be aware that a qr code, even if for example below it says "hackernews - the best news in the IT world" the qr code could still link to "https://news.xn--combinator-xwi.com" (edit : because ycombinator is a nice website it auto resolves the unicode char here : bad example tho but i dont have the time to recraft it and i guess you know unicode link/url tricks therefor i can just let it be the way i pasted it) did u spot the difference? Its not a regular "y" and just could get you on a fishing page. So ye even just know "urls" that you review on a qr code still can be dangerous if not typed by yourself. And than, for alot of users it prolly wouldn't event take that of a measure to trick them. Its not like the average Jane/John Doe does very good on url verification - else alot of scammers would go bancrupt.
Therefor i hope you understand you don't need a 0day. I also stated that in my answer but you seem to be so keen focusing on me listing some 0days (to disprove the initial article) that you kinda lost my point.
Also - sure everyone should keep his/her device updated - noone said anything else. Apart from that no i wouldn't recommend people to use whatsapp but that was't the point and im not actually sure why you mentioning it but here i said it : i wouldn't recommend it if that helps ¯\_(ツ)_/¯
Edit: not to forget - i for myself know that clicking on unknown links poses a certain risk and have several measures in place to reduce this risk.
>It's funny your warning about QR codes goes onto warn about PDF exploits. Yet you clicked the link to this article, by your own definition opening you up to "a whole different world of possible exploitations via whatever file is being returned". It's the nature of the internet to follow links, but our updated browsers keep us safe from exploits.
you really don't know what they did.
In the time of containerized OSs and virtualized-everything it's silly to guess.
"Never scan public QR codes" is functionally equivalent to "never type in a URL and never click on a link". Other than the smallish scan-specific attack surface that you mention and then largely dismiss, there's nothing that makes QR codes more dangerous than any other way of delivering links.
It's somewhere between impractical and impossible to evaluate a URL and know anything about its "safety". So if you can't make your Web browser impervious enough to tolerate basically any crap a server may send back to your satisfaction, then your only answer is a total walled garden.
Well we are as sadly so often in the world of only "black and white" discussion without ignoring gray areas.
While i pointed out that i think that the claim of public qr codes are always safe and cannot pose any danger is wrong, i also didn't state you should wall yourself in and handle like everything is f0rk3d.
You, as with everything in life, should evaluate whats worth risks and what not. Scanning a QR code in a museum linking an audio track to describe the exhibt, scanning a qr code in a restaurant for a menu, scanning a qr code from a sticker on a traffic light.
These are 3 completly different scenarios that can be weighted different and therefor not be answered with a single "yep good/bad" for every situation. My initial point regarding the article was that i don't think stating scanning public placed qr codes is always safe. People should not just NEVER scan a public qr, but they should understand possible risks, they should learn how to evaluate which risks are worth taking, and also learn what thinks they should look for. My point is that of make the public more informed.
Well i just listed the O.MG cable to show that there are alot of people not knowing that such things exist. My point is that of : people should be better informed about what vectors of attacks exist. So mentioning the cable (in relation to a coworker coming to my desk and asking about it) was just an example of how informed average Joe/Jane are and that i think this is the more important part - educate the public not just tell them not to worry.
The piece isn't about getting people not to worry, it's about not wasting the worry on things that aren't real threats. People have limited space in their brains for things to worry about.
>Personally, i think the worst part about it is handling a low probability as something that's not gonne happen. Thats, especially in IT-Sec, one of the worst practices.
If you are an online service provider, sure. Low probability means it's going to happen, especially as you scale with users.
For a small business IT team? You can't keep a clean sheet, the strategy is to reduce the probabilities of an incident and reducing its damage, but it will never be zero, if only because you have non-technical users that need to do actual work.
p(incident) is just yet another variable you need to do tradeoff engineering on, and obsessing over reducing it to 0 will probably compromise other tradeoffs like ease of use of the system.
It's a special case of ironic when in an attempt to get a specific variable to 0 (which is impossible with most variables anyways) you end up compromising that specific variable. So if you force users to use lots of passwords and password managers and MFA, and limit their capabilities, they end up circumventing your security systems and advice, so they introduce an issue (but of course it will be the users fault, and not the CISO's fault, their job is secure).
Well even tho i think at the end of your comment you went a bit out of the way, i get your point and i agree to a certain point.
You cannot reduce the risks to 0 - that's a matter of fact and i would never claim you could.
I tend to say its a question of cost/gain. If the cost the attacker has to pay (work/invest/...) is higher than the possible gain (data/funds/...) you are on a good track for your companies security.
Im btw not working for an ISP, rather something you would see as a smaller sized IT company. Therefor i also have certain points where i in theory could go alot harder on security, but i don't because its not feasible.
Another thing especially in that regard i find important is trying to educate your users, at least we work on that. We don't just enforce hard rules on them, but we also try to make sure they understand why we have these rules and mechanisms in place - not to annoy them but to protect them.
Finally, thats my favorite point of your article, "force users to use lots of passwords ".
Well our business has to undergo regular audits by partners which are lets say rather meticulous when it comes to the security of our systems. These enforce certain things on us we have to than enforce on our users even if we don't think its good.
So ye, now you can blame on me that i enforced something on our users, but keep in mind - it was also enforced on me - i even discussed certain things with these partners trying to explain to them why some measures sound cool on paper but in reality are just impractical - not that anyone would care. So we implement it.
Therefor the next time you argue that some security measure is just an CISO that doesn't really care about its users, maybe keep in mind that some things are forced upon us even tho we don't like and don't support them.
I can see why you would take "online service provider" to mean an ISP, but I meant it to include SaaS and apps like whatsapp, google, etc.. as well
>Therefor the next time you argue that some security measure is just an CISO that doesn't really care about its users
Oh I didn't mean to imply that, there's no doubt that IT admins that overimplement security policies care in general, the critique is not about motives, rather the efficiency. I don't argue that they don't care or even that they are wildly inefficient, just that they are suboptimal on this specific point by going overboard.
Wrote myself a script years ago that basically loops netstat -tulpn watch like for the same purpose - just wondering if your tool shows me more than that.
I really tried to get gemini to work properly in Agent mode. Tho it way to often wen't crazy, started rewriting files empty, commenting like "here you could implement the requested function" and many more stuff including running into permanent printing loops of stuff like "I've done that. What's next on the debugger? Okay, I've done that. What's next on the with? Okay, I've done that. What's next on the delete? Okay, I've done that. What's next on the in? Okay, I've done that. What's next on the instanceof? Okay, I've done that. What's next on the typeof? Okay, I've done that. What's next on the void? Okay, I've done that. What's next on the true? Okay, I've done that. What's next on the false? Okay, I've done that. What's next on the null? Okay, I've done that. What's next on the undefined? Okay, I've done that..." which went on for like 1hour (yes i waited to see how long it takes for them to cut it).
Its just really good yet.
I recently tried IntelliJs Junie and i have to say it works rather well.
I mean at the end of the day all of them need a human in the loop and the result is just as good as your prompt, tho with Junie i at least most of the time got something of a result, while with gemini 50% would have been a good rate.
Finally: Still dont see agentic coding for production stages - its just not there yet in terms of quality. For research and fun? Why not.
-> https://github.com/voodooEntity/ghost_trap
basically a github action that extends your README.md with a "polymorphic" prompt injection. I run some "llm"s against it and most cases they just produced garbage.
Thought about also creating a JS variant that you can add to your website that will (not visible for the user) also inject such prompt injections to stop web crwaling like you described