Ask HN: Anyone Interested in Capture the Flag (CTF) Events?

q3k · on Jan 12, 2019

I'm a semi-active member of the Dragon Sector CTF team [1]. We're a competitive security CTF team that's #1 on the ctftime.org ranking for 2018.

CTFs are as active as ever, but a lot of the high-stake ones (at least per ctftime.org) are very much focused on hard core exploitation challenges, and more and more actually include 0days and real-life challenges like browser expolitation (for instance, the 35C3 CTF had a VirtualBox 0day (GL acceleration bug), a logrotate 0day (race condition) and a patched Webkit and Chromium to exploit.

Honestly, the rising exploitation difficulty is what's slowly driving me away from traditional binary challenges - and so, I'm mostly focusing on either obscure architectures, hardware or other weird challenges.

There's more and more 'IoT' challenges as well - like exploiting vulnerable ESP8266 or ARM microcontroller code. I've created a somewhat 'IoT' challenge [2] for WCTF 2018, where you have to exploit a hardware flaw in a remote RISC-V device.

A lot of CTF people are on IRC - try #pwning on Freenode (PPP's channel), or #dragonsector :). If you have a local hackerspace, they might have a CTF team you can join.

[1] - https://dragonsector.pl/

[2] - https://hardflag.q3k.org/

legitbs_ctf · on Jan 12, 2019

I ran the DEF CON CTF from DEF CON 20 to DEF CON 25. Prior to that, I won a few.

I know order of the overflow is running a good game now. It’s different than what we did, but it’s a good direction, and I am hopeful it will continue to grow.

They (like we did) have a few selected prequalification events tbgat are generally pretty high quality. See https://www.oooverflow.io/dc-ctf-2019-quals/

PlaidCTF was an old favorite of ours as well. PPP has won several competitions, and seems to understand what makes a good challenge, what makes a challenge hard, and more importantly, what DOESNT make a good/interesting challenge.

Boston Key Party also runs a pretty good game. But a lot of them became Order, so who knows what will happen with them...

DARPA ran a CTF for autonomous systems called the Cyber Grand Challenge. It was neat, but I don’t know of any plans to do anything like it again. I believe they’re waiting to see what the community does to push the state of the art further. Source code for challenges and infrastructure are at https://github.com/cybergrandchallenge. There’s also a bunch of video at the darpatv YouTube channel.

We open sourced most of our challenges/frameworks after the fact. https://github.com/legitbs. This includes the compiler/emulator/manual for our 9-bit bytes custom architecture, clemency. That was probably our most ambitious year...

That’s all off the top of my head. If you care for more from a CTF has been, I’m happy to come back and answer more questions.

carom · on Jan 12, 2019

>seems to understand what makes a good challenge, what makes a challenge hard, and more importantly, what DOESNT make a good/interesting challenge

Any tips in this area?

deveynull · on Jan 13, 2019

Plaid Parliament has a writeup on this that could be considered borderline canonical [1]. From some other Plaid folk, Maxims [2] are great as well.

To sum it up, players should learn something and players should not have their time wasted.

[1] https://github.com/pwning/docs/blob/master/suggestions-for-r... [2] http://captf.com/maxims.html

q3k · on Jan 12, 2019

Like with any puzzle design. You can make any puzzle easily very difficult (by adding multiple steps, red herrings, ...). It's much more difficult to make it difficult _and_ rewarding. Make the puzzle solver only have to use knowledge acquired while examining the challenge and don't make them guess, and don't overflow them with information. Finally, test your challenges on your teammates.

Yen · on Jan 12, 2019

Like many things, this can be a matter of preference, but here's the rules of thumb I try to follow when writing challenges, and the things I appreciate when playing challenges.

* Avoid intentional red herrings, full stop. Your players have a limited amount of time in their lives, and a limited amount of time in your game. If you've got a plausible-looking path of investigation which actually serves to intentionally waste their time, it's super frustrating.

* Keep your challenge as focused as is reasonable. This avoids wasting your participants' time, as above. This also gets across the flavor or educational content of your challenge more effectively. Also, this does a lot to help prevent unintentional shortcuts around the intended solution.

* Make it unambiguous and obvious when a challenge has been solved. While not appropriate for all types of CTF, in CTFs I've run, we try to use ascii-based keys that have content related to the theme/solution. For example, a session-hijacking challenge might reward you with the flag "c00kies_r_d3licous". That said, some CTFs use randomly generated hexadecimal strings of a specific length, for rotating flags. Whatever you do, it's important to remain _consistent_ across all your CTF's challenges.

* Avoid "guess what the author was thinking" in your challenges. For example, if you use a freely available but obscure steganography program to hide a message in an image, using a 1-word dictionary password, you might think it's a relatively simple challenge, but your participants won't even know where to start. If they guess that what you were thinking was "stenography with a simple password", they'll likely have no more direct course to solving it than "download a bunch of stego programs, and brute-force <program,password> combinations". And, they have no real _reason_ to believe that's the correct course, as opposed to all kinds of other avenues of investigation.

* As specific examples of the above, I'd recommend basically never doing a cryptography challenge, without giving the participants an implementation of the cryptosystem, whether in source code or binary format. Similarly, most exploitation challenges should give out either source, binary, or both.

* Similarly, if you do a multi-stage challenge, it's helpful to make it clear when one stage is solved. In some cases, I've given a separate flag to each stage of the multi-stage challenge, so lesser-skilled teams can still score partial credit.

* Have a clear idea of which skill or piece of knowledge the challenge is testing for or educating about. In my opinion, some of the most fun and memorable challenges are ones in which I independently rediscover a well-known class of vulnerability, or CS concept.

* Playtest! Get at least one team member to try out your challenge, give you commentary on what they're thinking and investigating, and see if they solve it, how long it takes, and what they run into. Try not to give hints, except where it's reasonable to unstick and expedite the playtest process.

* Play in CTFs! Pay attention to what creates joy or frustration in you while you play.

For context, the CTFs I've run have been more focused on creating an enjoyable experience for a wide range of skill levels, from newbie to pro, rather than high-level, cutting edge competition. Both are entirely valid realms, and there's likely other interesting focuses for CTFs as well - just understand what you want the emphasis of your CTF to be. Depending on your focus, you might want to do the _opposite_ of some of this advice. But, I'd recommend being aware of these points, and intentionally choosing which direction you want to go.

Hope that helps.

legitbs_ctf · on Jan 12, 2019

I don't think I can add much more than what was included in this very well thought out answer. I will double down on "Make it unambiguous and obvious when a challenge has been solved." ALL of our flags started with "The Flag Is: ", and that string was NEVER allowed to show up outside of the answer. It made some classes of file carving challenges not possible to do well (because they could be solved with grep...), but I'd argue that those aren't great challenges to begin with. When we did come up with challenges like that, it forced us to be more thoughtful about what made the challenge difficult through being clever, instead of just by brute force red herrings.

Hack.lu in 2018 had a pretty bad challenge that left a bad taste in a lot of players mouths. https://twitter.com/fluxfingers/status/1053279841578086406 describes some of the thinking in making that challenge. I disagree with things like that, though.

Also, meta, but on challenge design: if you're gating your challenges, keep in mind that your final challenges may need to be opened sooner than the last hours of your competition. There's an expectation that challenges get harder as the game goes on, but also, the teams are getting tired, and the time left in the game is going down. We viewed challenges that remained completely unsolved at game close as minor failures. I think one year we didn't let the final challenges open as there was too little time left at the end of the game, so we rolled them over to the following year.

veganjay · on Jan 12, 2019

I played HackVent (https://www.hacking-lab.com) in December 2018 and it was a lot of fun. The same site also does HackyEaster which is coming up in a couple of months.

To find CTFs, https://ctftime.org/ is good and has already been mentioned. I've learned about a few on https://www.reddit.com/r/securityCTF/

If you are looking to learn - I highly recommend LiveOverflow's youtube series - https://www.youtube.com/watch?v=iyAyN3GFM7A&list=PLhixgUqwRT...

And a few good places to practice: http://overthewire.org/wargames/ and https://365.csaw.io/

subjectsigma · on Jan 12, 2019

CCC, which you linked to, still actively does CTFs every year, although only when Congress is in session. I found it incredibly difficult but maybe I just suck :) I think most events (ShmooCon, Blackhat, Defcon, BSides, etc) run CTFs during the conference which are accessible to all. I've found https://ctftime.org/ to be a good resource for finding competitions, not so much teams though.

abalaji · on Jan 12, 2019

If anyone is interested in a seeing a high-level real world CTF in action, I would recommend this video: https://www.youtube.com/watch?v=ozqOlUVKL1s. As someone who has experience with programming but not as much experience in reverse engineering / security, the entire channel has been quite amazing.

windowshopping · on Jan 12, 2019

Man I was all set to be like "Yeah I love CTF!!" and then I realized it had another meaning besides referring to my favorite childhood game.

ptd · on Jan 12, 2019

Memories of Quake and Unreal tournament for me!

ggerganov · on Jan 12, 2019

I recently made an unusual audio processing challenge where the goal is to recover the English text that a user is typing, just by analysing the recorded sound of their keyboard [0]. Not sure how hard it is. I am able to solve it with my own audio processing tools. If interested - give it a try.

[0] https://ggerganov.github.io/keytap-challenge/

ifoundthetao · on Jan 14, 2019

I am very much interested in this. Would you be willing to work with me offline on this? It's an area of research for me that I find fascinating, yet I have difficulty finding information for legitimate reproduction of the techniques that I hear about.