Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Veles: Open-source tool for binary data analysis (codisec.com)
128 points by LorenDB on July 25, 2024 | hide | past | favorite | 23 comments


The GitHub repo [1] was made a read-only archive on Sep 30, 2020. Are there forks picking up where the original author left off?

[1]: https://github.com/codilime/veles


Since this is an old tool and it fails to deal with large files, I made a completely new version from scratch a while ago, using a completely different rendering approach that's more like how you'd render a volumetric data set like an MRI scan. It loads the file and processes it into a 256x256x256 volumetric data set, which is then rendered using shaders. As a result, the file size doesn't matter for rendering, only the loading time depends on the file size. Unlike the original Veles, it also doesn't need any subsampling for huge files, but you need a powerful enough graphics card.

The source code is on github and unlike the original Veles, it doesn't have countless dependencies and build problems on modern systems: https://github.com/hackyourlife/veles


You should add some screenshots!


I added a few screenshots of different files right now.


See by yourself. There is only one fork wich fixed the msgpack encoding breakage: https://github.com/0xBattleSong/veles


Oh, I know something about this. Some people that made this are part of DragonSector [1] polish hacking group that broke DRMs in trains quite recently [2]. Sadly I joined company after there were long gone. Really surprised seeing this posted here as it was pointed out, the repo is pretty old and archived.

[1]: https://dragonsector.pl/

[2]: https://youtu.be/XrlrbfGZo2k


One person, IIRC, don't think anyone other than q3k was involved in CodiSec, and I think he left the company before proper development on Veles started (I think he was present when first talks about were happening and gave a lot of inspiration though, but isn't in credits for first open source version)


ImHex has some similar data pattern analysis tools. And as it is actively maintained, might be worthy to take a look at.


Not sure how new the tool is because it was new in 2017...https://news.ycombinator.com/item?id=15966021


There was a major push for development back in 2015-2016, back before it was open sourced, but AFAIK it went dead pretty soon as internal funding dried up.. Looking at github commit log, it seems it died by mid 2018.

First time I had to setup CI on a Mac, and what a horrible experience it was (Windows was comparatively easy, though finding out that starting with Windows Core wasn't best option took some time to learn)


> First time I had to setup CI on a Mac, and what a horrible experience it was

Could you share some details what was horrible?

I've created pipelines for Linux, macOS and Windows, and think that Windows is by far the worst of all three to setup anything related to CI.


One of the issues was making builds that worked on OS versions older than the build host, general annoyances of admining macOS, fragility of the environment (this was before image based upgrades)


Did you work for the parent company?


There was no "parent company" per se - codisec was a team inside CodiLime, starting out as an initiative to build more cybersecurity presence, including being the name we participated in various CTF contests. (This is all public info, you can find codisec at various CTF participation trackers too). And yes, I was there (worked at CodiLime late 2014 - early 2017), including in the early days of Veles (those who know my full name can easily find me in credits in the github repo, direct link with my account was lost when the repo was migrated into current one).

With regards to Veles, I mostly did infrastructure and tooling for CI/CD - so including the CMake stuff and making Jenkins setup for the builds. I think it was actually the first project on which I used Jenkinsfile, which was then very fresh thing. I was never much of a C++ fan so I didn't participate much in the editor code itself, especially with another project taking lion share of my attention later on.


The main page of codisec [0] lists that Veles has been retired. They since have a new project, CodiLime [1], which seems like is a general technology consulting/outsourcing company.

[0] https://codisec.com/

[1] https://codilime.com/


CodiLime was the actual company, CodiSec was "brand" for various security related activities, starting with CTF competitions where we used the name first, then it was used as brand for the team that worked on Veles (we continued to take part in CTFs)

None of the CodiSec members were among the management/founders/etc of CodiLime - we were AFAIK plain employees, though one of us was in fact mber of DragonSector which is partly how CodiSec started.


Neat, I was looking for such a tool a few days back and ended up not having the requisite google skills to find this (embarrassingly, it seems easy it find now)

I ended up writing my own script for visualizing my particular problem (trying to work out what's going on in the "mett" data frames in videos from my pixel phone, I was able to make some headway but I've gotten distracted by other projects, if anyone has info I'd appreciate it)


I was summoned? ... oh sorry!


I am honestly surprised this still shows up...

learnt so much CMakeFile pain...


Their linux build is in ubuntu 16.04 ...


Sorry, that was what was current LTS when I setup CI builds


When these tools come up, I like to post my little list of similar tools, for those interested. Apparently it's a hobby of mine:

* fq - like jq for binary data: https://github.com/wader/fq

* Kaitai Struct - https://kaitai.io/

* visualizer, for the above: https://github.com/kaitai-io/kaitai_struct_visualizer/

* HexFiend - a hex editor, but with "binary templates" feature : https://github.com/HexFiend/HexFiend

* binary templates, for the above: https://github.com/HexFiend/HexFiend/blob/master/templates/T...

* binspector - https://github.com/binspector/binspector

* binary-parsing - a collection of links to similar such tools : https://github.com/dloss/binary-parsing

* unblob - https://github.com/onekey-sec/unblob

* ImHex - https://github.com/WerWolv/ImHex

* GNU Poke - https://www.gnu.org/software/poke/

* dtrx - https://github.com/dtrx-py/dtrx/ dtrx

* feng - https://github.com/martinlindhe/feng

** they also have a similar list of prior art: https://github.com/martinlindhe/feng/blob/main/RESOURCES.md

And some previous HN threads for similar such tools:

* https://news.ycombinator.com/item?id=34434249 (unblob)

* https://news.ycombinator.com/item?id=34986042 (GNU Poke #1)

* https://news.ycombinator.com/item?id=36182873 (GNU Poke #2)


don't forget hachoir-wx




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: