I'm increasingly impressed with the progress that OpenBSD is making. They're hitting some real problems on the head and actually letting us know what the changes are in a concise form as they always have.
Compare this to a Ubuntu LTS release which is virtually impossible to determine a detailed change list for past the marketing and UI changes.
I found this ~14 minute interview with Theo de Raadt from ruBSD very interesting and apropos OpenBSD's approach to trail-blazing "painful" changes: https://www.youtube.com/watch?v=OXS8ljif9b8
tl;dw: Changes that must be done, must be DONE. The only way to see the ACTUAL impact is to just do it already and get feedback from real users. When the dust has settled, less brave projects can follow suit. I'm donating.
I've been using it since 2.4 as well but the progress is remarkable considering the team size and funding. Progress for me is the replacement of non-BSD licensed parts and top notch re-engineering of the old rusty bits as well as hardware support which is pretty damn solid and reliable.
Pain points for me: The package management and upgrades.
If I'm not mistaken (perfectly possible, I'm really no expert and only browsed OpenBSD documentation here and there)
- you have no decent binary packages
FreeBSD guys online complain a lot about the current pkgng changes, but the direction is great: Build from ports, install binaries elsewhere.
This is something I really really like, since I tend to avoid building stuff on production machines and since I don't want to (even temporarily - and that means you have to clean up!) pull in pure build time dependencies. FreeBSD solved this for my use case with pkgng and the ability to easily host your own repository. I have a build machine and run portshaker/poudriere to create exactly the (binary) packages I need. Installation on the targets is a breeze.
- consider the heartbleed issue
As far as I understand the OpenBSD project issued a patch (as in text file) and told you to rebuild the related parts of your base system (or everything)? FreeBSD has binary updates for these kind of things.
I'm not trying to bash OpenBSD or promote FreeBSD, I guess the general idea behind this post is that I'm interested in the project OpenBSD, but feel that running OpenBSD would probably too much tinkering than I'd like (cough I guess I'm saying that I'm lazy?).
But maybe, maybe, I'm wrong about one or both of these "issues"? Maybe there are 3rd party helpers that make everything very, very easy (just like portshaker/poudriere on FreeBSD are tools I install from ports, i.e. not part of base)? I'd be happy to be enlightened.
This is completely incorrect. OpenBSD has been all about building reproducible binary packages for quite a long time now, and the ports tree is regularly bulk-built and failing ports fixed or swept out of the way. The key difference from FreeBSD is that OpenBSD binary package sets are released every six months with the base system, and you are expected to upgrade them during a system upgrade.
It's as simple as `pkg_add -ui` if you point the PKG_PATH to a remote OpenBSD distribution mirror.
One advantage of upgrading ports with the base system is that caveats are all conveniently listed in the upgrade guide for that release. E.g. for 5.4->5.5: http://www.openbsd.org/faq/upgrade55.html#Pkgup
That said: I need more stuff than the ports tree contains (I .. checked NetBSD and FreeBSD, plus a couple of Linux distributions. I admit that I haven't checked all my packages against OpenBSD's ports tree, but I assume I'm correct). Plus, I do need the occasional update without waiting 6 month. How would I do that?
For comparison: On FreeBSD I use portshaker to create a ports tree that is a mixture of upstream (FreeBSD) and other sources (a git repository, in my case). If FreeBSD doesn't bump a package and PRs take a whiiiiile, I bump that thing in my overlay.
poudriere allows me to build (in bulk) all my relevant packages in jails, is clever enough to rebuild just the stuff that changed (or where I changed the config) and spits out a directory structure that represents a pkgng repository with (optional) signed binary packages.
pkgng is pointed to my repository (actually it supports multiple sources, apt like. Can PKG_PATH do that?), which contains now builds from upstream ports and my overlay.
How would I (hoping that you have a couple of minutes and can prove me wrong once more)
- build binary packages for ports that either don't exist upstream (stuff I ported) or are outdated upstream?
- distribute those in a decent way (PKG_PATH swapping..?)
I guess I'm left with the question whether we're talking about the same thing when we're saying 'decent binary packages'? I'm sure you can OpenBSD works well for the intended use - but 'decent' wasn't an objective attribute here, it was more a subjective 'decent for my needs' kind of thing. More details above.
In the end, pkg_* feel like ooold Solaris, or like using Slackware again. That's package management on a different level and feels like a step down from apt/yum/pacman/pkgng. Again: Probably it's just a matter of convenience and my laziness in the end, but so far I feel that's not for me.
Fair enough. Here's an upvote, that's probably the right answer.
That said: I read that page. Multiple times. It doesn't help me mapping my portshaker/poudriere setup into something in the OpenBSD world so far. On the other hand: It makes no sense to expect that answer from you either, so .. back to the first line, full circle. :)
I should note that most OpenBSD users don't like needless tinkering and therefore stick to the binary packages. Those who want more up to date packages run a -current snapshot. Stable packages are also available via M:Tier.
But if you want to tinker and build your packages, you can do that. That's what the ports tree is there for. That is how the OpenBSD packages are built.
> - build binary packages for ports that either don't exist upstream (stuff I ported) or are outdated upstream?
The package target for bsd.port.mk(5):
package Build a port package (or packages in a MULTI_PACKAGES case)
from the fake installation. Involves creating packaging
information from templates (see COMMENT, SUBST_VARS among
others) and invoking pkg_create(1) for each package in the
MULTI_PACKAGES list. If the repository already contains
up-to-date packages, they are not rebuilt.
> - distribute those in a decent way (PKG_PATH swapping..?)
It is very easy to distribute OpenBSD packages. You can use http, ftp, nfs.. Yes, PKG_PATH needs to be set, just like you need to configure the package manager to your own repo on any other system.
> actually it supports multiple sources, apt like. Can PKG_PATH do that?
pkg_add(1):
PKG_PATH If a given package name cannot be found, the directories
named by PKG_PATH are searched. It should contain a series
of entries separated by colons. Each entry consists of a
directory name. URL schemes such as FTP, HTTP, HTTPS, or
SCP are also appropriate. The current directory may be
indicated implicitly by an empty directory name, or
explicitly by a single period (`./').
> poudriere allows me to build (in bulk) all my relevant packages in jails
So OpenBSD doesn't have jails, but it is possible to do ports builds from a read-only ports tree using a non-privileged user, with systrace in place to filter accesses that shouldn't happen.
> In the end, pkg_* feel like ooold Solaris, or like using Slackware again. That's package management on a different level and feels like a step down from apt/yum/pacman/pkgng.
You sure have lots of feelings about a package manager you don't know. Have you even used it?
I haven't used pkgng and my experience with yum & pacman is limited, but apt (and aptitude) is what made me finally quit using Linux; I'm 100% OpenBSD now. pkg_add is a definite step up for me, whether using subjective feelings or the number of times it's broken things for me as the metric.
I appreciate the help people are offering here. I'm building a list of links (currently the Porter's handbook and various links around that) to read and decided that I need to dive deeper, give it a real spin (again) before commenting .. again.
If I understand the first part of your answer, or where it is coming from: I .. do understand that I can create packages. I don't understand how I'd have my own ports tree in a sensible way (do you have a couple of trees? Build package A from here, package B from there?), when you need stuff that just isn't ported/too 'niche'. Or if you need to bump a version of a port that is in the official tree.
I knew about make package, see above (same on/similar to systems I use today).
I posted my new understanding about PKG_PATH elsewhere in this thread, but I guess I DO have to admit that I should've hit the man page first. Admittedly I never used it for more than pointing to one single official mirror in the couple of VMs I played with. Anyway: Cool.
I'm not married to jails, I just like that poudriere offers a very nice way to build packages in an isolated environment. Security isn't the biggest concern here: One thing that this allows me to do for example is to run one version of FreeBSD and build packages in a jail of another version (or another arch, or both). At this point I assume there's an OpenBSD equivalent for this functionality as well though. :)
My feelings about pkg_* are honestly based on older experiences with FreeBSD and recent but discontinued explorations in NetBSD. My last OpenBSD installation is a while back and I might've given up quite quickly. Which of course might entirely be a fault of my character.
No offense intended, if you feel I attacked either you or a project you sympathize with I'm sorry about that.
I just felt that your knowledge is seriously outdated or not based on actual usage of the OpenBSD tool (which, by the way, has had a steady stream of improvements throughout its existence). And it looks like I was right, if your experience is indeed based on FreeBSD & NetBSD. OpenBSD has its own tool, which was written from scratch on and for OpenBSD by Marc Espie. It is not based on the code by Hubbard. It is not at all the same thing the other BSDs have.
As for your own ports tree -- well, I can't tell what is the best method for your scenario; I primarily use the binary packages (and sometimes compile things outside the ports tree). But the most obvious thing to do (which I've done) is to checkout the official ports tree, add your own ports and/or make your changes in it and use it as such. You might have to merge upstream changes if they've updated something you have touched. But if you want to track the official ports in addition to your own, then it makes all the sense to watch out for overlapping updates, no?
Also, if you're bumping ports because the OpenBSD porters haven't done so yet, it might be a good idea to send the diff to ports@.
As you correctly stated, my dislike for the pkg_* tools is probably without merit here and based on incorrect assumptions. I'll make sure to learn about the OpenBSD variants.
The ports tree.. Well, that's what I'd like portshaker [1] for. Yes, you can do that manually, but it's a pain.
Bumping ports: Same for FreeBSD, but unless OpenBSD has a much more responsive ports teams/mailing list, this won't usually work fast (I know, I'm extrapolating again). I have currently a meager list of four ports in my overlay that currently don't exist in FreeBSD or are outdated, but these exist for quite a while now ("weeks"). I try to be a good citizen and submit these things upstream, but I don't like to depend on someone having commit access.
> I don't understand how I'd have my own ports tree in a sensible way (do you have a couple of trees? Build package A from here, package B from there?), when you need stuff that just isn't ported/too 'niche'.
There's a /usr/ports/mystuff directory for building your own ports. Unless I'm missing something, I don't see why you'd need a whole new ports tree?
You post is formatted awkwardly so I'm only guessing at your issues. Forgive me if I am misunderstanding.
"you have no decent binary packages" - OpenBSD encourages the use of binary packages. The OS is shipped with them. You can use ports (compile from source) if you like, but packages are preferred. [0]
There are no binary updates for the base system (maybe in 3rd party, but not from OpenBSD). You do have to compile but there are only a few per release and not mandatory if they don't affect you. Here is the heartbleed patch [1]. Very simple and contains the instructions in the patch.
the most headache inducing issues by far are going to be file formats and network protocols that integrate 32bit unix timestamps, which are numerous.
NTP is one such example.
This is a much worse and much more fundamental issue than the y2k bug was, I hope people aren't writing off the severity, the time to start dealing with it is now.
Agreed, it's pretty scary. On the other hand, I'm sure similar problems were tackled for y2k with the 2-digit fields everywhere. If we built it, we can rebuild it. Although a lot of software and hardware will need replacing/upgrading/switching over to new protocols.
They were, however, given that a lot of things have life in the field of several decades, now is the time to start fixing it so it doesn't become a mad rush. Yes, it means less money for devs 23 years from now as they madly rush to update software, but I'd rather that money be spent for devs to make future programs awesome than to worry about our ghosts.
Indeed. I get worried when I see people deploying new embedded ARM systems these days to replace systems that are 30 years old when there is only 24 years to 2038...
If you install a 32-bit Linux, then time_t is still 32-bit, only on 64-bit Linux you get 64-bit time_t.
Although by 2038 I hope we won't have many 32-bit systems around...
Embedded devices are likely to be 32-bit for a very long time yet. I'd consider 100+ years to be a reasonable number!
64-bit devices are much more complicated (bus size, peripherals, part count) and therefore more expensive. If your entire task fits in a 32-bit space there is little motivation to use a 64-bit core.
It won't on full SoC devices but most ARM embedded devices of that sort have all sorts of peripherals which are bus connected so the bus will need to be external.
If it ain't broke...There are still 8 bit and 16 bit cores being produced today. Hoping products will become obsolete is one of the big reasons y2k was a problem.
I know Theo and others are opposed to it, but I still would like to see it on the Raspberry Pi. Kind of a missed opportunity to make it easy for lots of people to give OpenBSD a try.
It's not that Theo & co oppose it per se. It's that they are not interested in working on it, for a number of good reasons. If you are interested and willing to do the work to get it running (with free software), I'm sure somebody will take a look at the diffs you send to tech@. Chances are you'll also have to maintain the work.
This strategy really hit home for my dinkly little router. I don't have the disk or CPU resources to compile these patches. Is there no other way to get the updates other than wait for the next release?
You can run -current snapshots. Caveat: the system changes, packages change, things are not always in sync. You might have to upgrade to a newer snapshot if you want to add binary packages after a while. This doesn't affect you if you only use software from base, or only use packages that can be installed around the time you install the system.
Alternatively, you can get binary patches from M:Tier:
Get someone you trust to build it for you is my current solution[1]. I'm not really up on the plans for binary patches other than they are signing stuff now.
Regarding the 2k38 problem, 24 years from now will any 32-bit systems still be in production? Won't mean-time-to-failure handle this for us? Software wise, it might be a different story.
Not only will they be in production, they will still be built and designed.
Stuff like nuclear power-plants and bridges also have a slightly high mean-time-to-failure than my laptop. 24 years isn't that long for infrastructure.
My favorite operating system, unfortunately I had to uninstall it since I intend to learn Android development. If you haven't tried it yet & it looks like it has the software you need I invite you to give it a try.
Compare this to a Ubuntu LTS release which is virtually impossible to determine a detailed change list for past the marketing and UI changes.