I'd recommend neither. CSS has caught up, and the two large problems that these frameworks solved no longer exists: grids, and differences between browser implementations.
Additionally, semantic UI deserves the scorn of a thousand linguists for the most misleading (unsemantic? or even antisemtitic?) product name ever: there is nothing semantic about class="col-xs-12 col-m-3 col-x-1"
semantics aside (hehe), you described the grid class of bootstrap, not semantic ui. In semantic its something like "ten wide column" or "four wide column" etc.
I rarely use semantic for its grid though. Those are now easy problems to solve. I use it because it gets me a clean and pretty UI I can theme later to adjust the specifics (or get someone who is better at design than me to do it).
Additionally, semantic UI deserves the scorn of a thousand linguists for the most misleading (unsemantic? or even antisemtitic?) product name ever: there is nothing semantic about class="col-xs-12 col-m-3 col-x-1"