Hi, I'm Scott Clark, one of the founders of SigOpt (YC W15). Let me know if you have any questions about this post or SigOpt in general.
Jungtaek was recently a Research Engineering Intern with us and wrote this post. If you're a student looking for internships please check out our careers page [1]. Our platform is also free for academics [2]. You can find more of our research (including NIPS, ICML, AISTATS, etc papers) here [3].
We built an evaluation suite [1] and have some benchmarks [2] that we presented at some ICML workshops in 2016 comparing against some standard and open source methods. Members of our research team built or contributed to many of the popular open source libraries in Bayesian optimization (MOE, Bayesopt, Hyperopt, etc). In many of our other examples we benchmark against techniques we typically encounter in practice with our customers (grid and random search) [3] [4]. If you have particular problems you are interested in I'd be happy to connect you with members of the team that can explore a free POC to benchmark against your models [5].
Jungtaek was recently a Research Engineering Intern with us and wrote this post. If you're a student looking for internships please check out our careers page [1]. Our platform is also free for academics [2]. You can find more of our research (including NIPS, ICML, AISTATS, etc papers) here [3].
[1]: https://sigopt.com/careers
[2]: https://sigopt.com/edu
[3]: https://sigopt.com/research