https://plato.asu.edu/guide.html
Their website has just 3 cherry picked instances and claim complete dominance.
Often, the difference on "harder" problems is 10x or more.
I have problems that gurobi solves in 30 seconds that take 15 minutes or more for ~every non-commercial solver (or-tools, HIGHS, ipopt, etc).
But right now, this wouldn't even be interesting to me to use even if they actually were fronting commercial solvers, because they can't actually run it any faster and having this ".solve" API does nothing - pyomo already does that for me in practice.
This doesn't solve or provide guidance for the subtle problems in these otherwise opensource solvers... The first example requires the client to manually disambiguate equivalent variables to get a stable solution... Sure that's a pretty common problem everyone working with optimizers should be familiar with but they're also one of the hardest things to track down in a complex derived model.
Misc thoughts:
- I'm not familiar with the LABS problem, but the LABS benchmark page is interesting & compares against Gurobi. I'd be curious to see how an existing commercial non-mip approximate solver such as Hexaly (formerly LocalSolver) compares here.
- the other two benchmarks aren't very convincing as they don't compare against other methods or show running times
- the front page mentions peer reviewed methodology - consider linking to the publications
- good idea to have case studies of applications. I was a bit confused to see this listed under 'References' but on comparison the Gurobi & Hexaly marketing websites also do this (references -> case studies & references -> customer stories, respectively)
- re the client API, you may want to make the server URL have a default, so your trial users / customers don't have to specify it. It may be easier for you to roll out changes to your server URL in future if you can do it by changing the default server URL in a new version of your client library rather than requiring your customers to update their source code.
All the best!
Hexaly claims to go far beyond MIP. Amazon uses it for packing VMs into servers. This video by one of their research scientists was widely circulated at the time [1].
I work on combinatorial optimization too but a specific problem so we write the heuristics from scratch. Seems exact solvers are doing a lot more these days?
OR-tools is almost exclusively linear programming which according to its strict assumptions converges more or less trivially, assuming a correctly composed program.
Which means if you're paying for it "as a service" you all but deserve to lose that money.
> Different algorithms are better for different problems
So... why does your rhetorical style have such oppositional tone if you're just going to reaffirm the no free lunch theorem?