The graph shows a baseline 2% task success rate improving to to 8% task success rate, but the evals section details 100% success rates across the board.
I'm not sure what the effectiveness of this skill is from the readme. Is it 8% success, or 100% success?
The problem is there's really a lot of data out there and it's a lot of work to move it around, e.g. between S3 buckets. There's also a ton of GIS SAAS vendors who are pure rent-seekers: I'm looking at a newer offering charging $23 per month for 10GB storage. This has more utility than their offering in my opinion.
The good thing here is that it could keep data provenance because it's SQL over known datasets.
Either LLMs will be so good in a few months this will be redundant.
Or it won't be and LLMs are a dead end and there are better ways to build with LLMs