The UI for defining the scraper targeting rules is an A+ in my book. I've used a number of visual tools for this over the years but most try to do to much and the usefulness suffers. Yours doesn't.
Your price however feels off by about a factor of somewhere between 4 and 8. I haven't really got your system usage data to fully analyse but it feels too high a price. If your using full desktops and chrome browsers and the overhead is higher than I'm expecting based on past experience then perhaps the pricing is off by a factor of only 2
That also doesn't even account for the ability to take the scrape rules, reformat them to run with a headless scraper like scrapy when the website is suitable and is able to be scraped by scrapy, offering a reduced cost scraping option powered this way, throttled back to "human speed" is probably best priced around $1 or $2 a month per crawl thread.
I'm not saying the chrome browser powered crawling part couldn't be so inefficient you need to charge $20 to make 1 crawler pay for itself, but if that's the case, it's a growth limiter you should consider ways to optimise.
Right now your UI is about 10 times better than the open source Portia UI from scrapinghub, but your also 10x the cost of operating a Portia crawler. It's just something to keep in mind.
One question you should be prepared to answer is how you compare to alternatives:
- Services like Kimonolabs and Import.io
- Building a scraper with Scrapy
- Using Scrapehub
From what I understand the price point of the above is lower (some are free) while still offering the same features. What can you offer that they don't?
- Crawl & Extract Static, Dynamic, Single Page Applications.
- Fixed pricing for unlimited scraping
I used Kimonolabs but found it falling short of Import.io. Import.io is great but it felt very bloated and complicated to use.
If you already know how to use Scrapy or a free solution and it works for you then honestly you won't need it.
There's also a free plan where you can run any number of jobs but each job has a limit of 30 minutes and a wait line.
Further adding to the price point, you can create as many scraping jobs and run it as often as you want but in the free plan there's a wait line at peak load.
Page itself looks nice, but why you've decided to have a custom scroll? I think it would be so much better without it. Will try to look into app itself later.
No, it will not fly through thousands of pages at a blinding rate using multiple ip addresses, it goes about at a human like pace to avoid getting banned. If an ip is banned a new one may be assigned. If you need to go faster you will need concurrent jobs.
All the jobs use chrome browser, and supports any website.
Your price however feels off by about a factor of somewhere between 4 and 8. I haven't really got your system usage data to fully analyse but it feels too high a price. If your using full desktops and chrome browsers and the overhead is higher than I'm expecting based on past experience then perhaps the pricing is off by a factor of only 2
That also doesn't even account for the ability to take the scrape rules, reformat them to run with a headless scraper like scrapy when the website is suitable and is able to be scraped by scrapy, offering a reduced cost scraping option powered this way, throttled back to "human speed" is probably best priced around $1 or $2 a month per crawl thread.
I'm not saying the chrome browser powered crawling part couldn't be so inefficient you need to charge $20 to make 1 crawler pay for itself, but if that's the case, it's a growth limiter you should consider ways to optimise.
Right now your UI is about 10 times better than the open source Portia UI from scrapinghub, but your also 10x the cost of operating a Portia crawler. It's just something to keep in mind.
TL;DR? Nice product, but a bit expensive.