Missed this thread last week. Nice work! Great to see all the apps coming in recently 
Couple of Qs:
How is the data being stored? (What sort of data structure is being used). (as @happybeing noted, available as a datatype/in a structure readable by anyone/app? it would be great if we as a community could come up with some standardized data structures for search).
Do you have a typetag you’re using for the data (or plan to use one)?
Some other thoughts:
Toootally.
A centralized crawler/uploader/search app shouldn’t really be needed for SAFE, in my (super idealised safe-world) opinion. Is there anything blocking users from doing some crawling and uploading data sets? (eg. using a specific tag / or using a public MD with open read/insert as you suggest, that could enable anyone to crawl, and add to the data set, I think).
This also allows users to own their data that they crawled. Which is another great benefit of SAFE IMO.
Crawling and searching don’t necessarily need to be the same application/site (at least, as @neo points out, is the intention with APP rewards). They could very much be two separate things.
As long as the data is open/available and standardised in some way (which we should figure out just how that would be*), then there’s a lot of flexibility brought in for users / consumers / other devs to build upon (just as search indexes).
Heh, @Shane @AndyAlban sorry for the barrage of Qs here. I’ve been thinking about search on safe / what that might mean for some time (but never got round to tidying up my dabbles 
). Great to see work coming out in this area! I’d love to hear what you think on the above.
(* There are some threads on this: indexed data; an idea built around some older network data types, but a lot should still be applicable).