Post by dunx on Oct 29, 2005 14:09:51 GMT
As I have mentioned before, I have characters notes and so on stored in a wiki instance. It would be nice to supply a list of those pages I want to import, presumably taking the titles from the pages themselves, rather than having to do the menu button shuffle for every page.
I can see two ways to implement this:
1/ spider a target site: this would be the simplest from a user perspective, but would be very bad from a developer perspective. You need to start worrying about robots.txt files, and generally being a good citizen when scraping a site.
2/ provide a list of URLs. Worse from a user perspective, but still easier than manual import of every page. And it provides more control about what is imported.
In either case, it would be reasonable to restrict a particular import operation to one site.
I can see two ways to implement this:
1/ spider a target site: this would be the simplest from a user perspective, but would be very bad from a developer perspective. You need to start worrying about robots.txt files, and generally being a good citizen when scraping a site.
2/ provide a list of URLs. Worse from a user perspective, but still easier than manual import of every page. And it provides more control about what is imported.
In either case, it would be reasonable to restrict a particular import operation to one site.
