pastk/organicmaps
1
Fork 0
forked from organicmaps/organicmaps
Code Pull requests Activity

[crawler] Crawl more Wikitravel pages.

Browse source
This commit is contained in:
Yury Melnichek 2012-09-10 23:59:48 +02:00 • committed by Alex Zolotarev
parent 704b2fe733
commit 59ae727bef
1 changed files with 1 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
crawler/wikitravel-download-lists.sh
View file

@ -9,6 +9,7 @@ wget $LONGPAGES_URL"&limit=5000&offset=0" -O wikitravel-pages-0.html && slee
wget $LONGPAGES_URL"&limit=5000&offset=5000" -O wikitravel-pages-1.html && sleep 10s
wget $LONGPAGES_URL"&limit=5000&offset=10000" -O wikitravel-pages-2.html && sleep 10s
wget $LONGPAGES_URL"&limit=5000&offset=15000" -O wikitravel-pages-3.html && sleep 10s
wget $LONGPAGES_URL"&limit=5000&offset=20000" -O wikitravel-pages-4.html && sleep 10s
# Get all redirects.
wget $REDIRECTS_URL"&limit=5000&offset=0" -O wikitravel-redirects-0.html && sleep 10s