OL: Sitemap generation

To generate the Sitemap, execute this code on ol-home:

python sitemaps.py ol_dump_works_latest.txt.gz

the sitemaps.py is located at

/1/var/lib/openlibrary/deploy/openlibrary/openlibrary/data/sitemap.py

The last dump is available at: http://openlibrary.org/data/ol_dump_works_latest.txt.gz
for more details you can see https://openlibrary.org/developers/dumps.

After the sitemap is generated you need to place it in /1/var/lib/openlibrary/sitemaps
as defined in /olsystem/etc/nginx/sites-available/openlibrary.conf on ol-www1.


    location ~ ^/static/(docs|tour|sitemaps|jsondumps|images/shelfview|sampledump.txt.gz)(/.*)?$ {
        root /1/var/lib/openlibrary/sitemaps;
        autoindex on;
        rewrite ^/static/(.*)$ /$1 break;
    }

Note: use /1/var/tmp on ol-home, then you can rsync it from rsync://ol-home/var_1/tmp/