Site Map Generator is a handy tool that works on any platform to create site maps. It starts by crawling a website from a URL you provide and then produces an XML sitemap file. This file can be super useful for Google or other search engines when you use it with Google Webmaster Tools. Plus, it's free to use because it’s published under the GNU General Public License!
One of the cool things about this generator is that you don’t need shell access to your web server! It runs as a simple crawler on any computer with Python installed. The crawler will only follow links that are local, meaning it skips any links that go off-site.
This generator can even add <lastmod>
dates in your sitemap if your web server provides pages with 'Last-Modified' timestamps. If there’s an error while downloading or parsing a page, don’t worry! The crawler will keep going and try another page instead.
You’ll need Python version 2.4 or higher to run this script (the latest version as of now is 2.5, which you can grab from Python's official site). There’s no tricky installation process—just copy it into a directory and run it from there!
This script is great for smaller and medium-sized websites since it creates just one sitemap file. It maxes out at 50,000 URLs because that's what Google allows for sitemap files. By default, it only covers up to 1,000 URLs, but you can change that using the -m
option.
The command line syntax looks like this:
python sitemap_gen.py <options> <starting URL>
-h --help
: Print help info and exit.-b <ext> --block <ext>
: Exclude URLs with specific extensions (no leading dot). This comparison isn’t case-sensitive.-c <value> --changefreq <value>
: Set how often content changes (like always, daily, weekly).-p <prio> --priority <prio>
: Set priority between 0.0 and 1.0.-m <value> --max-urls <value>
: Set how many URLs to crawl (default is 1000, max is 50000).-o <file> --output-file <file>
: Name your generated sitemap file (default is sitemap.xml).If you're ready to give it a go, here’s how you might run the script:
python sitemap_gen.py -b doc -b bmp -o test_sitemap.xml http://www.your-site-name.com/index.html
Go to the Softpas website, press the 'Downloads' button, and pick the app you want to download and install—easy and fast!
SoftPas is your platform for the latest software and technology news, reviews, and guides. Stay up to date with cutting-edge trends in tech and software development.
Subscribe to newsletter
© Copyright 2024, SoftPas, All Rights Reserved.