Torrent Search plugin development guide

Gwendal Le Bihan

This document is a quick guide to plugin development for Torrent Search. Before going further into it, you should have some Python programming knowledge, and have some notions about parsing HTML (or XML) documents.

I've chosen to base this guide on example, explaining the steps to build the plugin.

If you have any comments about this document, please contact me by mail at gwendal.lebihan.dev@gmail.com.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".


Table of Contents
1. Prerequisites
2. Introduction
2.1. What does a plugin do exactly ?
2.2. What informations must be retrieved ?
2.3. Plugin files
3. Starting
3.1. Creating the files
3.1.1. Metadata file
3.1.2. Source file
3.2. Loading the plugin
3.3. Important note about system calls
4. Core of the plugin
4.1. Sending results to the application
4.2. Note about the result class
4.3. Retrieving the results
4.3.1. Making the request
4.3.2. Parsing the response
4.3.3. Localising and listing the results
4.3.4. Parsing a result
4.3.5. Back to the _run_search method
4.3.6. Case of multiple results pages
4.3.7. Last thing : counting the results
5. Publishing your plugin
6. Conclusion and recommendations
A. Full example code
B. The find_elements function
C. GNU Free Documentation License
List of Figures
3-1. Plugins list
List of Examples
B-1. find_elements example

Chapter 1. Prerequisites

Before starting to develop plugins for Torrent Search, you should obviously have Torrent Search installed. You can download it from Sourceforge. It's also a good idea to have taken a look to the documentation of the libxml2 library, or another XML/HTML parser. Examples in this document will be based on the libxml2 library.


Chapter 2. Introduction

2.1. What does a plugin do exactly ?

A plugin is given by the application a pattern to search for. Based on this, it will make a request to a web server to retrieve a list of torrents matching the pattern, with some informations about them. The plugin will progressively submit its results to the application, which will display them in the results list.


2.2. What informations must be retrieved ?

Some informations about the torrent files have to be retrieved by the plugin for its results to be taken into account by the application :

  • URL of the torrent file

  • Torrent title

  • File size

  • Date the torrent was added

  • Number of seeders/leechers

The torrent can optionnaly also submit the magnet link of the torrent. This should be done when possible, as it allows to filter duplicate results.


2.3. Plugin files

A plugin is constituted of 2 files :

  • A Python source file, containing the code of the plugin

  • An XML metadata file, containing informations about the plugin


Chapter 3. Starting

3.1. Creating the files

3.1.1. Metadata file

We will assume here that we are working in the folder ~/dev/torrent-search/myplugin. We are going to create a file named metadata.xml. Here is what it will look like :

   <?xml version="1.0"?>
   <plugin id="myplugin" version="0.1">
      <title>My Plugin</title>
      <released_time>2010-07-30 11:40:48</released_time>
      <author></author>
      <filename>MyPlugin.py</filename>
      <classname>MyPlugin</classname>
      <download_url></download_url>
   </plugin>
            

You do not have to fill the author and download_url fields, they will be useless for now.

Warning

Even if the actual date you'll put in the released_time does not matter, you have to put a correctly formatted date there, or your plugin will be ignored by the application.

The fields filename and classname must be correctly filled, respectively with the name of the file you will put your code in, and the name you will use for the class of your plugin.


3.1.2. Source file

We are now going to create the code file ~/dev/torrent-search/myplugin/MyPlugin.py.

Your plugin's must be a subclass of TorrentSearch.Plugin.Plugin and must implement a method called _run_search, which will take a pattern parameter.

We will also have to create a MyPluginResult class, subclass of TorrentSearch.Plugin.PluginResult and implementing a method called _do_get_link with no parameters. We will see what this is used for later.

So, here is what the file MyPlugin.py should look like for now :

   #! /usr/bin/python
   # -*- coding=utf-8 -*-
   
   import TorrentSearch
   
   class MyPluginResult(TorrentSearch.Plugin.PluginResult):
   
      def _do_get_link(self):
         pass
         
   
   class MyPlugin(TorrentSearch.Plugin.Plugin):
      
      def _run_search(self, pattern):
         pass
            

3.2. Loading the plugin

Now that we have created the files, let's see how to load our plugin in Torrent Search. There is a command-line parameter for that, --add-plugin, which allows us to load a plugin from an additional path. So we will load Torrent Search using the following command :

torrent-search --add-plugin ~/dev/torrent-search/MyPlugin

To check that our plugin has been loaded, let's go to the Search plugins tab of the Preferences dialog.

Figure 3-1. Plugins list

We can see that the plugin My Plugin has been loaded.

Before going further, I recommend to disable every other plugins, which makes it easier to see what our does plugin does.


3.3. Important note about system calls

Important

You must be very careful about system calls in your plugin (ie using the os module) : Torrent Search becoming a cross-platform application, the plugins have to conform to that and never use platform-dependant calls (like os.system).


Chapter 4. Core of the plugin

4.1. Sending results to the application

The TorrentSearch.Plugin.Plugin class has a method called add_result :

def add_result(self, result)
         

Each time a new result is found, we will have to make a call to this method, passing it an object of class MyPluginResult.


4.2. Note about the result class

We've seen earlier that the MyPluginResult class must implement a _do_get_link method. Why is that ?

Depending on the structure of the website you're building a plugin for, you may have to access several pages to get all the informations required about the torrent files. In many cases, you will be able to get all the informations for one page, except for the URL of the file. In that case, you are going to send the result to the application without the URL, in order to make the search faster. When a download will be requested on a file, the application will call the _do_get_link method, which will then retrieve the actual URL of the torrent file.

Note

We are now going to build our plugin. From now on, we will assume we are working on the plugin for http://thepiratebay.org/ website.


4.3. Retrieving the results

4.3.1. Making the request

The first thing we'll have to do is identifying the structure of the request URL. On "The Pirate Bay", if we're looking for "funny video", the request URL will be "http://thepiratebay.org/search/funny+video". So the first thing we'll hace to do in our _run_search method is to the get data from this URL.

   def _run_search(self, pattern):
      filename, msg = urllib.urlretrieve("http://thepiratebay.org/search/"+urllib.quote_plus(pattern))
            

Note

The urllib.quote_plus function will automatically replace spaces by the "+" sign and handle specials characters.

Note

The urllib.urlretrieve function is not the only way to perform the request. You may prefer usgin urllib.urlopen or a HTTP connection with the httplib or httplib2 libraries. It's up to you.


4.3.2. Parsing the response

Now that we have downloaded the results page, we are going to parse it. Using libxml2, we will first load the tree :

   tree = libxml2.htmlParseFile(filename, "utf-8")
            

Note

This will usually induce a lot of "HTML parser error" output. For debugging purpose, we will not want to see all these. I would suggest redirecting the standard error :

torrent-search --add-plugin ~/dev/torrent-search/MyPlugin 2>/dev/null

Note

During development phase, we will need to see the HTML source of the page. We will make a copy of it :

   os.system('cp "%s" "~/dev/torrent-search/MyPlugin/temp.html"'%filename)
            

Important

Do not forget to delete your temporary files when they are not needed any more !

   os.unlink(filename)
            

To sum it up, here is what the MyPlugin.py file looks like now :

   #! /usr/bin/python
   # -*- coding=utf-8 -*-
   
   import TorrentSearch, urllib, libxml2, os
   import TorrentSearch.htmltools
   
   class MyPluginResult(TorrentSearch.Plugin.PluginResult):
   
      def _do_get_link(self):
         pass
         
   
   class MyPlugin(TorrentSearch.Plugin.Plugin):
      
      def _run_search(self, pattern):
         filename, msg = urllib.urlretrieve("http://thepiratebay.org/search/"+urllib.quote_plus(pattern))
         os.system('cp "%s" "~/dev/torrent-search/MyPlugin/temp.html"'%filename)
         tree = libxml2.htmlParseFile(filename, "utf-8")
         os.unlink(filename)
            

4.3.3. Localising and listing the results

Taking a look at the HTML source, we can see that the results are located in a table element with an id property of searchResult. To find this element, we will use the TorrentSearch.htmltools.find_elements function.

   results_table = TorrentSearch.htmltools.find_elements(tree.getRootElement(), "table", id="searchResult")[0]
            

Now, each result is represented by one line in the table, so to list the results, we will simply do :

   results = TorrentSearch.htmltools.find_elements(results_table, "tr")
            

All we have to do now is parsing the content of these lines.


4.3.4. Parsing a result

Here is what a result's line looks like :

   <tr>
      <td class="vertTh">
         <center>
            <a href="/browse/200" title="More from this category">Video</a><br />

            (<a href="/browse/204" title="More from this category">Movie clips</a>)
         </center>
      </td>
      <td>
         <div class="detName"><a href="/torrent/5493621/Funny_video_collections" class="detLink" title="Details for Funny video collections">Funny video collections</a></div>
         <a href="http://torrents.thepiratebay.org/5493621/Funny_video_collections.5493621.TPB.torrent" title="Download this torrent">
            <img src="http://static.thepiratebay.org/img/dl.gif" class="dl" alt="Download" />
         </a>
         <a href="magnet:?xt=urn:btih:da6e502d9cf4c5a595a7c87df622e25426bff254&dn=Funny+video+collections&tr=http%3A%2F%2Fdenis.stalker.h3q.com%3A6969%2Fannounce" title="Download this torrent using magnet">
            <img src="http://static.thepiratebay.org/img/icon-magnet.gif" alt="Magnet link" />
         </a>
         <img src="http://static.thepiratebay.org/img/11x11p.png" />
         <img src="http://static.thepiratebay.org/img/11x11p.png" />
         <font class="detDesc">Uploaded 04-13&nbsp;07:35, Size 3.43&nbsp;GiB, ULed by <a class="detDesc" href="/user/saf_ahd/" title="Browse saf_ahd">saf_ahd</a></font>

      </td>
      <td align="right">9</td>
      <td align="right">22</td>
   </tr>
            

We are going to add a _parse_result method to our plugin, which will take care of parsing this data.

As we can see, the first cell (td) is of no interest to us. The second contains the links, title, size and date of the torrent. The third one contains the number of seeders, the fourth one the number of leechers. So here is how we will start our _parse_result method :

   def _parse_results(self, result_line):
      main_cell, seeders_cell, leechers_cell = TorrentSearch.htmltools.find_elements(result_line, "td")[1:]
            

As it's very simple, let's start by getting the number of seeders and leechers. For this, we just have to read the content of the corresponding cells, and convert it to integer :

   seeders = int(seeders_cell.getContent())
   leechers = int(leechers_cell.getContent())
            

Now let's take a look at the main cell. The link(a) in the <div class="detName"> element refers to the torrent details page. This page has no interest to us, as all the information we need is already present. However, the title of the torrent is the content of this link. To get the title, we'll do :

   title_link = TorrentSearch.htmltools.find_elements(main_cell, "a", **{"class": "detLink"})[0]
   title = title_link.getContent()
            

Following this, we have 2 links, one containing the link to the torrent line, the other the magnet link. To get them, we'll use the following code :

   torrent_link, magnet_link = TorrentSearch.htmltools.find_elements(main_cell, "a", 1)[:2]
            

Note

The third parameter here is used to limit the depth of the search, so that we will only get immediate children of the node.

Now let's get the URL of the links :

   torrent_url = torrent_link.prop('href')
   magnet_url = magnet_link.prop('href')
            

Note

Starting from version 0.8.0, plugins can submit the torrent hash value instead of the magnet link. Here is the new signature of the TorrentSearch.Plugin.PluginResult class constructor :

   def __init__(self, label, date, size, seeders, leechers, magnet_link=None, hashvalue=None)
            

All we have left to do is getting the file size and the date. This is a little bit more tricky than what we've seen so far. Both those informations are in the font element, and not a specific element for each information.


4.3.4.1. Parsing the date

The date you plugin has to submit is a datetime.date object. So we'll need to get the year, month and day of month for the date.

Unfortunately, on "The Pirate Bay", date can be formatted in different ways :

%d mins ago

%d being the number of minutes since the torrent was added, if it was less than an hour ago

Today %H:%M

%H:%M being the time the torrent was added

Y-day %H:%M

%H:%M being the time the torrent was added, if the torrent was added the day before

%m-%d %H:%M

%m-%d being the day the torrent was added (month and day) and %H:%M being the time the torrent was added, if the torrent was added during the current year

%m-%d %Y

%m-%d %Y begin the full date (month, day and year), if the torrent was added before the current year

One more thing is that the spaces are not really spaces, but non-breakable spaces (special HTML character &nbsp).

Let's go step by step :

  1. Get the content of the font element

       data = TorrentSearch.htmltools.find_elements(main_cell, "font")[0].getContent()
                         
  2. Convert the non-breakable spaces into normal spaces

       data = data.replace(chr(194)+chr(160),' ')
                         

    Note

    The non-breakable character is actually constituted of 2 characters of codes 194 and 160.

  3. Remove the leading "Uploaded "

       data = data[9:]
                         
  4. Search the end of the date string

       i = data.index(",")
       data = data[:i]
                         
  5. Parse the date string; we'll do this in a _parse_date method

       def _parse_date(self, data):
          try: # Date this year ?
             res=time.strptime(data,"%m-%d %H:%M")
             res=datetime.date(datetime.date.today().year,res.tm_mon,res.tm_mday)
          except:
             try: # Date before this year ?
                res=time.strptime(data,"%m-%d %Y")
                res=datetime.date(res.tm_year,res.tm_mon,res.tm_mday)
             except:
                if data.split(" ")[0]=="Y-day": # Yesterday ?
                   res=datetime.date.today()-datetime.timedelta(1)
                else: # Today
                   res=datetime.date.today()
          return res
                         

4.3.4.2. Parsing the size

The plugin must submit the size in the format "%f %u", where %f is a float number, and %u the unit (one of B, KB, MB, GB), for example "1.36 GB".

Here, we'll proceed the same way as for the date :

  1. Get the content of the font element

  2. Convert the non-breakable spaces into normal spaces

  3. Search the start of the size string

       i = data.index(",")
       data = data[i+1:]
                         
  4. Remove the leading " Size "

       data = data[6:]
                         
  5. Delete the "i" character

       data = data.replace("i","")
                         

To see how we put all of this together, refer to the full example code.


4.3.5. Back to the _run_search method

Now that we can parse the results, we can complete the _run_search method. It will now look like this :

   def _run_search(self, pattern):
      filename, msg = urllib.urlretrieve("http://thepiratebay.org/search/"+urllib.quote_plus(pattern))
      os.system('cp "%s" "~/dev/torrent-search/MyPlugin/temp.html"'%filename)
      tree = libxml2.htmlParseFile(filename, "utf-8")
      os.unlink(filename)
      
      results_table = TorrentSearch.htmltools.find_elements(tree.getRootElement(), "table", id="searchResult")[0]
      results = TorrentSearch.htmltools.find_elements(results_table, "tr")
      
      for result in results:
         try: # We want to continue parsing even if one result is badly formatted
            self._parse_result(result)
         except:
            pass
         if self.stop_search: # See note below
            return
            

Important

You can see that we make a test to know whether we should stop the search. This is important : in case a new search in run in the application, all plugins should stop the previous search in order to limit bandwidth and processor usage.


4.3.6. Case of multiple results pages

Very often, search engines will give more than one page of results. We are going to adapt the code the be able to handle this.

In our example, when multiple pages are present, we will have a "pager" at the bottom of the page. Unfortunately, the div element containing this pager doesn't have any particular attributes. We will still able to identify it easily, because it's the element immediately following the <div id="main-content"> element. So here is how we find this element :

   maindiv = TorrentSearch.htmltools.find_elements(tree.getRootElement(), "div", id="main-content")[0]
   pager = maindiv.nextElementSibling()
            

In the pager, we are going to look for a link going to the next page. If it exists, it will look like this :

   <a href="/search/video/4/99/100,200,300,400,600/"><img src="http://static.thepiratebay.org/img/next.gif" border="0" alt="Next"/></a>
            

We will be able to identify it easily thanks to alt attribute of the img element it contains :

   next_page_img = TorrentSearch.htmltools.find_elements(pager, "img", alt="Next")
   if next_page_img:
      next_page_link = next_page_img[0].parent
            

We will now rerun the _run_search method, passing it as a new argument the link to the next page. Here is the resulting code :

   def _run_search(self, pattern, page_url=""):
      if page_url == "": # First time we run the method
         page_url = "http://thepiratebay.org/search/"+urllib.quote_plus(pattern)
      filename, msg = urllib.urlretrieve(page_url)
      tree = libxml2.htmlParseFile(filename, "utf-8")
      os.unlink(filename)
      
      results_table = TorrentSearch.htmltools.find_elements(tree.getRootElement(), "table", id="searchResult")[0]
      results = TorrentSearch.htmltools.find_elements(results_table, "tr")
      
      for result in results:
         try: # We want to continue parsing even if one result is badly formatted
            self._parse_result(result)
         except:
            pass
         if self.stop_search:
            return
            
      maindiv = TorrentSearch.htmltools.find_elements(tree.getRootElement(), "div", id="main-content")[0]
      pager = maindiv.nextElementSibling()
      
      next_page_img = TorrentSearch.htmltools.find_elements(pager, "img", alt="Next")
      if next_page_img:
         next_page_link = next_page_img[0].parent
         next_page_url = urllib.basejoin(page_url, next_page_link.prop('href')) # urllib.basejoin handles relative links
         self._run_search(pattern, next_page_url) # We call the method again to parse the next page
            

We now have a fully working plugin !


4.3.7. Last thing : counting the results

This is not necessary, but when possible, it's better to get a total count of the results before starting to parse them. It must be stored in the results_count property of the plugin.

In our case, we will find this number in the first h2 element of the document :

   <h2><span>Search results: video</span>&nbsp;Displaying hits from 0 to 30 (approx 67 found)</h2>
            

To get this number, here is what we'll do :

   results_count_element = TorrentSearch.htmltools.find_elements(tree.getRootElement(), "h2")[0]
   data = results_count_element.getContent()
   i = data.index("(approx ")+8 # Get the position of the number in the string
   data = data[i:] # Remove useless data
   i = data.index(" ") # Get the end of the number
   data = data[:i] # Remove useless data
   self.results_count = eval(data)
            

This time, we're really done with the code !


Chapter 5. Publishing your plugin

Hey, we haven't done all this for nothing ! Now, we would like all the users of Torrent Search to be able to use our plugin.

To publish your plugin, contact me by mail at gwendal.lebihan.dev@gmail.com. It will be added to the database on the website, and will be part of the next plugins update.


Chapter 6. Conclusion and recommendations

What we've seen here is one way of developping a plugin. It isn't the only, and not necessarily the best one. The goal of the guide was to give some ideas on the plugin development process. Now, you're free to go, keeping in mind this few rules :

I recommend during development to go by small steps, and to print the informations retrieved.

Be careful about the website you choose to develop a plugin for : some make it easier to you than other, bust more importantly, some will not give you all the informations required by Torrent Search ! Don't lose time on those.

Have fun !


Appendix A. Full example code

   #! /usr/bin/python
   # -*- coding=utf-8 -*-
   
   import TorrentSearch, urllib, libxml2, os
   import TorrentSearch.htmltools
   
   class MyPluginResult(TorrentSearch.Plugin.PluginResult):
   
      def _do_get_link(self):
         pass
         
   
   class MyPlugin(TorrentSearch.Plugin.Plugin):
      
      def _run_search(self, pattern, page_url=""):
         if page_url == "":
            page_url = "http://thepiratebay.org/search/"+urllib.quote_plus(pattern)
         filename, msg = urllib.urlretrieve(page_url)
         tree = libxml2.htmlParseFile(filename, "utf-8")
         os.unlink(filename)
         
         try:
            results_count_element = TorrentSearch.htmltools.find_elements(tree.getRootElement(), "h2")[0]
            data = results_count_element.getContent()
            i = data.index("(approx ")+8
            data = data[i:]
            i = data.index(" ")
            data = data[:i]
            self.results_count = eval(data)
         except:
            pass
         
         results_table = TorrentSearch.htmltools.find_elements(tree.getRootElement(), "table", id="searchResult")[0]
         results = TorrentSearch.htmltools.find_elements(results_table, "tr")
         
         for result in results:
            try: # We want to continue parsing even if one result is badly formatted
               self._parse_result(result)
            except:
               pass
            if self.stop_search:
               return
               
         maindiv = TorrentSearch.htmltools.find_elements(tree.getRootElement(), "div", id="main-content")[0]
         pager = maindiv.nextElementSibling()
         
         next_page_img = TorrentSearch.htmltools.find_elements(pager, "img", alt="Next")
         if next_page_img:
            next_page_link = next_page_img[0].parent
            next_page_url = urllib.basejoin(page_url, next_page_link.prop('href'))
            self._run_search(pattern, next_page_url)
               
      def _parse_date(self, data):
         try: # Date this year ?
            res=time.strptime(data,"%m-%d %H:%M")
            res=datetime.date(datetime.date.today().year,res.tm_mon,res.tm_mday)
         except:
            try: # Date before this year ?
               res=time.strptime(data,"%m-%d %Y")
               res=datetime.date(res.tm_year,res.tm_mon,res.tm_mday)
            except:
               if data.split(" ")[0]=="Y-day": # Yesterday ?
                  res=datetime.date.today()-datetime.timedelta(1)
               else: # Today
                  res=datetime.date.today()
         return res
               
      def _parse_result(self, result_line):
         main_cell, seeders_cell, leechers_cell = TorrentSearch.htmltools.find_elements(result_line, "td")[1:]
      
         seeders = int(seeders_cell.getContent())
         leechers = int(leechers_cell.getContent())
         
         title_link = TorrentSearch.htmltools.find_elements(main_cell, "a", **{"class": "detLink"})[0]
         title = title_link.getContent()
         
         torrent_link, magnet_link = TorrentSearch.htmltools.find_elements(main_cell, "a", 1)[:2]
         torrent_url = torrent_link.prop('href')
         magnet_url = magnet_link.prop('href')
         
         date_size_data = TorrentSearch.htmltools.find_elements(main_cell, "font")[0].getContent()
         date_size_data = date_size_data.replace(chr(194)+chr(160),' ')
         
         date_data, size_data = date_size_data.split(',')[:2]
         
         date_data = date_data[9:]
         date = self._parse_date(date_data)
         
         size_data = size_data[6:]
         size = size_data.replace("i","")
         
         self.add_result(MyPluginResult(title, date, size, seeders, leechers, torrent_url, magnet_url))
         
      

Appendix B. The find_elements function

Torrent Search comes with a simple helper function find_elements, that can be found in the TorrentSearch.htmltools package, which allows to search for elements in an HTML document based on their name and properties.

def find_elements(node, elname = None, maxdepth = -1, **params)
      

node

The node (libxml2.xmlNode) under which to search

elname

The name of the elements to select (or None to select any element name)

maxdepth

How deep to walk under node (-1 for infinite depth)

**params

The properties of the elements to select

Example B-1. find_elements example

document = libxml2.htmlParseFile("somefile.html", "utf-8")
root_node = document.getRootElement()
elements = find_elements(root_node, "div", id="main")
         

In this example, we will select all elements named div with a property id equal to "main".

Warning

class is a reserved word in Python. The following line will give an error :

elements = find_elements(root_node, "div", class="main")

Instead, you will have to use the followihg syntax :

elements = find_elements(root_node, "div", **{"class": "main"})

Appendix C. GNU Free Documentation License

Version 1.3, 3 November 2008

Copyright © 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc.

Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other functional and useful document “free” in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of “copyleft”, which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. Such a notice grants a world-wide, royalty-free license, unlimited in duration, to use that work under the conditions stated herein. The “Document”, below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as “you”. You accept the license if you copy, modify or distribute the work in a way requiring permission under copyright law.

A “Modified Version” of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A “Secondary Section” is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document’s overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The “Invariant Sections” are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License. If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant. The Document may contain zero Invariant Sections. If the Document does not identify any Invariant Sections then there are none.

The “Cover Texts” are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words.

A “Transparent” copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup, or absence of markup, has been arranged to thwart or discourage subsequent modification by readers is not Transparent. An image format is not Transparent if used for any substantial amount of text. A copy that is not “Transparent” is called “Opaque”.

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML, PostScript or PDF designed for human modification. Examples of transparent image formats include PNG, XCF and JPG. Opaque formats include proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML, PostScript or PDF produced by some word processors for output purposes only.

The “Title Page” means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, “Title Page” means the text near the most prominent appearance of the work’s title, preceding the beginning of the body of the text.

The “publisher” means any person or entity that distributes copies of the Document to the public.

A section “Entitled XYZ” means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. (Here XYZ stands for a specific section name mentioned below, such as “Acknowledgements”, “Dedications”, “Endorsements”, or “History”.) To “Preserve the Title” of such a section when you modify the Document means that it remains a section “Entitled XYZ” according to this definition.

The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. These Warranty Disclaimers are considered to be included by reference in this License, but only as regards disclaiming warranties: any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License.

2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

3. COPYING IN QUANTITY

If you publish printed copies (or copies in media that commonly have printed covers) of the Document, numbering more than 100, and the Document’s license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Document, free of added material. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.

  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has fewer than five), unless they release you from this requirement.

  3. State on the Title page the name of the publisher of the Modified Version, as the publisher.

  4. Preserve all the copyright notices of the Document.

  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.

  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.

  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document’s license notice.

  8. Include an unaltered copy of this License.

  9. Preserve the section Entitled “History”, Preserve its Title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section Entitled “History” in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.

  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the “History” section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.

  11. For any section Entitled “Acknowledgements” or “Dedications”, Preserve the Title of the section, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.

  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.

  13. Delete any section Entitled “Endorsements”. Such a section may not be included in the Modified Version.

  14. Do not retitle any existing section to be Entitled “Endorsements” or to conflict in title with any Invariant Section.

  15. Preserve any Warranty Disclaimers.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version’s license notice. These titles must be distinct from any other section titles.

You may add a section Entitled “Endorsements”, provided it contains nothing but endorsements of your Modified Version by various parties — for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice, and that you preserve all their Warranty Disclaimers.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections Entitled “History” in the various original documents, forming one section Entitled “History”; likewise combine any sections Entitled “Acknowledgements”, and any sections Entitled “Dedications”. You must delete all sections Entitled “Endorsements”.

6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, is called an “aggregate” if the copyright resulting from the compilation is not used to limit the legal rights of the compilation’s users beyond what the individual works permit. When the Document is included in an aggregate, this License does not apply to the other works in the aggregate which are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one half of the entire aggregate, the Document’s Cover Texts may be placed on covers that bracket the Document within the aggregate, or the electronic equivalent of covers if the Document is in electronic form. Otherwise they must appear on printed covers that bracket the whole aggregate.

8. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License, and all the license notices in the Document, and any Warranty Disclaimers, provided that you also include the original English version of this License and the original versions of those notices and disclaimers. In case of a disagreement between the translation and the original version of this License or a notice or disclaimer, the original version will prevail.

If a section in the Document is Entitled “Acknowledgements”, “Dedications”, or “History”, the requirement (section 4) to Preserve its Title (section 1) will typically require changing the actual title.

9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense, or distribute it is void, and will automatically terminate your rights under this License.

However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.

Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.

Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, receipt of a copy of some or all of the same material does not give you any rights to use it.

10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See Copyleft.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License “or any later version” applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation. If the Document specifies that a proxy can decide which future versions of this License can be used, that proxy’s public statement of acceptance of a version permanently authorizes you to choose that version for the Document.

11. RELICENSING

“Massive Multiauthor Collaboration Site” (or “MMC Site”) means any World Wide Web server that publishes copyrightable works and also provides prominent facilities for anybody to edit those works. A public wiki that anybody can edit is an example of such a server. A “Massive Multiauthor Collaboration” (or “MMC”) contained in the site means any set of copyrightable works thus published on the MMC site.

“CC-BY-SA” means the Creative Commons Attribution-Share Alike 3.0 license published by Creative Commons Corporation, a not-for-profit corporation with a principal place of business in San Francisco, California, as well as future copyleft versions of that license published by that same organization.

“Incorporate” means to publish or republish a Document, in whole or in part, as part of another Document.

An MMC is “eligible for relicensing” if it is licensed under this License, and if all works that were first published under this License somewhere other than this MMC, and subsequently incorporated in whole or in part into the MMC, (1) had no cover texts or invariant sections, and (2) were thus incorporated prior to November 1, 2008.

The operator of an MMC Site may republish an MMC contained in the site under CC-BY-SA on the same site at any time before August 1, 2009, provided the MMC is eligible for relicensing.

ADDENDUM: How to use this License for your documents

To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:

Copyright © YEAR YOUR NAME

Permission is granted to copy, distribute and/or modify this document under the
terms of the GNU Free Documentation License, Version 1.3 or any later version
published by the Free Software Foundation; with no Invariant Sections, no
Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in
the section entitled “GNU Free Documentation License”.

If you have Invariant Sections, Front-Cover Texts and Back-Cover Texts, replace the “with… Texts.” line with this:

with the Invariant Sections being LIST THEIR TITLES, with the Front-Cover Texts
being LIST, and with the Back-Cover Texts being LIST.

If you have Invariant Sections without Cover Texts, or some other combination of the three, merge those two alternatives to suit the situation.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.