How to Download a PDK

From Symbian Developer Community

See also: Product Development Kit Quick Start

The Symbian PDKs contain a large volume of files, grouped into many separately-downloadable zip files. This page describes a Python utility (downloadkit.py) which downloads and extracts the kit archives for you.

Overview

The installation of a full Symbian PDK is really fairly simple, but it involves downloading and unzipping more than 25 zip files. The downloadkit.py script automates this process: it downloads the archive files for a specified build in order, unzipping them in a separate thread (in parallel).

It works by opening the appropriate download page on the Developer Website, then reading the code for the web page to find the downloadable files and the URLs for each download. Once it has that list, it will work through the items in sequence, downloading each file and then running a separate thread to unzip the file while downloading the next one.

To access the download pages it needs to ask for your developer.symbian.org user id and password, so that it can log in to the website and get access to the right download pages. It's been tested with both member and non-member accounts, for PDKs since 2.0.0, 3.0.b, and 3.0.e.

Warning A few of the files have click-through licenses - for example the (non essential) rnd*.zip archives. The script won't accept click-through licenses on your behalf; if you attempt to access these files without first accepting the agreements you will get error messages like this:   Downloading src_rndonly_samsung.zip *** src_rndonly_samsung.zip is subject to the SYMBIAN FOUNDATION RESEARCH & DEVELOPMENT LICENSE which you have not yet accepted   The solution is to try downloading one of them manually through the website, accept the click-through license agreement, then run the script again.

Installing the utility

downloadkit.py is supplied in the "utilities" repository, so the best way to get the latest version is to pull it from our Mercurial server. As a bonus, this will also get you the latest "clone_all_packages" script used to get the platform source code from Mercurial.

If this is the very first time, we suggest that you type these commands:

mkdir c:\Symbian
hg clone https://developer.symbian.org/oss/MCL/utilities c:\Symbian\utilities

If you've done it before, then you just need to pull with an update

hg pull --update -R c:\Symbian\utilities

You will need to download and install Python to run the script, and 7-zip, because WinZip cannot cope with the large numbers of files in some of the zip files.

What if I don't have Mercurial?

You really should consider installing Mercurial (we recommend using TortoiseHg, which includes Mercurial). However, if that's not for you, then you can still get the utility directly from our Mercurial server as a zip file.

1. Download the zip file from https://developer.symbian.org/oss/MCL/utilities/archive/tip.zip - it will be called something like "utilities-c63eca238256.zip".
2. Unpack the file into c:\Symbian - it will create a directory with the same name as the zip file.
3. Rename c:\Symbian\utilities-* to c:\Symbian\utilities

You'll still need to install Python (version 2.6.3 or later) and 7-zip though, because they are used by downloadkit.py.

Downloading a PDK

To download and extract a kit you simply create an empty directory in which to store it, navigate to that directory, and run the script with the kit version number as an argument. Note that downloadkit.py is a Python script - so you will need to already have Python 2.5 installed on your computer.

For example, to download and extract PDK 3.0.g into g:\PDK\3.0.g, use these commands:

g:
mkdir \PDK\3.0.g
cd \PDK\3.0.g
c:\Symbian\utilities\downloadkit\downloadkit.py 3.0.g

The script will prompt you for your developer.symbian.org user id and password, then download/extract the kit.

Tip Downloading large files can take a long time, during which the script may appear to be frozen; you can use the --progress option to get a progress notification every 20 seconds during file download.

There are a number of other command line options, as listed below:

Usage: downloadkit.py [options] version
 
Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -n, --dryrun          print the files to be downloaded, the 7z commands, and
                        the recommended deletions
  --nosrc               Don't download any of the source code available
                        directly from Mercurial
  --nounzip             Just download, don't unzip or delete any files
  --nodelete            Do not delete files after unzipping
  --progress            Report download progress
  -u USER, --username=USER
                        login to website as USER
  -p PWD, --password=PWD
                        specify the account password
  --debug               debug HTML traffic (not recommended!)
  --webhost=SITE        use alternative website (for testing!)
 

Implementation Details

The "screen scraping" is done using a Python library called BeautifulSoup. The web pages and download URLs are accessed using the standard Python urllib2 library, which is needed because the developer.symbian.org website makes extensive use of session cookies and redirection.

The script looks for downloadable files with ".zip" or ".xml" extensions, then takes the following actions

When the script decides to download a file, it first checks to see if the file is already present with the correct MD5 checksum: this avoids downloading a file again unnecessarily if you run the script more than once. It also computes the checksum of the file as it downloads, and will report a warning if the checksum doesn't match the value in the release_metadata.xml file.

The script always had to read the download pages, to find the precise URLs for each file, and always downloads the release_metadata.xml file to get the checksums. A website login is needed to do that, which is why you always need to provide a username and password, even with the "--dryrun" option.

What if the Script Fails?

We are fairly confident of this script - it has been tested with both member and non-member accounts, for PDKs since 2.0.0, 3.0.b, and 3.0.e. However if it does fail, you can download the kits manually from the developer website (and please report the issue as a defect in our Bug tracker , or as a comment to this page). The "--dryrun" option may be helpful in this circumstance: it lists the URLs, the filenames and suggested actions which the script would normally do on your behalf.

Can I run the script again if things break part way through?

It is safe to run the script again if you had problems the first time. The script examines the MD5 checksum of existing local files before downloading, so if you run the script twice is should not download the really large binaries_*.zip files again. The large src_*.zip files are deleted after unpacking, so they will be downloaded again. If you have persistent network issues, then use the "--nounzip" option to avoid deleting those files.

The PDK download page for the latest kit in each Symbian platform version are linked from the Tools & Kits entry point (you can access older builds from the Build and Integration page). Extracting the kits involves manually following the behaviour described in #Implementation Details above.

My organisation requires me to use a Proxy Server

The script includes a check to see if it can contact the https://developer.symbian.org login page - if it can't, then you will see a message that says:

 
Do you need to use a proxy server to access the developer.symbian.org website?
 

If you are required to use a proxy server to get from a corporate network to the outside world, then you will need to use Python 2.6.3 or later to get an essential bugfix in the urllib2 library.

Assuming you have a suitable version of Python, you need to set environment variables to tell downloadkit.py that it should use a proxy.

 
set http_proxy XXX
set https_proxy YYY
 

where XXX is the address or name of your proxy server for HTTP, and YYY is the address or name of the proxy server for HTTPS (which might be different). Q: How can people identify the proxy server name? Browser settings?

It's still not working

If all else fails, try the "--debug" option. This will print out the HTTP requests and responses that show what is really happening behind the scenes. Note that this output includes your password, so please be careful to remove or disguise the password before showing the debug output to anyone else. It will also create downloadpage.html, which is the result of trying to read the download page.

Open Defects

Further Information & Improvements

Please leave a comment on this page if you have specific suggestions for improvements, or better still, wish to contribute some changes! Discussion can also be posted on the forum as a response to this forum post.

How we test the script

We test new PDK download pages with this script before we make them public. To do this, we use the --webhost option to point to our development server, and there's some code which deals with the extra layer of HTTP authentication. We don't currently test for everything which could go wrong, particularly the issues about proxy servers. If you would like to volunteer to test new versions of downloadkit.py, please get in touch!

Sign in to comment…

James Nash (cirrus) said…

Traceback (most recent call last):
  File "c:\Symbian\utilities\downloadkit\downloadkit.py", line 398, in <module>
    login(True)
  File "c:\Symbian\utilities\downloadkit\downloadkit.py", line 65, in login
    response = urllib2.urlopen(req)
  File "C:\APPS\actpython\lib\urllib2.py", line 124, in urlopen
    return _opener.open(url, data, timeout)
  File "C:\APPS\actpython\lib\urllib2.py", line 383, in open
    response = self._open(req, data)
  File "C:\APPS\actpython\lib\urllib2.py", line 401, in _open
    '_open', req)
  File "C:\APPS\actpython\lib\urllib2.py", line 361, in _call_chain
    result = func(*args)
  File "C:\APPS\actpython\lib\urllib2.py", line 1138, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "C:\APPS\actpython\lib\urllib2.py", line 1105, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 1] _ssl.c:480: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol>

>>> import _ssl
# C:\APPS\actpython\lib\encodings\cp850.pyc matches C:\APPS\actpython\lib\encodings\cp850.py
import encodings.cp850 # precompiled from C:\APPS\actpython\lib\encodings\cp850.pyc
import _socket # dynamically loaded from C:\APPS\actpython\DLLs\_socket.pyd
import _ssl # dynamically loaded from C:\APPS\actpython\DLLs\_ssl.pyd

William roberts said…

William roberts said…

James Nash (cirrus) said…

J:\>python c:\Symbian\utilities\downloadkit\downloadkit.py --debug 3.0.g
username: XXX
Password:
Traceback (most recent call last):
  File "c:\Symbian\utilities\downloadkit\downloadkit.py", line 422, in <module>
    login(True)
  File "c:\Symbian\utilities\downloadkit\downloadkit.py", line 83, in login
    response = urllib2.urlopen(req)
  File "C:\APPS\actpython\lib\urllib2.py", line 124, in urlopen
    return _opener.open(url, data, timeout)
  File "C:\APPS\actpython\lib\urllib2.py", line 383, in open
    response = self._open(req, data)
  File "C:\APPS\actpython\lib\urllib2.py", line 401, in _open
    '_open', req)
  File "C:\APPS\actpython\lib\urllib2.py", line 361, in _call_chain
    result = func(*args)
  File "C:\APPS\actpython\lib\urllib2.py", line 1138, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "C:\APPS\actpython\lib\urllib2.py", line 1105, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 1] _ssl.c:480: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol>

William roberts said…

Jasper said…

William roberts said…

Sampsavo said…