Welcome to Pyslet’s documentation!¶

Contents:

IMS Global Learning Consortium Specifications¶

The section contains modules that implement specifications published by the IMS Global Learning Consortium. For more information see http://www.imsglobal.org/

Contents:

IMS Content Packaging (version 1.2)¶

The IMS Content Packaging specification defines methods for packaging and organizing resources and their associated metadata for transmission between systems. There is a small amount of information on Wikipedia about content packaging in general, see http://en.wikipedia.org/wiki/Content_package. The main use of IMS Content Packaging in the market place is through the SCORM profile. Content Packaging is also used as the basis for the new IMS Common Cartridge, and a method of packaging assessment materials using the speicifcation is also described by IMS QTI version 2.1.

Official information about the specification is available from the IMS GLC: http://www.imsglobal.org/content/packaging/index.html

Example¶

The following example script illustrates the use of this module. The script takes two arguments, a resource file to be packaged (such as an index.html file) and the path to save the zipped package to. The script creates a new package containing a single resource with the entry point set to point to the resource file. It also adds any other files in the same directory as the resource file, using the python os.walk function to include files in sub-directories too. The ContentPackage.IgnoreFilePath() method is used to ensure that hidden files are not added:

#! /usr/bin/env python

import sys, os, os.path, shutil
from pyslet.imscpv1p2 import ContentPackage, PathInPath
from pyslet.rfc2396 import URIFactory

def main():
        if len(sys.argv)!=3:
                print "Usage: makecp <resource file> <package file>"
                return
        resFile=sys.argv[1]
        pkgFile=sys.argv[2]
        pkg=ContentPackage()
        try:
                if os.path.isdir(resFile):
                        print "Resource entry point must be a file, not a directory."
                        return
                resHREF=URIFactory.URLFromPathname(resFile)
                srcDir,srcFile=os.path.split(resFile)
                r=pkg.manifest.root.Resources.ChildElement(pkg.manifest.root.Resources.ResourceClass)
                r.href=str(resHREF.Relative(URIFactory.URLFromPathname(os.path.join(srcDir,'imsmanifest.xml'))))
                r.type=='webcontent'
                for dirpath,dirnames,filenames in os.walk(srcDir):
                        for f in filenames:
                                srcPath=os.path.join(dirpath,f)
                                if pkg.IgnoreFilePath(srcPath):
                                        print "Skipping: %s"%srcPath
                                        continue
                                dstPath=os.path.join(pkg.dPath,PathInPath(srcPath,srcDir))
                                # copy the file
                                dName,fName=os.path.split(dstPath)
                                if not os.path.isdir(dName):
                                        os.makedirs(dName)
                                print "Copying: %s"%srcPath
                                shutil.copy(srcPath,dstPath)
                                pkg.File(r,URIFactory.URLFromPathname(dstPath))
                if os.path.exists(pkgFile):
                        if raw_input("Are you sure you want to overwrite %s? (y/n) "%pkgFile).lower()!='y':
                                return
                pkg.manifest.Update()
                pkg.ExportToPIF(pkgFile)
        finally:
                pkg.Close()

if __name__ == "__main__":
        main()

Note the use of the try:... finally: construct to ensure that the ContentPackage object is properly closed when it is finished with. Note also the correct way to create elements within the manifest, using the dependency safe *Class attributes:

r=pkg.manifest.root.Resources.ChildElement(pkg.manifest.root.Resources.ResourceClass)

This line creates a new resource element as a child of the (required) Resources element.

At the end of the script the ManifestDocument is updated on the disk using the inherited Update() method. The package can then be exported to the zip file format.

Reference¶

class pyslet.imscpv1p2.ContentPackage(dPath=None)¶

Represents a content package.

When constructed with no arguments a new package is created. A temporary folder to hold the contents of the package is created and will not be cleaned up until the Close() method is called.

Alternatively, you can pass an operating system or virtual file path to a content package directory, to an imsmanifest.xml file or to a Package Interchange Format file. In the latter case, the file is unzipped into a temporary folder to facilitate manipulation of the package contents.

A new manifest file is created and written to the file system when creating a new package, or if it is missing from an existing package or directory.

ManifestDocumentClass¶

the default class for representing the Manifest file

alias of ManifestDocument

dPath = None¶: the VirtualFilePath to the package’s directory

manifest = None¶

The ManifestDocument object representing the imsmanifest.xml file.

The file is read (or created) on construction.

fileTable = None¶

The fileTable is a dictionary that maps package relative file paths to the File objects that represent them in the manifest.

It is possible for a file to be referenced multiple times (although dependencies were designed to take care of most cases it is still possible for two resources to share a physical file, or even for a resource to contain multiple references to the same file.) Therefore, the dictionary values are lists of File objects.

If a file path maps to an empty list then a file exists in the package which is not referenced by any resource. In some packages it is commone for auxiliary files such as supporting schemas to be included in packages without a corresponding File object so an empty list does not indicate that the file can be removed safely. These files are still included when packaging the content package for interchange.

Finally, if a file referred to by a File object in the manifest is missing an entry is still created in the fileTable. You can walk the keys of the fileTable testing if each file exists to determine if some expected files are missing from the package.

The keys in fileTable are VirtualFilePath instances. To convert a string to an appropriate instance use the FilePath() method.

FilePath(*path)¶: Converts a string into a pyslet.vfs.VirtualFilePath instance suitable for using as a key into the fileTable. The conversion is done using the file system of the content package’s directory, dPath.

SetIgnoreFiles(ignoreFiles)¶

Sets the regular expression used to determine if a file should be ignored.

Some operating systems and utilities create hidden files or other spurious data inside the content package directory. For example, Apple’s OS X creates .DS_Store files and the svn source control utility creates .svn directories. The files shouldn’t generally be included in exported packages as they may confuse the recipient (who may be using a system on which these files and directories are not hidden) and be deemed to violate the specification, not to mention adding unnecessarily to the size of the package and perhaps even leaking information unintentionally.

To help avoid this type of problem the class uses a regular expression to determine if a file should be considered part of the package. When listing directories, the names of the files found are compared against this regular expression and are ignored if they match.

By default, the pattern is set to match all directories and files with names beginning ‘.’ so you will not normally need to call this method.

IgnoreFile(f)¶

Compares a file or directory name against the pattern set by SetIgnoreFiles().

f is a unicode string.

IgnoreFilePath(fPath)¶

Compares a file path against the pattern set by SetIgnoreFiles()

The path is normalised before comparison and any segments consisting of the string ‘..’ are skipped. The method returns True if any of the remaining path components matches the ignore pattern. In other words, if the path describes a file that is is in a directory that should be ignored it will also be ignored.

The path can be relative or absolute. Relative paths are not made absolute prior to comparison so this method is not affected by the current directory, even if the current diretory would itself be ignored.

RebuildFileTable()¶: Rescans the file system and manifest and rebuilds the fileTable.

PackagePath(fPath)¶

Converts an absolute file path into a canonical package-relative path

Returns None if fPath is not inside the package.

ExportToPIF(zPath)¶

Exports the content package, saving the zipped package in zPath

zPath is overwritten by this operation.

In order to make content packages more interoperable this method goes beyond the basic zip specification and ensures that pathnames are always UTF-8 encoded when added to the archive. When creating instances of ContentPackage from an existing archive the reverse transformation is performed. When exchanging PIF files between systems with different native file path encodings, encoding erros may occurr.

GetUniqueFile(suggestedPath)¶

Returns a unique file path suitable for creating a new file in the package.

suggestedPath is used to provide a suggested path for the file. This may be relative (to the root and manifest) or absolute but it must resolve to a file (potentially) in the package. The suggestedPath should either be a VirtualFilePath (of the same type as the content package’s dPath) or a string suitable for conversion to a VirtualFilePath.

When suggestedPath is relative, it is forced to lower-case. This is consistent with the behaviour of normcase on systems that are case insensitive. The trouble with case insensitive file systems is that it may be impossible to unpack a content package created on a case sensitive system and store it on a case insenstive one. By channelling all file storage through this method (and constructing any URIs after the file has been stored) the resulting packages will be more portable.

If suggestedPath already corresponds to a file already in the package, or to a file already referred to in the manifest, then a random string is added to it while preserving the suggested extension in order to make it unique.

The return result is always normalized and returned relative to the package root.

File(resource, href)¶

Returns a new File object attached to resource

href is the URI of the file expressed relative to the resource element in the manifest. Although this is normally the same as the URI expressed relative to the package, a resource may have an xml:base attribute that alters the base for resolving relative URIs.

href may of course be an absolute URI to an external resource. If an absolute URI is given to a local file it must be located inside the package.

Attempting to add a File object representing the manifest file iteself will raise CPFilePathError.

The fileTable is updated automatically by this method.

FileCopy(resource, srcURL)¶

Returns a new File object copied into the package from srcURL, attached to resource.

The file is copied to the same directory as the resource’s entry point or to the main package directory if the resource has no entry point.

The File object is actually created with the File() method.

Note that if srcURL points to a missing file then no file is copied to the package but the associated File is still created. It will point to a missing file.

DeleteFile(href)¶

Removes the file at href from the file system

This method also removes any file references to it from resources in the manifest. href may be given relative to the package root directory. The entry in fileTable is also removed.

CPFileTypeError is raised if the file is not a regular file

CPFilePathError is raised if the file is an IgnoreFile(), the manifest itself or outside of the content package.

CPProtocolError is raised if the indicated file is not in the local file system.

GetPackageName()¶

Returns a human readable name for the package

The name is determined by the method used to create the object. The purpose is to return a name that would be intuitive to the user if it were to be used as the name of the package directory or the stem of a file name when exporting to a PIF file.

Note that the name is returned as a unicode string suitable for showing to the user and may need to be encoded before being used in file path operations.

Close()¶

Closes the content package, removing any temporary files.

This method must be called to clean up any temporary files created when processing the content package. Temporary files are created inside a special temporary directory created using the builtin python tempdir.mkdtemp function. They are not automatically cleaned up when the process exits or when the garbage collector disposes of the object. Use of try:... finally: to clean up the package is recommended. For example:

pkg=ContentPackage("MyPackage.zip")
try:
        # do stuff with the content package here
finally:
        pkg.Close()

class pyslet.imscpv1p2.ManifestDocument(**args)¶

Bases: pyslet.xmlnames20091208.XMLNSDocument

Represents the imsmanifest.xml file itself.

Buildong on pyslet.xmlnames20091208.XMLNSDocument this class is used for parsing and writing manifest files.

The constructor defines three additional prefixes using MakePrefix(), mapping xsi onto XML schema, imsmd onto the IMS LRM namespace and imsqti onto the IMS QTI 2.1 namespace. It also adds a schemaLocation attribute. The elements defined by the pyslet.imsmdv1p2p1 and pyslet.imsqtiv2p1 modules are added to the classMap to ensure that metadata from those schemas are bound to the special classes defined there.

defaultNS = None¶: the default namespace is set to IMSCP_NAMESPACE

GetElementClass(name)¶

Overrides pyslet.xmlnames20091208.XMLNSDocument.GetElementClass() to look up name.

The class contains a mapping from (namespace,element name) pairs to class objects representing the elements. Any element not in the class map returns XMLNSElement() instead.

Constants¶

The following constants are used for setting and interpreting XML documents that conform to the Content Packaging specification

pyslet.imscpv1p2.IMSCP_NAMESPACE = 'http://www.imsglobal.org/xsd/imscp_v1p1'¶

str(object=’‘) -> string

Return a nice string representation of the object. If the argument is a string, the return value is the same object.

pyslet.imscpv1p2.IMSCP_SCHEMALOCATION = 'http://www.imsglobal.org/xsd/imscp_v1p1.xsd'¶

str(object=’‘) -> string

Return a nice string representation of the object. If the argument is a string, the return value is the same object.

pyslet.imscpv1p2.IMSCPX_NAMESPACE = 'http://www.imsglobal.org/xsd/imscp_extensionv1p2'¶

str(object=’‘) -> string

Return a nice string representation of the object. If the argument is a string, the return value is the same object.

Elements¶

class pyslet.imscpv1p2.CPElement(parent, name=None)¶

Bases: pyslet.xmlnames20091208.XMLNSElement

Base class for all elements defined by the Content Packaging specification.

class pyslet.imscpv1p2.Manifest(parent)¶

Bases: pyslet.imscpv1p2.CPElement

Represents the manifest element, the root element of the imsmanifest file.

MetadataClass¶

the default class to represent the metadata element

alias of Metadata

OrganizationsClass¶

the default class to represent the organizations element

alias of Organizations

ResourcesClass¶

the default class to represent the resources element

alias of Resources

ManifestClass¶

the default class to represent child manifest elements

alias of Manifest

Metadata = None¶: the manifest’s metadata element

Organizations = None¶: the organizations element

Resources = None¶: the resources element

Manifest = None¶: a list of child manifest elements

class pyslet.imscpv1p2.Metadata(parent)¶

Bases: pyslet.imscpv1p2.CPElement

Represents the Metadata element.

SchemaClass¶

the default class to represent the schema element

alias of Schema

SchemaVersionClass¶: alias of SchemaVersion

Schema = None¶: the optional schema element

SchemaVersion = None¶: the optional schemaversion element

class pyslet.imscpv1p2.Schema(parent, name=None)¶

Bases: pyslet.imscpv1p2.CPElement

Represents the schema element.

class pyslet.imscpv1p2.SchemaVersion(parent, name=None)¶

Bases: pyslet.imscpv1p2.CPElement

Represents the schemaversion element.

class pyslet.imscpv1p2.Organizations(parent)¶

Bases: pyslet.imscpv1p2.CPElement

Represents the organizations element.

OrganizationClass¶

the default class to represent the organization element

alias of Organization

Organization = None¶: a list of organization elements

class pyslet.imscpv1p2.Organization(parent, name=None)¶

Bases: pyslet.imscpv1p2.CPElement

Represents the organization element.

class pyslet.imscpv1p2.Resources(parent)¶

Bases: pyslet.imscpv1p2.CPElement

Represents the resources element.

ResourceClass¶

the default class to represent the resource element

alias of Resource

Resource = None¶: the list of resources in the manifest

class pyslet.imscpv1p2.Resource(parent)¶

Bases: pyslet.imscpv1p2.CPElement

Represents the resource element.

MetadataClass¶

the default class to represent the metadata element

alias of Metadata

FileClass¶

the default class to represent the file element

alias of File

DependencyClass¶

the default class to represent the dependency element

alias of Dependency

type = None¶: the type of the resource

href = None¶: the href pointing at the resource’s entry point

Metadata = None¶: the resource’s optional metadata element

File = None¶: a list of file elements associated with the resource

Dependency = None¶: a list of dependencies of this resource

GetEntryPoint()¶

Returns the File object that is identified as the entry point.

If there is no entry point, or no File object with a matching href, then None is returned.

SetEntryPoint(f)¶

Set’s the File object that is identified as the resource’s entry point.

The File must already exist and be associated with the resource.

class pyslet.imscpv1p2.File(parent)¶

Bases: pyslet.imscpv1p2.CPElement

Represents the file element.

href = None¶: the href used to locate the file object

PackagePath(cp)¶

Returns the normalized file path relative to the root of the content package, cp.

If the href does not point to a local file then None is returned. Otherwise, this function calculates an absolute path to the file and then calls the content package’s ContentPackage.PackagePath() method.

class pyslet.imscpv1p2.Dependency(parent)¶

Bases: pyslet.imscpv1p2.CPElement

Represents the dependency element.

identifierref = None¶: the identifier of the resource in this dependency

Utilities¶

pyslet.imscpv1p2.PathInPath(childPath, parentPath)¶

Utility function that returns childPath expressed relative to parentPath

This function processes file system paths, not the path components of URI.

Both paths are normalized to remove any redundant navigational segments before any processing, the resulting path will not contain these either.

If childPath is not contained in parentPath then None is returned.

If childPath and parentPath are equal an empty string is returned.

IMS Question and Test Interoperability (version 1.2)¶

The IMS Question and Test Interoperability (QTI) specification version 1.2 was finalized in 2002. After a gap of 1-2 years work started on a major revision, culminating in version 2 of the specification, published first in 2005. For information about the history of the specification see http://en.wikipedia.org/wiki/QTI - official information about the specification is available from the IMS GLC: http://www.imsglobal.org/question/index.html

The purpose of this module is to allow documents in QTI v1 format to be parsed and then transformed into objects representing the QTI v2 data model where more sophisticated processing can be performed. Effectively, the native model of assessment items in Pyslet (and in the PyAssess package it supersedes) is QTI v2 and this module simply provides an import capability for legacy data marked up as QTI v1 items.

Class methods or functions with names beginning MigrateV2 use a common pattern for performing the conversion. Errors and warnings are logged during conversion to a list passed in as the log parameter.

Core Types and Utilities¶

This module contains a number core classes used to support the standard.

Enumerations¶

Where the DTD defines enumerated attribute values we define special enumeration classes. These follow a common pattern in which the values are represented by constant members of the class. The classes are not designed to be instantiated but they do define class methods for decoding and encoding from and to text strings.

class pyslet.qtiv1.core.Action¶

Bases: pyslet.xsdatatypes20041028.Enumeration

Action enumeration (for pyslet.qtiv1.common.SetVar:

(Set | Add | Subtract | Multiply | Divide )  'Set'

Defines constants for the above action types. Usage example:

Action.Add

Note that:

Action.DEFAULT == Action.Set

For more methods see Enumeration

class pyslet.qtiv1.core.Area¶

Bases: pyslet.xsdatatypes20041028.Enumeration

Area enumeration:

(Ellipse | Rectangle | Bounded )  'Ellipse'

Defines constants for the above area types. Usage example:

Area.Rectangle

Note that:

Area.DEFAULT == Area.Ellipse

For more methods see Enumeration

pyslet.qtiv1.core.MigrateV2AreaCoords(area, value, log)¶

Returns a tuple of (shape,coords object) representing the area.

area is one of the Area constants.
value is the string containing the content of the element to which

the area applies.

This conversion is generous because the separators have never been well defined and in some cases content uses a mixture of space and ‘,’.

Note also that the definition of rarea was updated in the 1.2.1 errata and that affects this algorithm. The clarification on the definition of ellipse from radii to diameters might mean that some content ends up with hotspots that are too small but this is safer than hotspots that are too large.

Example:

import pyslet.qtiv1.core as qticore1
import pyslet.qtiv2.core as qticore2
import pyslet.html40_1991224 as html
log=[]
shape,coords=qticore1.MigrateV2AreaCoords(qticore1.Area.Ellipse,"10,10,2,2",log)
# returns (qticore2.Shape.circle, html.Coords([10, 10, 1]) )

Note that Ellipse was deprecated in QTI version 2:

import pyslet.qtiv1.core as qticore1
import pyslet.html40_1991224 as html
log=[]
shape,coords=qticore1.MigrateV2AreaCoords(qticore1.Area.Ellipse,"10,10,2,4",log)
print log
# outputs the following...

['Warning: ellipse shape is deprecated in version 2']

class pyslet.qtiv1.core.FeedbackStyle¶

Bases: pyslet.xsdatatypes20041028.Enumeration

feedbackstyle enumeration:

(Complete | Incremental | Multilevel | Proprietary )  'Complete'

Defines constants for the above feedback style. Usage example:

FeedbackStyle.Decimal

Note that:

FeedbackStyle.DEFAULT == FeedbackStyle.Complete

For more methods see Enumeration

class pyslet.qtiv1.core.FeedbackType¶

Bases: pyslet.xsdatatypes20041028.Enumeration

feedbacktype enumeration:

(Response | Solution | Hint )  'Response'

Defines constants for the above types of feedback. Usage example:

FeedbackType.Decimal

Note that:

FeedbackType.DEFAULT == FeedbackType.Response

For more methods see Enumeration

class pyslet.qtiv1.core.FIBType¶

Bases: pyslet.xsdatatypes20041028.Enumeration

Fill-in-the-blank type enumeration:

(String | Integer | Decimal | Scientific )  'String'

Defines constants for the above fill-in-the-blank types. Usage example:

FIBType.Decimal

Note that:

FIBType.DEFAULT == FIBType.String

For more methods see Enumeration

class pyslet.qtiv1.core.MDOperator¶

Bases: pyslet.xsdatatypes20041028.Enumeration

Metadata operator enumeration for pyslet.qtiv1.sao.SelectionMetadata:

(EQ | NEQ | LT | LTE | GT | GTE )

Defines constants for the above operators. Usage example:

MDOperator.EQ

Lower-case aliases of the constants are provided for compatibility.

For more methods see Enumeration

class pyslet.qtiv1.core.NumType¶

Bases: pyslet.xsdatatypes20041028.Enumeration

numtype enumeration:

(Integer | Decimal | Scientific )  'Integer'

Defines constants for the above numeric types. Usage example:

NumType.Scientific

Note that:

NumType.DEFAULT == NumType.Integer

For more methods see Enumeration

class pyslet.qtiv1.core.Orientation¶

Bases: pyslet.xsdatatypes20041028.Enumeration

Orientation enumeration:

(Horizontal | Vertical )  'Horizontal'

Defines constants for the above orientation types. Usage example:

Orientation.Horizontal

Note that:

Orientation.DEFAULT == Orientation.Horizontal

For more methods see Enumeration

pyslet.qtiv1.core.MigrateV2Orientation(orientation)¶

Maps a v1 orientation onto the corresponding v2 constant.

Raises KeyError if orientation is not one of the Orientation constants.

class pyslet.qtiv1.core.PromptType¶

Bases: pyslet.xsdatatypes20041028.Enumeration

Prompt type enumeration:

(Box | Dashline | Asterisk | Underline )

Defines constants for the above prompt types. Usage example:

PromptType.Dashline

For more methods see Enumeration

class pyslet.qtiv1.core.RCardinality¶

Bases: pyslet.xsdatatypes20041028.Enumeration

rcardinality enumeration:

(Single | Multiple | Ordered )  'Single'

Defines constants for the above cardinality types. Usage example:

RCardinality.Multiple

Note that:

RCardinality.DEFAULT == RCardinality.Single

For more methods see Enumeration

pyslet.qtiv1.core.MigrateV2Cardinality(rCardinality)¶

Maps a v1 cardinality onto the corresponding v2 constant.

Raises KeyError if rCardinality is not one of the RCardinality constants.

pyslet.qtiv1.core.TestOperator = <class pyslet.qtiv1.core.MDOperator at 0x7fe07aac6808>¶: A simple alias of MDOperator defined for pyslet.qtiv1.outcomes.VariableTest

class pyslet.qtiv1.core.VarType¶

Bases: pyslet.xsdatatypes20041028.Enumeration

vartype enumeration:

(Integer | String | Decimal | Scientific | Boolean | Enumerated | Set )  'Integer'

Defines constants for the above view types. Usage example:

VarType.String

Note that:

VarType.DEFAULT == VarType.Integer

For more methods see Enumeration

pyslet.qtiv1.core.MigrateV2VarType(vartype, log)¶

Returns the v2 BaseType representing the v1 vartype.

Note that we reduce both Decimal and Scientific to the float types. In version 2 the BaseType values were chosen to map onto the typical types available in most programming languages. The representation of the number in decimal or exponent form is considered to be part of the interaction or the presentation rather than part of the underlying processing model. Although there clearly are use cases where retaining this distinction would have been an advantage the quality of implementation was likely to be poor and use cases that require a distinction are now implemented in more cumbersome, but probably more interoperable ways.

Note also that the poorly defined Set type in version 1 maps to an identifier in version 2 on the assumption that the cardinality will be upgraded as necessary.

Raises KeyError if vartype is not one of the VarType constants.

class pyslet.qtiv1.core.View¶

Bases: pyslet.xsdatatypes20041028.Enumeration

View enumeration:

(All | Administrator | AdminAuthority | Assessor | Author | Candidate |
InvigilatorProctor | Psychometrician | Scorer | Tutor )  'All'

Defines constants for the above view types. Usage example:

View.Candidate

Note that:

View.DEFAULT == View.All

In addition to the constants defined in the specification we add two aliases which are in common use:

(Invigilator | Proctor)

For more methods see Enumeration

pyslet.qtiv1.core.MigrateV2View(view, log)¶

Returns a list of v2 view values representing the v1 view.

The use of a list as the return type enables mapping of the special value ‘All’, which has no direct equivalent in version 2 other than providing all the defined views.

Raises KeyError if view is not one of the View constants.

This function will log warnings when migrating the following v1 values: Administrator, AdminAuthority, Assessor and Psychometrician

Utility Functions¶

pyslet.qtiv1.core.MakeValidName(name)¶: This function takes a string that is supposed to match the production for Name in XML and forces it to comply by replacing illegal characters with ‘_’. If name starts with a valid name character but not a valid name start character, it is prefixed with ‘_’ too.

pyslet.qtiv1.core.ParseYesNo(src)¶

Returns a True/False parsed from a “Yes” / “No” string.

This function is generous in what it accepts, it will accept mixed case and strips surrounding space. It returns True if the resulting string matches “yes” and False otherwise.

Reverses the transformation defined by FormatYesNo().

pyslet.qtiv1.core.FormatYesNo(value)¶

Returns “Yes” if value is True, “No” otherwise.

Reverses the transformation defined by ParseYesNo().

Constants¶

pyslet.qtiv1.core.QTI_SOURCE = 'QTIv1'¶

str(object=’‘) -> string

Return a nice string representation of the object. If the argument is a string, the return value is the same object.

Exceptions¶

class pyslet.qtiv1.core.QTIError¶

Bases: exceptions.Exception

All errors raised by this module are derived from QTIError.

class pyslet.qtiv1.core.QTIUnimplementedError¶

Bases: pyslet.qtiv1.core.QTIError

A feature of QTI v1 that is not yet implemented by this module.

Abstract Elements¶

class pyslet.qtiv1.core.QTIElement(parent, name=None)¶

Bases: pyslet.xml20081126.structures.Element

Base class for all elements defined by the QTI specification

DeclareMetadata(label, entry, definition=None)¶

Declares a piece of metadata to be associated with the element.

Most QTIElements will be contained by some type of metadata container that collects metadata in a format suitable for easy lookup and export to other metadata formats. The default implementation simply passes the call to the parent element or, if there is no parent, the declaration is ignored.

For more information see MetadataContainer.

class pyslet.qtiv1.core.ObjectMixin¶

Mix-in class for elements that can be inside ObjectBank:

(section | item)+

class pyslet.qtiv1.core.SectionItemMixin¶

Mix-in class for objects that can be in section objects:

(itemref | item | sectionref | section)*

class pyslet.qtiv1.core.SectionMixin¶

Bases: pyslet.qtiv1.core.SectionItemMixin

Mix-in class for objects that can be in assessment objects:

(sectionref | section)+

Common Classes¶

This module contains the common data elements defined in section 3.6 of the binding document. The doc string of each element defined by IMS is introduced with a quote from that document to provide context. For more information see: http://www.imsglobal.org/question/qtiv1p2/imsqti_asi_bindv1p2.html

Content Model¶

Perhaps the biggest change between version 1 and version 2 of the specification was the content model. There were attempts to improve the original model through the introduction of the flow concept in version 1.2 but it wasn’t until the externally defined HTML content model was formally adopted in version 2 that some degree of predictability in rendering became possible.

class pyslet.qtiv1.common.ContentMixin¶

Mixin class for handling all content-containing elements.

This class is used by all elements that behave as content, the default implementation provides an additional contentChildren member that should be used to collect any content-like children.

contentChildren = None¶: the list of content children

ContentMixin(childClass)¶

Creates a new ContentMixin child of this element.

This factory method is called by the parser when it finds an element that is derived from ContentMixin. By default we accept any type of content but derived classes override this behaviour to limit the range of elements to match their content models.

GetContentChildren()¶: Returns an iterable of the content children.

IsInline()¶

True if this element can be inlined, False if it is block level

The default implementation returns True if all contentChildren can be inlined.

InlineChildren()¶: True if all of this element’s contentChildren can all be inlined.

ExtractText()¶

Returns a tuple of (<text string>, <lang>).

Sometimes it is desirable to have a plain text representation of a content object. For example, an element may permit arbitrary content but a synopsis is required to set a metadata value.

Our algorithm for determining the language of the text is to first check if the language has been specified for the context. If it has then that language is used. Otherwise the first language attribute encountered in the content is used as the language. If no language is found then None is returned as the second value.

MigrateV2Content(parent, childType, log, children=None)¶

Migrates this content element to QTIv2.

The resulting QTIv2 content is added to parent.

childType indicates whether the context allows block, inline or a mixture of element content types (flow). It is set to one of the following HTML classes: pyslet.html40_19991224.BlockMixin, pyslet.html40_19991224.InlineMixin or pyslet.html40_19991224.FlowMixin.

The default implementation adds each of children or, if children is None, each of the local contentChildren. The algorithm handles flow elements by creating <p> elements where the context permits. Nested flows are handled by the addition of <br/>.

class pyslet.qtiv1.common.Material(parent)¶

Bases: pyslet.qtiv1.common.QTICommentContainer, pyslet.qtiv1.common.ContentMixin

This is the container for any content that is to be displayed by the question-engine. The supported content types are text (emphasized or not), images, audio, video, application and applet. The content can be internally referenced to avoid the need for duplicate copies. Alternative information can be defined - this is used if the primary content cannot be displayed:

<!ELEMENT material (qticomment? , (mattext | matemtext | matimage |
        mataudio | matvideo | matapplet | matapplication | matref | matbreak
        | mat_extension)+ , altmaterial*)>
<!ATTLIST material
        label CDATA  #IMPLIED
        xml:lang CDATA  #IMPLIED >

class pyslet.qtiv1.common.AltMaterial(parent)¶

Bases: pyslet.qtiv1.common.QTICommentContainer, pyslet.qtiv1.common.ContentMixin

This is the container for alternative content. This content is to be displayed if, for whatever reason, the primary content cannot be rendered. Alternative language implementations of the host <material> element are also supported using this structure:

<!ELEMENT altmaterial (qticomment? ,
        (mattext | matemtext | matimage | mataudio | matvideo |
        matapplet | matapplication | matref | matbreak | mat_extension)+)>
<!ATTLIST altmaterial  xml:lang CDATA  #IMPLIED >

class pyslet.qtiv1.common.MatThingMixin¶

Bases: pyslet.qtiv1.common.ContentMixin

An abstract class used to help identify the mat* elements.

class pyslet.qtiv1.common.PositionMixin¶

Mixin to define the positional attributes

width       CDATA  #IMPLIED
height      CDATA  #IMPLIED
y0          CDATA  #IMPLIED
x0          CDATA  #IMPLIED

class pyslet.qtiv1.common.MatText(parent)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.PositionMixin, pyslet.qtiv1.common.MatThingMixin

The <mattext> element contains any text that is to be displayed to the users

<!ELEMENT mattext (#PCDATA)>
<!ATTLIST mattext
        texttype    CDATA  'text/plain'
        label               CDATA  #IMPLIED
        charset             CDATA  'ascii-us'
        uri                 CDATA  #IMPLIED
        xml:space   (preserve | default )  'default'
        xml:lang    CDATA  #IMPLIED
        entityref   ENTITY  #IMPLIED
        width               CDATA  #IMPLIED
        height              CDATA  #IMPLIED
        y0                  CDATA  #IMPLIED
        x0                  CDATA  #IMPLIED >

inlineWrapper = None¶: an inline html object used to wrap inline elements

class pyslet.qtiv1.common.MatEmText(parent)¶

Bases: pyslet.qtiv1.common.MatText

The <matemtext> element contains any emphasized text that is to be displayed to the users. The type of emphasis is dependent on the question-engine rendering the text:

<!ELEMENT matemtext (#PCDATA)>
<!ATTLIST matemtext
        texttype    CDATA  'text/plain'
        label               CDATA  #IMPLIED
        charset             CDATA  'ascii-us'
        uri                 CDATA  #IMPLIED
        xml:space   (preserve | default )  'default'
        xml:lang    CDATA  #IMPLIED
        entityref   ENTITY  #IMPLIED
        width               CDATA  #IMPLIED
        height              CDATA  #IMPLIED
        y0                  CDATA  #IMPLIED
        x0                  CDATA  #IMPLIED >

class pyslet.qtiv1.common.MatBreak(parent)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.MatThingMixin

The element that is used to insert a break in the flow of the associated material. The nature of the ‘break’ is dependent on the display-rendering engine:

<!ELEMENT matbreak EMPTY>

ExtractText()¶: Returns a simple line break

class pyslet.qtiv1.common.MatImage(parent)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.PositionMixin, pyslet.qtiv1.common.MatThingMixin

The <matimage> element is used to contain image content that is to be displayed to the users:

<!ELEMENT matimage (#PCDATA)>
<!ATTLIST matimage
        imagtype    CDATA  'image/jpeg'
        label               CDATA  #IMPLIED
        height              CDATA  #IMPLIED
        uri                 CDATA  #IMPLIED
        embedded    CDATA  'base64'
        width               CDATA  #IMPLIED
        y0                  CDATA  #IMPLIED
        x0                  CDATA  #IMPLIED
        entityref   ENTITY #IMPLIED >

ExtractText()¶: We cannot extract text from matimage so we return a simple string.

class pyslet.qtiv1.common.MatAudio(parent)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.MatThingMixin

The <mataudio> element is used to contain audio content that is to be displayed to the users:

<!ELEMENT mataudio (#PCDATA)>
<!ATTLIST mataudio
        audiotype   CDATA  'audio/base'
        label               CDATA  #IMPLIED
        uri                 CDATA  #IMPLIED
        embedded    CDATA  'base64'
        entityref   ENTITY  #IMPLIED >

ExtractText()¶: We cannot extract text from mataudio so we return a simple string.

class pyslet.qtiv1.common.MatVideo(parent)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.PositionMixin, pyslet.qtiv1.common.MatThingMixin

The <matvideo> element is used to contain video content that is to be displayed to the users:

<!ELEMENT matvideo (#PCDATA)>
<!ATTLIST matvideo
        videotype   CDATA  'video/avi'
        label               CDATA  #IMPLIED
        uri                 CDATA  #IMPLIED
        width               CDATA  #IMPLIED
        height              CDATA  #IMPLIED
        y0                  CDATA  #IMPLIED
        x0                  CDATA  #IMPLIED
        embedded    CDATA  'base64'
        entityref   ENTITY  #IMPLIED >

ExtractText()¶: We cannot extract text from matvideo so we return a simple string.

class pyslet.qtiv1.common.MatApplet(parent)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.PositionMixin, pyslet.qtiv1.common.MatThingMixin

The <matapplet> element is used to contain applet content that is to be displayed to the users. Parameters that are to be passed to the applet being launched should be enclosed in a CDATA block within the content of the <matapplet> element:

<!ELEMENT matapplet (#PCDATA)>
<!ATTLIST matapplet
        label               CDATA  #IMPLIED
        uri                 CDATA  #IMPLIED
        y0                  CDATA  #IMPLIED
        height              CDATA  #IMPLIED
        width               CDATA  #IMPLIED
        x0                  CDATA  #IMPLIED
        embedded    CDATA  'base64'
        entityref   ENTITY  #IMPLIED >

ExtractText()¶: We cannot extract text from matapplet so we return a simple string.

class pyslet.qtiv1.common.MatApplication(parent)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.MatThingMixin

The <matapplication> element is used to contain application content that is to be displayed to the users. Parameters that are to be passed to the application being launched should be enclosed in a CDATA block within the content of the <matapplication> element:

<!ELEMENT matapplication (#PCDATA)>
<!ATTLIST matapplication
        apptype             CDATA  #IMPLIED
        label               CDATA  #IMPLIED
        uri                 CDATA  #IMPLIED
        embedded    CDATA  'base64'
        entityref   ENTITY  #IMPLIED >

ExtractText()¶: We cannot extract text from matapplication so we return a simple string.

class pyslet.qtiv1.common.MatRef(parent)¶

Bases: pyslet.qtiv1.common.MatThingMixin, pyslet.qtiv1.core.QTIElement

The <matref> element is used to contain a reference to the required material. This material will have had an identifier assigned to enable such a reference to be reconciled when the instance is parsed into the system. <matref> should only be used to reference a material component and not a <material> element (the element <material_ref> should be used for the latter):

<!ELEMENT matref EMPTY>
<!ATTLIST matref linkrefid CDATA  #REQUIRED >

class pyslet.qtiv1.common.MatExtension(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.MatThingMixin

The extension facility to enable proprietary types of material to be included with the corresponding data object:

<!ELEMENT mat_extension ANY>

class pyslet.qtiv1.common.FlowMixin¶

Mix-in class to identify all flow elements:

( flow | flow_mat | flow_label)

class pyslet.qtiv1.common.FlowMatContainer(parent)¶

Bases: pyslet.qtiv1.common.QTICommentContainer, pyslet.qtiv1.common.ContentMixin

Abstract class used to represent objects that contain flow_mat:

<!ELEMENT XXXXXXXXXX (qticomment? , (material+ | flow_mat+))>

class pyslet.qtiv1.common.FlowMat(parent)¶

Bases: pyslet.qtiv1.common.FlowMatContainer, pyslet.qtiv1.common.FlowMixin

This element allows the materials to be displayed to the users to be grouped together using flows. The manner in which these flows are handled is dependent upon the display-engine:

<!ELEMENT flow_mat (qticomment? , (flow_mat | material | material_ref)+)>
<!ATTLIST flow_mat  class CDATA  'Block' >

IsInline()¶: flowmat is always treated as a block if flowClass is specified, otherwise it is treated as a block unless it is an only child.

MigrateV2Content(parent, childType, log)¶

flow typically maps to a div element.

A flow with a specified class always becomes a div.

class pyslet.qtiv1.common.PresentationMaterial(parent)¶

Bases: pyslet.qtiv1.common.FlowMatContainer

This is material that must be presented to set the context of the parent evaluation. This could be at the Section level to contain common question material that is relevant to all of the contained Sections/Items. All the contained material must be presented:

<!ELEMENT presentation_material (qticomment? , flow_mat+)>

Our interpretation is generous here, we also accept <material> by default from FlowMatContainer. This element is one of the newer definitions in QTI v1, after the introduction of <flow>. It excludes <material> because it was assumed there would no legacy content. Adoption of flow was poor and it was replaced with direct inclusion of the html model in version 2 (where content is either inline or block level and flow is a general term to describe both for contexts where either is allowed).

class pyslet.qtiv1.common.Reference(parent)¶

Bases: pyslet.qtiv1.common.QTICommentContainer, pyslet.qtiv1.common.ContentMixin

The container for all of the materials that can be referenced by other structures e.g. feedback material, presentation material etc. The presentation of this material is under the control of the structure that is referencing the material. There is no implied relationship between any of the contained material components:

<!ELEMENT reference (qticomment? , (material | mattext | matemtext | matimage | mataudio |
        matvideo | matapplet | matapplication | matbreak | mat_extension)+)>

ContentMixin(childClass)¶: We override this method to prevent references from being included.

class pyslet.qtiv1.common.MaterialRef(parent)¶

Bases: pyslet.qtiv1.core.QTIElement

The <material_ref> element is used to contain a reference to the required full material block. This material will have had an identifier assigned to enable such a reference to be reconciled when the instance is parsed into the system:

<!ELEMENT material_ref EMPTY>
<!ATTLIST material_ref  linkrefid CDATA  #REQUIRED >

Metadata Model¶

class pyslet.qtiv1.common.MetadataContainerMixin¶

A mix-in class used to hold dictionaries of metadata.

There is a single dictionary maintained to hold all metadata values, each value is a list of tuples of the form (value string, defining element). Values are keyed on the field label or tag name with any leading qmd_ prefix removed.

class pyslet.qtiv1.common.QTIMetadata(parent)¶

Bases: pyslet.qtiv1.core.QTIElement

The container for all of the vocabulary-based QTI-specific meta-data. This structure is available to each of the four core ASI data structures:

<!ELEMENT qtimetadata (vocabulary? , qtimetadatafield+)>

class pyslet.qtiv1.common.Vocabulary(parent)¶

Bases: pyslet.qtiv1.core.QTIElement

The vocabulary to be applied to the associated meta-data fields. The vocabulary is defined either using an external file or it is included as a comma separated list:

<!ELEMENT vocabulary (#PCDATA)>
<!ATTLIST vocabulary
        uri                 CDATA  #IMPLIED
        entityref   ENTITY  #IMPLIED
        vocab_type  CDATA  #IMPLIED >

class pyslet.qtiv1.common.QTIMetadataField(parent)¶

Bases: pyslet.qtiv1.core.QTIElement

The structure responsible for containing each of the QTI-specific meta-data fields:

<!ELEMENT qtimetadatafield  (fieldlabel , fieldentry)>
<!ATTLIST qtimetadatafield  xml:lang CDATA  #IMPLIED >

class pyslet.qtiv1.common.FieldLabel(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement

Used to contain the name of the QTI-specific meta-data field:

<!ELEMENT fieldlabel (#PCDATA)>

class pyslet.qtiv1.common.FieldEntry(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement

Used to contain the actual data entry of the QTI-specific meta-data field named using the associated ‘fieldlabel’ element:

<!ELEMENT fieldentry (#PCDATA)>

Objectives & Rubric¶

class pyslet.qtiv1.common.Objectives(parent)¶

Bases: pyslet.qtiv1.common.FlowMatContainer

The objectives element is used to store the information that describes the educational aims of the Item. These objectives can be defined for each of the different ‘view’ perspectives. This element should not be used to contain information specific to an Item because the question-engine may not make this information available to the Item during the actual test:

<!ELEMENT objectives (qticomment? , (material+ | flow_mat+))>
<!ATTLIST objectives  view  (All | Administrator | AdminAuthority | Assessor | Author |
                        Candidate | InvigilatorProctor | Psychometrician | Scorer |
                        Tutor ) 'All' >

MigrateV2(v2Item, log)¶: Adds rubric representing these objectives to the given item’s body

LRMMigrateObjectives(lom, log)¶: Adds educational description from these objectives.

class pyslet.qtiv1.common.Rubric(parent)¶

Bases: pyslet.qtiv1.common.FlowMatContainer

The rubric element is used to contain contextual information that is important to the element e.g. it could contain standard data values that might or might not be useful for answering the question. Different sets of rubric can be defined for each of the possible ‘views’. The material contained within the rubric must be displayed to the participant:

<!ELEMENT rubric (qticomment? , (material+ | flow_mat+))>
<!ATTLIST rubric  view      (All | Administrator | AdminAuthority | Assessor | Author |
                        Candidate | InvigilatorProctor | Psychometrician | Scorer |
                        Tutor ) 'All' >

Response Processing Model¶

class pyslet.qtiv1.common.DecVar(parent)¶

Bases: pyslet.qtiv1.core.QTIElement

The <decvar> element permits the declaration of the scoring variables

<!ELEMENT decvar (#PCDATA)>
<!ATTLIST decvar  varname CDATA  'SCORE' ::
        vartype             (Integer |  String |  Decimal |  Scientific |  Boolean |
                Enumerated | Set )  'Integer'
        defaultval  CDATA  #IMPLIED
        minvalue    CDATA  #IMPLIED
        maxvalue    CDATA  #IMPLIED
        members     CDATA  #IMPLIED
        cutvalue    CDATA  #IMPLIED >

ContentChanged()¶: The decvar element is supposed to be empty but QTI v1 content is all over the place.

class pyslet.qtiv1.common.InterpretVar(parent)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ContentMixin

The <interpretvar> element is used to provide statistical interpretation information about the associated variables:

<!ELEMENT interpretvar (material | material_ref)>
<!ATTLIST interpretvar
        view        (All | Administrator | AdminAuthority | Assessor | Author | Candidate |
                InvigilatorProctor | Psychometrician | Scorer | Tutor )  'All'
                varname CDATA  'SCORE' >

class pyslet.qtiv1.common.SetVar(parent)¶

Bases: pyslet.qtiv1.core.QTIElement

The <setvar> element is responsible for changing the value of the scoring variable as a result of the associated response processing test:

<!ELEMENT setvar (#PCDATA)>
<!ATTLIST setvar  varname CDATA  'SCORE'
        action     (Set | Add | Subtract | Multiply | Divide )  'Set' >

class pyslet.qtiv1.common.DisplayFeedback(parent)¶

Bases: pyslet.qtiv1.core.QTIElement

The <displayfeedback> element is responsible for assigning an associated feedback to the response processing if the ‘True’ state is created through the associated response processing condition test:

<!ELEMENT displayfeedback (#PCDATA)>
<!ATTLIST displayfeedback
        feedbacktype        (Response | Solution | Hint )  'Response'
        linkrefid           CDATA  #REQUIRED >

class pyslet.qtiv1.common.ConditionVar(parent)¶

Bases: pyslet.qtiv1.core.QTIElement

The conditional test that is to be applied to the user’s response. A wide range of separate and combinatorial test can be applied:

<!ELEMENT conditionvar (not | and | or | unanswered | other | varequal | varlt |
        varlte | vargt | vargte | varsubset | varinside | varsubstring | durequal |
        durlt | durlte | durgt | durgte | var_extension)+>

class pyslet.qtiv1.common.ExtendableExpressionMixin¶: Abstract mixin class to indicate an expression, including var_extension

class pyslet.qtiv1.common.ExpressionMixin¶

Bases: pyslet.qtiv1.common.ExtendableExpressionMixin

Abstract mixin class to indicate an expression excluding var_extension

class pyslet.qtiv1.common.VarThing(parent)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExpressionMixin

Abstract class for var* elements

<!ATTLIST *
        respident   CDATA #REQUIRED
        index               CDATA  #IMPLIED >

class pyslet.qtiv1.common.VarEqual(parent)¶

Bases: pyslet.qtiv1.common.VarThing

The <varequal> element is the test of equivalence. The data for the test is contained within the element’s PCDATA string and must be the same as one of the <response_label> values (this were assigned using the ident attribute):

<!ELEMENT varequal (#PCDATA)>
<!ATTLIST varequal
        case  (Yes | No )  'No'
        respident CDATA  #REQUIRED"
        index CDATA  #IMPLIED >

class pyslet.qtiv1.common.VarInequality(parent)¶

Bases: pyslet.qtiv1.common.VarThing

Abstract class for varlt, varlte, vargt and vargte.

MigrateV2Inequality()¶: Returns the class to use in qtiv2

class pyslet.qtiv1.common.VarLT(parent)¶

Bases: pyslet.qtiv1.common.VarInequality

The <varlt> element is the ‘less than’ test. The data for the test is contained within the element’s PCDATA string and is assumed to be numerical in nature:

<!ELEMENT varlt (#PCDATA)>
<!ATTLIST varlt
        respident   CDATA  #REQUIRED"
        index               CDATA  #IMPLIED >

class pyslet.qtiv1.common.VarLTE(parent)¶

Bases: pyslet.qtiv1.common.VarInequality

The <varlte> element is the ‘less than or equal’ test. The data for the test is contained within the element’s PCDATA string and is assumed to be numerical in nature:

<!ELEMENT varlte (#PCDATA)>
<!ATTLIST varlte
        respident CDATA  #REQUIRED"
        index CDATA  #IMPLIED >

class pyslet.qtiv1.common.VarGT(parent)¶

Bases: pyslet.qtiv1.common.VarInequality

The <vargt> element is the ‘greater than’ test. The data for the test is contained within the element’s PCDATA string and is assumed to be numerical in nature:

<!ELEMENT vargt (#PCDATA)>
<!ATTLIST vargt
        respident CDATA  #REQUIRED"
        index CDATA  #IMPLIED >

class pyslet.qtiv1.common.VarGTE(parent)¶

Bases: pyslet.qtiv1.common.VarInequality

The <vargte> element is the ‘greater than or equal to’ test. The data for the test is contained within the element’s PCDATA string and is assumed to be numerical in nature:

<!ELEMENT vargte (#PCDATA)>
<!ATTLIST vargte
        respident CDATA  #REQUIRED"
        index CDATA  #IMPLIED >

class pyslet.qtiv1.common.VarSubset(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExpressionMixin

The <varsubset> element is the ‘member of a list/set’ test. The data for the test is contained within the element’s PCDATA string. The set is a comma separated list with no enclosing parentheses:

<!ELEMENT varsubset (#PCDATA)>
<!ATTLIST varsubset
        respident CDATA  #REQUIRED"
        setmatch     (Exact | Partial )  'Exact'
        index CDATA  #IMPLIED >

class pyslet.qtiv1.common.VarSubString(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExpressionMixin

The <varsubstring> element is used to determine if a given string is a substring of some other string:

<!ELEMENT varsubstring (#PCDATA)>
<!ATTLIST varsubstring
        index CDATA  #IMPLIED
        respident CDATA  #REQUIRED"
        case  (Yes | No )  'No' >

class pyslet.qtiv1.common.VarInside(parent)¶

Bases: pyslet.qtiv1.common.VarThing

The <varinside> element is the ‘xy-co-ordinate inside an area’ test. The data for the test is contained within the element’s PCDATA string and is a set of co-ordinates that define the area:

<!ELEMENT varinside (#PCDATA)>
<!ATTLIST varinside
        areatype     (Ellipse | Rectangle | Bounded )  #REQUIRED
        respident CDATA  #REQUIRED"
        index CDATA  #IMPLIED >

class pyslet.qtiv1.common.DurEqual(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExpressionMixin

The <durequal> element is the ‘duration equal to’ test i.e. a test on the time taken to make the response:

<!ELEMENT durequal (#PCDATA)>
<!ATTLIST durequal
        index CDATA  #IMPLIED
        respident CDATA  #REQUIRED" >

class pyslet.qtiv1.common.DurLT(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExpressionMixin

The <durlt> element is the ‘duration less than’ test i.e. a test on the time taken to make the response:

<!ELEMENT durlt (#PCDATA)>
<!ATTLIST durlt
        index               CDATA  #IMPLIED
        respident   CDATA  #REQUIRED" >

class pyslet.qtiv1.common.DurLTE(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExpressionMixin

The <durlte> element is the ‘duration less than or equal to’ test i.e. a test on the time taken to make the response:

<!ELEMENT durlte (#PCDATA)>
<!ATTLIST durlte
        index               CDATA  #IMPLIED
        respident   CDATA  #REQUIRED" >

class pyslet.qtiv1.common.DurGT(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExpressionMixin

The <durgt> element is the ‘duration greater than’ test i.e. a test on the time taken to make the response:

<!ELEMENT durgt (#PCDATA)>
<!ATTLIST durgt
        index               CDATA  #IMPLIED
        respident   CDATA  #REQUIRED" >

class pyslet.qtiv1.common.DurGTE(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExpressionMixin

The <durgte> element is the ‘duration greater than or equal to’ test i.e. a test on the time taken to make the response:

<!ELEMENT durgte (#PCDATA)>
<!ATTLIST durgte
        index               CDATA  #IMPLIED
        respident   CDATA  #REQUIRED" >

class pyslet.qtiv1.common.Not(parent)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExpressionMixin

The <not> element inverts the logical test outcome that is required. In the case of the <varequal> element produces a ‘not equals’ test:

<!ELEMENT not (and | or | not | unanswered | other | varequal | varlt | varlte |
        vargt | vargte | varsubset | varinside | varsubstring | durequal | durlt |
        durlte | durgt | durgte)>

class pyslet.qtiv1.common.And(parent)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExpressionMixin

The <and> element is used to create the Boolean ‘AND’ operation between the two or more enclosed tests. The result ‘True’ is returned if all of the tests return a ‘True’ value:

<!ELEMENT and (not | and | or | unanswered | other | varequal | varlt | varlte |
        vargt | vargte | varsubset | varinside | varsubstring | durequal | durlt |
        durlte | durgt | durgte)+>

class pyslet.qtiv1.common.Or(parent)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExpressionMixin

The <or> element is used to create the Boolean ‘OR’ operation between the two or more enclosed tests. The result ‘True’ is returned if one or more of the tests return a ‘True’ value:

<!ELEMENT or (not | and | or | unanswered | other | varequal | varlt | varlte |
        vargt | vargte | varsubset | varinside | varsubstring | durequal | durlt |
        durlte | durgt | durgte)+>

class pyslet.qtiv1.common.Unanswered(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExpressionMixin

The <unanswered> element is the condition to be applied if a response is not received for the Item i.e. it is unanswered:

<!ELEMENT unanswered (#PCDATA)>
<!ATTLIST unanswered  respident CDATA  #REQUIRED" >

class pyslet.qtiv1.common.Other(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExpressionMixin

The <other> element is used to trigger the condition when all of the other tests have not returned a ‘True’ state:

<!ELEMENT other (#PCDATA)>

class pyslet.qtiv1.common.VarExtension(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement, pyslet.qtiv1.common.ExtendableExpressionMixin

This element contains proprietary extensions to be applied to condition tests. This enables vendors to create their own conditional tests to be used on the participant responses:

<!ELEMENT var_extension ANY>

Miscellaneous Classes¶

class pyslet.qtiv1.common.QTICommentContainer(parent)¶

Bases: pyslet.qtiv1.core.QTIElement

Basic element to represent all elements that can contain a comment as their first child:

<!ELEMENT XXXXXXXXXXXX (qticomment? , ....... )>

class pyslet.qtiv1.common.QTIComment(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement

This element contains the comments that are relevant to the host element. The comment is contained as a string:

<!ELEMENT qticomment (#PCDATA)>
<!ATTLIST qticomment  xml:lang CDATA  #IMPLIED >

class pyslet.qtiv1.common.Duration(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement

The duration permitted for the completion of a particular activity. The duration is defined as per the ISO8601 standard. The information is entered as a string:

<!ELEMENT duration (#PCDATA)>

class pyslet.imsqtiv1p2p1.QTIDocument(**args)¶

Bases: pyslet.xml20081126.structures.Document

Class for working with QTI documents.

We turn off the parsing of external general entities to prevent a missing DTD causing the parse to fail. This is a significant limitation as it is possible that some sophisticated users have used general entities to augment the specification or to define boiler-plate code. If this causes problems then you can turn the setting back on again for specific instances of the parser that will be used with that type of data.

XMLParser(entity)¶: Adds some options to the basic XMLParser to improve QTI compatibility.

GetElementClass(name)¶

Returns the class to use to represent an element with the given name.

This method is used by the XML parser. The class object is looked up in the classMap, if no specialized class is found then the general pyslet.xml20081126.Element class is returned.

RegisterMatThing(matThing)¶: Registers a MatThing instance in the dictionary of matThings.

FindMatThing(linkRefID)¶

Returns the mat<thing> element with label matching the linkRefID.

The specification says that material_ref should be used if you want to refer a material object, not matref, however this rule is not universally observed so if we don’t find a basic mat<thing> we will search the material objects too and return a Material instance instead.

RegisterMaterial(material)¶: Registers a Material instance in the dictionary of labelled material objects.

FindMaterial(linkRefID)¶

Returns the material element with label matching linkRefID.

Like FindMatThing() this method will search for instances of MatThingMixin if it can’t find a Material element to match. The specification is supposed to be strict about matching the two types of reference but errors are common, even in the official example set.

MigrateV2(cp)¶

Converts the contents of this document to QTI v2

The output is stored into the content package passed in cp. Errors and warnings generated by the migration process are added as annotations to the resulting resource objects in the content package.

The function returns a list of 4-tuples, one for each object migrated.

Each tuple comprises ( <QTI v2 Document>, <LOM Metadata>, <log>, <Resource> )

QuesTestInterop Elements¶

class pyslet.imsqtiv1p2p1.QuesTestInterop(parent)¶

Bases: pyslet.qtiv1.common.QTICommentContainer

The <questestinterop> element is the outermost container for the QTI contents i.e. the container of the Assessment(s), Section(s) and Item(s):

<!ELEMENT questestinterop (qticomment? , (objectbank | assessment | (section | item)+))>

MigrateV2()¶

Converts this element to QTI v2

Returns a list of tuples of the form: ( <QTIv2 Document>, <Metadata>, <List of Log Messages> ).

One tuple is returned for each of the objects found. In QTIv2 there is no equivalent of QuesTestInterop. The baseURI of each document is set from the baseURI of the QuesTestInterop element using the object identifier to derive a file name.

Object Bank Elements¶

class pyslet.imsqtiv1p2p1.ObjectBank(parent)¶

Bases: pyslet.qtiv1.common.MetadataContainerMixin, pyslet.qtiv1.common.QTICommentContainer

This is the container for the Section(s) and/or Item(s) that are to be grouped as an object-bank. The object-bank is assigned its own unique identifier and can have the full set of QTI-specific meta-data:

<!ELEMENT objectbank (qticomment? , qtimetadata* , (section | item)+)>
<!ATTLIST objectbank  ident CDATA  #REQUIRED >

Assessment Elements¶

class pyslet.imsqtiv1p2p1.Assessment(parent)¶

Bases: pyslet.qtiv1.common.QTICommentContainer

The Assessment data structure is used to contain the exchange of test data structures. It will always contain at least one Section and may contain meta-data, objectives, rubric control switches, assessment-level processing, feedback and selection and sequencing information for sections:

<!ELEMENT assessment (qticomment? ,
        duration? ,
        qtimetadata* ,
        objectives* ,
        assessmentcontrol* ,
        rubric* ,
        presentation_material? ,
        outcomes_processing* ,
        assessproc_extension? ,
        assessfeedback* ,
        selection_ordering? ,
        reference? ,
        (sectionref | section)+
        )>
<!ATTLIST assessment  ident CDATA  #REQUIRED
                                           %I_Title;
                                           xml:lang CDATA  #IMPLIED >

MigrateV2(output)¶

Converts this assessment to QTI v2

For details, see QuesTestInterop.MigrateV2.

class pyslet.imsqtiv1p2p1.AssessmentControl(parent)¶

Bases: pyslet.qtiv1.common.QTICommentContainer

The control switches that are used to enable or disable the display of hints, solutions and feedback within the Assessment:

<!ELEMENT assessmentcontrol (qticomment?)>
<!ATTLIST assessmentcontrol
        hintswitch  (Yes | No )  'Yes'
        solutionswitch  (Yes | No )  'Yes'
        view        (All | Administrator | AdminAuthority | Assessor | Author |
                        Candidate | InvigilatorProctor | Psychometrician | Scorer |
                        Tutor ) 'All'
        feedbackswitch  (Yes | No )  'Yes' >

class pyslet.imsqtiv1p2p1.AssessProcExtension(parent, name=None)¶

Bases: pyslet.qtiv1.core.QTIElement

This is used to contain proprietary alternative Assessment-level processing functionality:

<!ELEMENT assessproc_extension ANY>

class pyslet.imsqtiv1p2p1.AssessFeedback(parent)¶

Bases: pyslet.qtiv1.common.QTICommentContainer, pyslet.qtiv1.common.ContentMixin

The container for the Assessment-level feedback that is to be presented as a result of Assessment-level processing of the user responses:

<!ELEMENT assessfeedback (qticomment? , (material+ | flow_mat+))>
<!ATTLIST assessfeedback
        view        (All | Administrator | AdminAuthority | Assessor | Author |
                        Candidate | InvigilatorProctor | Psychometrician | Scorer |
                        Tutor ) 'All'
        ident CDATA  #REQUIRED
        title CDATA  #IMPLIED >

IMS Question and Test Interoperability (version 2.1)¶

The IMS Question and Test Interoperability specification version 2.1 has yet to be finalized and is currently only available as a “Public Draft Specification” from the IMS GLC website: http://www.imsglobal.org/question/index.html

Version 2.1 is an extension of the pre-existing version 2.0 which was finalized in 2005. For more information on the history of the specification see http://en.wikipedia.org/wiki/QTI

This module implements version 2.1 of the specification in anticipation of the finalization of the specification by the consortium.

Items¶

class pyslet.qtiv2.items.AssessmentItem(parent)¶

Bases: pyslet.qtiv2.core.QTIElement, pyslet.qtiv2.core.DeclarationContainer

An assessment item encompasses the information that is presented to a candidate and information about how to score the item:

<xsd:attributeGroup name="assessmentItem.AttrGroup">
        <xsd:attribute name="identifier" type="string.Type" use="required"/>
        <xsd:attribute name="title" type="string.Type" use="required"/>
        <xsd:attribute name="label" type="string256.Type" use="optional"/>
        <xsd:attribute ref="xml:lang"/>
        <xsd:attribute name="adaptive" type="boolean.Type" use="required"/>
        <xsd:attribute name="timeDependent" type="boolean.Type" use="required"/>
        <xsd:attribute name="toolName" type="string256.Type" use="optional"/>
        <xsd:attribute name="toolVersion" type="string256.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="assessmentItem.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="responseDeclaration" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="outcomeDeclaration" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="templateDeclaration" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="templateProcessing" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="stylesheet" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="itemBody" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="responseProcessing" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="modalFeedback" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

SortDeclarations()¶: Sort each of the variable declaration lists so that they are in identifier order. This is not essential but it does help ensure that output is predictable. This method is called automatically when reading items from XML files.

RenderHTML(itemState, htmlParent=None)¶

Renders this item in html, adding nodes to htmlParent. The state of the item (e.g., the values of any controls and template variables), is taken from itemState, a variables.ItemSessionState instance.

The result is the top-level div containing the item added to the htmlParent. If htmlParent is None then a parentless div is created. If the item has no itemBody then an empty Div is returned.

AddToContentPackage(cp, lom, dName=None)¶: Adds a resource and associated files to the content package.

Tests¶

class pyslet.qtiv2.tests.AssessmentTest(parent)¶

Bases: pyslet.qtiv2.core.QTIElement, pyslet.qtiv2.core.DeclarationContainer

A test is a group of assessmentItems with an associated set of rules that determine which of the items the candidate sees, in what order, and in what way the candidate interacts with them. The rules describe the valid paths through the test, when responses are submitted for response processing and when (if at all) feedback is to be given:

<xsd:attributeGroup name="assessmentTest.AttrGroup">
        <xsd:attribute name="identifier" type="string.Type" use="required"/>
        <xsd:attribute name="title" type="string.Type" use="required"/>
        <xsd:attribute name="toolName" type="string256.Type" use="optional"/>
        <xsd:attribute name="toolVersion" type="string256.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="assessmentTest.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="outcomeDeclaration" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="timeLimits" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="testPart" minOccurs="1" maxOccurs="unbounded"/>
                <xsd:element ref="outcomeProcessing" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="testFeedback" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

SortDeclarations()¶: Sort the outcome declarations so that they are in identifier order. This is not essential but it does help ensure that output is predictable. This method is called automatically when reading items from XML files.

RegisterPart(part)¶: Registers a testPart, asssessmentSection or assessmentItemRef in parts.

GetPart(identifier)¶: Returns the testPart, assessmentSection or assessmentItemRef with the given identifier.

Navigation and Submission¶

class pyslet.qtiv2.tests.NavigationMode¶

Bases: pyslet.xsdatatypes20041028.Enumeration

The navigation mode determines the general paths that the candidate may take. A testPart in linear mode restricts the candidate to attempt each item in turn. Once the candidate moves on they are not permitted to return. A testPart in nonlinear mode removes this restriction - the candidate is free to navigate to any item in the test at any time:

<xsd:simpleType name="navigationMode.Type">
        <xsd:restriction base="xsd:NMTOKEN">
                <xsd:enumeration value="linear"/>
                <xsd:enumeration value="nonlinear"/>
        </xsd:restriction>
</xsd:simpleType>

Defines constants for the above modes. Usage example:

NavigationMode.linear

Note that:

NavigationMode.DEFAULT == None

For more methods see Enumeration

class pyslet.qtiv2.tests.SubmissionMode¶

Bases: pyslet.xsdatatypes20041028.Enumeration

The submission mode determines when the candidate’s responses are submitted for response processing. A testPart in individual mode requires the candidate to submit their responses on an item-by-item basis. In simultaneous mode the candidate’s responses are all submitted together at the end of the testPart:

<xsd:simpleType name="submissionMode.Type">
        <xsd:restriction base="xsd:NMTOKEN">
                <xsd:enumeration value="individual"/>
                <xsd:enumeration value="simultaneous"/>
        </xsd:restriction>
</xsd:simpleType>

Defines constants for the above modes. Usage example:

SubmissionMode.individual

Note that:

SubmissionMode.DEFAULT == None

For more methods see Enumeration

class pyslet.qtiv2.tests.TestPart(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

Each test is divided into one or more parts which may in turn be divided into sections, sub-sections, and so on:

<xsd:attributeGroup name="testPart.AttrGroup">
        <xsd:attribute name="identifier" type="identifier.Type" use="required"/>
        <xsd:attribute name="navigationMode" type="navigationMode.Type" use="required"/>
        <xsd:attribute name="submissionMode" type="submissionMode.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="testPart.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="preCondition" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="branchRule" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="itemSessionControl" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="timeLimits" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="assessmentSection" minOccurs="1" maxOccurs="unbounded"/>
                <xsd:element ref="testFeedback" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

CheckPreConditions(state)¶: Returns True if this testPart’s pre-conditions are satisfied or if there are no pre-conditions in effect.

GetBranchTarget(state)¶: Returns the identifier of the testPart to branch to, or the pre-defined EXIT_TEST identifier. If there is no branch rule in effect then None is returned. state is a variables.TestSessionState instance used to evaluate the branch rule expressions.

Test Structure¶

class pyslet.qtiv2.tests.Selection(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

The selection class specifies the rules used to select the child elements of a section for each test session:

<xsd:attributeGroup name="selection.AttrGroup">
        <xsd:attribute name="select" type="integer.Type" use="required"/>
        <xsd:attribute name="withReplacement" type="boolean.Type" use="optional"/>
        <xsd:anyAttribute namespace="##other"/>
</xsd:attributeGroup>

<xsd:group name="selection.ContentGroup">
        <xsd:sequence>
        <xsd:any namespace="##any" minOccurs="0" maxOccurs="unbounded" processContents="skip"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.tests.Ordering(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

The ordering class specifies the rule used to arrange the child elements of a section following selection. If no ordering rule is given we assume that the elements are to be ordered in the order in which they are defined:

<xsd:attributeGroup name="ordering.AttrGroup">
        <xsd:attribute name="shuffle" type="boolean.Type" use="required"/>
        <xsd:anyAttribute namespace="##other"/>
</xsd:attributeGroup>

<xsd:group name="ordering.ContentGroup">
        <xsd:sequence>
        <xsd:any namespace="##any" minOccurs="0" maxOccurs="unbounded" processContents="skip"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.tests.SectionPart(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

Sections group together individual item references and/or sub-sections. A number of common parameters are shared by both types of child element:

<xsd:attributeGroup name="sectionPart.AttrGroup">
        <xsd:attribute name="identifier" type="identifier.Type" use="required"/>
        <xsd:attribute name="required" type="boolean.Type" use="optional"/>
        <xsd:attribute name="fixed" type="boolean.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="sectionPart.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="preCondition" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="branchRule" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="itemSessionControl" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="timeLimits" minOccurs="0" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

CheckPreConditions(state)¶: Returns True if this item or section’s pre-conditions are satisfied or if there are no pre-conditions in effect.

GetBranchTarget(state)¶: Returns the identifier of the next item or section to branch to, or one of the pre-defined EXIT_* identifiers. If there is no branch rule in effect then None is returned. state is a variables.TestSessionState instance used to evaluate the branch rule expressions.

class pyslet.qtiv2.tests.AssessmentSection(parent)¶

Bases: pyslet.qtiv2.tests.SectionPart

Represents assessmentSection element

<xsd:attributeGroup name="assessmentSection.AttrGroup">
        <xsd:attributeGroup ref="sectionPart.AttrGroup"/>
        <xsd:attribute name="title" type="string.Type" use="required"/>
        <xsd:attribute name="visible" type="boolean.Type" use="required"/>
        <xsd:attribute name="keepTogether" type="boolean.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="assessmentSection.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="sectionPart.ContentGroup"/>
                <xsd:element ref="selection" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="ordering" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="rubricBlock" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:group ref="sectionPart.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.tests.AssessmentItemRef(parent)¶

Bases: pyslet.qtiv2.tests.SectionPart

Items are incorporated into the test by reference and not by direct aggregation:

<xsd:attributeGroup name="assessmentItemRef.AttrGroup">
        <xsd:attributeGroup ref="sectionPart.AttrGroup"/>
        <xsd:attribute name="href" type="uri.Type" use="required"/>
        <xsd:attribute name="category" use="optional">
                <xsd:simpleType>
                        <xsd:list itemType="identifier.Type"/>
                </xsd:simpleType>
        </xsd:attribute>
</xsd:attributeGroup>

<xsd:group name="assessmentItemRef.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="sectionPart.ContentGroup"/>
                <xsd:element ref="variableMapping" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="weight" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="templateDefault" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

GetItem()¶: Returns the AssessmentItem referred to by this reference.

Content Model¶

class pyslet.qtiv2.content.ItemBody(parent)¶

Bases: pyslet.qtiv2.content.BodyElement

The item body contains the text, graphics, media objects, and interactions that describe the item’s content and information about how it is structured:

<xsd:attributeGroup name="itemBody.AttrGroup">
        <xsd:attributeGroup ref="bodyElement.AttrGroup"/>
</xsd:attributeGroup>

<xsd:group name="itemBody.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="block.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

RenderHTML(parent, profile, itemState)¶: Overrides BodyElement.RenderHTML(), the result is always a Div with class set to “itemBody”. Unlike other such method parent may by None, in which case a new parentless Div is created.

class pyslet.qtiv2.content.BodyElement(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

The root class of all content objects in the item content model is the bodyElement. It defines a number of attributes that are common to all elements of the content model:

<xsd:attributeGroup name="bodyElement.AttrGroup">
        <xsd:attribute name="id" type="identifier.Type" use="optional"/>
        <xsd:attribute name="class" use="optional">
                <xsd:simpleType>
                        <xsd:list itemType="styleclass.Type"/>
                </xsd:simpleType>
        </xsd:attribute>
        <xsd:attribute ref="xml:lang"/>
        <xsd:attribute name="label" type="string256.Type" use="optional"/>
</xsd:attributeGroup>

RenderHTML(parent, profile, itemState)¶

Renders this element in html form, adding nodes to parent. This method effectively overrides html40_19991224.XHTMLElement.RenderHTML enabling QTI and XHTML elements to be mixed freely.

The state of the item (e.g., the values of any controls), is taken from itemState, a variables.ItemSessionState instance.

RenderHTMLChildren(parent, profile, itemState)¶: Renders this element’s children to an external document represented by the parent node

Basic Classes¶

Many of the basic classes are drawn directly from the html40_19991224 module, as a result there are slight modifications to some of the abstract base class definitions. See InlineMixin, BlockMixin and FlowMixin; there is no class corresponding to the objectFlow concept (see Object for more information). There is also no representation of the static base classes used to exclude interactions or any of the other basic container classes, these are all handled directly by their equivalent html abstractions.

class pyslet.qtiv2.content.FlowContainerMixin¶

Mixin class used for objects that can contain flows.

PrettyPrint()¶

Deteremins if this flow-container-like object should be pretty printed.

This is similar to the algorithm we use in HTML flow containers, suppressing pretty printing if we have inline elements (ignoring non-trivial data). This could be refactored in future.

XHMTL Elements¶

Again, these classes are defined in the accompanying html40_19991224 module, however we do define some profiles here to make it easier to constraint general HTML content to the profile defined here.

pyslet.qtiv2.content.TextElements = {'em': ('id', 'class', 'label'), 'pre': ('id', 'class', 'label'), 'code': ('id', 'class', 'label'), 'h2': ('id', 'class', 'label'), 'h3': ('id', 'class', 'label'), 'h1': ('id', 'class', 'label'), 'h6': ('id', 'class', 'label'), 'kbd': ('id', 'class', 'label'), 'h5': ('id', 'class', 'label'), 'span': ('id', 'class', 'label'), 'dfn': ('id', 'class', 'label'), 'var': ('id', 'class', 'label'), 'samp': ('id', 'class', 'label'), 'cite': ('id', 'class', 'label'), 'blockquote': ('id', 'class', 'label'), 'acronym': ('id', 'class', 'label'), 'h4': ('id', 'class', 'label'), 'br': ('id', 'class', 'label'), 'address': ('id', 'class', 'label'), 'strong': ('id', 'class', 'label'), 'q': ('id', 'class', 'label'), 'p': ('id', 'class', 'label'), 'div': ('id', 'class', 'label'), 'abbr': ('id', 'class', 'label')}¶: Basic text formatting elements

pyslet.qtiv2.content.ListElements = {'dl': ('id', 'class', 'label'), 'ol': ('id', 'class', 'label'), 'dd': ('id', 'class', 'label'), 'li': ('id', 'class', 'label'), 'ul': ('id', 'class', 'label'), 'dt': ('id', 'class', 'label')}¶: Elements required for lists

pyslet.qtiv2.content.ObjectElements = {'object': ('id', 'class', 'label', 'data', 'type', 'width', 'height'), 'param': ('id', 'class', 'label', 'name', 'value', 'valuetype', 'type')}¶: The object element

pyslet.qtiv2.content.PresentationElements = {'caption': ('id', 'class', 'label'), 'tfoot': ('id', 'class', 'label'), 'th': ('id', 'class', 'label', 'headers', 'scope', 'abbr', 'axis', 'rowspan', 'colspan'), 'colgroup': ('id', 'class', 'label', 'span'), 'table': ('id', 'class', 'label', 'summary'), 'td': ('id', 'class', 'label', 'headers', 'scope', 'abbr', 'axis', 'rowspan', 'colspan'), 'thead': ('id', 'class', 'label'), 'tr': ('id', 'class', 'label'), 'col': ('id', 'class', 'label', 'span'), 'tbody': ('id', 'class', 'label')}¶: Tables

pyslet.qtiv2.content.ImageElement = {'img': ('id', 'class', 'label', 'src', 'alt', 'longdesc', 'height', 'width')}¶: Images

pyslet.qtiv2.content.HypertextElement = {'a': ('id', 'class', 'label', 'href', 'type')}¶: Hyperlinks

pyslet.qtiv2.content.HTMLProfile = {'em': ('id', 'class', 'label'), 'pre': ('id', 'class', 'label'), 'code': ('id', 'class', 'label'), 'h2': ('id', 'class', 'label'), 'h3': ('id', 'class', 'label'), 'h1': ('id', 'class', 'label'), 'h6': ('id', 'class', 'label'), 'kbd': ('id', 'class', 'label'), 'h5': ('id', 'class', 'label'), 'table': ('id', 'class', 'label', 'summary'), 'span': ('id', 'class', 'label'), 'img': ('id', 'class', 'label', 'src', 'alt', 'longdesc', 'height', 'width'), 'caption': ('id', 'class', 'label'), 'tr': ('id', 'class', 'label'), 'tbody': ('id', 'class', 'label'), 'param': ('id', 'class', 'label', 'name', 'value', 'valuetype', 'type'), 'li': ('id', 'class', 'label'), 'dfn': ('id', 'class', 'label'), 'tfoot': ('id', 'class', 'label'), 'th': ('id', 'class', 'label', 'headers', 'scope', 'abbr', 'axis', 'rowspan', 'colspan'), 'var': ('id', 'class', 'label'), 'td': ('id', 'class', 'label', 'headers', 'scope', 'abbr', 'axis', 'rowspan', 'colspan'), 'samp': ('id', 'class', 'label'), 'cite': ('id', 'class', 'label'), 'thead': ('id', 'class', 'label'), 'dl': ('id', 'class', 'label'), 'blockquote': ('id', 'class', 'label'), 'acronym': ('id', 'class', 'label'), 'dd': ('id', 'class', 'label'), 'object': ('id', 'class', 'label', 'data', 'type', 'width', 'height'), 'h4': ('id', 'class', 'label'), 'br': ('id', 'class', 'label'), 'address': ('id', 'class', 'label'), 'dt': ('id', 'class', 'label'), 'strong': ('id', 'class', 'label'), 'abbr': ('id', 'class', 'label'), 'a': ('id', 'class', 'label', 'href', 'type'), 'ol': ('id', 'class', 'label'), 'colgroup': ('id', 'class', 'label', 'span'), 'q': ('id', 'class', 'label'), 'p': ('id', 'class', 'label'), 'div': ('id', 'class', 'label'), 'col': ('id', 'class', 'label', 'span'), 'ul': ('id', 'class', 'label')}¶: The full HTML profile defined by QTI

Interactions¶

class pyslet.qtiv2.interactions.Interaction(parent)¶

Bases: pyslet.qtiv2.content.BodyElement

Interactions allow the candidate to interact with the item. Through an interaction, the candidate selects or constructs a response:

<xsd:attributeGroup name="interaction.AttrGroup">
        <xsd:attributeGroup ref="bodyElement.AttrGroup"/>
        <xsd:attribute name="responseIdentifier" type="identifier.Type" use="required"/>
</xsd:attributeGroup>

class pyslet.qtiv2.interactions.InlineInteraction(parent)¶

Bases: pyslet.html40_19991224.InlineMixin, pyslet.qtiv2.interactions.Interaction

Abstract class for interactions that appear inline.

class pyslet.qtiv2.interactions.BlockInteraction(parent)¶

Bases: pyslet.html40_19991224.BlockMixin, pyslet.qtiv2.interactions.Interaction

An interaction that behaves like a block in the content model. Most interactions are of this type:

<xsd:group name="blockInteraction.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="prompt" minOccurs="0" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.Prompt(parent)¶

Bases: pyslet.qtiv2.content.BodyElement

The prompt used in block interactions

<xsd:group name="prompt.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="inlineStatic.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.Choice(parent)¶

Bases: pyslet.qtiv2.content.BodyElement

Many of the interactions involve choosing one or more predefined choices

<xsd:attributeGroup name="choice.AttrGroup">
        <xsd:attributeGroup ref="bodyElement.AttrGroup"/>
        <xsd:attribute name="identifier" type="identifier.Type" use="required"/>
        <xsd:attribute name="fixed" type="boolean.Type" use="optional"/>
        <xsd:attribute name="templateIdentifier" type="identifier.Type" use="optional"/>
        <xsd:attribute name="showHide" type="showHide.Type" use="optional"/>
</xsd:attributeGroup>

class pyslet.qtiv2.interactions.AssociableChoice(parent)¶

Bases: pyslet.qtiv2.interactions.Choice

Other interactions involve associating pairs of predefined choices

<xsd:attributeGroup name="associableChoice.AttrGroup">
        <xsd:attributeGroup ref="choice.AttrGroup"/>
        <xsd:attribute name="matchGroup" use="optional">
                <xsd:simpleType>
                        <xsd:list itemType="identifier.Type"/>
                </xsd:simpleType>
        </xsd:attribute>
</xsd:attributeGroup>

Simple Interactions¶

class pyslet.qtiv2.interactions.ChoiceInteraction(parent)¶

Bases: pyslet.qtiv2.interactions.BlockInteraction

The choice interaction presents a set of choices to the candidate. The candidate’s task is to select one or more of the choices, up to a maximum of maxChoices:

<xsd:attributeGroup name="choiceInteraction.AttrGroup">
        <xsd:attributeGroup ref="blockInteraction.AttrGroup"/>
        <xsd:attribute name="shuffle" type="boolean.Type" use="required"/>
        <xsd:attribute name="maxChoices" type="integer.Type" use="required"/>
        <xsd:attribute name="minChoices" type="integer.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="choiceInteraction.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="blockInteraction.ContentGroup"/>
                <xsd:element ref="simpleChoice" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.OrderInteraction(parent)¶

Bases: pyslet.qtiv2.interactions.BlockInteraction

In an order interaction the candidate’s task is to reorder the choices, the order in which the choices are displayed initially is significant:

<xsd:attributeGroup name="orderInteraction.AttrGroup">
        <xsd:attributeGroup ref="blockInteraction.AttrGroup"/>
        <xsd:attribute name="shuffle" type="boolean.Type" use="required"/>
        <xsd:attribute name="minChoices" type="integer.Type" use="optional"/>
        <xsd:attribute name="maxChoices" type="integer.Type" use="optional"/>
        <xsd:attribute name="orientation" type="orientation.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="orderInteraction.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="blockInteraction.ContentGroup"/>
                <xsd:element ref="simpleChoice" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.SimpleChoice(parent)¶

Bases: pyslet.qtiv2.content.FlowContainerMixin, pyslet.qtiv2.interactions.Choice

A SimpleChoice is a choice that contains flow objects; it must not contain any nested interactions:

<xsd:group name="simpleChoice.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="flowStatic.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.AssociateInteraction(parent)¶

Bases: pyslet.qtiv2.interactions.BlockInteraction

An associate interaction is a blockInteraction that presents candidates with a number of choices and allows them to create associations between them:

<xsd:attributeGroup name="associateInteraction.AttrGroup">
        <xsd:attributeGroup ref="blockInteraction.AttrGroup"/>
        <xsd:attribute name="shuffle" type="boolean.Type" use="required"/>
        <xsd:attribute name="maxAssociations" type="integer.Type" use="required"/>
        <xsd:attribute name="minAssociations" type="integer.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="associateInteraction.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="blockInteraction.ContentGroup"/>
                <xsd:element ref="simpleAssociableChoice" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.MatchInteraction(parent)¶

Bases: pyslet.qtiv2.interactions.BlockInteraction

A match interaction is a blockInteraction that presents candidates with two sets of choices and allows them to create associates between pairs of choices in the two sets, but not between pairs of choices in the same set:

<xsd:attributeGroup name="matchInteraction.AttrGroup">
        <xsd:attributeGroup ref="blockInteraction.AttrGroup"/>
        <xsd:attribute name="shuffle" type="boolean.Type" use="required"/>
        <xsd:attribute name="maxAssociations" type="integer.Type" use="required"/>
        <xsd:attribute name="minAssociations" type="integer.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="matchInteraction.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="blockInteraction.ContentGroup"/>
                <xsd:element ref="simpleMatchSet" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.SimpleAssociableChoice(parent)¶

Bases: pyslet.qtiv2.content.FlowContainerMixin, pyslet.qtiv2.interactions.AssociableChoice

associableChoice is a choice that contains flowStatic objects, it must not contain nested interactions:

<xsd:attributeGroup name="simpleAssociableChoice.AttrGroup">
        <xsd:attributeGroup ref="associableChoice.AttrGroup"/>
        <xsd:attribute name="matchMax" type="integer.Type" use="required"/>
        <xsd:attribute name="matchMin" type="integer.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="simpleAssociableChoice.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="flowStatic.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.SimpleMatchSet(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

Contains an ordered set of choices for the set

<xsd:group name="simpleMatchSet.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="simpleAssociableChoice" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.GapMatchInteraction(parent)¶

Bases: pyslet.qtiv2.interactions.BlockInteraction

A gap match interaction is a blockInteraction that contains a number gaps that the candidate can fill from an associated set of choices:

<xsd:attributeGroup name="gapMatchInteraction.AttrGroup">
        <xsd:attributeGroup ref="blockInteraction.AttrGroup"/>
        <xsd:attribute name="shuffle" type="boolean.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="gapMatchInteraction.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="blockInteraction.ContentGroup"/>
                <xsd:group ref="gapChoice.ElementGroup" minOccurs="1" maxOccurs="unbounded"/>
                <xsd:group ref="blockStatic.ElementGroup" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.Gap(parent)¶

Bases: pyslet.html40_19991224.InlineMixin, pyslet.qtiv2.interactions.AssociableChoice

A gap is an inline element that must only appear within a gapMatchInteraction

<xsd:attributeGroup name="gap.AttrGroup">
        <xsd:attributeGroup ref="associableChoice.AttrGroup"/>
        <xsd:attribute name="required" type="boolean.Type" use="optional"/>
</xsd:attributeGroup>

class pyslet.qtiv2.interactions.GapChoice(parent)¶

Bases: pyslet.qtiv2.interactions.AssociableChoice

The choices that are used to fill the gaps in a gapMatchInteraction are either simple runs of text or single image objects, both derived from gapChoice:

<xsd:attributeGroup name="gapChoice.AttrGroup">
        <xsd:attributeGroup ref="associableChoice.AttrGroup"/>
        <xsd:attribute name="matchMax" type="integer.Type" use="required"/>
        <xsd:attribute name="matchMin" type="integer.Type" use="optional"/>
</xsd:attributeGroup>

class pyslet.qtiv2.interactions.GapText(parent)¶

Bases: pyslet.qtiv2.interactions.GapChoice

A simple run of text to be inserted into a gap by the user, may be subject to variable value substitution with printedVariable:

<xsd:group name="gapText.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="printedVariable" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.GapImg(parent)¶

Bases: pyslet.qtiv2.interactions.GapChoice

A gap image contains a single image object to be inserted into a gap by the candidate:

<xsd:attributeGroup name="gapImg.AttrGroup">
        <xsd:attributeGroup ref="gapChoice.AttrGroup"/>
        <xsd:attribute name="objectLabel" type="string.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="gapImg.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="object" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

Text-based Interactions¶

class pyslet.qtiv2.interactions.InlineChoiceInteraction(parent)¶

Bases: pyslet.qtiv2.interactions.InlineInteraction

An inline choice is an inlineInteraction that presents the user with a set of choices, each of which is a simple piece of text:

<xsd:attributeGroup name="inlineChoiceInteraction.AttrGroup">
        <xsd:attributeGroup ref="inlineInteraction.AttrGroup"/>
        <xsd:attribute name="shuffle" type="boolean.Type" use="required"/>
        <xsd:attribute name="required" type="boolean.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="inlineChoiceInteraction.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="inlineChoice" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.InlineChoice(parent)¶

Bases: pyslet.qtiv2.interactions.Choice

A simple run of text to be displayed to the user, may be subject to variable value substitution with printedVariable:

<xsd:group name="inlineChoice.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="printedVariable" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.StringInteractionMixin¶

Abstract mix-in class for interactions based on free-text input. String interactions can be bound to numeric response variables, instead of strings, if desired:

<xsd:attributeGroup name="stringInteraction.AttrGroup">
        <xsd:attribute name="base" type="integer.Type" use="optional"/>
        <xsd:attribute name="stringIdentifier" type="identifier.Type" use="optional"/>
        <xsd:attribute name="expectedLength" type="integer.Type" use="optional"/>
        <xsd:attribute name="patternMask" type="string.Type" use="optional"/>
        <xsd:attribute name="placeholderText" type="string.Type" use="optional"/>
</xsd:attributeGroup>

class pyslet.qtiv2.interactions.TextEntryInteraction(parent)¶

Bases: pyslet.qtiv2.interactions.StringInteractionMixin, pyslet.qtiv2.interactions.InlineInteraction

A textEntry interaction is an inlineInteraction that obtains a simple piece of text from the candidate.

class pyslet.qtiv2.interactions.TextFormat¶

Bases: pyslet.xsdatatypes20041028.Enumeration

Used to control the format of the text entered by the candidate:

<xsd:simpleType name="textFormat.Type">
        <xsd:restriction base="xsd:NMTOKEN">
                <xsd:enumeration value="plain"/>
                <xsd:enumeration value="preFormatted"/>
                <xsd:enumeration value="xhtml"/>
        </xsd:restriction>
</xsd:simpleType>

Defines constants for the above formats. Usage example:

TextFormat.plain

Note that:

TextFormat.DEFAULT == TextFormat.plain

For more methods see Enumeration

class pyslet.qtiv2.interactions.ExtendedTextInteraction(parent)¶

Bases: pyslet.qtiv2.interactions.StringInteractionMixin, pyslet.qtiv2.interactions.BlockInteraction

An extended text interaction is a blockInteraction that allows the candidate to enter an extended amount of text:

<xsd:attributeGroup name="extendedTextInteraction.AttrGroup">
        <xsd:attributeGroup ref="blockInteraction.AttrGroup"/>
        <xsd:attributeGroup ref="stringInteraction.AttrGroup"/>
        <xsd:attribute name="maxStrings" type="integer.Type" use="optional"/>
        <xsd:attribute name="minStrings" type="integer.Type" use="optional"/>
        <xsd:attribute name="expectedLines" type="integer.Type" use="optional"/>
        <xsd:attribute name="format" type="textFormat.Type" use="optional"/>
</xsd:attributeGroup>

class pyslet.qtiv2.interactions.HottextInteraction(parent)¶

Bases: pyslet.qtiv2.interactions.BlockInteraction

The hottext interaction presents a set of choices to the candidate represented as selectable runs of text embedded within a surrounding context, such as a simple passage of text:

<xsd:attributeGroup name="hottextInteraction.AttrGroup">
        <xsd:attributeGroup ref="blockInteraction.AttrGroup"/>
        <xsd:attribute name="maxChoices" type="integer.Type" use="required"/>
        <xsd:attribute name="minChoices" type="integer.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="hottextInteraction.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="blockInteraction.ContentGroup"/>
                <xsd:group ref="blockStatic.ElementGroup" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.Hottext(parent)¶

Bases: pyslet.html40_19991224.FlowMixin, pyslet.qtiv2.interactions.Choice

A hottext area is used within the content of an hottextInteraction to provide the individual choices:

<xsd:group name="hottext.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="inlineStatic.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

Graphical Interactions¶

class pyslet.qtiv2.interactions.GapImg(parent)

Bases: pyslet.qtiv2.interactions.GapChoice

A gap image contains a single image object to be inserted into a gap by the candidate:

<xsd:attributeGroup name="gapImg.AttrGroup">
        <xsd:attributeGroup ref="gapChoice.AttrGroup"/>
        <xsd:attribute name="objectLabel" type="string.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="gapImg.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="object" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.GapImg(parent)

Bases: pyslet.qtiv2.interactions.GapChoice

A gap image contains a single image object to be inserted into a gap by the candidate:

<xsd:attributeGroup name="gapImg.AttrGroup">
        <xsd:attributeGroup ref="gapChoice.AttrGroup"/>
        <xsd:attribute name="objectLabel" type="string.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="gapImg.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="object" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.interactions.GapImg(parent)

Bases: pyslet.qtiv2.interactions.GapChoice

A gap image contains a single image object to be inserted into a gap by the candidate:

<xsd:attributeGroup name="gapImg.AttrGroup">
        <xsd:attributeGroup ref="gapChoice.AttrGroup"/>
        <xsd:attribute name="objectLabel" type="string.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="gapImg.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="object" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

Item Variables¶

This module contains the basic run-time data model. Although the specification does contain elements to represent the values of variables set at runtime the XML schema sometimes relies too much on context for an efficient implementation. For example, a <value> element is always a value of a specific base type but the base type is rarely specified on the value element itself as it is normally implicit in the context. such as a variable declaration.

Although the expression model does contain an element that provides a more complete representation of single values (namely <baseValue>) we decide to make the distinction in this module with ValueElement representing the element and the abstract Value being used as the root of the runtime object model.

For example, to get the default value of a variable from a variable declaration you’ll use the GetDefaultValue() method and it will return a Value instance which could be of any cardinality or base type.

class pyslet.qtiv2.variables.VariableDeclaration(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

Item variables are declared by variable declarations... The purpose of the declaration is to associate an identifier with the variable and to identify the runtime type of the variable’s value:

<xsd:attributeGroup name="variableDeclaration.AttrGroup">
        <xsd:attribute name="identifier" type="identifier.Type" use="required"/>
        <xsd:attribute name="cardinality" type="cardinality.Type" use="required"/>
        <xsd:attribute name="baseType" type="baseType.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="variableDeclaration.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="defaultValue" minOccurs="0" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

GetDefaultValue()¶: Returns a Value instance representing either the default value or an appropriately typed NULL value if there is no default defined.

class pyslet.qtiv2.variables.ValueElement(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

A class that can represent a single value of any baseType in variable declarations and result reports:

<xsd:attributeGroup name="value.AttrGroup">
        <xsd:attribute name="fieldIdentifier" type="identifier.Type" use="optional"/>
        <xsd:attribute name="baseType" type="baseType.Type" use="optional"/>
</xsd:attributeGroup>

class pyslet.qtiv2.variables.DefaultValue(parent)¶

Bases: pyslet.qtiv2.variables.DefinedValue

An optional default value for a variable. The point at which a variable is set to its default value varies depending on the type of item variable.

class pyslet.qtiv2.variables.Cardinality¶

Bases: pyslet.xsdatatypes20041028.Enumeration

An expression or itemVariable can either be single-valued or multi-valued. A multi-valued expression (or variable) is called a container. A container contains a list of values, this list may be empty in which case it is treated as NULL. All the values in a multiple or ordered container are drawn from the same value set:

<xsd:simpleType name="cardinality.Type">
        <xsd:restriction base="xsd:NMTOKEN">
                <xsd:enumeration value="multiple"/>
                <xsd:enumeration value="ordered"/>
                <xsd:enumeration value="record"/>
                <xsd:enumeration value="single"/>
        </xsd:restriction>
</xsd:simpleType>

Defines constants for the above carinalities. Usage example:

Cardinality.multiple

There is no default:

Cardinality.DEFAULT == None

For more methods see Enumeration

class pyslet.qtiv2.variables.BaseType¶

Bases: pyslet.xsdatatypes20041028.Enumeration

A base-type is simply a description of a set of atomic values (atomic to this specification). Note that several of the baseTypes used to define the runtime data model have identical definitions to those of the basic data types used to define the values for attributes in the specification itself. The use of an enumeration to define the set of baseTypes used in the runtime model, as opposed to the use of classes with similar names, is designed to help distinguish between these two distinct levels of modelling:

<xsd:simpleType name="baseType.Type">
        <xsd:restriction base="xsd:NMTOKEN">
                <xsd:enumeration value="boolean"/>
                <xsd:enumeration value="directedPair"/>
                <xsd:enumeration value="duration"/>
                <xsd:enumeration value="file"/>
                <xsd:enumeration value="float"/>
                <xsd:enumeration value="identifier"/>
                <xsd:enumeration value="integer"/>
                <xsd:enumeration value="pair"/>
                <xsd:enumeration value="point"/>
                <xsd:enumeration value="string"/>
                <xsd:enumeration value="uri"/>
        </xsd:restriction>
</xsd:simpleType>

Defines constants for the above base types. Usage example:

BaseType.float

There is no default:

BaseType.DEFAULT == None

For more methods see Enumeration

class pyslet.qtiv2.variables.Mapping(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

A special class used to create a mapping from a source set of any baseType (except file and duration) to a single float:

<xsd:attributeGroup name="mapping.AttrGroup">
        <xsd:attribute name="lowerBound" type="float.Type" use="optional"/>
        <xsd:attribute name="upperBound" type="float.Type" use="optional"/>
        <xsd:attribute name="defaultValue" type="float.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="mapping.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="mapEntry" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

ContentChanged()¶

Builds an internal dictionary of the values being mapped.

In order to fully specify the mapping we need to know the baseType of the source values. (The targets are always floats.) We do this based on our parent, orphan Mapping elements are treated as mappings from source strings.

MapValue(value)¶: Maps an instance of Value with the same base type as the mapping to an instance of Value with base type float.

class pyslet.qtiv2.variables.MapEntry(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

An entry in a Mapping

<xsd:attributeGroup name="mapEntry.AttrGroup">
        <xsd:attribute name="mapKey" type="valueType.Type" use="required"/>
        <xsd:attribute name="mappedValue" type="float.Type" use="required"/>
</xsd:attributeGroup>

mapKey = None¶: The source value

mappedValue = None¶: The mapped value

Response Variables¶

class pyslet.qtiv2.variables.ResponseDeclaration(parent)¶

Bases: pyslet.qtiv2.variables.VariableDeclaration

Response variables are declared by response declarations and bound to interactions in the itemBody:

<xsd:group name="responseDeclaration.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="variableDeclaration.ContentGroup"/>
                <xsd:element ref="correctResponse" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="mapping" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="areaMapping" minOccurs="0" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

GetCorrectValue()¶: Returns a Value instance representing either the correct response value or an appropriately typed NULL value if there is no correct value.

GetStageDimensions()¶

For response variables with point type, returns a pair of integer values: width,height

In HTML, shapes (including those used in the AreaMapping) can use relative coordinates. To interpret relative coordinates we need to know the size of the stage used to interpret the point values. For a response variable that is typically the size of the image or object used in the interaction.

This method searches for the interaction associated with the response and obtains the width and height of the corresponding object.

[TODO: currently returns 100,100]

class pyslet.qtiv2.variables.CorrectResponse(parent)¶

Bases: pyslet.qtiv2.variables.DefinedValue

A response declaration may assign an optional correctResponse. This value may indicate the only possible value of the response variable to be considered correct or merely just a correct value.

class pyslet.qtiv2.variables.AreaMapping(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

A special class used to create a mapping from a source set of point values to a target set of float values:

<xsd:attributeGroup name="areaMapping.AttrGroup">
        <xsd:attribute name="lowerBound" type="float.Type" use="optional"/>
        <xsd:attribute name="upperBound" type="float.Type" use="optional"/>
        <xsd:attribute name="defaultValue" type="float.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="areaMapping.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="areaMapEntry" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

MapValue(value, width, height)¶

Maps an instance of Value with point base type to an instance of Value with base type float.

value is a Value of base type point
width is the integer width of the object on which the area is defined
height is the integer height of the object on which the area is defined

The width and height of the object are required because HTML allows relative values to be used when defining areas.

class pyslet.qtiv2.variables.AreaMapEntry(parent)¶

Bases: pyslet.qtiv2.core.QTIElement, pyslet.qtiv2.core.ShapeElementMixin

An AreaMapping is defined by a set of areaMapEntries, each of which maps an area of the coordinate space onto a single float:

<xsd:attributeGroup name="areaMapEntry.AttrGroup">
        <xsd:attribute name="shape" type="shape.Type" use="required"/>
        <xsd:attribute name="coords" type="coords.Type" use="required"/>
        <xsd:attribute name="mappedValue" type="float.Type" use="required"/>
</xsd:attributeGroup>

mappedValue = None¶: The mapped value

Outcome Variables¶

class pyslet.qtiv2.variables.OutcomeDeclaration(parent)¶

Bases: pyslet.qtiv2.variables.VariableDeclaration

Outcome variables are declared by outcome declarations

<xsd:attributeGroup name="outcomeDeclaration.AttrGroup">
        <xsd:attributeGroup ref="variableDeclaration.AttrGroup"/>
        <xsd:attribute name="view" use="optional">
                <xsd:simpleType>
                        <xsd:list itemType="view.Type"/>
                </xsd:simpleType>
        </xsd:attribute>
        <xsd:attribute name="interpretation" type="string.Type" use="optional"/>
        <xsd:attribute name="longInterpretation" type="uri.Type" use="optional"/>
        <xsd:attribute name="normalMaximum" type="float.Type" use="optional"/>
        <xsd:attribute name="normalMinimum" type="float.Type" use="optional"/>
        <xsd:attribute name="masteryValue" type="float.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="outcomeDeclaration.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="variableDeclaration.ContentGroup"/>
                <xsd:group ref="lookupTable.ElementGroup" minOccurs="0" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.variables.LookupTable(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

An abstract class associated with an outcomeDeclaration used to create a lookup table from a numeric source value to a single outcome value in the declared value set:

<xsd:attributeGroup name="lookupTable.AttrGroup">
        <xsd:attribute name="defaultValue" type="valueType.Type" use="optional"/>
</xsd:attributeGroup>

default = None¶: a Value instance representing the default

class pyslet.qtiv2.variables.MatchTable(parent)¶

Bases: pyslet.qtiv2.variables.LookupTable

A matchTable transforms a source integer by finding the first matchTableEntry with an exact match to the source:

<xsd:group name="matchTable.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="matchTableEntry" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

ContentChanged()¶: Builds an internal dictionary of the values being mapped.

Lookup(value)¶: Maps an instance of Value with integer base type to an instance of Value with the base type of the match table.

class pyslet.qtiv2.variables.MatchTableEntry(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

sourceValue: The source integer that must be matched exactly.
targetValue: The target value that is used to set the outcome when a match is found

<xsd:attributeGroup name="matchTableEntry.AttrGroup">
        <xsd:attribute name="sourceValue" type="integer.Type" use="required"/>
        <xsd:attribute name="targetValue" type="valueType.Type" use="required"/>
</xsd:attributeGroup>

class pyslet.qtiv2.variables.InterpolationTable(parent)¶

Bases: pyslet.qtiv2.variables.LookupTable

An interpolationTable transforms a source float (or integer) by finding the first interpolationTableEntry with a sourceValue that is less than or equal to (subject to includeBoundary) the source value:

<xsd:group name="interpolationTable.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="interpolationTableEntry" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

ContentChanged()¶: Builds an internal table of the values being mapped.

Lookup(value)¶: Maps an instance of Value with integer or float base type to an instance of Value with the base type of the interpolation table.

class pyslet.qtiv2.variables.InterpolationTableEntry(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

sourceValue: The lower bound for the source value to match this entry.
includeBoundary: Determines if an exact match of sourceValue matches this entry. If true, the default, then an exact match of the value is considered a match of this entry.
targetValue: The target value that is used to set the outcome when a match is found

<xsd:attributeGroup name="interpolationTableEntry.AttrGroup">
        <xsd:attribute name="sourceValue" type="float.Type" use="required"/>
        <xsd:attribute name="includeBoundary" type="boolean.Type" use="optional"/>
        <xsd:attribute name="targetValue" type="valueType.Type" use="required"/>
</xsd:attributeGroup>

Template Variables¶

class pyslet.qtiv2.variables.TemplateDeclaration(parent)¶

Bases: pyslet.qtiv2.variables.VariableDeclaration

Template declarations declare item variables that are to be used specifically for the purposes of cloning items

<xsd:attributeGroup name="templateDeclaration.AttrGroup">
        <xsd:attributeGroup ref="variableDeclaration.AttrGroup"/>
        <xsd:attribute name="paramVariable" type="boolean.Type" use="optional"/>
        <xsd:attribute name="mathVariable" type="boolean.Type" use="optional"/>
</xsd:attributeGroup>

Runtime Object Model¶

class pyslet.qtiv2.variables.SessionState¶

Bases: object

Abstract class used as the base class for namespace-like objects used to track the state of an item or test session. Instances can be used as if they were dictionaries of Value.

GetDeclaration(varName)¶

Returns the declaration associated with varName or None if the variable is one of the built-in variables. If varName is not a variable KeyError is raised. To test for the existence of a variable just use the object as you would a dictionary:

# state is a SessionState instance
if 'RESPONSE' in state:
        print "RESPONSE declared!" 

IsResponse(varName)¶: Return True if varName is the name of a response variable.

IsOutcome(varName)¶: Return True if varName is the name of an outcome variable.

IsTemplate(varName)¶: Return True if varName is the name of a template variable.

__getitem__(varName)¶: Returns the Value instance corresponding to varName or raises KeyError if there is no variable with that name.

__setitem__(varName, value)¶

Sets the value of varName to the Value instance value.

The baseType and cardinality of value must match those expected for the variable.

This method does not actually update the dictionary with the value instance but instead, it copies the value of value into the Value instance already stored in the session. The side-effect of this implementation is that a previous look-up will be updated by a subsequent assignment:

# state is a SessionState instance
state['RESPONSE']=IdentifierValue('Hello')
r1=state['RESPONSE']
state['RESPONSE']=IdentifierValue('Bye')
r2=state['RESPONSE']
r1==r2          # WARNING: r1 has been updated so still evaluates to True!

__weakref__¶: list of weak references to the object (if defined)

class pyslet.qtiv2.variables.ItemSessionState(item)¶

Bases: pyslet.qtiv2.variables.SessionState

Represents the state of an item session. item is the item from which the session should be created.

On construction, all declared variables (included built-in variables) are added to the session with NULL values, except the template variables which are set to their defaults.

In addition to the variables defined by the specification we add meta variables corresponding to response and outcome defaults, these have the same name as the variable but with ”.DEFAULT” appended. Similarly, we define names for the correct values of response variables using ”.CORRECT”. The values of these meta-variables are all initialised from the item definition on construction.

SelectClone()¶

Item templates describe a range of possible items referred to as clones.

If the item used to create the session object is an item template then you must call SelectClone before beginning the candidate’s session with BeginSession().

The main purpose of this method is to run the template processing rules. These rules update the values of the template variables and may also alter correct responses and default outcome (or response) values.

BeginSession()¶

Called at the start of an item session. According to the specification:

“The session starts when the associated item first becomes eligible for delivery to the candidate”

The main purpose of this method is to set the outcome values to their defaults.

BeginAttempt(htmlParent=None)¶

Called at the start of an attempt.

This method sets the default RESPONSE values and completionStatus if this is the first attempt and increments numAttempts accordingly.

SaveSession(params, htmlParent=None)¶: Called when we wish to save unsubmitted values.

SubmitSession(params, htmlParent=None)¶: Called when we wish to submit values (i.e., end an attempt).

EndAttempt()¶: Called at the end of an attempt. Invokes response processing if present.

IsResponse(varName)¶

Return True if varName is the name of a response variable.

We add handling of the built-in response variables numAttempts and duration.

IsOutcome(varName)¶

Return True if varName is the name of an outcome variable.

We add handling of the built-in outcome variable completionStatus.

class pyslet.qtiv2.variables.TestSessionState(form)¶

Bases: pyslet.qtiv2.variables.SessionState

Represents the state of a test session. The keys are the names of the variables including qualified names that can be used to look up the value of variables from the associated item session states. form is the test form from which the session should be created.

On construction, all declared variables (included built-in variables) are added to the session with NULL values.

test = None¶: the tests.AssessmentTest that this session is an instance of

t = None¶: the time of the last event

salt = None¶: a random string of bytes used to add entropy to the session key

key = None¶: A key representing this session in its current state, this key is initialised to a random value and changes as each event is received. The key must be supplied when triggering subsequent events. The key is designed to be unguessable and unique so a caller presenting the correct key when triggering an event can be securely assumed to be the owner of the existing session.

prevKey = None¶: The key representing the previous state. This can be used to follow session state transitions back through a chain of states back to the beginning of the session (i.e., for auditing).

keyMap = None¶: A mapping of keys previously used by this session. A caller presenting an expired key when triggering an event generates a SessionKeyExpired exception. This condition might indicate that a session response was not received (e.g., due to a connection failure) and that the session should be re-started with the previous response.

GetCurrentTestPart()¶: Returns the current test part or None if the test is finished.

GetCurrentQuestion()¶: Returns the current question or None if the test is finished.

BeginSession(key, htmlParent=None)¶

Called at the start of a test session. Represents a ‘Start Test’ event.

The main purpose of this method is to set the outcome values to their defaults and to select the first question.

GetNamespace(varName)¶

Takes a variable name varName and returns a tuple of namespace/varName.

The resulting namespace will be a dictionary or a dictionary-like object from which the value of the returned varName object can be looked up.

IsResponse(varName)¶: Return True if varName is the name of a response variable. The test-level duration values are treated as built-in responses and return True.

__len__()¶: Returns the total length of all namespaces combined.

__getitem__(varName)¶: Returns the Value instance corresponding to varName or raises KeyError if there is no variable with that name.

class pyslet.qtiv2.variables.Value¶

Bases: object

Represents a single value in the processing model.

This class is the heart of the QTI processing model. This is an abstract base class of a class hierarchy that represents the various types of value that may be encountered when processing.

baseType = None¶

One of the BaseType constants or None if the baseType is unknown.

An unknown baseType acts like a wild-card. It means that the baseType is not determined and could potentially be any of the BaseType values. This distinction has implications for the way evaluation is done. A value with a baseType of None will not raise TypeErrors during evaluation if the cardinalities match the context. This allows expressions which contain types bound only at runtime to be evaluated for validity checking.

value = None¶

The value of the variable. The following representations are used for values of single cardinality:

NULL value: Represented by None
boolean: One of the built-in Python values True and False
directedPair: A tuple of strings (<source identifier>, <destination identifier>)
duration: real number of seconds
file: a file like object (supporting seek)
float: real number
identifier: A text string
integer: A plain python integer (QTI does not support long integer values)
pair: A sorted tuple of strings (<identifier A>, <identifier B>). We sort the identifiers in a pair by python’s native string sorting to ensure that pair values are comparable.
point: A tuple of integers (<x-coordinate>, <y-coordinate>)
string: A python string
uri: An instance of URI

For containers, we use the following structures:

ordered: A list of one of the above value types.
multiple:: A dictionary with keys that are one of the above value types and values that indicate the frequency of that value in the container.
record:: A dictionary with keys that are the field identifiers and values that Value instances.

SetValue(value)¶

Sets the value.

All single values can be set from a single text string corresponding to their XML schema defined lexical values (without character level escaping). If v is a single Value instance then the following always leaves v unchanged:

v.SetValue(unicode(v))

Value instances can also be set from values of the appropriate type as described in value. For base types that are represented with tuples we also accept and convert lists.

Containers values cannot be set from strings.

ValueError(value)¶: Raises a ValueError with a debug-friendly message string.

Cardinality()¶

Returns the cardinality of this value. One of the Cardinality constants.

By default we return None - indicating unknown cardinality. This can only be the case if the value is a NULL.

IsNull()¶: Returns True is this value is NULL, as defined by the QTI specification.

classmethod NewValue(cardinality, baseType=None)¶: Creates a new value instance with cardinality and baseType.

classmethod CopyValue(value)¶: Creates a new value instance copying value.

class pyslet.qtiv2.variables.SingleValue¶

Bases: pyslet.qtiv2.variables.Value

Represents all values with single cardinality.

classmethod NewValue(baseType, value=None)¶: Creates a new instance of a single value with baseType and value

class pyslet.qtiv2.variables.BooleanValue(value=None)¶

Bases: pyslet.qtiv2.variables.SingleValue

Represents single values of type BaseType.boolean.

SetValue(value)¶

If value is a string it will be decoded according to the rules for representing boolean values. Booleans and integers can be used directly in the normal python way but other values will raise ValueError. To take advantage of a non-zero test you must explicitly force it to be a boolean. For example:

# x is a value of unknown type with non-zero test implemented
v=BooleanValue()
v.SetValue(True if x else False)

class pyslet.qtiv2.variables.DirectedPairValue(value=None)¶

Bases: pyslet.qtiv2.variables.SingleValue

Represents single values of type BaseType.directedPair.

SetValue(value, nameCheck=False)¶

See comment on Identifier.SetValue() for usage of nameCheck.

Note that if value is a string then nameCheck is ignored and identifier validation is always performed.

class pyslet.qtiv2.variables.DurationValue(value=None)¶

Bases: pyslet.qtiv2.variables.FloatValue

Represents single value of type BaseType.duration.

class pyslet.qtiv2.variables.FileValue¶

Bases: pyslet.qtiv2.variables.SingleValue

Represents single value of type BaseType.file.

contentType = None¶: The content type of the file, a pyslet.rfc2616.HTTPMediaType instance.

fileName = None¶: The file name to use for the file.

SetValue(value, type='application/octet-stream', name='data.bin')¶

Sets a file value from a file like object or a string.

There are some important and subtle distinctions in this method.

If value is a Unicode text string then it is parsed according to the MIME-like format defined in the QTI specification. The values of type and name are only used as defaults if those values cannot be read from the value’s headers.

If value is a plain string then it is assumed to represent the file’s data directly, type and name are used to interpret the data. Other file type objects are set in the same way.

class pyslet.qtiv2.variables.FloatValue(value=None)¶

Bases: pyslet.qtiv2.variables.SingleValue

Represents single value of type BaseType.float.

SetValue(value)¶

This method will not convert integers to float values, you must do this explicitly if you want automatic conversion, for example

# x is a numeric value that may be float or integer
v=FloatValue()
v.SetValue(float(x))

class pyslet.qtiv2.variables.IdentifierValue(value=None)¶

Bases: pyslet.qtiv2.variables.SingleValue

Represents single value of type BaseType.identifier.

SetValue(value, nameCheck=True)¶

In general, to speed up computation we do not check the validity of identifiers unless parsing the value from a string representation (such as a value read from an XML input document).

As values of baseType identifier are represented natively as strings we cannot tell if this method is being called with an existing, name-checked value or a new value being parsed from an external source. To speed up computation you can suppress the name check in the first case by setting nameCheck to False (the default is True).

class pyslet.qtiv2.variables.IntegerValue(value=None)¶

Bases: pyslet.qtiv2.variables.SingleValue

Represents single value of type BaseType.integer.

SetValue(value)¶: Note that integers and floats are distinct types in QTI: we do not accept floats where we would expect integers or vice versa. However, integers are accepted from long or plain integer values provided they are within the ranges specified in the QTI specification: -2147483648...2147483647.

class pyslet.qtiv2.variables.PairValue(value=None)¶

Bases: pyslet.qtiv2.variables.DirectedPairValue

Represents single values of type BaseType.pair.

SetValue(value, nameCheck=True)¶: Overrides DirectedPair’s implementation to force a predictable ordering on the identifiers.

class pyslet.qtiv2.variables.PointValue(value=None)¶

Bases: pyslet.qtiv2.variables.SingleValue

Represents single value of type BaseType.point.

class pyslet.qtiv2.variables.StringValue(value=None)¶

Bases: pyslet.qtiv2.variables.SingleValue

Represents single value of type BaseType.string.

class pyslet.qtiv2.variables.URIValue(value=None)¶

Bases: pyslet.qtiv2.variables.SingleValue

Represents single value of type BaseType.uri.

SetValue(value)¶: Sets a uri value from a string or another URI instance.

class pyslet.qtiv2.variables.Container(baseType=None)¶

Bases: pyslet.qtiv2.variables.Value

An abstract class for all container types.

By default containers are empty (and are treated as NULL values). You can force the type of an empty container by passing a baseType constant to the constructor. This will cause the container to generate TypeError if used in a context where the specified baseType is not allowed.

GetValues()¶: Returns an iterable of the container’s values.

classmethod NewValue(cardinality, baseType=None)¶: Creates a new container with cardinality and baseType.

class pyslet.qtiv2.variables.OrderedContainer(baseType=None)¶

Bases: pyslet.qtiv2.variables.Container

Represents containers with ordered Cardinality.

SetValue(value, baseType=None)¶

Sets the value of this container from a list, tuple or other iterable. The list must contain valid representations of baseType, items may be None indicating a NULL value in the list. In accordance with the specification’s multiple operator NULL values are ignored.

If the input list of values empty, or contains only NULL values then the resulting container is empty.

If baseType is None the base type specified when the container was constructed is assumed.

GetValues()¶: Returns an iterable of values in the ordered container.

class pyslet.qtiv2.variables.MultipleContainer(baseType=None)¶

Bases: pyslet.qtiv2.variables.Container

Represents containers with multiple Cardinality.

SetValue(value, baseType=None)¶

Sets the value of this container from a list, tuple or other iterable. The list must contain valid representations of baseType, items may be None indicating a NULL value in the list. In accordance with the specification’s multiple operator NULL values are ignored.

If the input list of values is empty, or contains only NULL values then the resulting container is empty.

If baseType is None the base type specified when the container was constructed is assumed.

GetValues()¶: Returns an iterable of values in the ordered container.

class pyslet.qtiv2.variables.RecordContainer¶

Bases: pyslet.qtiv2.variables.Container

Represents containers with record Cardinality.

SetValue(value)¶

Sets the value of this container from an existing dictionary in which the keys are the field identifiers and the values are Value instances. You cannot parse containers from strings.

Records are always treated as having a wild-card base type.

If the input value contains any keys which map to None or to a NULL value then these fields are omitted from the resulting value.

__getitem__(fieldIdentifier)¶: Returns the Value instance corresponding to fieldIdentifier or raises KeyError if there is no field with that name.

__setitem__(fieldIdentifier, value)¶

Sets the value in the named field to value.

We add some special behaviour here. If value is None or is a NULL value then we remove the field with the give name. In other words:

r=RecordContainer()
r['pi']=FloatValue(3.14)
r['pi']=FloatValue()     # a NULL value
print r['pi']            # raises KeyError

Response Processing¶

Generalized Response Processing¶

class pyslet.qtiv2.processing.ResponseProcessing(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

Response processing is the process by which the Delivery Engine assigns outcomes based on the candidate’s responses:

<xsd:attributeGroup name="responseProcessing.AttrGroup">
        <xsd:attribute name="template" type="uri.Type" use="optional"/>
        <xsd:attribute name="templateLocation" type="uri.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="responseProcessing.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="responseRule.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

Run(state)¶

Runs response processing using the values in state.

state is an ItemSessionState instance.

class pyslet.qtiv2.processing.ResponseRule(parent, name=None)¶

Bases: pyslet.qtiv2.core.QTIElement

Abstract class to represent all response rules.

Run(state)¶: Abstract method to run this rule using the values in state.

class pyslet.qtiv2.processing.ResponseCondition(parent)¶

Bases: pyslet.qtiv2.processing.ResponseRule

If the expression given in a responseIf or responseElseIf evaluates to true then the sub-rules contained within it are followed and any following responseElseIf or responseElse parts are ignored for this response condition:

<xsd:group name="responseCondition.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="responseIf" minOccurs="1" maxOccurs="1"/>
                <xsd:element ref="responseElseIf" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="responseElse" minOccurs="0" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.processing.ResponseIf(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

A responseIf part consists of an expression which must have an effective baseType of boolean and single cardinality. If the expression is true then the sub-rules are processed, otherwise they are skipped (including if the expression is NULL):

<xsd:group name="responseIf.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
                <xsd:group ref="responseRule.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

Run(state)¶

Run this test and, if True, any resulting rules.

Returns True if the condition evaluated to True.

class pyslet.qtiv2.processing.ResponseElseIf(parent)¶

Bases: pyslet.qtiv2.processing.ResponseIf

Represents the responseElse element, see ResponseIf

<xsd:group name="responseElseIf.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
                <xsd:group ref="responseRule.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.processing.ResponseElse(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

Represents the responseElse element, see ResponseCondition

<xsd:group name="responseElse.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="responseRule.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

Run(state)¶: Runs the sub-rules.

class pyslet.qtiv2.processing.SetOutcomeValue(parent)¶

Bases: pyslet.qtiv2.processing.ResponseRule

The setOutcomeValue rule sets the value of an outcome variable to the value obtained from the associated expression:

<xsd:attributeGroup name="setOutcomeValue.AttrGroup">
        <xsd:attribute name="identifier" type="identifier.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="setOutcomeValue.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.processing.StopProcessing¶

Bases: pyslet.qtiv2.core.QTIError

Raised when a rule which stops processing is encountered.

class pyslet.qtiv2.processing.ExitResponse(parent, name=None)¶

Bases: pyslet.qtiv2.processing.ResponseRule

The exit response rule terminates response processing immediately (for this invocation). It does this by raising StopProcessing:

<xsd:complexType name="exitResponse.Type"/>

Template Processing¶

class pyslet.qtiv2.processing.TemplateProcessing(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

Template processing consists of one or more templateRules that are followed by the cloning engine or delivery system in order to assign values to the template variables:

<xsd:group name="templateProcessing.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="templateRule.ElementGroup" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

<xsd:complexType name="templateProcessing.Type" mixed="false">
        <xsd:group ref="templateProcessing.ContentGroup"/>
</xsd:complexType>

Run(state)¶

Runs template processing rules using the values in state.

state is an ItemSessionState instance.

class pyslet.qtiv2.processing.TemplateRule(parent, name=None)¶

Bases: pyslet.qtiv2.core.QTIElement

Abstract class to represent all template rules.

Run(state)¶: Abstract method to run this rule using the values in state.

class pyslet.qtiv2.processing.TemplateCondition(parent)¶

Bases: pyslet.qtiv2.processing.TemplateRule

If the expression given in the templateIf or templateElseIf evaluates to true then the sub-rules contained within it are followed and any following templateElseIf or templateElse parts are ignored for this template condition:

<xsd:group name="templateCondition.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="templateIf" minOccurs="1" maxOccurs="1"/>
                <xsd:element ref="templateElseIf" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="templateElse" minOccurs="0" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.processing.TemplateIf(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

A templateIf part consists of an expression which must have an effective baseType of boolean and single cardinality. If the expression is true then the sub-rules are processed, otherwise they are skipped (including if the expression is NULL):

<xsd:group name="templateIf.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
                <xsd:group ref="templateRule.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

Run(state)¶

Run this test and, if True, any resulting rules.

Returns True if the condition evaluated to True.

class pyslet.qtiv2.processing.TemplateElseIf(parent)¶

Bases: pyslet.qtiv2.processing.TemplateIf

Represents the templateElse element, see templateIf

<xsd:group name="templateElseIf.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
                <xsd:group ref="templateRule.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.processing.TemplateElse(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

Represents the templateElse element, see TemplateCondition

<xsd:group name="templateElse.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="templateRule.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

Run(state)¶: Runs the sub-rules.

class pyslet.qtiv2.processing.SetTemplateValue(parent)¶

Bases: pyslet.qtiv2.processing.TemplateRule

The setTemplateValue rule sets the value of a template variable to the value obtained from the associated expression:

<xsd:attributeGroup name="setTemplateValue.AttrGroup">
        <xsd:attribute name="identifier" type="identifier.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="setTemplateValue.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.processing.SetCorrectResponse(parent)¶

Bases: pyslet.qtiv2.processing.TemplateRule

The setCorrectResponse rule sets the correct value of a response variable to the value obtained from the associated expression:

<xsd:attributeGroup name="setCorrectResponse.AttrGroup">
        <xsd:attribute name="identifier" type="identifier.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="setCorrectResponse.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.processing.SetDefaultValue(parent)¶

Bases: pyslet.qtiv2.processing.TemplateRule

The setDefaultValue rule sets the default value of a response or outcome variable to the value obtained from the associated expression:

<xsd:attributeGroup name="setDefaultValue.AttrGroup">
        <xsd:attribute name="identifier" type="identifier.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="setDefaultValue.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.processing.ExitTemplate(parent, name=None)¶

Bases: pyslet.qtiv2.processing.TemplateRule

The exit template rule terminates template processing immediately. It does this by raising StopProcessing:

<xsd:complexType name="exitTemplate.Type"/>

Pre-conditions and Branching¶

class pyslet.qtiv2.processing.TestPartCondition(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

Evaluate(state)¶

Evaluates the condition using the values in state.

state is a TestSessionState instance.

class pyslet.qtiv2.processing.PreCondition(parent)¶

Bases: pyslet.qtiv2.processing.TestPartCondition

A preCondition is a simple expression attached to an assessmentSection or assessmentItemRef that must evaluate to true if the item is to be presented:

<xsd:group name="preCondition.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.processing.BranchRule(parent)¶

Bases: pyslet.qtiv2.processing.TestPartCondition

A branch-rule is a simple expression attached to an assessmentItemRef, assessmentSection or testPart that is evaluated after the item, section, or part has been presented to the candidate:

<xsd:attributeGroup name="branchRule.AttrGroup">
        <xsd:attribute name="target" type="identifier.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="branchRule.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.processing.TemplateDefault(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

Overrides the default value of a template variable based on the test context in which the template is instantiated:

<xsd:attributeGroup name="templateDefault.AttrGroup">
        <xsd:attribute name="templateIdentifier" type="identifier.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="templateDefault.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

Run(itemState, testState)¶: Updates the value of a template variable in itemState based on the values in testState.

Expressions¶

class pyslet.qtiv2.expressions.Expression(parent, name=None)¶

Bases: pyslet.qtiv2.core.QTIElement

Abstract class for all expression elements.

Evaluate(state)¶: Evaluates this expression in the context of the session state.

IntegerOrTemplateRef(state, value)¶: Given a value of type integerOrTemplateRef this method returns the corresponding integer by looking up the value, if necessary, in state. If value is a variable reference to a variable with NULL value then None is returned.

FloatOrTemplateRef(state, value)¶: Given a value of type floatOrTemplateRef this method returns the corresponding float by looking up the value, if necessary, in state. If value is a variable reference to a variable with NULL value then None is returned.

StringOrTemplateRef(state, value)¶: Given a value of type stringOrTemplateRef this method returns the corresponding string by looking up the value, if necessary, in state. If value is a variable reference to a variable with NULL value then None is returned. Note that unlike the integer and float expansions this expansion will not raise an error if value is a syntactically valid reference to a non-existent template variable, as per this condition in the specification.

“if a string attribute appears to be a reference to a template variable but there is no variable with the given name it should be treated simply as string value”

class pyslet.qtiv2.expressions.NOperator(parent)¶

Bases: pyslet.qtiv2.expressions.Expression

An abstract class to help implement operators which take multiple sub-expressions.

EvaluateChildren(state)¶: Evaluates all child expressions, returning an iterable of Value instances.

class pyslet.qtiv2.expressions.UnaryOperator(parent)¶

Bases: pyslet.qtiv2.expressions.Expression

An abstract class to help implement unary operators.

Built-in General Expressions¶

class pyslet.qtiv2.expressions.BaseValue(parent)¶

Bases: pyslet.qtiv2.expressions.Expression

The simplest expression returns a single value from the set defined by the given baseType

<xsd:attributeGroup name="baseValue.AttrGroup">
        <xsd:attribute name="baseType" type="baseType.Type" use="required"/>
</xsd:attributeGroup>

<xsd:complexType name="baseValue.Type">
        <xsd:simpleContent>
                <xsd:extension base="xsd:string">
                        <xsd:attributeGroup ref="baseValue.AttrGroup"/>
                </xsd:extension>
        </xsd:simpleContent>
</xsd:complexType>

class pyslet.qtiv2.expressions.Variable(parent)¶

Bases: pyslet.qtiv2.expressions.Expression

This expression looks up the value of an itemVariable that has been declared in a corresponding variableDeclaration or is one of the built-in variables:

<xsd:attributeGroup name="variable.AttrGroup">
        <xsd:attribute name="identifier" type="identifier.Type" use="required"/>
        <xsd:attribute name="weightIdentifier" type="identifier.Type" use="optional"/>
</xsd:attributeGroup>

class pyslet.qtiv2.expressions.Default(parent)¶

Bases: pyslet.qtiv2.expressions.Expression

This expression looks up the declaration of an itemVariable and returns the associated defaultValue or NULL if no default value was declared:

<xsd:attributeGroup name="default.AttrGroup">
        <xsd:attribute name="identifier" type="identifier.Type" use="required"/>
</xsd:attributeGroup>

class pyslet.qtiv2.expressions.Correct(parent)¶

Bases: pyslet.qtiv2.expressions.Expression

This expression looks up the declaration of a response variable and returns the associated correctResponse or NULL if no correct value was declared:

<xsd:attributeGroup name="correct.AttrGroup">
        <xsd:attribute name="identifier" type="identifier.Type" use="required"/>
</xsd:attributeGroup>

class pyslet.qtiv2.expressions.MapResponse(parent)¶

Bases: pyslet.qtiv2.expressions.Expression

This expression looks up the value of a response variable and then transforms it using the associated mapping, which must have been declared. The result is a single float:

<xsd:attributeGroup name="mapResponse.AttrGroup">
        <xsd:attribute name="identifier" type="identifier.Type" use="required"/>
</xsd:attributeGroup>

class pyslet.qtiv2.expressions.MapResponsePoint(parent)¶

Bases: pyslet.qtiv2.expressions.Expression

This expression looks up the value of a response variable that must be of base-type point, and transforms it using the associated areaMapping:

<xsd:attributeGroup name="mapResponsePoint.AttrGroup">
        <xsd:attribute name="identifier" type="identifier.Type" use="required"/>
</xsd:attributeGroup>

class pyslet.qtiv2.expressions.Null(parent, name=None)¶

Bases: pyslet.qtiv2.expressions.Expression

null is a simple expression that returns the NULL value - the null value is treated as if it is of any desired baseType

<xsd:complexType name="null.Type"/>

class pyslet.qtiv2.expressions.RandomInteger(parent)¶

Bases: pyslet.qtiv2.expressions.Expression

Selects a random integer from the specified range [min,max] satisfying min + step * n for some integer n:

<xsd:attributeGroup name="randomInteger.AttrGroup">
        <xsd:attribute name="min" type="integerOrTemplateRef.Type" use="required"/>
        <xsd:attribute name="max" type="integerOrTemplateRef.Type" use="required"/>
        <xsd:attribute name="step" type="integerOrTemplateRef.Type" use="optional"/>
</xsd:attributeGroup>

class pyslet.qtiv2.expressions.RandomFloat(parent)¶

Bases: pyslet.qtiv2.expressions.Expression

Selects a random float from the specified range [min,max]

<xsd:attributeGroup name="randomFloat.AttrGroup">
        <xsd:attribute name="min" type="floatOrTemplateRef.Type" use="required"/>
        <xsd:attribute name="max" type="floatOrTemplateRef.Type" use="required"/>
</xsd:attributeGroup>

Expressions Used only in Outcomes Processing¶

Operators¶

class pyslet.qtiv2.expressions.Multiple(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The multiple operator takes 0 or more sub-expressions all of which must have either single or multiple cardinality:

<xsd:group name="multiple.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Ordered(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The multiple operator takes 0 or more sub-expressions all of which must have either single or multiple cardinality:

<xsd:group name="ordered.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.ContainerSize(parent)¶

Bases: pyslet.qtiv2.expressions.UnaryOperator

The containerSize operator takes a sub-expression with any base-type and either multiple or ordered cardinality:

<xsd:group name="containerSize.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.IsNull(parent)¶

Bases: pyslet.qtiv2.expressions.UnaryOperator

The isNull operator takes a sub-expression with any base-type and cardinality

<xsd:group name="isNull.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Index(parent)¶

Bases: pyslet.qtiv2.expressions.UnaryOperator

The index operator takes a sub-expression with an ordered container value and any base-type

<xsd:attributeGroup name="index.AttrGroup">
        <xsd:attribute name="n" type="integer.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="index.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.FieldValue(parent)¶

Bases: pyslet.qtiv2.expressions.UnaryOperator

The field-value operator takes a sub-expression with a record container value. The result is the value of the field with the specified fieldIdentifier:

<xsd:attributeGroup name="fieldValue.AttrGroup">
        <xsd:attribute name="fieldIdentifier" type="identifier.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="fieldValue.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Random(parent)¶

Bases: pyslet.qtiv2.expressions.UnaryOperator

The random operator takes a sub-expression with a multiple or ordered container value and any base-type. The result is a single value randomly selected from the container:

<xsd:group name="random.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Member(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The member operator takes two sub-expressions which must both have the same base-type. The first sub-expression must have single cardinality and the second must be a multiple or ordered container. The result is a single boolean with a value of true if the value given by the first sub-expression is in the container defined by the second sub-expression:

<xsd:group name="member.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Delete(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The delete operator takes two sub-expressions which must both have the same base-type. The first sub-expression must have single cardinality and the second must be a multiple or ordered container. The result is a new container derived from the second sub-expression with all instances of the first sub-expression removed:

<xsd:group name="delete.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Contains(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The contains operator takes two sub-expressions which must both have the same base-type and cardinality – either multiple or ordered. The result is a single boolean with a value of true if the container given by the first sub-expression contains the value given by the second sub-expression and false if it doesn’t:

<xsd:group name="contains.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.SubString(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The substring operator takes two sub-expressions which must both have an effective base-type of string and single cardinality. The result is a single boolean with a value of true if the first expression is a substring of the second expression and false if it isn’t:

<xsd:attributeGroup name="substring.AttrGroup">
        <xsd:attribute name="caseSensitive" type="boolean.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="substring.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Not(parent)¶

Bases: pyslet.qtiv2.expressions.UnaryOperator

The not operator takes a single sub-expression with a base-type of boolean and single cardinality. The result is a single boolean with a value obtained by the logical negation of the sub-expression’s value:

<xsd:group name="not.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.And(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The and operator takes one or more sub-expressions each with a base-type of boolean and single cardinality. The result is a single boolean which is true if all sub-expressions are true and false if any of them are false:

<xsd:group name="and.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Or(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The or operator takes one or more sub-expressions each with a base-type of boolean and single cardinality. The result is a single boolean which is true if any of the sub-expressions are true and false if all of them are false:

<xsd:group name="or.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.AnyN(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The anyN operator takes one or more sub-expressions each with a base-type of boolean and single cardinality. The result is a single boolean which is true if at least min of the sub-expressions are true and at most max of the sub-expressions are true:

<xsd:attributeGroup name="anyN.AttrGroup">
        <xsd:attribute name="min" type="integerOrTemplateRef.Type" use="required"/>
        <xsd:attribute name="max" type="integerOrTemplateRef.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="anyN.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Match(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The match operator takes two sub-expressions which must both have the same base-type and cardinality. The result is a single boolean with a value of true if the two expressions represent the same value and false if they do not:

<xsd:group name="match.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.StringMatch(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The stringMatch operator takes two sub-expressions which must have single and a base-type of string. The result is a single boolean with a value of true if the two strings match:

<xsd:attributeGroup name="stringMatch.AttrGroup">
        <xsd:attribute name="caseSensitive" type="boolean.Type" use="required"/>
        <xsd:attribute name="substring" type="boolean.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="stringMatch.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.PatternMatch(parent)¶

Bases: pyslet.qtiv2.expressions.UnaryOperator

The patternMatch operator takes a sub-expression which must have single cardinality and a base-type of string. The result is a single boolean with a value of true if the sub-expression matches the regular expression given by pattern and false if it doesn’t:

<xsd:attributeGroup name="patternMatch.AttrGroup">
        <xsd:attribute name="pattern" type="stringOrTemplateRef.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="patternMatch.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Equal(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The equal operator takes two sub-expressions which must both have single cardinality and have a numerical base-type. The result is a single boolean with a value of true if the two expressions are numerically equal and false if they are not:

<xsd:attributeGroup name="equal.AttrGroup">
        <xsd:attribute name="toleranceMode" type="toleranceMode.Type" use="required"/>
        <xsd:attribute name="tolerance" use="optional">
                <xsd:simpleType>
                        <xsd:list itemType="floatOrTemplateRef.Type"/>
                </xsd:simpleType>
        </xsd:attribute>
        <xsd:attribute name="includeLowerBound" type="boolean.Type" use="optional"/>
        <xsd:attribute name="includeUpperBound" type="boolean.Type" use="optional"/>
</xsd:attributeGroup>

<xsd:group name="equal.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.ToleranceMode¶

Bases: pyslet.xsdatatypes20041028.Enumeration

When comparing two floating point numbers for equality it is often desirable to have a tolerance to ensure that spurious errors in scoring are not introduced by rounding errors. The tolerance mode determines whether the comparison is done exactly, using an absolute range or a relative range:

<xsd:simpleType name="toleranceMode.Type">
        <xsd:restriction base="xsd:NMTOKEN">
                <xsd:enumeration value="absolute"/>
                <xsd:enumeration value="exact"/>
                <xsd:enumeration value="relative"/>
        </xsd:restriction>
</xsd:simpleType>

Defines constants for the above modes. Usage example:

ToleranceMode.exact

The default value is exact:

ToleranceMode.DEFAULT == ToleranceMode.exact

For more methods see Enumeration

class pyslet.qtiv2.expressions.EqualRounded(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The equalRounded operator takes two sub-expressions which must both have single cardinality and have a numerical base-type. The result is a single boolean with a value of true if the two expressions are numerically equal after rounding and false if they are not:

<xsd:attributeGroup name="equalRounded.AttrGroup">
        <xsd:attribute name="roundingMode" type="roundingMode.Type" use="required"/>
        <xsd:attribute name="figures" type="integerOrTemplateRef.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="equalRounded.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.RoundingMode¶

Bases: pyslet.xsdatatypes20041028.Enumeration

Numbers are rounded to a given number of significantFigures or decimalPlaces:

<xsd:simpleType name="roundingMode.Type">
        <xsd:restriction base="xsd:NMTOKEN">
                <xsd:enumeration value="decimalPlaces"/>
                <xsd:enumeration value="significantFigures"/>
        </xsd:restriction>
</xsd:simpleType>

Defines constants for the above modes. Usage example:

RoundingMode.decimalPlaces

The default value is significantFigures:

RoundingMode.DEFAULT == RoundingMode.significantFigures

For more methods see Enumeration

class pyslet.qtiv2.expressions.Inside(parent)¶

Bases: pyslet.qtiv2.expressions.UnaryOperator, pyslet.qtiv2.core.ShapeElementMixin

The inside operator takes a single sub-expression which must have a baseType of point. The result is a single boolean with a value of true if the given point is inside the area defined by shape and coords. If the sub-expression is a container the result is true if any of the points are inside the area:

<xsd:attributeGroup name="inside.AttrGroup">
        <xsd:attribute name="shape" type="shape.Type" use="required"/>
        <xsd:attribute name="coords" type="coords.Type" use="required"/>
</xsd:attributeGroup>

<xsd:group name="inside.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.LT(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The lt operator takes two sub-expressions which must both have single cardinality and have a numerical base-type. The result is a single boolean with a value of true if the first expression is numerically less than the second and false if it is greater than or equal to the second:

<xsd:group name="lt.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.GT(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The gt operator takes two sub-expressions which must both have single cardinality and have a numerical base-type. The result is a single boolean with a value of true if the first expression is numerically greater than the second and false if it is less than or equal to the second:

<xsd:group name="gt.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.LTE(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The lte operator takes two sub-expressions which must both have single cardinality and have a numerical base-type. The result is a single boolean with a value of true if the first expression is numerically less than or equal to the second and false if it is greater than the second:

<xsd:group name="lte.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.GTE(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The gte operator takes two sub-expressions which must both have single cardinality and have a numerical base-type. The result is a single boolean with a value of true if the first expression is numerically less than or equal to the second and false if it is greater than the second:

<xsd:group name="durationGTE.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.DurationLT(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The durationLT operator takes two sub-expressions which must both have single cardinality and base-type duration. The result is a single boolean with a value of true if the first duration is shorter than the second and false if it is longer than (or equal) to the second:

<xsd:group name="durationLT.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.DurationGTE(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The durationGTE operator takes two sub-expressions which must both have single cardinality and base-type duration. The result is a single boolean with a value of true if the first duration is longer (or equal, within the limits imposed by truncation) than the second and false if it is shorter than the second:

<xsd:group name="durationGTE.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Sum(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The sum operator takes 1 or more sub-expressions which all have single cardinality and have numerical base-types. The result is a single float or, if all sub-expressions are of integer type, a single integer that corresponds to the sum of the numerical values of the sub-expressions:

<xsd:group name="sum.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Product(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The product operator takes 1 or more sub-expressions which all have single cardinality and have numerical base-types. The result is a single float or, if all sub-expressions are of integer type, a single integer that corresponds to the product of the numerical values of the sub-expressions:

<xsd:group name="product.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Subtract(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The subtract operator takes 2 sub-expressions which all have single cardinality and numerical base-types. The result is a single float or, if both sub-expressions are of integer type, a single integer that corresponds to the first value minus the second:

<xsd:group name="subtract.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Divide(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The divide operator takes 2 sub-expressions which both have single cardinality and numerical base-types. The result is a single float that corresponds to the first expression divided by the second expression:

<xsd:group name="divide.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Power(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The power operator takes 2 sub-expression which both have single cardinality and numerical base-types. The result is a single float that corresponds to the first expression raised to the power of the second:

<xsd:group name="power.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.IntegerDivide(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The integer divide operator takes 2 sub-expressions which both have single cardinality and base-type integer. The result is the single integer that corresponds to the first expression (x) divided by the second expression (y) rounded down to the greatest integer (i) such that i<=(x/y):

<xsd:group name="integerDivide.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.IntegerModulus(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The integer modulus operator takes 2 sub-expressions which both have single cardinality and base-type integer. The result is the single integer that corresponds to the remainder when the first expression (x) is divided by the second expression (y). If z is the result of the corresponding integerDivide operator then the result is x-z*y:

<xsd:group name="integerModulus.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="2" maxOccurs="2"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Truncate(parent)¶

Bases: pyslet.qtiv2.expressions.UnaryOperator

The truncate operator takes a single sub-expression which must have single cardinality and base-type float. The result is a value of base-type integer formed by truncating the value of the sub-expression towards zero:

<xsd:group name="truncate.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.Round(parent)¶

Bases: pyslet.qtiv2.expressions.UnaryOperator

The round operator takes a single sub-expression which must have single cardinality and base-type float. The result is a value of base-type integer formed by rounding the value of the sub-expression. The result is the integer n for all input values in the range [n-0.5,n+0.5):

<xsd:group name="round.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.IntegerToFloat(parent)¶

Bases: pyslet.qtiv2.expressions.UnaryOperator

The integer to float conversion operator takes a single sub-expression which must have single cardinality and base-type integer. The result is a value of base type float with the same numeric value:

<xsd:group name="integerToFloat.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="1" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

class pyslet.qtiv2.expressions.CustomOperator(parent)¶

Bases: pyslet.qtiv2.expressions.NOperator

The custom operator provides an extension mechanism for defining operations not currently supported by this specification:

<xsd:attributeGroup name="customOperator.AttrGroup">
        <xsd:attribute name="class" type="identifier.Type" use="optional"/>
        <xsd:attribute name="definition" type="uri.Type" use="optional"/>
        <xsd:anyAttribute namespace="##other"/>
</xsd:attributeGroup>

<xsd:group name="customOperator.ContentGroup">
        <xsd:sequence>
                <xsd:group ref="expression.ElementGroup" minOccurs="0" maxOccurs="unbounded"/>
        </xsd:sequence>
</xsd:group>

Core Types and Utilities¶

This module contains a number core classes used to support the standard.

Constants¶

pyslet.qtiv2.core.IMSQTI_NAMESPACE = 'http://www.imsglobal.org/xsd/imsqti_v2p1'¶: The namespace used to recognise elements in XML documents.

pyslet.qtiv2.core.IMSQTI_SCHEMALOCATION = 'http://www.imsglobal.org/xsd/imsqti_v2p1.xsd'¶: The location of the QTI 2.1 schema file on the IMS website.

pyslet.qtiv2.core.IMSQTI_ITEM_RESOURCETYPE = 'imsqti_item_xmlv2p1'¶: The resource type to use for the QTI 2.1 items when added to content packages.

XML Basics¶

class pyslet.qtiv2.core.QTIElement(parent, name=None)¶

Bases: pyslet.xmlnames20091208.XMLNSElement

Basic element to represent all QTI elements

AddToCPResource(cp, resource, beenThere)¶

We need to add any files with URL’s in the local file system to the content package.

beenThere is a dictionary we use for mapping URLs to File objects so that we don’t keep adding the same linked resource multiple times.

This implementation is a little more horrid, we avoid circular module references by playing dumb about our children. HTML doesn’t actually know anything about QTI even though QTI wants to define children for some XHTML elements so we pass the call only to “CP-Aware” elements.

class pyslet.qtiv2.core.QTIDocument(**args)¶

Bases: pyslet.xmlnames20091208.XMLNSDocument

Used to represent all documents representing information from the QTI v2 specification.

AddToContentPackage(cp, metadata, dName=None)¶

Copies this QTI document into a content package and returns the resource ID used.

An optional directory name can be specified in which to put the resource files.

Exceptions¶

class pyslet.qtiv2.core.QTIError¶

Bases: exceptions.Exception

Abstract class used for all QTI v2 exceptions.

class pyslet.qtiv2.core.DeclarationError¶

Bases: pyslet.qtiv2.core.QTIError

Error raised when a variable declaration is invalid.

class pyslet.qtiv2.core.ProcessingError¶

Bases: pyslet.qtiv2.core.QTIError

Error raised when an invalid processing element is encountered.

class pyslet.qtiv2.core.SelectionError¶

Bases: pyslet.qtiv2.core.QTIError

Error raised when there is a problem with creating test forms.

Basic Data Types¶

Basic data types in QTI v2 are a mixture of custom types and basic types defined externally, for example, by XMLSchema.

The external types used are:

boolean

Represented by python’s boolean values True and False. See DecodeBoolean() and EncodeBoolean()

coords

Defined as part of support for HTML. See Coords

date

Although QTI draws on the definitions in XML schema it restricts values to those from the nontimezoned timeline. This restriction is effectively implemented in the basic Date class.

datetime:

See DecodeDateTime() and EncodeDateTime()

duration:

Earlier versions of QTI drew on the ISO8601 representation of duration but QTI v2 simplifies this with a basic representation in seconds bound to XML Schema’s double type which we, in turn, represent with python’s float. See DecodeDouble() and EncodeDouble()

float:

implemented by python’s float. Note that this is defined as having “machine-level double precision” and the python specification goes on to warn that “You are at the mercy of the underlying machine architecture”. See DecodeDouble() and EncodeDouble()

identifier:

represented by python’s (unicode) string. The type is effectively just the NCName from the XML namespace specification. See pyslet.xmlnames20091208.IsValidNCName().

pyslet.qtiv2.core.ValidateIdentifier(value, prefix='_')¶

Decodes an identifier from a string:

<xsd:simpleType name="identifier.Type">
        <xsd:restriction base="xsd:NCName"/>
</xsd:simpleType>

This function takes a string that is supposed to match the production for NCName in XML and forces it to comply by replacing illegal characters with ‘_’, except the ‘:’ which is replaced with a hyphen for compatibility with previous versions of the QTI migraiton script. If name starts with a valid name character but not a valid name start character, it is prefixed with ‘_’ too, but the prefix string used can be overridden.

integer:

XML schema’s integer, implemented by python’s integer. See DecodeInteger() and EncodeInteger()

language:

Currently implemented as a simple python string.

length:

Defined as part of support for HTML. See LengthType

mimeType:

Currently implemented as a simple python string

string:

XML schema string becomes python’s unicode string

string256:

Length restriction not yet implemented, see string above.

styleclass:

Inherited from HTML, implemented with a simple (unicode) string.

uri:

In some instances this is implemented as a simple (unicode) string, for example, in cases where a URI is being used as global identifier. In contexts where the URI will need to be interpreted it is implemented with instances of pyslet.rfc2396.URI.

QTI-specific types:

class pyslet.qtiv2.core.Orientation¶

Bases: pyslet.xsdatatypes20041028.Enumeration

Orientation attribute values provide a hint to rendering systems that an element has an inherent vertical or horizontal interpretation:

<xsd:simpleType name="orientation.Type">
        <xsd:restriction base="xsd:NMTOKEN">
                <xsd:enumeration value="horizontal"/>
                <xsd:enumeration value="vertical"/>
        </xsd:restriction>
</xsd:simpleType>

Defines constants for the above orientations. Usage example:

Orientation.horizontal

Note that:

Orientation.DEFAULT == None

For more methods see Enumeration

class pyslet.qtiv2.core.Shape¶

Bases: pyslet.xsdatatypes20041028.Enumeration

A value of a shape is always accompanied by coordinates and an associated image which provides a context for interpreting them:

<xsd:simpleType name="shape.Type">
        <xsd:restriction base="xsd:NMTOKEN">
                <xsd:enumeration value="circle"/>
                <xsd:enumeration value="default"/>
                <xsd:enumeration value="ellipse"/>
                <xsd:enumeration value="poly"/>
                <xsd:enumeration value="rect"/>
        </xsd:restriction>
</xsd:simpleType>

Defines constants for the above types of Shape. Usage example:

Shape.circle

Note that:

Shape.DEFAULT == Shape.default

For more methods see Enumeration

class pyslet.qtiv2.core.ShowHide¶

Bases: pyslet.xsdatatypes20041028.Enumeration

Used to control content visibility with variables

<xsd:simpleType name="showHide.Type">
        <xsd:restriction base="xsd:NMTOKEN">
                <xsd:enumeration value="hide"/>
                <xsd:enumeration value="show"/>
        </xsd:restriction>
</xsd:simpleType>

Note that ShowHide.DEFAULT == ShowHide.show

class pyslet.qtiv2.core.View¶

Bases: pyslet.xsdatatypes20041028.Enumeration

Used to represent roles when restricting view:

<xsd:simpleType name="view.Type">
        <xsd:restriction base="xsd:NMTOKEN">
                <xsd:enumeration value="author"/>
                <xsd:enumeration value="candidate"/>
                <xsd:enumeration value="proctor"/>
                <xsd:enumeration value="scorer"/>
                <xsd:enumeration value="testConstructor"/>
                <xsd:enumeration value="tutor"/>
        </xsd:restriction>
</xsd:simpleType>

Defines constants for the above views. Usage example:

View.candidate

There is no default view. Views are represented in XML as space-separated lists of values. Typical usage:

view=View.DecodeValueDict("tutor scorer")
# returns...
{ View.tutor:'tutor', View.scorer:'scorer' }
View.EncodeValueDict(view)
# returns...
"scorer tutor"

For more methods see Enumeration

The QTI specification lists valueType as a basic data type. In pyslet this is implemented as a core part of the processing model. See pyslet.qtiv2.variables.Value for details.

Meta-data and Usage Data¶

class pyslet.qtiv2.metadata.QTIMetadata(parent)¶

Bases: pyslet.qtiv2.core.QTIElement

A new category of meta-data for the recording of QTI specific information. It is designed to be treated as an additional top-level category to augment the LOM profile:

<xsd:group name="qtiMetadata.ContentGroup">
        <xsd:sequence>
                <xsd:element ref="itemTemplate" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="timeDependent" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="composite" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="interactionType" minOccurs="0" maxOccurs="unbounded"/>
                <xsd:element ref="feedbackType" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="solutionAvailable" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="toolName" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="toolVersion" minOccurs="0" maxOccurs="1"/>
                <xsd:element ref="toolVendor" minOccurs="0" maxOccurs="1"/>
        </xsd:sequence>
</xsd:group>

IMS Basic Learning Tools Interoperability (version 1.0)¶

The IMS Basic Learning Tools Interoperability (BLTI) specification was released in 2010. The purpose of the specification is to provide a link between tool consumers (such as Learning Management Systems and portals) and Tools (such as specialist assessment management systems). Official information about the specification is available from the IMS GLC: http://www.imsglobal.org/lti/

This module requires the oauth module to be installed. The oauth module is available from http://pypi.python.org/pypi/oauth/1.0.1

This module is written from the point of view of the Tool Provider. Consumers are modeled by the BLTIConsumer class which does nothing more than implement the recommended algorithm for checking the Nonces as recommended in the specification.

Typical usage would be in an HTTP request handler, this code is modelled on the code provided with the oauth module:

import pyslet.imsbltiv1p0 as blti

class BLTIRequestHandler(BaseHTTPRequestHandler):
        def __init__(self, *args, **kwargs):
                self.tp=blti.BLTIToolProvider()
                self.tp.LoadFromFile(open('consumers.txt'))
                BaseHTTPRequestHandler.__init__(self, *args, **kwargs)

        def do_GET(self):
                postdata = None
                if self.command=='POST':
                        try:
                                length = int(self.headers.getheader('content-length'))
                                postdata = self.rfile.read(length)
                        except:
                                pass
                parts=urlparse.urlsplit(self.path)
                if not parts.scheme:
                        scheme='http' # change here for https
                        if not parts.netloc:
                                netloc=self.headers['Host']
                        else:
                                netloc=parts.netloc
                        url=urlparse.urlunsplit([scheme,netloc]+list(parts[2:]))
                else:
                        url=self.path
                try:
                        consumer,params=self.tp.Launch(self.command, url, headers=self.headers, query_string=postdata)
                        self.ReturnPage("LTI Provider authorized request with parameters: \n%s"%str(params))
                except blti.BLTIOAuthParameterError:
                        self.ReturnUnauthorized("Missing or incomplete authentication parameters in request")
                except blti.BLTIAuthenticationError, err:
                        self.ReturnUnauhtorized("Access denied: "+str(err))
                return

        def do_POST(self):
                return self.do_GET()

        def ReturnPage(self,msg):
                self.send_response(200,'OK')
                self.send_header('Content-Type','text/plain')
                self.send_header('Content-Length',str(len(msg)))
                self.end_headers()
                self.wfile.write(msg)

        def ReturnUnauthorized(self,msg):
                self.send_response(403,"Unauthorized")
                self.end_headers()
                self.wfile.write(msg)

Reference¶

class pyslet.imsbltiv1p0.BLTIError¶

Bases: exceptions.Exception

Base class for BLTI errors.

class pyslet.imsbltiv1p0.BLTIAuthenticationError¶

Bases: pyslet.imsbltiv1p0.BLTIError

Error raised when a launch request cannot be authorized.

Supporting Standards¶

The section contains modules that implement supporting specifications that are not specific, or at least have no special interest in the domains of Learning, Education and Training.

Contents:

The Open Data Protocol (OData)¶

This sub-package defines functions and classes for working with OData, a data access protocol based on Atom and Atom Pub: http://www.odata.org/

This sub-package only deals with version 2 of the protocol at the moment.

OData is not an essential part of supporting the Standards for Learning, Education and Training (SLET) that gives pyslet its name, though I have actively promoted its use in these communities. As technical standards move towards using REST-ful web services it makes sense to converge around some common patterns for common use cases. Many of the protocols now being worked on are much more like basic data-access layers spread over the web between two co-operating systems. HTTP on its own is often good enough for these applications but when the data lends itself to tabular representations I think the OData standard is the best protocol available.

The purpose of this group of modules is to make is easy to use the conventions of the OData protocol as a general purpose data-access layer (DAL) for Python applications. To get started, look at the Data Consumers section which gives a high-level overview of the API with examples that use Microsoft’s Northwind data-service.

If you are interested in writing an OData provider, or you simply want to use these classes to implement a data access layer for your own application then look in OData Providers.

Data Consumers¶

Warning: the OData client doesn’t support certificate validation when accessing servers through https URLs. This feature is coming soon...

Introduction¶

Let’s start with a simple illustration of how to consume data using the DAL API by walking through the use of the OData client.

The client implementation uses Python’s logging module to provide logging, when learning about the client it may help to turn logging up to “INFO” as it makes it clearer what the client is doing. “DEBUG” would show exactly what is passing over the wire.:

>>> import logging
>>> logging.basicConfig(level=logging.INFO)

To create a new client simply instantiate a Client object. You can pass the URL of the service root you wish to connect to directly to the constructor which will then call the service to download the list of feeds and the metadata document from which it will set the Client.model.

>>> from pyslet.odata2.client import Client
>>> c = Client("http://services.odata.org/V2/Northwind/Northwind.svc/")
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/ HTTP/1.1
INFO:root:Finished Response, status 200
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/$metadata HTTP/1.1
INFO:root:Finished Response, status 200
>>>

The Client.feeds attribute is a dictionary mapping the exposed feeds (by name) onto EntitySet instances. This makes it easy to open the feeds as EDM collections. In your code you’d typically use the with statement when opening the collection but for clarity we’ll continue on the python command line:

>>> products = c.feeds['Products'].OpenCollection()
>>> for p in products: print p
...
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products HTTP/1.1
INFO:root:Finished Response, status 200
1
2
3
... [and so on]
...
20
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products?$skiptoken=20 HTTP/1.1
INFO:root:Finished Response, status 200
21
22
23
... [and so on]
...
76
77
>>>

Note that products behaves like a dictionary, iterating through it iterates through the keys in the dictionary. In this case these are the keys of the entities in the collection of products. Notice that the client logs several requests to the server interspersed with the printed output. Subsequent requests use $skiptoken because the server is limiting the maximum page size. These calls are made as you iterate through the collection allowing you to iterate through very large collections.

The keys alone are of limited interest, let’s try a similar loop but this time we’ll print the product names as well:

>>> for k, p in products.iteritems(): print k, p['ProductName'].value
...
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products HTTP/1.1
INFO:root:Finished Response, status 200
1 Chai
2 Chang
3 Aniseed Syrup
...
...
20 Sir Rodney's Marmalade
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products?$skiptoken=20 HTTP/1.1
INFO:root:Finished Response, status 200
21 Sir Rodney's Scones
22 Gustaf's Knäckebröd
23 Tunnbröd
...
...
76 Lakkalikööri
77 Original Frankfurter grüne Soße
>>>

Sir Rodney’s Scones sound interesting, we can grab an individual record in the usual way:

>>> scones = products[21]
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products(21) HTTP/1.1
INFO:root:Finished Response, status 200
>>> for k, v in scones.data_items(): print k, v.value
...
ProductID 21
ProductName Sir Rodney's Scones
SupplierID 8
CategoryID 3
QuantityPerUnit 24 pkgs. x 4 pieces
UnitPrice 10.0000
UnitsInStock 3
UnitsOnOrder 40
ReorderLevel 5
Discontinued False
>>>

Well, I’ve simply got to have some of these, let’s use one of the navigation properties to load information about the supplier:

>>> supplier = scones['Supplier'].GetEntity()
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products(21)/Supplier HTTP/1.1
INFO:root:Finished Response, status 200
>>> for k, v in supplier.data_items(): print k, v.value
...
SupplierID 8
CompanyName Specialty Biscuits, Ltd.
ContactName Peter Wilson
ContactTitle Sales Representative
Address 29 King's Way
City Manchester
Region None
PostalCode M14 GSD
Country UK
Phone (161) 555-4448
Fax None
HomePage None

Attempting to load a non existent entity results in a KeyError of course:

>>> p = products[211]
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products(211) HTTP/1.1
INFO:root:Finished Response, status 404
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/pyslet/odata2/client.py", line 165, in __getitem__
        raise KeyError(key)
KeyError: 211
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/pyslet/odata2/client.py", line 165, in __getitem__
        raise KeyError(key)
KeyError: 211

Finally, when we’re done, it is a good idea to close the open collection:

>>> products.close()

The Data Access Layer in Depth¶

In the introduction we created an OData Client object using a URL, but in general the way you connect to a data service will vary depending on the implementation. The Client class itself isn’t actually part of the DAL API itself.

The API starts with a model of the data service. The model is typically parsed from an XML file. For the OData client the XML file is obtained from the service’s $metadata URL. Here’s an extract from the Northwind $metadata file showing the definition of the data service, I’ve removed the XML namespace definitions for brevity:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<edmx:Edmx Version="1.0">
        <edmx:DataServices m:DataServiceVersion="1.0">
        <Schema Namespace="NorthwindModel">
                <EntityType Name="Category">
                        <!-- rest of the definitions go here... -->

Each element is represented by an object in Pyslet, the starting point for the API is the DataServices object. A DataServices object can contain multiple Schema elements, which in turn can contain multiple EntityContainer elements which in turn can contain multiple EntitySet elements. The following diagram illustrates these relationships and compares them with approximate equivalent concepts in a typical SQL-scenario.

In the OData client example we used a short-cut to get to the EntitySet objects we were interested in by using the feeds property of the client itself. However, we could have used the model directly as follows, continuing with the same session:

>>> c.model
<pyslet.odata2.metadata.Edmx object at 0x10140a9d0>
>>> c.model.DataServices
<pyslet.odata2.metadata.DataServices object at 0x107fdb990>
>>> for s in c.model.DataServices.Schema: print s.name
...
NorthwindModel
ODataWeb.Northwind.Model
>>> c.model.DataServices['ODataWeb.Northwind.Model']
<pyslet.odata2.csdl.Schema object at 0x10800cd90>
>>> c.model.DataServices['ODataWeb.Northwind.Model']['NorthwindEntities']
<pyslet.odata2.metadata.EntityContainer object at 0x10800cdd0>
>>> c.model.DataServices['ODataWeb.Northwind.Model']['NorthwindEntities']['Products']
<pyslet.odata2.metadata.EntitySet object at 0x10800f150>
>>> c.feeds['Products']
<pyslet.odata2.metadata.EntitySet object at 0x10800f150>

As you can see, the same EntitySet object can be obtained by looking it up in the parent container which behaves like a dictionary, this in turn can be looked up in the parent Schema which in turn can be looked up in the DataServices enclosing object. Elements of the model also support deep references using dot-concatenation of names which makes the code easier to read:

>>> print c.model.DataServices['ODataWeb.Northwind.Model']['NorthwindEntities']['Products'].GetFQName()
ODataWeb.Northwind.Model.NorthwindEntities.Products
>>> c.model.DataServices['ODataWeb.Northwind.Model.NorthwindEntities.Products']
<pyslet.odata2.metadata.EntitySet object at 0x10800f150>

When writing an application that would normally use a single database you should pass an EntityCollection object to it as a data source rather than the DataServices ancestor. It is best not to pass an implementation-specific class like the OData Client as that will make the application dependent on a particular type of data source.

Entity Sets¶

The following attributes are useful for consumers of the API (and should be treated as read only)

name

The name of the entity set

entityTypeName

The name of the entity set’s EntityType

entityType

The EntityType object that defines the properties for entities in this set.

keys

A list of the names of the keys for this EntitySet. For example:

>>> print products.keys
[u'ProductID']

For entity types with compound keys this list will contain multiple items of course.

The following methods are useful for consumers of the API.

GetFQName()

Returns the fully qualified name of the entity set, suitable for looking up the entity set in the enclosing DataServices object.

GetLocation()

Returns a pyslet.rfc2396.URI object that represents this entity set:

>>> print products.GetLocation()
http://services.odata.org/V2/Northwind/Northwind.svc/Products

(If there is no base URL available this will be a relative URI.)

OpenCollection()

Returns a pyslet.odata2.csdl.EntityCollection object that can be used to access the entities in the set.

NavigationTarget()

Returns the target entity set of a named navigation property.

NavigationMultiplicity()

Returns a tuple of multiplicity constants for the named navigation property. Constants for these values are defined in pyslet.odata2.csdl.Multiplicity, for example:

>>> from pyslet.odata2.csdl import Multiplicity, EncodeMultiplicity
>>> print Multiplicity.ZeroToOne, Multiplicity.One, Multiplicity.Many
0 1 2
>>> products.NavigationMultiplicity('Supplier')
(2, 0)
>>> map(lambda x:EncodeMultiplicity(x),products.NavigationMultiplicity('Supplier'))
['*', '0..1']

IsEntityCollection()

Returns True if the named navigation property points to a collection of entities or a single entity. In Pyslet, you can treat all navigation properties as collections. In the above example the collection of Supplier entities obtained by following the ‘Supplier’ navigation property of a Product entity will have at most 1 member.

Entity Collections¶

To continue with database analogy above, if EntitySets are like SQL Tables EntityCollections are somewhat like the database cursors that you use to actually read data - the difference is that EntityCollections can only read entities from a single EntitySet.

An EntityCollection may consume physical resources (like a database connection) and so must be closed with its close() method when you’re done. They support the context manager protocol to make this easier so you can use them in with statements to make clean-up easier:

with c.feeds['Products'].OpenCollection() as products:
        if 42 in products:
                print "Found it!"

The close method is called automatically when the with statement exits.

Entity collections also behave like a python dictionary of Entity instances keyed on a value representing the Entity’s key property or properties. The keys are either single values (as in the above code example) or tuples in the case of compound keys. The order of the values in the tuple is taken from the order of the PropertyRef definitions in the model.

There are two ways to obtain an EntityCollection object. You can open an entity set directly or you can open a collection by navigating from a specific entity through a named navigation property. Although dictionary-like there are some differences with true dictionaries.

When you have opened a collection from the base entity set the following rules apply:

collection[key]: Returns a new Entity instance by looking up the key in the collection. As a result, subsequent calls will return a different object, but with the same key!
collection[key] = new_entity: For an existing entity this is essentially a no-operation. This form of assignment cannot be used to create a new entity in the collection because the act of inserting the entity may alter its key (for example, when the entity set represents a database table with an auto-generated primary key). See below for information on how to create and update entities.
del collection[key]: In contrast, del will remove an entity from the collection completely.

When an EntityCollection represents a collection of entities obtained by navigation then these rules are updated as follows:

collection[key]: Normally returns a new Entity instance by looking up the key in the collection but when the navigation property has been expanded it will return a cached Entity (so subsequent calls will return the same object without looking up the key in the data source again).
collection[key]=existingEntity: Provided that key is the key of existingEntity this will add an existing entity to this collection, effectively creating a link from the entity you were navigating from to an existing entity.
del collection[key]: Removes the entity with key from this collection. The entity is not deleted from its EntitySet, is merely unlinked from the entity you were navigating from.

The following attribute is useful for consumers of the API (and should be treated as read only)

entity_set: The EntitySet of this collection. In the case of a collection opened through navigation this is the base entity set.

In addition to all the usual dictionary methods like len, itervalues and so on, the following methods are useful for consumers of the API:

GetLocation()

Returns a pyslet.rfc2396.URI object that represents this entity collection.

GetTitle()

Returns a user-friendly title to represent this entity collection.

new_entity()

Creates a new entity suitable for inserting into this collection. The entity does not exist until it is inserted with insert_entity.

CopyEntity()

Creates a new entity by copying all non-key properties from another entity. The entity does not exist until it is inserted with insert_entity.

insert_entity()

Inserts an entity previously created by new_entity or CopyEntity. When inserting an entity any active filter is ignored.

Warning: an active filter may result in a paradoxical KeyError:

import pyslet.odata2.core as core
with people.OpenCollection() as collection:
        collection.set_filter(
            core.CommonExpression.from_str("startswith(Name,'D')"))
        new_entity = collection.new_entity()
        new_entity['Key'].set_from_value(1)
        new_entity['Name'].set_from_value(u"Steve")
        collection.insert_entity(new_entity)
        # new_entity now exists in the base collection but...
        e1 = collection[1]
        # ...raises KeyError as new_entity did not match the filter!

It is recommended that collections used to insert entities are not filtered.

update_entity()

Updates an existing entity following changes to the Entity’s values. You can’t update the values of key properties. To change the key you will need to create a new entity with CopyEntity, insert the new entity and then remove the old one. Like insert_entity, the current filter is ignored.

set_page()

Sets the top and skip values for this collection, equivalent to the $top and $skip options in OData. This value only affects calls to iterpage. See Paging for more information.

iterpage()

Iterates through a subset of the entities returned by itervalues defined by the top and skip values. See Paging for more information.

set_filter()

Sets the filter for this collection, equivalent to the $filter option in OData. Once set this value effects all future entities returned from the collection (with the exception of new_entity). See Filtering Collections for more information.

set_orderby()

Sets the filter for this collection, equivalent to the $orderby option in OData. Once set this value effects all future iterations through the collection. See Ordering Collections for more information.

Expand()

Sets expand and select options for this collection, equivalent to the $expand and $select system query options in OData. Once set these values effect all future entities returned from the collection (with the exception of new_entity). See Expand and Select for more information.

Paging¶

Supported from build 0.4.20140215 onwards

The $top/$skip options in OData are a useful way to restrict the amount of data that an OData server returns. The collection dictionary always behaves as if it contains all entities so the value returned by len doesn’t change if you set top and skip values and nor does the set of entities returned by itervalues and similar methods.

In most cases, the server will impose a reasonable maximum on each request using server-enforced paging. However, you may wish to set a smaller top value or simply have more control over the automatic paging implemented by the default iterators.

To iterate through a single page of entities you’ll start by using the the set_page() method to specify values for top and, optinally, skip. You must then use the iterpage() method to iterate through the entities in just that page. The set_next boolean parameter indicates whether or not the next call to iterpage iterates over the same page or the next page of the collection.

To continue the example above, in which products is an open collection from the Northwind data service:

>>> products.set_page(5,50)
>>> for p in products.iterpage(True): print p.key(), p['ProductName'].value
...
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products?$skip=50&$top=5 HTTP/1.1
INFO:root:Finished Response, status 200
51 Manjimup Dried Apples
52 Filo Mix
53 Perth Pasties
54 Tourtière
55 Pâté chinois
>>> for p in products.iterpage(True): print p.key(), p['ProductName'].value
...
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products?$skip=55&$top=5 HTTP/1.1
INFO:root:Finished Response, status 200
56 Gnocchi di nonna Alice
57 Ravioli Angelo
58 Escargots de Bourgogne
59 Raclette Courdavault
60 Camembert Pierrot

In some cases, the server will restrict the page size and fewer entities will be returned than expected, in these cases the skiptoken is used automatically when the next page is requested:

>>> products.set_page(30, 50)
>>> for p in products.iterpage(True): print p.key(), p['ProductName'].value
...
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products?$skip=50&$top=30 HTTP/1.1
INFO:root:Finished Response, status 200
51 Manjimup Dried Apples
52 Filo Mix
53 Perth Pasties
... [and so on]
...
69 Gudbrandsdalsost
70 Outback Lager
>>> for p in products.iterpage(True): print p.key(), p['ProductName'].value
...
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products?$top=30&$skiptoken=70 HTTP/1.1
INFO:root:Finished Response, status 200
71 Flotemysost
72 Mozzarella di Giovanni
73 Röd Kaviar
74 Longlife Tofu
75 Rhönbräu Klosterbier
76 Lakkalikööri
77 Original Frankfurter grüne Soße

Filtering Collections¶

By default, an entity collection contains all items in the entity set or, if the collection was obtained by navigation, all items linked to the entity by the property being navigated. Filtering a collection (potentially) selects a sub-set of the these entities based on a filter expression.

Filter expressions are set using the set_filter() method of the collection. Once a filter is set, the dictionary methods, and iterpage, will only return entities that match the filter.

The easiest way to set a filter is to compile one directly from a string representation using OData’s query language. For example:

>>> import pyslet.odata2.core as core
>>> filter = core.CommonExpression.from_str("substringof('one',ProductName)")
>>> products.set_filter(filter)
>>> for p in products.itervalues(): print p.key(), p['ProductName'].value
...
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products?$filter=substringof('one'%2CProductName) HTTP/1.1
INFO:root:Finished Response, status 200
21 Sir Rodney's Scones
32 Mascarpone Fabioli

To remove a filter, set the filter expression to None:

>>> products.set_filter(None)

Ordering Collections¶

Like OData and python dictionaries, this API does not specify a default order in which entities will be returned by the iterators. However, unlike python dictionaries you can control this order using an orderby option.

OrderBy expressions are set using the set_orderby() method of the collection. Once an order by expression is set, the dictionary methods, and iterpage, will return entities in the order specified.

The easiest way to define an ordering is to compile one directly from a string representation using OData’s query language. For example:

>>> ordering=core.CommonExpression.OrderByFromString("ProductName desc")
>>> products.set_orderby(ordering)
>>> for p in products.itervalues(): print p.key(), p['ProductName'].value
...
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products?$orderby=ProductName%20desc HTTP/1.1
INFO:root:Finished Response, status 200
47 Zaanse koeken
64 Wimmers gute Semmelknödel
63 Vegie-spread
50 Valkoinen suklaa
7 Uncle Bob's Organic Dried Pears
23 Tunnbröd
... [and so on]
...
56 Gnocchi di nonna Alice
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products?$orderby=ProductName%20desc&$skiptoken='Gnocchi%20di%20nonna%20Alice',56 HTTP/1.1
INFO:root:Finished Response, status 200
15 Genen Shouyu
33 Geitost
71 Flotemysost
... [and so on]
...
40 Boston Crab Meat
3 Aniseed Syrup
17 Alice Mutton

To remove an ordering, set the orderby expression to None:

>>> products.Orderby(None)

Expand and Select¶

Expansion and selection are two interrelated concepts in the API. Expansion allows you to follow specified navigation properties retrieving the entities they link to in the same way that simple and complex property values are retrieved.

Expand options are represented by nested dictionaries of strings. For example, to expand the Supplier navigation property of Products you would use a dictionary like this:

expansion={'Supplier':None}

The value in the dictionary is either None, indicating no further expansion, or another dictionary specifying the expansion to apply to any linked Suppliers:

>>> products.Expand({'Supplier':None}, None)
>>> scones = products[21]
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products(21)?$expand=Supplier HTTP/1.1
INFO:root:Finished Response, status 200
>>> supplier=scones['Supplier'].GetEntity()
>>> for k, v in supplier.data_items(): print k, v.value
...
SupplierID 8
CompanyName Specialty Biscuits, Ltd.
ContactName Peter Wilson
ContactTitle Sales Representative
Address 29 King's Way
City Manchester
Region None
PostalCode M14 GSD
Country UK
Phone (161) 555-4448
Fax None
HomePage None

A critical point to note is that applying an expansion to a collection means that linked entities are retrieved at the same time as the entity they are linked to and cached. In the example above, the GetEntity call does not generate a call to the server. Compare this with the same code executed without the expansion:

>>> products.Expand(None, None)
>>> scones = products[21]
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products(21) HTTP/1.1
INFO:root:Finished Response, status 200
>>> supplier = scones['Supplier'].GetEntity()
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products(21)/Supplier HTTP/1.1
INFO:root:Finished Response, status 200

The select option complements expansion, narrowing down the simple and complex properties that are retrieved from the data source. You specify a select option in a similar way, using nested dictionaries. Simple and complex properties must always map to None, for a more complex example with navigation properties see below. Suppose we are only interested in the product name:

>>> products.Expand(None, {'ProductName':None})
>>> scones = products[21]
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products(21)?$select=ProductID%2CProductName HTTP/1.1
INFO:root:Finished Response, status 200
>>> for k, v in scones.data_items(): print k, v.value
...
ProductID 21
ProductName Sir Rodney's Scones
SupplierID None
CategoryID None
QuantityPerUnit None
UnitPrice None
UnitsInStock None
UnitsOnOrder None
ReorderLevel None
Discontinued None

In Pyslet, the values of the key properties are always retrieved, even if they are not selected. This is required to maintain the dictionary-like behaviour of the collection. An entity retrieved this way has NULL values for any properties that weren’t retrieved. The Selected() method allows you to determine if a value is NULL in the data source or NULL because it is not selected:

>>> for k, v in scones.data_items():
...  if scones.Selected(k): print k, v.value
...
ProductID 21
ProductName Sir Rodney's Scones

The expand and select options can be combined in complex ways:

>>> products.Expand({'Supplier':None}, {'ProductName':None, 'Supplier':{'Phone':None}})
>>> scones = products[21]
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products(21)?$expand=Supplier&$select=ProductID%2CProductName%2CSupplier%2FPhone%2CSupplier%2FSupplierID HTTP/1.1
INFO:root:Finished Response, status 200
>>> supplier = scones['Supplier'].GetEntity()
>>> for k, v in scones.data_items():
...  if scones.Selected(k): print k, v.value
...
ProductID 21
ProductName Sir Rodney's Scones
>>> for k, v in supplier.data_items():
...  if supplier.Selected(k): print k, v.value
...
SupplierID 8
Phone (161) 555-4448

Entity Objects¶

Continuing further with the database analogy and Entity is like a single record.

Entity instances behave like a read-only dictionary mapping property names onto their values. The values are either SimpleValue, Complex or DeferredValue instances. All property values are created on construction and cannot be assigned. To update a SimpleValue, whether it is a direct child or part of a Complex value, use its set_from_value() method:

entity['Name'].set_from_value("Steve")
entity['Address']['City'].set_from_value("Cambridge")

The following attributes are useful for consumers of the API (and should be treated as read only):

entity_set: The EntitySet to which this entity belongs.
type_def: The EntityType which defines this entity’s type.
exists: True if this entity exists in the collection, i.e., it was returned by one of the dictionary methods of an entity collection such as itervalues or [key] look-up.

The following methods are useful for consumers of the API:

key()

Returns the entity’s key, as a single python value or tuple in the case of compound keys

GetLocation()

Returns a pyslet.rfc2396.URI object that represents this entity:

>>> print scones.GetLocation()
http://services.odata.org/V2/Northwind/Northwind.svc/Products(21)

DataKeys()

Iterates over the simple and complex property names:

>>> list(scones.DataKeys())
[u'ProductID', u'ProductName', u'SupplierID', u'CategoryID', u'QuantityPerUnit', u'UnitPrice', u'UnitsInStock', u'UnitsOnOrder', u'ReorderLevel', u'Discontinued']

data_items()

Iterates over tuples of simple and complex property (name,value) pairs. See above for examples of usage.

Selected()

Tests if the given data property is selected.

NavigationKeys()

Iterates over the navigation property names:

>>> list(scones.NavigationKeys())
[u'Category', u'Order_Details', u'Supplier']

NavigationItems()

Iterates over tuples of navigation property (name,DeferredValue) pairs.

IsNavigationProperty()

Tests if a navigation property with the given name exists

The following methods can be used only on entities that exists, i.e., entities that have been returned from one of the collection’s dictionary methods:

Update(): Normally you’ll use the the Update method of an open EntityCollection but in cases where the originating collection is no longer open this method can be used as a convenience method for opening the base collection, updating the entity and then closing the collection collection again.
Delete(): Deletes this entity from the base entity set. If you already have the base entity set open it is more efficient to use the del operator but if the collection is no longer open or the entity was obtained from a collection opened through navigation then this method can be used to delete the entity.

The following method can only be used on entities that don’t exist, i.e., entities returned from the collection’s new_entity or CopyEntity methods that have not been inserted.

set_key(): Sets the entity’s key

SimpleValue¶

Simple property values are represented by (sub-classes of) SimpleValue, they share a number of common methods:

IsNull()

Returns True if this value is NULL. This method is also used by Python’s non-zero test so:

if entity['Property']:
        print entity['Property'].value
        # prints even if value is 0

will print the Property value of entity if it is non-NULL. In particular, it will print empty strings or other representations of zero. If you want to exclude these from the test you should test the value attribute directly:

if entity['Property'].value:
        print entity['Property'].value
        # will not print if value is 0

set_from_value()

Updates the value, coercing the argument to the correct type and range checking its value.

SetFromSimpleValue()

Updates the value from another SimpleValue, if the types match then the value is simply copied, otherwise the value is coerced using set_from_value.

SetFromLiteral()

Updates the value by parsing it from a (unicode) string. This is the opposite to the unicode function. The literal form is the form used when serializing the value to XML (but does not include XML character escaping).

SetNull()

Updates the value to NULL

The value attribute is always an immutable value in python and so can be used as a key in your own dictionaries. The following list describes the mapping from the EDM-defined simple types to their corresponding native Python types.

Edm.Boolean:

one of the Python constants True or False

Edm.Byte, Edm.SByte, Edm.Int16, Edm.Int32:

int

Edm.Int64:

long

Edm.Double, Edm.Single:

python float

Edm.Decimal:

python Decimal instance (from the built-in decimal module)

Edm.DateTime, Edm.DateTimeOffset:

py:class:pyslet.iso8601.TimePoint instance

This is a custom object in Pyslet, see Working with Dates for more information.

Edm.Time:

py:class:pyslet.iso8601.Time instance

Early versions of the OData specification incorrectly mapped this type to the XML Schema duration. The use of a Time object to represent it, rather than a duration, reflects this correction.

See Working with Dates for more information.

Edm.Binary:

raw string

Edm.String:

unicode string

Edm.Guid:

Python UUID instance (from the built-in uuid module)

Complex¶

Complex values behave like dictionaries of data properties. They do not have keys or navigation properties. They are never NULL, IsNull and the Python non-zero test will always return True.

SetNull(): Although a Complex value can never be NULL, this method will set all of its data properties (recursively if necessary) to NULL

DeferredValue¶

Navigation properties are represented as DeferredValue instances. All deferred values can be treated as an entity collection and opened in a similar way to an entity set:

>>> sconeSuppliers=scones['Supplier'].OpenCollection()
>>> for s in sconeSuppliers.itervalues(): print s['CompanyName'].value
...
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products(21)/Supplier HTTP/1.1
INFO:root:Finished Response, status 200
Specialty Biscuits, Ltd.
>>>

For reading, a collection opened from a deferred value behaves in exactly the same way as a collection opened from a base entity set. However, for writing there are some difference described above in Entity Collections.

If you use the dictionary methods to update the collection the changes are made straight away by accessing the data source directly. If you want to make a number of changes simultaneously, or you want to link entities to entities that don’t yet exist, then you should use the BindEntity method described below instead. This method defers the changes until the parent entity is updated (or inserted, in the case of non-existent entities.)

Read-only attributes useful to data consumers:

name: The name of the navigation property
from_entity: The parent entity of this navigation property
pDef: The NavigationProperty that defines this navigation property in the model.
isRequired: True if the target of this property has multiplicity 1, i.e., it is required.
isCollection: True if the target of this property has multiplicity *
isExpanded: True if this navigation property has been expanded. Expanded navigation keep a cached version of the target collection. Although you can open it and use it in the same way any other collection the values returned are returned from the cache and not by accessing the data source.

Methods useful to data consumers:

OpenCollection(): Returns an pyslet.odata2.csdl.EntityCollection object that can be used to access the target entities.
GetEntity(): Convenience method that returns the entity that is the target of the link when the target has multiplicity 1 or 0..1. If no entity is linked by the association then None is returned.
BindEntity(): Marks the target entity for addition to this navigation collection on next update or insert. If this navigation property is not a collection then the target entity will replace any existing target of the link.
Target(): The target entity set of this navigation property.

Working with Dates¶

In the EDM there are two types of date, DateTime and DateTimeOffset. The first represents a time-point in an implicit zone and the second represents a time-point with the zone offset explicitly set.

Both types are represented by the custom :py:class:pyslet.iso8601.TimePoint` class in Pyslet.

time module from build 0.4.20140217 onwards

Interacting with Python’s time module is done using the struct_time type, or lists that have values corresponding to those in struct_time:

>>> import time
>>> orders = c.feeds['Orders'].OpenCollection()
>>> orders.set_page(5)
>>> top = list(orders.iterpage())
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Orders?$skip=0&$top=5 HTTP/1.1
INFO:root:Finished Response, status 200
>>> print top[0]['OrderDate'].value
1996-07-04T00:00:00
>>> t = [None]*9
>>> top[0]['OrderDate'].value.UpdateStructTime(t)
>>> t
[1996, 7, 4, 0, 0, 0, 3, 186, -1]
>>> time.strftime("%a, %d %b %Y %H:%M:%S",t)
'Thu, 04 Jul 1996 00:00:00'

You can set values obtained from the time module in a similar way:

>>> import pyslet.iso8601 as iso
>>> t = time.gmtime(time.time())
>>> top[0]['OrderDate'].set_from_value(iso.TimePoint.FromStructTime(t))
>>> print top[0]['OrderDate'].value
2014-02-17T21:51:41

But if you just want a timestamp use one of the built-in factory methods:

>>> top[0]['OrderDate'].set_from_value(iso.TimePoint.FromNowUTC())
>>> print top[0]['OrderDate'].value
2014-02-17T21:56:23

In future versions, look out for better support for datetime and calendar module conversion methods.

Working with Media Resources¶

OData is based on Atom and the Atom Publishing Protocol (APP) and inherits the concept of media resources and media link entries from those specifications.

In OData, an entity can be declared as a media link entry indicating that the main purpose of the entity is to hold a media stream. If the entity with the following URL is a media link entry:

http://host/Documents(123)

then the following URL provides access to the associated media resource:

http://host/Documents(123)/$value

In the DAL this behaviour is modelled by operations on the collection containing the entities. The methods you’ll use are:

:py:meth:`~pyslet.odata2.core.EntityCollection.is_medialink_collection`

Returns True if the entities are media link entries

read_stream()

Reads information about a stream, optionally copying the stream’s data to a file-like object.

new_stream()

Creates a new media resource, copying the stream’s data from a file-like object.

This method implicitly creates an associated media link entry and returns the resulting Entity object. By its nature, APP does not guarantee the URL that will be used to store a posted resource. The implication for OData is that you can’t specify the key that will be used for the media resource’s entry, though this method does allow you to supply a hint.

udpate_stream()

Updates a media resource, copying the stream’s new data from a file-like object.

If a collection is a collection of media link entries then the behaviour of :py:meth:~pyslet.odata2.core.EntityCollection.insert_entity` is modified as entities are created implicitly when a new stream is added to the collection. In this case, insert_entity creates an empty stream of type application/octet-stream and then merges the property values from the entity being inserted into the new media link entry created for the stream.

OData Providers¶

The approach to writing a data access layer (DAL) taken by Pyslet is to use the Entity Data Model (EDM), and the extensions defined by OData, and to encapsulate them in an API defined by a set of abstract Python classes. The Data Consumers section goes through this API from the point of view of the consumer and provides a good primer for understanding what is required from a provider.

Pyslet includes three derived classes that implement the API in a variety of different storage scenarios:

OData Client - an implementation of the DAL that makes calls over the web to an OData server. Defined in the module pyslet.odatav2.client and used in the examples in the section Data Consumers.
In-memory data service - an implementation of the DAL that stores all entities and associations in python dictionaries. Defined in the module pyslet.odatav2.memds.
SQL data service - an implementation of the DAL that maps on to python’s database API. Defined in the module pyslet.odatav2.sqlds. In practice, the classes defined by this module will normally need to be sub-classed to deal with database-specific issues but a full implementation for SQLite is provided and a quick look at the source code for that should give you courage to tackle any modifications necessary for your favourite database. Using this DAL API is much easier than having to do these tweaks when they are distributed throughout your code in embedded SQL-statements.

A high-level plan for writing an OData provider would therefore be:

Identify the underlying DAL class that is most suited to your needs or, if there isn’t one, create a new DAL implementation using the existing implementations as a guide.
Create a metadata document to describe your data model
Write a test program that uses the DAL classes directly to validate that your model and the DAL implementation are working correctly
Create pyslet.odata2.server.Server that is bound to your model test it with an OData client to ensure that it works as expected.
Finally, create a sub-class of the server with any specific customisations needed by your application: mainly to implement your applications authentication and authorization model. (For a read-only service there may be nothing to do here.)

Of course, if all you want to do is use these interfaces as a DAL in your own application you can stop at item 3 above.

Sample Project: InMemory Data Service¶

The sample code for this service is in the samples directory in the Pyslet distribution.

This project demonstrates how to construct a simple OData service based on the InMemoryEntityContainer class. We don’t need any customisations, this class does everything we need ‘out of the box’.

Step 1: Creating the Metadata Model¶

Unlike other frameworks for implementing OData services Pyslet starts with the metadata model, it is not automatically generated: you must write it yourself!

Fortunately, there are plenty of examples you can use as a template. In this sample project we’ll write a very simple memory cache capable of storing a key-value pair. Here’s our data model:

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<edmx:Edmx Version="1.0" xmlns:edmx="http://schemas.microsoft.com/ado/2007/06/edmx"
        xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
        <edmx:DataServices m:DataServiceVersion="2.0">
                <Schema Namespace="MemCacheSchema" xmlns="http://schemas.microsoft.com/ado/2006/04/edm">
                        <EntityContainer Name="MemCache" m:IsDefaultEntityContainer="true">
                                <EntitySet Name="KeyValuePairs" EntityType="MemCacheSchema.KeyValuePair"/>
                        </EntityContainer>
                        <EntityType Name="KeyValuePair">
                                <Key>
                                        <PropertyRef Name="Key"/>
                                </Key>
                                <Property Name="Key" Type="Edm.String" Nullable="false" MaxLength="256"
                                        Unicode="true" FixedLength="false"/>
                                <Property Name="Value" Type="Edm.String" Nullable="false" MaxLength="8192"
                                        Unicode="true" FixedLength="false"/>
                                <Property Name="Expires" Type="Edm.DateTime" Nullable="false"
                                        Precision="3"/>
                        </EntityType>
                </Schema>
        </edmx:DataServices>
</edmx:Edmx>

Our model has one defined EntityType called KeyValuePair and one EntitySet called KeyValuePairs in a container called MemCache. The idea behind the model is that each key-value pair is inserted with an expires time, after which it is safe to clean it up.

For simplicity, we’ll save this model to a file and load it from the file when our script starts up. Here’s the source code:

import pyslet.odata2.metadata as edmx

def LoadMetadata():
        """Loads the metadata file from the current directory."""
        doc=edmx.Document()
        with open('MemCacheSchema.xml','rb') as f:
                doc.Read(f)
        return doc

The metadata module contains a Document object and the definitions of the elements in the edmx namespace that enable us to read the XML file.

Step 2: Test the Model¶

Let’s write a simple test function to test our model:

def TestData(memCache):
        with memCache.OpenCollection() as collection:
                for i in xrange(26):
                        e=collection.new_entity()
                        e.set_key(str(i))
                        e['Value'].set_from_value(unichr(0x41+i))
                        e['Expires'].set_from_value(iso.TimePoint.FromUnixTime(time.time()+10*i))
                        collection.insert_entity(e)

def TestModel():
        """Read and write some key value pairs"""
        doc=LoadMetadata()
        container=InMemoryEntityContainer(doc.root.DataServices['MemCacheSchema.MemCache'])
        memCache=doc.root.DataServices['MemCacheSchema.MemCache.KeyValuePairs']
        TestData(memCache)
        with memCache.OpenCollection() as collection:
                for e in collection.itervalues():
                        print "%s: %s (expires %s)"%(e['Key'].value,e['Value'].value,str(e['Expires'].value))

Our function comes in two parts (for reasons that will become clear later). The first function takes an EntitySet object and creates 26 key-value pairs with increasing expiry times.

The main function loads the metadata model, creates the InMemoryEntityContainer object, calls the first function to create the test data and then opens the KeyValuePairs collection itself to check that everything is in order. Here’s the output from a sample run:

>>> import memcache
>>> memcache.TestModel()
Y (expires 2014-02-17T22:26:21)
Z (expires 2014-02-17T22:26:31)
U (expires 2014-02-17T22:25:41)
V (expires 2014-02-17T22:25:51)
W (expires 2014-02-17T22:26:01)
X (expires 2014-02-17T22:26:11)
B (expires 2014-02-17T22:22:31)
A (expires 2014-02-17T22:22:21)
D (expires 2014-02-17T22:22:51)
C (expires 2014-02-17T22:22:41)
F (expires 2014-02-17T22:23:11)
E (expires 2014-02-17T22:23:01)
H (expires 2014-02-17T22:23:31)
G (expires 2014-02-17T22:23:21)
J (expires 2014-02-17T22:23:51)
I (expires 2014-02-17T22:23:41)
L (expires 2014-02-17T22:24:11)
K (expires 2014-02-17T22:24:01)
N (expires 2014-02-17T22:24:31)
M (expires 2014-02-17T22:24:21)
P (expires 2014-02-17T22:24:51)
O (expires 2014-02-17T22:24:41)
R (expires 2014-02-17T22:25:11)
Q (expires 2014-02-17T22:25:01)
T (expires 2014-02-17T22:25:31)
S (expires 2014-02-17T22:25:21)

It is worth pausing briefly here to look at the InMemoryEntityContainer object. When we construct this object we pass in the EntityContainer and it creates all the necessary storage for the EntitySets (and AssociationSets, if required) that it contains. It also binds internal implementations of the EntityCollection object to the model so that, in future, the EntitySet can be opened using the same API described previously in Data Consumers. From this point on we don’t need to refer to the container again as we can just open the EntitySet directly from the model. That object is the heart of our application, blink and you’ve missed it.

Step 4: Link the Data Source to the OData Server¶

OData runs over HTTP so we need to assign a service root URL for the server to run on. We define a couple of constants to help with this:

SERVICE_PORT=8080
SERVICE_ROOT="http://localhost:%i/"%SERVICE_PORT

We’re also going to use a separate thread to run the server, a global variable helps here. We’re using Pythons wsgi interface for the server which requires a callable object to handle requests. The Server object implements callable behaviour to enable this:

import logging, threading
from wsgiref.simple_server import make_server

cacheApp=None           #: our Server instance

def runCacheServer():
        """Starts the web server running"""
        server=make_server('',SERVICE_PORT,cacheApp)
        logging.info("Starting HTTP server on port %i..."%SERVICE_PORT)
        # Respond to requests until process is killed
        server.serve_forever()

The final part of server implementation involves loading the model, creating the server object and then spawning the server thread:

def main():
        """Executed when we are launched"""
        doc=LoadMetadata()
        container=InMemoryEntityContainer(doc.root.DataServices['MemCacheSchema.MemCache'])
        server=Server(serviceRoot=SERVICE_ROOT)
        server.SetModel(doc)
        # The server is now ready to serve forever
        global cacheApp
        cacheApp=server
        t=threading.Thread(target=runCacheServer)
        t.setDaemon(True)
        t.start()
        logging.info("MemCache starting HTTP server on %s"%SERVICE_ROOT)

The Server object just takes the serviceRoot as a parameter on construction and has a SetModel() method which is used to assign the metadata document to it. That’s all you need to do to create it, it uses the same API described in Data Consumers to consume the data source and expose it via the OData protocol.

At this stage we can test it via the terminal and a browser:

>>> import memcache
>>> memcache.main()
>>>

At this point the server is running in a separate thread, listening on port 8080. A quick check from the browser shows this to be the case, when I hit http://localhost:8080/KeyValuePairs Firefox recognises that the document is an Atom feed and displays the feed title. The page source shows:

<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xml:base="http://localhost:8080/">
        <id>http://localhost:8080/KeyValuePairs</id>
        <title type="text">MemCacheSchema.MemCache.KeyValuePairs</title>
        <updated>2014-02-17T22:41:51Z</updated>
        <link href="http://localhost:8080/KeyValuePairs" rel="self"/>
</feed>

Looks like it is working!

Step 5: Customise the Server¶

We don’t need to do much to customise our server, we’ll assume that it is only ever going to be exposed to clients we trust and so authentication is not required or will be handled by some intermediate proxy.

However, we do want to clean up expired entries automatically. Let’s add one last function to our code:

CLEANUP_SLEEP=10

def CleanupForever(memCache):
        """Runs a loop continuously cleaning up expired items"""
        expires=core.PropertyExpression(u"Expires")
        now=edm.DateTimeValue()
        t=core.LiteralExpression(now)
        filter=core.BinaryExpression(core.Operator.lt)
        filter.operands.append(expires)
        filter.operands.append(t)
        while True:
                now.set_from_value(iso.TimePoint.FromNowUTC())
                logging.info("Cleanup thread running at %s",str(now.value))
                with memCache.OpenCollection() as cacheEntries:
                        cacheEntries.set_filter(filter)
                        expiredList=list(cacheEntries)
                        if expiredList:
                                logging.info("Cleaning %i cache entries",len(expiredList))
                                for expired in expiredList:
                                        del cacheEntries[expired]
                        cacheEntries.set_filter(None)
                        logging.info("Cleanup complete, %i cache entries remain",len(cacheEntries))
                time.sleep(CLEANUP_SLEEP)

This function starts by building a filter expression manually. Filter expressions are just simple trees of expression objects. We start with a PropertyExpression that references a property named Expires and a literal expression with a date-time value. DateTimeValue is just a sub-class of SimpleValue which was introduced in Data Consumers. Previously we’ve only seen simple values that are part of an entity but in this case we create a standalone value to use in the expression. Finally, the filter expression is created as a BinaryExpression using the less than operator and the operands appended. The resulting expression tree looks like this:

Each time around the loop we can just update the value of the literal expression with the current time.

This function takes an EntitySet as a parameter so we can open it to get the collection and then apply the filter. Once filtered, all matching cache entries are loaded into a list before being deleted from the collection, one by one.

Finally, we remove the filter and report the number of remaining entries before sleeping ready for the next run.

We’ll call this function right after main, so we’ve got one thread running the server and the main thread running the cleanup loop.

Now we can test, we start by firing up our server application:

$ ./memcache.py
INFO:root:MemCache starting HTTP server on http://localhost:8080/
INFO:root:Cleanup thread running at 2014-02-17T23:03:34
INFO:root:Cleanup complete, 0 cache entries remain
INFO:root:Starting HTTP server on port 8080...
INFO:root:Cleanup thread running at 2014-02-17T23:03:44
INFO:root:Cleanup complete, 0 cache entries remain

Unfortunately, we need more than a simple browser to test the application properly. We want to know that the key value pairs are being created properly and for that we need a client capable of writing to the service. Fortunately, Pyslet has an OData consumer, so we open the interpreter in a new terminal and start interacting with our server:

>>> from pyslet.odata2.client import Client
>>> c=Client("http://localhost:8080/")

As soon as we start the client our server registers hits:

INFO:root:Cleanup thread running at 2014-02-17T23:06:34
INFO:root:Cleanup complete, 0 cache entries remain
127.0.0.1 - - [17/Feb/2014 23:06:34] "GET / HTTP/1.1" 200 360
127.0.0.1 - - [17/Feb/2014 23:06:34] "GET /$metadata HTTP/1.1" 200 1040
INFO:root:Cleanup thread running at 2014-02-17T23:06:44
INFO:root:Cleanup complete, 0 cache entries remain

Entering the data manually would be tedious but we already wrote a suitable function for adding test data. Because both the data source and the OData client adhere to the same API we can simply pass the EntitySet to our TestData function:

>>> import memcache
>>> memcache.TestData(c.feeds['KeyValuePairs'])

As we do this, the server window goes crazy as each of the POST requests comes through:

INFO:root:Cleanup thread running at 2014-02-17T23:08:14
INFO:root:Cleanup complete, 0 cache entries remain
127.0.0.1 - - [17/Feb/2014 23:08:23] "POST /KeyValuePairs HTTP/1.1" 201 717
... [and so on]
...
127.0.0.1 - - [17/Feb/2014 23:08:24] "POST /KeyValuePairs HTTP/1.1" 201 720
INFO:root:Cleanup thread running at 2014-02-17T23:08:24
INFO:root:Cleaning 1 cache entries
INFO:root:Cleanup complete, 19 cache entries remain
127.0.0.1 - - [17/Feb/2014 23:08:24] "POST /KeyValuePairs HTTP/1.1" 201 720
127.0.0.1 - - [17/Feb/2014 23:08:24] "POST /KeyValuePairs HTTP/1.1" 201 720
127.0.0.1 - - [17/Feb/2014 23:08:24] "POST /KeyValuePairs HTTP/1.1" 201 720
127.0.0.1 - - [17/Feb/2014 23:08:24] "POST /KeyValuePairs HTTP/1.1" 201 720
127.0.0.1 - - [17/Feb/2014 23:08:24] "POST /KeyValuePairs HTTP/1.1" 201 720
127.0.0.1 - - [17/Feb/2014 23:08:24] "POST /KeyValuePairs HTTP/1.1" 201 720
INFO:root:Cleanup thread running at 2014-02-17T23:08:34
INFO:root:Cleaning 1 cache entries
INFO:root:Cleanup complete, 24 cache entries remain

We can then watch the data gradually decay as each entry times out in turn. We can easily repopulate the cache, this time let’s catch it in a browser by navigating to:

http://localhost:8080/KeyValuePairs('25')?$format=json

The result is:

{"d":{"__metadata":{"uri":"http://localhost:8080/KeyValuePairs('25')
","type":"MemCacheSchema.KeyValuePair"},"Key":"25","Value":"Z","
Expires":"/Date(1392679105162)/"}}

We can pick the value our directly with a URL like:

http://localhost:8080/KeyValuePairs('25')/Value/$value

This returns the simple string ‘Z’.

Conclusion¶

It is easy to write an OData server using Pyslet!

A SQL-Backed Data Service¶

The sample code for this service is in the samples directory in the Pyslet distribution.

This project demonstrates how to construct a simple OData service based on the SQLiteEntityContainer class.

We don’t need any customisations, this class does everything we need ‘out of the box’. If you want to use a database other than SQLite you will need to create a new implementation of the generic SQLEntityContainer. See the reference documentation for sqlds for details on what is involved. You shouldn’t have to change much!

Step 1: Creating the Metadata Model¶

If you haven’t read the Sample Project: InMemory Data Service yet it is a good idea to do that to get a primer on how providers work. The actual differences between writing a SQL-backed service and one backed by the in-memory implementation are minimal. I haven’t repeated code here if it is essentially the same as the code shown in the previous example, but remember that the full working source is available in the samples directory.

For this project, I’ve chosen to write an OData service that exposes weather data for my home town of Cambridge, England. The choice of data set is purely because I have access to over 325,000 data points stretching back to 1995 thanks to the excellent Weather Station website run by the University of Cambridge’s Digital Technology Group: http://www.cl.cam.ac.uk/research/dtg/weather/

We start with our metadata model, which we write by hand. There are two entity sets. The first contains the actual data readings from the weather station and the second contains notes relating to known inaccuracies in the data. I’ve included a navigation property so that it is easy to see which note, if any, applies to a data point.

Here’s the model:

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<edmx:Edmx Version="1.0" xmlns:edmx="http://schemas.microsoft.com/ado/2007/06/edmx"
        xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
        <edmx:DataServices m:DataServiceVersion="2.0">
                <Schema Namespace="WeatherSchema" xmlns="http://schemas.microsoft.com/ado/2006/04/edm">
                        <EntityContainer Name="CambridgeWeather" m:IsDefaultEntityContainer="true">
                                <EntitySet Name="DataPoints" EntityType="WeatherSchema.DataPoint"/>
                                <EntitySet Name="Notes" EntityType="WeatherSchema.Note"/>
                                <AssociationSet Name="DataPointNotes" Association="WeatherSchema.DataPointNote">
                                        <End Role="DataPoint" EntitySet="DataPoints"/>
                                        <End Role="Note" EntitySet="Notes"/>
                                </AssociationSet>
                        </EntityContainer>
                        <EntityType Name="DataPoint">
                                <Key>
                                        <PropertyRef Name="TimePoint"/>
                                </Key>
                                <Property Name="TimePoint" Type="Edm.DateTime" Nullable="false" Precision="0" m:FC_TargetPath="SyndicationUpdated" m:FC_KeepInContent="true"/>
                                <Property Name="Temperature" Type="Edm.Single" m:FC_TargetPath="SyndicationTitle" m:FC_KeepInContent="true"/>
                                <Property Name="Humidity" Type="Edm.Byte"/>
                                <Property Name="DewPoint" Type="Edm.Single"/>
                                <Property Name="Pressure" Type="Edm.Int16"/>
                                <Property Name="WindSpeed" Type="Edm.Single"/>
                                <Property Name="WindDirection" Type="Edm.String" MaxLength="3" Unicode="false"/>
                                <Property Name="WindSpeedMax" Type="Edm.Single"/>
                                <Property Name="SunRainStart" Type="Edm.Time" Precision="0"></Property>
                                <Property Name="Sun" Type="Edm.Single"/>
                                <Property Name="Rain" Type="Edm.Single"/>
                                <NavigationProperty Name="Note" Relationship="WeatherSchema.DataPointNote"
                                        FromRole="DataPoint" ToRole="Note"/>
                        </EntityType>
                        <EntityType Name="Note">
                                <Key><PropertyRef Name="ID"></PropertyRef></Key>
                                <Property Name="ID" Type="Edm.Int32" Nullable="false"/>
                                <Property Name="StartDate" Type="DateTime" Nullable="false" Precision="0"/>
                                <Property Name="EndDate" Type="DateTime" Nullable="false" Precision="0"/>
                                <Property Name="Details" Type="Edm.String" MaxLength="1024" Nullable="false" FixedLength="false"/>
                                <NavigationProperty Name="DataPoints" Relationship="WeatherSchema.DataPointNote"
                                        FromRole="Note" ToRole="DataPoint"/>
                        </EntityType>
                        <Association Name="DataPointNote">
                                <End Role="DataPoint" Type="WeatherSchema.DataPoint" Multiplicity="*"/>
                                <End Role="Note" Type="WeatherSchema.Note" Multiplicity="0..1"/>
                        </Association>
                </Schema>
        </edmx:DataServices>
</edmx:Edmx>

I’ve added two feed customisations to this model. The TimePoint field of the data point will be echoed in the Atom ‘updated’ field and the Temperature field will become the Atom title. This will make my OData service more interesting to look at in a standard browser.

As before, we’ll save the model to a file and load it when our script starts up.

To link the model to a SQLite database back-end we need to create an instance of SQLiteEntityContainer:

SAMPLE_DB='weather.db'

def MakeContainer(doc,drop=False):
        if drop and os.path.isfile(SAMPLE_DB):
                os.remove(SAMPLE_DB)
        create=not os.path.isfile(SAMPLE_DB)
        container=SQLiteEntityContainer(file_path=SAMPLE_DB,containerDef=doc.root.DataServices['WeatherSchema.CambridgeWeather'])
        if create:
                container.create_all_tables()

This function handles the only SQL-specific part of our project. When we create a SQLite container we have to pass two keyword arguments: rather than just the container definition as we did for the in-memory implementation. We don’t need to return a value because the SQL implementation is bound to the model that was passed in doc.

The code above automatically creates the tables if the database doesn’t exist yet. This is fine if you are starting from scratch but if you want to expose an existing database you’ll need to work backwards from your existing schema when creating the model. Anyway, letting Pyslet create your SQL tables for you neglects your DBA who will almost certainly want to create indexes to optimise performance and tweak the model to get the best out of your platform. The automatically generated SQL script is supposed to be a starting point, not the complete solution.

For example, the data set I used for this project has over 300,000 records in it. At the end of this exercise I had an OData server capable of serving this information from a SQLite database but example URLs were taking 10s or more on my laptop to load. I created an index on the Temperature column using the SQLite command line and the page load times were instantaneous:

sqlite> create index TIndex ON DataPoints(Temperature);

Modelling an Existing Database¶

For simple data properties it should be fairly easy to map to the EDM. Here is the way Pyslet maps simple types in the EDM to SQL types:

EDM Type	SQL Equivalent
Edm.Binary	BINARY(MaxLength) if FixedLength specified
Edm.Binary	VARBINARY(MaxLength) if no FixedLength
Edm.Boolean	BOOLEAN
Edm.Byte	SMALLINT
Edm.DateTime	TIMESTAMP
Edm.DateTimeOffset	CHARACTER(20), ISO 8601 string representation is used
Edm.Decimal	DECIMAL(Precision,Scale), defaults 10,0
Edm.Double	FLOAT
Edm.Guid	BINARY(16)
Edm.Int16	SMALLINT
Edm.Int32	INTEGER
Edm.Int64	BIGINT
Edm.SByte	SMALLINT
Edm.Single	REAL
Edm.String	CHAR(MaxLength) or VARCHAR(MaxLength)
Edm.String	NCHAR(MaxLength) or NVARCHAR(MaxLength) if Unicode=”true”
Edm.Time	TIME

Navigation properties, and complex properties do not map as easily but they can still be modelled. To start with, look at the way the SQLite implementation turns our model into a SQL CREATE TABLE statement:

>>> import weather
>>> doc=weather.LoadMetadata()
>>> weather.MakeContainer(doc)
>>> dataPoints=doc.root.DataServices['WeatherSchema.CambridgeWeather.DataPoints'].OpenCollection()
>>> print dataPoints.CreateTableQuery()[0]
CREATE TABLE "DataPoints" ("TimePoint" TIMESTAMP NOT NULL,
"Temperature" REAL, "Humidity" SMALLINT, "DewPoint" REAL, "Pressure"
SMALLINT, "WindSpeed" REAL, "WindDirection" TEXT, "WindSpeedMax"
REAL, "SunRainStart" REAL, "Sun" REAL, "Rain" REAL,
"DataPointNotes_ID" INTEGER, PRIMARY KEY ("TimePoint"), CONSTRAINT
"DataPointNotes" FOREIGN KEY ("DataPointNotes_ID") REFERENCES
"Notes"("ID"))

After all the data properties there’s an additional property called DataPointNotes_ID which is a foreign key into into the Notes table. This was created automatically to model the association set that links the two EntitySets in the container.

Pyslet generates foreign keys for the following types of association:

0..1 to 1	With UNIQUE and NOT NULL constraints
* to 1	With a NOT NULL constraint only
* to 0..1	No additional constraints

When these relationships are reversed the foreign key is of course created in the target table.

What if your foreign key has a different name, say, NoteID? Pyslet gives you the chance to override all name mappings. To fix up this part of the model you need to create a derived class of the base class SQLEntityContainer and override the mangle_name() method.

In this case, the method would have been called like this:

quotedName=container.mangle_name((u"DataPoints",u"DataPointNotes",u"ID"))

There is a single argument consisting of a tuple. The first item is the name of the EntitySet (SQL TABLE) and the subsequent items complete a kind of ‘path’ to the value. Foreign keys have a path comprising of the AssociationSet name followed by the name of the key field in the target EntitySet. The default implementation just joins the path with an underscore character. The method must return a suitably quoted value to use for the column name. To complete the example, here is how our subclass might implement this method to ensure that the foreign key is called ‘NoteID’ instead of ‘DataPointNotes_ID’:

def mangle_name(self,source_path):
        if source_path==(u"DataPoints",u"DataPointNotes",u"ID"):
                return self.quote_identifier(u'NoteID')
        else:
                return super(MyCustomerContainer,self).mangle_name(source_path)

You may be wondering why we don’t expose the foreign key field in the model. Some libraries might force you to expose the foreign key in order to expose the navigation property but Pyslet takes the opposite approach. The whole point of navigation properties is to hide away details like foreign keys. If you really want to access the value you can always use an expansion and select the key field in the target entity. Exposing it in the source entity just tempts you in to writing code that ‘knows’ about your model for example, if we had exposed the foreign key in our example as a simple property we might have been tempted to do something like this:

noteID=dataPoint['DataPointNotes_ID'].value
if noteID is not None:
        note=noteCollection[noteID]
        # do something with the note

When we should be doing something like this:

note=dataPoint['Note']
if note is not None:
        # do something with the note

Complex types are handled in the same way as foreign keys, the path being comprised of the name(s) of the complex field(s) terminated by the name of a simple property. For example, if you have a complex type called Address and two properties of type Address called “Home” and “Work” you might end up with SQL that looked like this:

CREATE TABLE Employee (
        ...
        Home_Street NVARCHAR(50),
        Home_City NVARCHAR(50),
        Home_Phone NVARCHAR(50),
        Work_Street NVARCHAR(50),
        Work_City NVARCHAR(50),
        Work_Phone NVARCHAR(50)
        ...
        )

You often see SQL written like this anyway so if you want to tweak the mapping to put a Complex type in your model you can.

Finally, we need to deal with the symmetric relationships, 1 to 1 and * to *. These are modelled by separate tables. 1 to 1 relationships are best avoided, the advantages over combining the two entities into a single larger entity are marginal given OData’s $select option which allows you to pick a subset of the fields anyway. If you have them in your SQL schema already you might consider creating a view to combine them before attempting to map them to the metadata model.

Either way, both types of symmetric relationships get mapped to a table with the name of the AssociationSet. There are two sets of foreign keys, one for each of the EntitySets being joined. The paths are rather complex and are explained in detail in SQLAssociationCollection.

Step 2: Test the Model¶

Before we add the complication of using our model with a SQL database, let’s test it out using the same in-memory implementation we used before:

def DryRun():
        doc=LoadMetadata()
        container=InMemoryEntityContainer(doc.root.DataServices['WeatherSchema.CambridgeWeather'])
        weatherData=doc.root.DataServices['WeatherSchema.CambridgeWeather.DataPoints']
        weatherNotes=doc.root.DataServices['WeatherSchema.CambridgeWeather.Notes']
        LoadData(weatherData,SAMPLE_DIR)
        LoadNotes(weatherNotes,'weathernotes.txt',weatherData)
        return doc.root.DataServices['WeatherSchema.CambridgeWeather']

SAMPLE_DIR here is the name of a directory containing data from the weather station. The implementation of the LoadData function is fairly ordinary, parsing the daily text files from the station and adding them to the DataPoints entity set.

The implementation of the LoadNotes function is more interesting as it demonstrates use of the API for binding entities together using navigation properties:

def LoadNotes(weatherNotes,fileName,weatherData):
        with open(fileName,'r') as f:
                id=1
                with weatherNotes.OpenCollection() as collection, weatherData.OpenCollection() as data:
                        while True:
                                line=f.readline()
                                if len(line)==0:
                                        break
                                elif line[0]=='#':
                                        continue
                                noteWords=line.split()
                                if noteWords:
                                        note=collection.new_entity()
                                        note['ID'].set_from_value(id)
                                        start=iso.TimePoint(
                                                date=iso.Date.from_str(noteWords[0]),
                                                time=iso.Time(hour=0,minute=0,second=0))
                                        note['StartDate'].set_from_value(start)
                                        end=iso.TimePoint(
                                                date=iso.Date.from_str(noteWords[1]).Offset(days=1),
                                                time=iso.Time(hour=0,minute=0,second=0))
                                        note['EndDate'].set_from_value(end)
                                        note['Details'].set_from_value(string.join(noteWords[2:],' '))
                                        collection.insert_entity(note)
                                        # now find the data points that match
                                        data.set_filter(core.CommonExpression.from_str("TimePoint ge datetime'%s' and TimePoint lt datetime'%s'"%(unicode(start),unicode(end))))
                                        for dataPoint in data.values():
                                                dataPoint['Note'].BindEntity(note)
                                                data.update_entity(dataPoint)
                                        id=id+1
        with weatherNotes.OpenCollection() as collection:
                collection.set_orderby(core.CommonExpression.OrderByFromString('StartDate desc'))
                for e in collection.itervalues():
                        with e['DataPoints'].OpenCollection() as affectedData:
                                print "%s-%s: %s (%i data points affected)"%(unicode(e['StartDate'].value),
                                        unicode(e['EndDate'].value),e['Details'].value,len(affectedData))

The function opens collections for both Notes and DataPoints. For each uncommented line in the source file it creates a new Note entity, then, it adds a filter to the collection of data points that narrows down the collection to all the data points affected by the note and then iterates through them binding the note to the data point and updating the entity (to commit the change to the data source). Here’s a sample of the output on a dry-run of a small sample of the data from November 2007:

2007-12-25T00:00:00-2008-01-03T00:00:00: All sensors inaccurate (0 data points affected)
2007-11-01T00:00:00-2007-11-23T00:00:00: rain sensor over reporting rainfall following malfunction (49 data points affected)

You may wonder why we use the values function, rather than itervalues in the loop that updates the data points. itervalues would certainly have been more efficient but, just like native Python dictionaries, it is a bad idea to modify the data source when iterating as unpredictable things may happen. The concept is extended by this API to cover the entire container: a thread should not modify the container while iterating through a collection.

Of course, this API has been designed for parallel use so there is always the chance that another thread or process is modifying the data source outside of your control. Behaviour in that case is left to be implementation dependent - storage engines have widely differing policies on what to do in these cases.

If you have large amounts of data to iterate through you should consider using list(collection.iterpage(True)) instead. For a SQL data souurce this has the disadvantage of executing a new query for each page rather than spooling data from a single SELECT but it provides control over page size (and hence memory usage in your client) and is robust to modifications.

As an aside, if you change the call from values to itervalues in the sample you may well discover a bug in the SQLite driver in Python 2.7. The bug means that a commit on a database connection while you are fetching data on another cursor causes subsequent data access commands to fail. It’s a bit technical, but the details are here: http://bugs.python.org/issue10513

Having tested the model using the in-memory provider we can implement a full test using the SQL back-end we created in MakeContainer above. This test function prints the 30 strongest wind gusts in the database, along with any linked note:

def TestModel(drop=False):
        doc=LoadMetadata()
        container=MakeContainer(doc,drop)
        weatherData=doc.root.DataServices['WeatherSchema.CambridgeWeather.DataPoints']
        weatherNotes=doc.root.DataServices['WeatherSchema.CambridgeWeather.Notes']
        if drop:
                LoadData(weatherData,SAMPLE_DIR)
                LoadNotes(weatherNotes,'weathernotes.txt',weatherData)
        with weatherData.OpenCollection() as collection:
                collection.set_orderby(core.CommonExpression.OrderByFromString('WindSpeedMax desc'))
                collection.set_page(30)
                for e in collection.iterpage():
                        note=e['Note'].GetEntity()
                        if e['WindSpeedMax'] and e['Pressure']:
                                print "%s: Pressure %imb, max wind speed %0.1f knots (%0.1f mph); %s"%(unicode(e['TimePoint'].value),
                                        e['Pressure'].value,e['WindSpeedMax'].value,e['WindSpeedMax'].value*1.15078,
                                        note['Details'] if note is not None else "")

Here’s some sample output:

>>> weather.TestModel()
2002-10-27T10:30:00: Pressure 988mb, max wind speed 74.0 knots (85.2 mph);
2004-03-20T15:30:00: Pressure 993mb, max wind speed 72.0 knots (82.9 mph);
2007-01-18T14:30:00: Pressure 984mb, max wind speed 70.0 knots (80.6 mph);
... [ and so on ]
...
2007-01-11T10:30:00: Pressure 998mb, max wind speed 58.0 knots (66.7 mph);
2007-01-18T07:30:00: Pressure 980mb, max wind speed 58.0 knots (66.7 mph);
1996-02-18T04:30:00: Pressure 998mb, max wind speed 56.0 knots (64.4 mph); humidity and dewpoint readings may be inaccurate, particularly high humidity readings
2000-12-13T01:30:00: Pressure 991mb, max wind speed 56.0 knots (64.4 mph);
2002-10-27T13:00:00: Pressure 996mb, max wind speed 56.0 knots (64.4 mph);
2004-01-31T17:30:00: Pressure 983mb, max wind speed 56.0 knots (64.4 mph);

Notice that the reading from 1996 has a related note.

Step 4: Link the Data Source to the OData Server¶

This data set is designed to be updated by some offline process that polls the weather station for the latest readings and adds them to the database behind the scenes. Unlike the memory-cache example, the OData interface should be read-only so we use the ReadOnlyServer sub-class of the OData server:

def runWeatherServer(weatherApp=None):
        """Starts the web server running"""
        server=make_server('',SERVICE_PORT,weatherApp)
        logging.info("HTTP server on port %i running"%SERVICE_PORT)
        # Respond to requests until process is killed
        server.serve_forever()

def main():
        """Executed when we are launched"""
        doc=LoadMetadata()
        container=MakeContainer(doc)
        server=ReadOnlyServer(serviceRoot=SERVICE_ROOT)
        server.SetModel(doc)
        t=threading.Thread(target=runWeatherServer,kwargs={'weatherApp':server})
        t.setDaemon(True)
        t.start()
        logging.info("Starting HTTP server on %s"%SERVICE_ROOT)
        t.join()

Once the script is running we test in a browser. I’ve loaded the full data set into the server, how many data points? Here’s how we can find out, in our browser we go to:

http://localhost:8080/DataPoints/$count

The result is 325213. Firefox recognises that the feeds are in Atom format and renders the feed customisations we made earlier.

When we access this page with logging turned up to INFO we get the following output on the console, interspersed with the simple HTTP server output:

INFO:root:SELECT COUNT(*) FROM "DataPoints"; []
127.0.0.1 - - [21/Feb/2014 22:57:01] "GET /DataPoints/$count HTTP/1.1" 200 6
INFO:root:SELECT "TimePoint", "Temperature", "Humidity", "DewPoint", "Pressure", "WindSpeed", "WindDirection", "WindSpeedMax", "SunRainStart", "Sun", "Rain", "Temperature" AS o_1, "TimePoint" FROM "DataPoints" ORDER BY o_1 DESC, "TimePoint" ASC; []
127.0.0.1 - - [21/Feb/2014 22:57:18] "GET /DataPoints?$orderby=Temperature%20desc&$top=30 HTTP/1.1" 200 31006

You may wonder what those square brackets are doing at the end of the SQL statements. They’re actually used for logging the parameter values when the query has been parameterised. If we add a filter you’ll see what they do:

http://localhost:8080/DataPoints?$filter=Temperature%20gt%20-100&$orderby=Temperature%20asc&$top=30

And here’s the output on the console:

INFO:root:SELECT "TimePoint", "Temperature", "Humidity", "DewPoint", "Pressure", "WindSpeed", "WindDirection", "WindSpeedMax", "SunRainStart", "Sun", "Rain", "Temperature" AS o_1, "TimePoint" FROM "DataPoints" WHERE ("Temperature" > ?) ORDER BY o_1 DESC, "TimePoint" ASC; [-100]
127.0.0.1 - - [21/Feb/2014 16:35:09] "GET /DataPoints?$filter=Temperature%20gt%20-100&$orderby=Temperature%20desc&$top=30 HTTP/1.1" 200 31006

Yes, all Pyslet queries are fully parameterized for security and performance!

Sample Project: Custom Data Service¶

Which DAL Implementation?¶

Transient Data¶

If your data is relatively small and transient then you could use the in memory implementation of the DAL API directly. This is the easiest route to creating a new OData provider as you won’t need to override any of of the implementations.

Look at the example project Sample Project: InMemory Data Service to see how easy it is to create a useful in-memory key-value store.

SQL¶

If your data is currently in a SQL database, or if you intend to write a read-only data source and you could easily put your data into a SQL database, then you should use the Python DB ABI-based implementation as a starting point.

If your data is in a database other than a SQLite database you will have to provide a few tweaks by deriving a new class from SQLEntityContainer. This can’t be helped, the DB API does a good job at dealing with most issues, such as variation in parameterization conventions and expected data types, but SQL connection parameters and the occasional differences in the SQL syntax mean there is likely to be a small amount of work to do.

A look at the customisations required for SQLiteEntityContainer where a handful of methods have had to be overridden should point the way. You may want to override the default SQLEntityCollection object too where functions and operators from the the expression language can be mapped on to parameterized SQL queries.

Once you have a class that can connect to your chosen database move on to A SQL-Backed Data Service.

Customer Provider¶

Writing a customer provider isn’t as hard as you might think, provided your data set is of a mangeable size then you can use the built-in behaviour of the base classes to take care of almost all the API’s needs. You just need to expose the entity values themselves by implementing a couple of methods!

Look at the example project Sample Project: Custom Data Service to see how you can write a simple application that exposes a download-directory to the web using OData (providing a little more metadata than is easily obtainable from plain HTTP.)

An OData Proxy¶

Finally, the OData client implementation of the DAL API opens the possibility of writing an OData proxy server. Why would you do this?

One of the big challenges for the OData protocol is web-security in the end user’s browser. By supporting JSON over the wire OData sends out a clear signal that using it directly from a Javascript on a web page should be possible. But in practice, this only works well for unauthenticated (and hence read-only) OData services. If you want to write more exciting applications you leave yourself open to all manner of browser-based attacks that could expose your data to unauthorised bad guys. To mitigate these risks browsers are increasingly locking down the browser to make it harder for cross-site exploits to happen, which is a good thing. The downside is that it makes it harder for your web-application to talk to an OData server unless they are both hosted on the same domain.

An OData proxy can be co-located with your application to overcome this problem. A dumb proxy is probably best implemented by the web-server, rather than a full-blown web application but the classes defined in this package are a good starting point for writing a more intelligent proxy such as one that checks for a valid session with your application before proxying the request.

The implementation isn’t trivial because the identities of the entities created by the client (as reported by GetLocation()) are the URLs of the entities as they appear in the remote data service whereas the OData proxy needs to serve up entities with identities with URLs that appear under its service root. As a result, you need to create a copy of the client’s model and implement proxy classes that implement the API by pulling and pushing entities into the client. This isn’t as much work as it sounds and you probably want to do it anyway so that your proxy can add value, such as hiding parts of the model that shouldn’t be proxied, adding constraints for authorisation, etc.

I’m the process of developing a set of proxy classes to act as a good starting point for this type of application. Watch this space, or reach out to me via the Pyslet home page.

OData Reference¶

The basic API for the DAL is defined by the Entity Data Model (EDM) defined in pyslet.odata2.csdl, which is extended by some core OData-specific features defined in pyslet.odata2.core and pyslet.odata2.metadata. With these three modules it is possible to create derived classes that implement the Data Access Layer API in a variety of different storage scenarios.

Entity Data Model (EDM)¶

This module defines functions and classes for working with data based on Microsoft’s Entity Data Model (EDM) as documented by the Conceptual Schema Definition Language and associated file format: http://msdn.microsoft.com/en-us/library/dd541474.aspx

The classes in this model fall in to two categories. The data classes represent the actual data objects, like simple and complex values, entities and collections. The metadata classes represent the elements of the metadata model like entity types, property definitions, associations, entity sets and so on. The metadata elements have direct XML representations, the data classes do not.

Data Model¶

class pyslet.odata2.csdl.EntityCollection(entity_set, **kwargs)¶

Bases: pyslet.odata2.csdl.DictionaryLike, pyslet.pep8.PEP8Compatibility

Represents a collection of entities from an EntitySet.

To use a database analogy, EntitySet’s are like tables whereas EntityCollections are more like the database cursors that you use to execute data access commands. An entity collection may consume physical resources (like a database connection) and so should be closed with the close() method when you’re done.

Entity collections support the context manager protocol in python so you can use them in with statements to make clean-up easier:

with entity_set.OpenCollection() as collection:
        if 42 in collection:
                print "Found it!"

The close method is called automatically when the with statement exits.

Entity collections also behave like a python dictionary of Entity instances keyed on a value representing the Entity’s key property or properties. The keys are either single values (as in the above code example) or tuples in the case of compound keys. The order of the values in the tuple is taken from the order of the PropertyRef definitions in the metadata model. You can obtain an entity’s key from the Entity.key() method.

When an EntityCollection represents an entire entity set you cannot use dictionary assignment to modify the collection. You must use insert_entity() instead where the reasons for this restriction are expanded on.

For consistency with python dictionaries the following statement is permitted, though it is effectively a no-operation:

etColl[key]=entity

The above statement raises KeyError if entity is not a member of the entity set. If key does not match the entity’s key then ValueError is raised.

Although you can’t add an entity with assignment you can delete an entity with the delete operator:

del etColl[key]

Deletes the entity with key from the entity set.

These two operations have a different meaning when a collection represents the subset of entities obtained through navigation. See NavigationCollection for details.

Notes for data providers

Derived classes MUST call super in their __init__ method to ensure the proper construction of the parent collection class. The proper way to do this is:

class MyCollection(EntityCollection):

        def __init__(self,paramA,paramsB,**kwargs):
                # paramA and paramB are examples of how to consume
                # private keyword arguments in this method so that they
                # aren't passed on to the next __init__
                super(MyCollection,self).__init__(**kwargs)

All collections require a named entity_set argument, an EntitySet instance from which all entities in the collection are drawn.

Derived classes MUST also override itervalues(). The implementation of itervalues must return an iterable object that honours the value of the expand query option, the current filter and the orderby rules.

Derived classes SHOULD also override __getitem__() and __len__() as the default implementations are very inefficient, particularly for non-trivial entity sets.

Writeable data sources must override py:meth:__delitem__.

If a particular operation is not supported for some data-service specific reason then NotImplementedError must be raised.

Writeable entity collections SHOULD override clear() as the default implementation is very inefficient.

entity_set = None¶: the entity set from which the entities are drawn

expand = None¶: the expand query option in effect

select = None¶: the select query option in effect

filter = None¶: a filter or None for no filter (see CheckFilter())

orderby = None¶: a list of orderby rules or None for no ordering

skip = None¶: the skip query option in effect

top = None¶: the top query option in effect

topmax = None¶: the provider-enforced maximum page size in effect

inlinecount = None¶

True if inlinecount option is in effect

The inlinecount option is used to alter the representation of the collection and, if set, indicates that the __len__ method will be called before iterating through the collection itself.

GetLocation()¶

Returns the location of this collection as a URI instance.

By default, the location is given as the location of the entity_set from which the entities are drawn.

GetTitle()¶

Returns a user recognisable title for the collection.

By default this is the fully qualified name of the entity set in the metadata model.

Expand(expand, select=None)¶

Sets the expand and select query options for this collection.

The expand query option causes the named navigation properties to be expanded and the associated entities to be loaded in to the entity instances before they are returned by this collection.

expand is a dictionary of expand rules. Expansions can be chained, represented by the dictionary entry also being a dictionary:

# expand the Customer navigation property...
{ 'Customer': None }
# expand the Customer and Invoice navigation properties
{ 'Customer':None, 'Invoice':None }
# expand the Customer property and then the Orders property within Customer
{ 'Customer': {'Orders':None} }

The select query option restricts the properties that are set in returned entities. The select option is a similar dictionary structure, the main difference being that it can contain the single key ‘*’ indicating that all data properties are selected.

SelectKeys()¶

Sets the select rule to select the key property/properties only.

Any expand rule is removed.

expand_entities(entityIterable)¶

Utility method for data providers.

Given an object that iterates over all entities in the collection, returns a generator function that returns expanded entities with select rules applied according to expand and select rules.

Data providers should use a better method of expanded entities if possible as this implementation simply iterates through the entities and calls Entity.Expand() on each one.

set_filter(filter)¶: Sets the filter object for this collection, see CheckFilter().

filter_entities(entityIterable)¶

Utility method for data providers.

Given an object that iterates over all entities in the collection, returns a generator function that returns only those entities that pass through the current filter object.

Data providers should use a better method of filtering entities if possible as this implementation simply iterates through the entities and calls CheckFilter() on each one.

CheckFilter(entity)¶

Checks entity against the current filter object and returns True if it passes.

This method is really a placeholder. Filtering is not covered in the CSDL model itself but is a feature of the OData pyslet.odata2.core module.

See pyslet.odata2.core.EntityCollectionMixin.CheckFilter() for more. The implementation in the case class simply raises NotImplementedError if a filter has been set.

set_orderby(orderby)¶

Sets the orderby rules for this collection.

orderby is a list of tuples, each consisting of:

( an order object as used by :py:meth:`CalculateOrderKey` , 1 | -1 )

CalculateOrderKey(entity, orderObject)¶

Given an entity and an order object returns the key used to sort the entity.

This method is really a placeholder. Ordering is not covered in the CSDL model itself but is a feature of the OData pyslet.odata2.core module.

See pyslet.odata2.core.EntityCollectionMixin.CalculateOrderKey() for more. The implementation in the case class simply raises NotImplementedError.

order_entities(entityIterable)¶

Utility method for data providers.

Given an object that iterates over the entities in random order, returns a generator function that returns the same entities in sorted order (according to the orderby object).

This implementation simply creates a list and then sorts it based on the output of CalculateOrderKey() so is not suitable for use with long lists of entities. However, if no ordering is required then no list is created.

SetInlineCount(inlinecount)¶: Sets the inline count flag for this collection.

new_entity()¶

Returns a new py:class:Entity instance suitable for adding to this collection.

The properties of the entity are set to their defaults, or to null if no default is defined (even if the property is marked as not nullable).

The entity is not considered to exist until it is actually added to the collection. At this point we deviate from dictionary-like behaviour, Instead of using assignment you must call insert_entity().:

e=collection.new_entity()
e["ID"]=1000
e["Name"]="Fred"
assert 1000 not in collection
collection[1000]=e          # raises KeyError

The correct way to add the entity is:

collection.insert_entity(e)

The first block of code is prone to problems as the key 1000 may violate the collection’s key allocation policy so we raise KeyError when assignment is used to insert a new entity to the collection. This is consistent with the concept behind OData and Atom where new entities are POSTed to collections and the ID and resulting entity are returned to the caller on success because the service may have modified them to satisfy service-specific constraints.

CopyEntity(entity)¶

Creates a new entity copying the value from entity

The key is not copied and is initially set to NULL.

insert_entity(entity)¶

Inserts entity into this entity set.

After a successful call to insert_entity:

entity is updated with any auto-generated values such as

an autoincrement correct key.
exists is set to True for entity

Data providers must override this method if the collection is writable.

If the call is unsuccessful then entity should be discarded as its associated bindings may be in a misleading state (when compared to the state of the data source itself).

A general ConstraintError will be raised when the insertion violates model constraints (including an attempt to create two entities with duplicate keys).

update_entity(entity)¶

Updates entity which must already be in the entity set.

Data providers must override this method if the collection is writable.

update_bindings(entity)¶

Iterates through the Entity.NavigationItems() and generates appropriate calls to create/update any pending bindings.

Unlike the Update() method, which updates all data and navigation values simultaneously, this method can be used to selectively update just the navigation properties.

set_page(top, skip=0, skiptoken=None)¶

Sets the page parameters that determine the next page returned by iterpage().

The skip and top query options are integers which determine the number of entities returned (top) and the number of entities skipped (skip).

skiptoken is an opaque token previously obtained from a call to next_skiptoken() on a similar collection which provides an index into collection prior to any additional skip being applied.

TopMax(topmax)¶

Sets the maximum page size for this collection.

Data consumers should use set_page() to control paging, however data providers can use this method to force the collection to limit the size of a page to at most topmax entities. When topmax is in force and is less than the top value set in set_page(), next_skiptoken() will return a suitable value for identifying the next page in the collection immediately after a complete iteration of iterpage().

Provider enforced paging is optional, if it is not supported NotImplementedError must be raised.

iterpage(set_next=False)¶

Returns an iterable subset of the values returned by itervalues()

The subset is defined by the top, skip and skiptoken values set with set_page()

If set_next is True then the page is automatically advanced so that the next call to iterpage iterates over the next page.

Data providers should override this implementation for a more efficient implementation. The default implementation simply wraps itervalues().

next_skiptoken()¶: Following a complete iteration of the generator returned by iterpage(), this method returns the skiptoken which will generate the next page or None if all requested entities were returned.

itervalues()¶

Iterates over the collection.

The collection is filtered as defined by set_filter() and sorted according to any rules defined by set_orderby().

Entities are also expanded and selected according to the rules defined by Expand.

Data providers must override this implementation which, by default, returns no entities (simulating an empty collection).

class pyslet.odata2.csdl.Entity(entity_set)¶

Bases: pyslet.odata2.csdl.TypeInstance

Represents a single instance of an EntityType.

Entity instance must only be created by data providers, a child class may be used with data provider-specific functionality. Data consumers should use the EntityCollection.new_entity() or EntityCollection.CopyEntity methods to create instances.

entity_set is the entity set this entity belongs to

Entity instances extend TypeInstance‘s dictionary-like behaviour to include all properties. As a result the dictionary values are one of SimpleValue, Complex or py:class:DeferredValue instances.

Property values are created on construction and cannot be assigned directly. To update a simple value use the value’s SimpleValue.SetFromPyVaue() method:

e['Name'].set_from_value("Steve")
        # update simple property Name
e['Address']['City'].set_from_value("Cambridge")
        # update City in complex property Address

A simple valued property that is NULL is still a SimpleValue instance, though it will behave as 0 in tests:

e['Name'].set_from_value(None)    # set to NULL
if e['Name']:
        print "Will not print!"

Navigation properties are represented as DeferredValue instances. A deferred value can be opened in a similar way to an entity set:

# open the collection obtained from navigation property Friends
with e['Friends'].OpenCollection() as friends:
        # iterate through all the friends of entity e
        for friend in friends:
                print friend['Name']

A convenience method is provided when the navigation property points to a single entity (or None) by definition:

mum=e['Mother'].GetEntity()     # may return None

In the EDM one or more properties are marked as forming the entity’s key. The entity key is unique within the entity set. On construction, an Entity instance is marked as being ‘non-existent’, exists is set to False. This is consistent with the fact that the data properties of an entity are initialised to their default values, or NULL if there is no default specified in the model. Entity instances returned as values in collection objects have exists set to True.

If an entity does not exist, OpenCollection will fail if called on one of its navigation properties with NonExistentEntity.

You can use IsEntityCollection() to determine if a property will return an EntityCollection without the cost of accessing the data source itself.

exists = None¶: whether or not the instance exists in the entity set

selected = None¶: the set of selected property names or None if all properties are selected

__iter__()¶

Iterates over the property names, including the navigation properties.

Unlike native Python dictionaries, the order in which the properties are iterated over is defined. The regular property names are yielded first, followed by the navigation properties. Within these groups properties are yielded in the order they were declared in the metadata model.

DataKeys()¶

Iterates through the names of this entity’s data properties only

The order of the names is always the order they are defined in the metadata model.

data_items()¶

Iterator that yields tuples of (key,value) for this entity’s data properties only.

The order of the items is always the order they are defined in the metadata model.

merge(fromvalue)¶

Sets this entity’s value from fromvalue which must be a TypeInstance instance. In other words, it may be either an Entity or a Complex value.

There is no requirement that fromvalue be of the same type, but it must be broadly compatible, which is defined as:

Any named property present in both the current value and fromvalue must be of compatible types.

Any named property in the current value which is not present in fromvalue is left unchanged by this method.

Null values in fromvalue are not copied.

NavigationKeys()¶

Iterates through the names of this entity’s navigation properties only.

The order of the names is always the order they are defined in the metadata model.

NavigationItems()¶

Iterator that yields tuples of (key,deferred value) for this entity’s navigation properties only.

The order of the items is always the order they are defined in the metadata model.

CheckNavigationConstraints(ignoreEnd=None)¶

For entities that do not yet exist, checks that each of the required navigation properties has been bound (with DeferredValue.BindEntity()).

If a required navigation property has not been bound then NavigationConstraintError is raised.

If the entity already exists, EntityExists is raised.

For data providers, ignoreEnd may be set to an association set end bound to this entity’s entity set. Any violation of the related association is ignored.

IsNavigationProperty(name)¶: Returns true if name is the name of a navigation property, False otherwise.

IsEntityCollection(name)¶: Returns True if name is the name of a navigation property that points to an entity collection, False otherwise.

Update()¶

Updates this entity following modification.

You can use select rules to provide a hint about which fields have been updated. By the same logic, you cannot update a property that is not selected!

The default implementation opens a collection object from the parent entity set and calls EntityCollection.update_entity().

Delete()¶

Deletes this entity from the parent entity set.

The default implementation opens a collection object from the parent entity set and uses the del operator.

Data providers must ensure that the entity’s exists flag is set to False after deletion.

key()¶

Returns the entity key as a single python value or a tuple of python values for compound keys.

The order of the values is always the order of the PropertyRef definitions in the associated EntityType’s key.

set_key(key)¶

Sets this entity’s key from a single python value or tuple.

The entity must be non-existent or EntityExists is raised.

auto_key(base=None)¶

Sets the key to a random value

base: An optional key suggestion which can be used to influence the choice of automatically generated key.

KeyDict()¶: Returns the entity key as a dictionary mapping key property names onto SimpleValue instances.

Expand(expand, select=None)¶

Expands and selects properties of the entity according to the given expand and select rules (if any).

Data consumers will usually apply expand rules to a collection which will then automatically ensure that all entities returned by the collection have been expanded.

If, as a result of select, a non-key property is unselected then its value is set to NULL. (Properties that comprise the key are never NULL.)

If a property that is being expanded is also subject to one or more selection rules these are passed along with any chained Expand method call.

The selection rules in effect are saved in the select member and can be tested using Selected().

Selected(name)¶

Returns true if the property name is selected in this entity.

You should not rely on the value of a unselected property, in most cases it will be set to NULL.

ETag()¶: Returns a list of EDMValue instance values to use for optimistic concurrency control or None if the entity does not support it (or if all concurrency tokens are NULL or unselected).

ETagValues()¶: Returns a list of EDMValue instance values that may be used for optimistic concurrency control. The difference between this method and ETag() is that this method returns all values even if they are NULL or unselected. If there are no concurrency tokens then an empty list is returned.

generate_ctoken()¶

Returns a hash object representing this entity’s value.

The hash is a SHA256 obtained by concatenating the literal representations of all data properties (strings are UTF-8 encoded) except the keys and properties which have Fixed concurrency mode.

SetConcurrencyTokens()¶

A utility method for data providers.

Sets all ETagValues() using the following algorithm:

Binary values are set directly from the output of

generate_ctoken()
String values are set from the hexdigest of the output

generate_ctoken()
Integer values are incremented.
DateTime and DateTimeOffset values are set to the current

time in UTC (and nudged by 1s if necessary)
Guid values are set to a new random (type 4) UUID.

Any other type will generate a ValueError.

ETagIsStrong()¶

Returns True if this entity’s etag is a strong entity tag as defined by RFC2616:

A "strong entity tag" MAY be shared by two entities of a
resource only if they are equivalent by octet equality.

The default implementation returns False which is consistent with the implementation of generate_ctoken() as that does not include the key fields.

class pyslet.odata2.csdl.SimpleValue(pDef=None)¶

Bases: pyslet.odata2.csdl.EDMValue

An abstract class that represents a value of a simple type in the EDMModel.

This class is not designed to be instantiated directly, use one of the factory methods in EdmValue to construct one of the specific child classes.

typeCode = None¶: the SimpleType code

mtype = None¶: an optional pyslet.rfc2616.MediaType representing this value

value = None¶

The actual value or None if this instance represents a NULL value

The python type used for value depends on typeCode as follows:

Edm.Boolean: one of the Python constants True or False
Edm.Byte, Edm.SByte, Edm.Int16, Edm.Int32: int
Edm.Int64: long
Edm.Double, Edm.Single: python float
Edm.Decimal: python Decimal instance (from decimal module)
Edm.DateTime, Edm.DateTimeOffset: py:class:pyslet.iso8601.TimePoint instance
Edm.Time: py:class:pyslet.iso8601.Time instance (not a Duration, note corrected v2 specification of OData)
Edm.Binary: raw string
Edm.String: unicode string
Edm.Guid: python UUID instance (from uuid module)

For future compatibility, this attribute should only be updated using set_from_value() or one of the other related methods.

SimpleCast(typeCode)¶

Returns a new SimpleValue instance created from typeCode

The value of the new instance is set using Cast()

Cast(targetValue)¶

Updates and returns targetValue a SimpleValue instance.

The value of targetValue is replaced with a value cast from this instance’s value.

If the types are incompatible a TypeError is raised, if the values are incompatible then ValueError is raised.

NULL values can be cast to any value type.

SetFromSimpleValue(new_value)¶: The reverse of the Cast() method, sets this value to the value of new_value casting as appropriate.

__eq__(other)¶: Instances compare equal only if they are of the same type and have values that compare equal.

__unicode__()¶

Formats this value into its literal form.

NULL values cannot be represented in literal form and will raise ValueError.

SetFromLiteral(value)¶

Decodes a value from the value’s literal form.

You can get the literal form of a value using the unicode function.

SetNull()¶: Sets the value to NULL

set_from_value(new_value)¶: Sets the value from a python variable coercing new_value if necessary to ensure it is of the correct type for the value’s typeCode.

SetRandomValue(base=None)¶

Sets a random value based

base: a SimpleValue instance of the same type that may be used as a base or stem or the random value generated or may be ignored, depending on the value type.

classmethod Copy(value)¶: Constructs a new SimpleValue instance by copying value

class pyslet.odata2.csdl.NumericValue(pDef=None)¶

Bases: pyslet.odata2.csdl.SimpleValue

An abstract class that represents all numeric simple values.

The literal forms of numeric values are parsed in a two-stage process. Firstly the utility class Parser is used to obtain a numeric tuple and then the value is set using SetFromNumericLiteral()

All numeric types may have their value set directly from int, long, float or Decimal.

Integer representations are rounded towards zero using the python int or long functions when necessary.

SetToZero()¶: Set this value to the default representation of zero

SetFromNumericLiteral(numericValue)¶: Decodes a value from a numeric tuple as returned by Parser.ParseNumericLiteral().

class pyslet.odata2.csdl.FloatValue(pDef=None)¶

Bases: pyslet.odata2.csdl.NumericValue

Abstract class that represents one of Edm.Double or Edm.Single.

Values can be set from int, long, float or Decimal.

There is no hard-and-fast rule about the representation of float in Python and we may refuse to accept values that fall within the accepted ranges defined by the CSDL if float cannot hold them. That said, you won’t have this problem in practice.

The derived classes SingleValue and DoubleValue only differ in the Max value used when range checking.

Values are formatted using Python’s default unicode conversion.

Primitive SimpleTypes¶

Simple values can be created directly using one of the type-specific classes below.

class pyslet.odata2.csdl.BinaryValue(pDef=None)¶

Bases: pyslet.odata2.csdl.SimpleValue

Represents a SimpleValue of type Edm.Binary.

Binary literals allow content in the following form:

[A-Fa-f0-9][A-Fa-f0-9]*

Binary values can be set from any Python type, though anything other than a binary string is set to its pickled representation. There is no reverse facility for reading an object from the pickled value.

class pyslet.odata2.csdl.BooleanValue(pDef=None)¶

Bases: pyslet.odata2.csdl.SimpleValue

Represents a simple value of type Edm.Boolean

Boolean literals are one of:

true | false

Boolean values can be set from their Python equivalents and from any int, long, float or Decimal where the non-zero test is used to set the value.

class pyslet.odata2.csdl.ByteValue(pDef=None)¶

Bases: pyslet.odata2.csdl.NumericValue

Represents a simple value of type Edm.Byte

Byte literals must not have a sign, decimal point or exponent.

Byte values can be set from an int, long, float or Decimal

class pyslet.odata2.csdl.DateTimeValue(pDef=None)¶

Bases: pyslet.odata2.csdl.SimpleValue

Represents a simple value of type Edm.DateTime

DateTime literals allow content in the following form:

yyyy-mm-ddThh:mm[:ss[.fffffff]]

DateTime values can be set from an instance of iso8601.TimePoint or type int, long, float or Decimal.

Any zone specifier is ignored. There is no conversion to UTC, the value simply becomes a local time in an unspecified zone. This is a weakness of the EDM, it is good practice to limit use of the DateTime type to UTC times.

When set from a numeric value, the value must be non-negative. Unix time is assumed. See the FromUnixTime() factory method of TimePoint for information.

If a property definition was set on construction then the defined precision is used when representing the value as a unicode string. For example, if the property has precision 3 then the output of the unicode conversion will appear in the following form:

1969-07-20T20:17:40.000

class pyslet.odata2.csdl.DateTimeOffsetValue(pDef=None)¶

Bases: pyslet.odata2.csdl.SimpleValue

Represents a simple value of type Edm.DateTimeOffset

DateTimeOffset literals are defined in terms of the XMLSchema lexical representation.

DateTimeOffset values can be set from an instance of iso8601.TimePoint or type int, long, float or Decimal.

TimePoint instances must have a zone specifier. There is no automatic assumption of UTC.

When set from a numeric value, the value must be non-negative. Unix time in UTC assumed. See the FromUnixTime() factory method of TimePoint for information.

If a property definition was set on construction then the defined precision is used when representing the value as a unicode string. For example, if the property has precision 3 then the output of the unicode conversion will appear in the following form:

1969-07-20T15:17:40.000-05:00

It isn’t completely clear if the canonical representation of UTC using ‘Z’ instead of an offset is intended or widely supported so we always use an offset:

1969-07-20T20:17:40.000+00:00

class pyslet.odata2.csdl.DecimalValue(pDef=None)¶

Bases: pyslet.odata2.csdl.NumericValue

Represents a simple value of type Edm.Decimal

Decimal literals must not use exponent notation and there must be no more than 29 digits to the left and right of the decimal point.

Decimal values can be set from int, long, float or Decimal values.

class pyslet.odata2.csdl.DoubleValue(pDef=None)¶

Bases: pyslet.odata2.csdl.FloatValue

Represents a simple value of type Edm.Double

Max = 1.7976931348623157e+308¶

the largest positive double value

This value is set dynamically on module load, theoretically it may be set lower than the maximum allowed by the specification if Python’s native float is of insufficient precision but this is unlikely to be an issue.

MaxD = Decimal('1.79769313486E+308')¶: the largest positive double value converted to decimal form

class pyslet.odata2.csdl.GuidValue(pDef=None)¶

Bases: pyslet.odata2.csdl.SimpleValue

Represents a simple value of type Edm.Guid

Guid literals allow content in the following form: dddddddd-dddd-dddd-dddd-dddddddddddd where each d represents [A-Fa-f0-9].

Guid values can also be set directly from either binary or hex strings. Binary strings must be of length 16 and are passed as raw bytes to the UUID constructor, hexadecimal strings can be string or unicode strings and must be of length 32 characters.

class pyslet.odata2.csdl.Int16Value(pDef=None)¶

Bases: pyslet.odata2.csdl.NumericValue

Represents a simple value of type Edm.Int16

class pyslet.odata2.csdl.Int32Value(pDef=None)¶

Bases: pyslet.odata2.csdl.NumericValue

Represents a simple value of type Edm.Int32

class pyslet.odata2.csdl.Int64Value(pDef=None)¶

Bases: pyslet.odata2.csdl.NumericValue

Represents a simple value of type Edm.Int64

class pyslet.odata2.csdl.SByteValue(pDef=None)¶

Bases: pyslet.odata2.csdl.NumericValue

Represents a simple value of type Edm.SByte

class pyslet.odata2.csdl.SingleValue(pDef=None)¶

Bases: pyslet.odata2.csdl.FloatValue

Represents a simple value of type Edm.Single

Max = 3.4028234663852886e+38¶

the largest positive single value

This value is set dynamically on module load, theoretically it may be set lower than the maximum allowed by the specification if Python’s native float is of insufficient precision but this is very unlikely to be an issue unless you’ve compiled Python on in a very unusual environment.

MaxD = Decimal('3.40282346639E+38')¶: the largest positive single value converted to Decimal

SetFromNumericLiteral(numericValue)¶: Decodes a Single value from a Numeric literal.

class pyslet.odata2.csdl.StringValue(pDef=None)¶

Bases: pyslet.odata2.csdl.SimpleValue

Represents a simple value of type Edm.String”

The literal form of a string is the string itself.

Values may be set from any string or object which supports the native unicode function.

class pyslet.odata2.csdl.TimeValue(pDef=None)¶

Bases: pyslet.odata2.csdl.SimpleValue

Represents a simple value of type Edm.Time

Time literals allow content in the form:

hh:mm:ss.sss

Time values can be set from an instance of pyslet.iso8601.Time, int, long, float or Decimal.

When set from a numeric value the value must be in the range 0..86399.9̅ and is treated as an elapsed time in seconds since midnight.

If a property definition was set on construction then the defined precision is used when representing the value as a unicode string. For example, if the property has precision 3 then the output of the unicode conversion will appear in the following form:

20:17:40.000

Complex Types¶

class pyslet.odata2.csdl.Complex(pDef=None)¶

Bases: pyslet.odata2.csdl.EDMValue, pyslet.odata2.csdl.TypeInstance

Represents a single instance of a ComplexType.

IsNull()¶: Complex values are never NULL

SetNull()¶: Sets all simple property values to NULL recursively

merge(new_value)¶

Sets this value from new_value which must be a Complex instance.

There is no requirement that new_value is of the same type, but it must be broadly compatible, which is defined as:

Any named property present in both the current value and new_value must be of compatible types.

Any named property in the current value which is not present in new_value is left unchanged by this method.

Null values are not merged.

Navigation: Deferred Values¶

class pyslet.odata2.csdl.DeferredValue(name, from_entity)¶

Bases: object

Represents the value of a navigation property.

name = None¶: the name of the associated navigation property

from_entity = None¶: the entity that contains this value

pDef = None¶: the definition of the navigation property

isRequired = None¶: True if this deferred value represents a (single) required entity

isCollection = None¶: True if this deferred value represents a collection

isExpanded = None¶

True if this deferred value has been expanded.

An expanded navigation property will return a read-only ExpandedEntityCollection when OpenCollection() is called.

bindings = None¶: The list of entity instances or keys to bind to from_entity when it is inserted or next updated.

Target()¶: Returns the target entity set of this navigation (without opening the collection).

GetEntity()¶

Returns a single entity instance or None.

If this deferred value represents an entity collection then NavigationError is raised.

OpenCollection()¶

Opens the collection associated with this navigation property.

Returns an EntityCollection instance which must be closed when it is no longer needed. This is best achieved with the Python with statement using the collection’s context-manager behaviour. For example:

with customer['Orders'].OpenCollection() as orders:
        # do something with the orders

SetExpansion(expandedCollection)¶

Sets the expansion for this deferred value to the ExpandedEntityCollection given.

If expandedCollection is None then the expansion is removed and future calls to OpenColection() will yield a (dynamically created) entity collection.

ExpandCollection(expand, select)¶

A convenience function of use to data providers.

Expands this navigation property, further expanding the resulting collection of entities using the given expand and select options (see EntityCollection.Expand() for details).

BindEntity(target)¶

Binds a target entity to this navigation property.

target is either the entity you’re binding or its key in the target entity set. For example:

customer['Orders'].bind(1)

binds the entity represented by ‘customer’ to the Order entity with key 1.

Just as for updates to data property values, the binding information is saved and acted upon when the entity is next updated or, for non-existent entities, inserted into the entity set.

If you attempt to bind to a target entity that doesn’t exist the target entity will be created automatically when the source entity is updated or inserted.

CheckNavigationConstraint()¶

Checks if this navigation property isRequired and raises NavigationConstraintError if it has not been bound with BindEntity().

This method is only intended to be called on non-existent entities.

update_bindings()¶

Iterates through bindings and generates appropriate calls to update the collection.

Unlike the parent Entity’s Entity.Update() method, which updates all data and navigation values simultaneously, this method can be used to selectively update a single navigation property.

ClearBindings()¶: Removes any (unsaved) entity bindings from this navigation property.

class pyslet.odata2.csdl.NavigationCollection(from_entity, name, **kwargs)¶

Bases: pyslet.odata2.csdl.EntityCollection

Represents the collection of entities returned by a navigation property.

These collections behave in the same way as entity collections opened from the base EntitySet with the following exceptions:

etColl[key]=entity

Adds a link to entity from the source entity used to open the navigation collection. If key does not match entity‘s key then ValueError is raised. The entity must already exist and be a member of the base entity set, otherwise KeyError is raised.

This class is used even if the navigation property is declared to return a single entity, rather than a collection. In this case assignment will only work if the collection is currently empty. To replace an existing link use replace().

del etColl[key]

Deletes the link from the source entity to the entity with key. If no such link exists, KeyError is raised.

Thie behaviour differs from the base EntityCollection behaviour where the del operator removes the entity completely from the entity container. In this case the entity still exists in the parent entity set, only the link is removed.

Notes for data providers

On construction:

entity_set is the entity set containing the target entities,

the collection behaves like a subset of this entity set. It is passed to super

Named arguments specific to this class:

from_entity is the source entity being navigated
name is the name of the navigation property being navigated

Writeable collections must override the __setitem__() method.

name = None¶: the name of the navigation property

from_entity = None¶: the source entity

from_end = None¶: the AssociationSetEnd that represents the source of this association

pDef = None¶: the navigation property’s definition in the metadata model

insert_entity(entity)¶: Inserts a new entity into the target entity set and simultaneously creates a link to it from the source entity.

replace(entity)¶

This method replaces all links with a link to the single item, entity. If the collection was empty then this is equivalent to __setitem__(entity.key(),entity).

Although for some collections this is equivalent to clear() followed by __setitem__, this method must be used to combine these operations into a single call when the collection is required to contain exactly one link at all times.

class pyslet.odata2.csdl.ExpandedEntityCollection(entityList, **kwargs)¶

Bases: pyslet.odata2.csdl.NavigationCollection

A special sub-class of NavigationCollection used when a navigation property has been expanded.

An expanded entity collection is a read-only, cached view of the entities linked from the source entity.

Warning: although you may apply a filter and orderby rules to an expanded collection these are evaluated on the local copy and are not passed to the data source. As a result, there may be differences in the way these options behave due to different expression semantics.

Note for data providers:

The named argument entityList passed to this constructor is a simple python list of the entities the expanded collection contains. Internally a dictionary of the entities is built to speed up access by key.

Supporting Classes¶

class pyslet.odata2.csdl.EDMValue(pDef=None)¶

Bases: pyslet.pep8.PEP8Compatibility

Abstract class to represent a value in the EDMModel.

This class is used to wrap or ‘box’ instances of a value. In particular, it can be used in a context where that value can have either a simple or complex type.

pDef = None¶: An optional Property instance from the metadata model defining this value’s type

__nonzero__()¶: EDMValue instances are treated as being non-zero if IsNull() returns False.

IsNull()¶: Returns True if this object is Null.

classmethod NewValue(pDef)¶

Constructs an instance of the correct child class of EDMValue to represent a value defined by Property instance pDef.

We support a special case for creating a type-less NULL. If you pass None for pDef then a type-less SipmleValue is instantiated.

classmethod NewSimpleValue(typeCode)¶: Constructs an instance of the correct child class of EDMValue to represent an (undeclared) simple value of SimpleType typeCode.

classmethod NewSimpleValueFromValue(value)¶

Constructs an instance of the correct child class of EDMValue to hold value.

value may be any of the types listed in SimpleValue.

class pyslet.odata2.csdl.TypeInstance(type_def=None)¶

Bases: pyslet.odata2.csdl.DictionaryLike, pyslet.pep8.PEP8Compatibility

Abstract class to represents a single instance of a ComplexType or EntityType.

Behaves like a read-only dictionary mapping property names onto EDMValue instances. (You can change the value of a property using the methods of EDMValue and its descendants.)

Unlike regular Python dictionaries, iteration over the of keys in the dictionary (the names of the properties) is always done in the order in which they are declared in the type definition.

type_def = None¶: the definition of this type

Metadata Model¶

class pyslet.odata2.csdl.CSDLElement(parent, name=None)¶

Bases: pyslet.xmlnames20091208.XMLNSElement, pyslet.pep8.PEP8Compatibility

All elements in the metadata model inherit from this class.

class pyslet.odata2.csdl.Schema(parent)¶

Bases: pyslet.odata2.csdl.NameTableMixin, pyslet.odata2.csdl.CSDLElement

Represents the Edm root element.

Schema instances are based on NameTableMixin allowing you to look up the names of declared Associations, ComplexTypes, EntityTypes, EntityContainers and Functions using dictionary-like methods.

name = None¶: the declared name of this schema

Association = None¶: a list of Association instances

ComplexType = None¶: a list of ComplexType instances

EntityType = None¶: a list of EntityType instances

class pyslet.odata2.csdl.EntityContainer(parent)¶

Bases: pyslet.odata2.csdl.NameTableMixin, pyslet.odata2.csdl.CSDLElement

Models an entity container in the metadata model.

An EntityContainer inherits from NameTableMixin to enable it to behave like a scope. The EntitySet instances and AssociationSet instances it contains are declared within the scope.

name = None¶: the declared name of the container

Documentation = None¶: the optional Documentation

EntitySet = None¶: a list of EntitySet instances

AssociationSet = None¶: a list of AssociationSet instances

class pyslet.odata2.csdl.EntitySet(parent)¶

Bases: pyslet.odata2.csdl.CSDLElement

Represents an EntitySet in the metadata model.

name = None¶: the declared name of the entity set

entityTypeName = None¶: the name of the entity type of this set’s elements

entityType = None¶: the EntityType of this set’s elements

keys = None¶: a list of the names of this entity set’s keys in their declared order

navigation = None¶: a mapping from navigation property names to AssociationSetEnd instances

linkEnds = None¶: A mapping from AssociationSetEnd instances that reference this entity set to navigation property names (or None if this end of the association is not bound to a named navigation property)

unboundPrincipal = None¶

An AssociationSetEnd that represents our end of an association with an unbound principal or None if all principals are bound.

What does that mean? It means that there is an association set bound to us where the other role has a multiplicity of 1 (required) but our entity type does not have a navigation property bound to the association. As a result, our entities can only be created by a deep insert from the principal (the entity set at the other end of the association).

Clear as mud? An example may help. Suppose that each Order entity must have an associated Customer but (perhaps perversely) there is no navigation link from Order to Customer, only from Customer to Order. Attempting to create an Order in the base collection of Orders will always fail:

with Orders.OpenCollection() as collection:
    order=collection.new_entity()
    # set order fields here
    collection.insert_entity(order)
    # raises ConstraintError as order is not bound to a customer

Instead, you have to create new orders from a Customer entity:

with Customers.OpenCollection() as collectionCustomers:
    # get the existing customer
    customer=collectionCustomers['ALFKI']
    with customer['Orders'].OpenCollection() as collectionOrders:
        # create a new order
        order=collectionOrders.new_entity()
        # ... set order details here
        collectionOrders.insert_entity(order)

You can also use a deep insert:

with Customers.OpenCollection() as collectionCustomers,
        Orders.OpenCollection() as collectionOrders:
    customer=collectionCustomers.new_entity()
    # set customer details here
    order=collectionOrders.new_entity()
    # set order details here
    customer['Orders'].BindEntity(order)
    collectionCustomers.insert_entity(customer)

For the avoidance of doubt, an entity set can’t have two unbound principals because if it did you would never be able to create entities in it!

Documentation = None¶: the optional Documentation

GetFQName()¶: Returns the fully qualified name of this entity set.

GetLocation()¶: Returns a pyslet.rfc2396.URI instance representing the location for this entity set.

SetLocation()¶

Sets the location of this entity set by resolving a relative path consisting of:

[ EntityContainer.name '.' ] name

The resolution of URIs is done in accordance with the XML specification, so is affected by any xml:base attributes set on parent elements or by the original base URI used to load the metadata model. If no base URI can be found then the location remains expressed in relative terms.

GetKey(keylike)¶

Extracts a key value suitable for using as a key in an EntityCollection based on this entity set.

Keys are represented as python values (as described in SimpleValue) or as tuples of python values in the case of compound keys. The order of the values in a compound key is the order in which the Key properties are defined in the corresponding EntityType definition.

If keylike is already in the correct format for this entity type then it is returned unchanged.

If the key is single-valued and keylike is a tuple containing a single value then the single value is returned without the tuple wrapper.

If keylike is a dictionary, or an Entity instance, which maps property names to values (or to SimpleValue instances) the key is calculated from it by extracting the key properties. As a special case, a value mapped with a dictionary key of the empty string is assumed to be the value of the key property for an entity type with a single-valued key, but only if the key property’s name is not itself in the dictionary.

If keylike cannot be turned in to a valid key the KeyError is raised.

extract_key(keyvalue)¶

Extracts a key value from keylike.

Unlike GetKey, this method attempts to convert the data in keyvalue into the correct format for the key. For compound keys keyvalue must be a suitable list or tuple or compatible iterable supporting the len method. Dictionaries are not supported.

If keyvalue cannot be converted into a suitable representation of the key then None is returned.

GetKeyDict(key)¶

Given a key from this entity set, returns a key dictionary.

The result is a mapping from named properties to SimpleValue instances. As a special case, if a single property defines the entity key it is represented using the empty string, not the property name.

bind(entityCollectionBinding, **extraArgs)¶

Binds this entity set to a specific class or callable used by OpenCollection()

entityCollectionBinding must be a class (or other callable) that returns an EntityCollection instance, by default we are bound to the default EntityCollection class which behaves like an empty collection.

extraArgs is a python dict of named arguments to pass to the binding callable

OpenCollection()¶: Returns an EntityCollection instance suitable for accessing the entities themselves.

BindNavigation(name, entityCollectionBinding, **extraArgs)¶

Binds the navigation property name to a class or callable used by OpenNavigation()

entityCollectionBinding must be a class (or other callable) that returns a NavigationCollection instance. By default we are bound to the default NavigationCollection class which behaves like an empty collection.

extraArgs is a python dict of named arguments to pass to the binding callable

OpenNavigation(name, sourceEntity)¶: Returns a NavigationCollection instance suitable for accessing the entities obtained by navigating from sourceEntity, an Entity instance, via the navigation property with name.

NavigationTarget(name)¶: Returns the target entity set of navigation property name

NavigationMultiplicity(name)¶

Returns the Multiplicity of both the source and the target of the named navigation property, as a tuple, for example, if customers is an entity set from the sample OData service:

customers.NavigationMultiplicity['Orders']==(Multiplicity.ZeroToOne,Multiplicity.Many)

IsEntityCollection(name)¶: Returns True if more than one entity is possible when navigating the named property.

class pyslet.odata2.csdl.AssociationSet(parent)¶

Bases: pyslet.odata2.csdl.CSDLElement

Represents an association set in the metadata model.

The purpose of the association set is to bind the ends of an association to entity sets in the container.

Contrast this with the association element which merely describes the association between entity types.

At first sight this part of the entity data model can be confusing but imagine an entity container that contains two entity sets that have the same entity type. Any navigation properties that reference this type will need to be explicitly bound to one or other of the entity sets in the container.

As an aside, it isn’t really clear if the model was intended to be used this way. It may have been intended that the entity type in the definition of an entity set should be unique within the scope of the entity container.

name = None¶: the declared name of this association set

associationName = None¶: the name of the association definition

association = None¶: the Association definition

Documentation = None¶: the optional Documentation

class pyslet.odata2.csdl.AssociationSetEnd(parent)¶

Bases: pyslet.odata2.csdl.CSDLElement

Represents the links between two actual sets of entities in the metadata model.

The GetQualifiedName() method defines the identity of this element. The built-in Python hash function returns a hash based on this value and the associated comparison functions are also implemented enabling these elements to be added to ordinary Python dictionaries.

Oddly, role names are sometimes treated as optional but it can make it a challenge to work out which end of the association is which when we are actually using the model if one or both are missing. The algorithm we use is to use role names if either are given, otherwise we match the entity types. If these are also identical then the choice is arbitrary. To prevent confusion missing role names are filled in when the metadata model is loaded.

name = None¶: the role-name given to this end of the link

entitySetName = None¶: name of the entity set this end links to

entity_set = None¶: EntitySet this end links to

associationEnd = None¶: AssociationEnd that defines this end of the link

otherEnd = None¶: the other AssociationSetEnd of this link

Documentation = None¶: the optional Documentation

GetQualifiedName()¶

A utility function to return a qualified name.

The qualified name comprises the name of the parent AssociationSet and the role name.

class pyslet.odata2.csdl.Type(parent)¶

Bases: pyslet.odata2.csdl.NameTableMixin, pyslet.odata2.csdl.CSDLElement

An abstract class for both Entity and Complex types.

Types inherit from NameTableMixin to allow them to behave as scopes in their own right. The named properties are declared in the type’s scope enabling you so use them as dictionaries to look up property definitions.

Because of the way nested scopes work, this means that you can concatenate names to do a deep look up, for example, if Person is a defined type:

Person['Address']['City'] is Person['Address.City']

name = None¶: the declared name of this type

baseType = None¶: the name of the base-type for this type

Property = None¶: a list of Property

GetFQName()¶: Returns the full name of this type, including the schema namespace prefix.

class pyslet.odata2.csdl.EntityType(parent)¶

Bases: pyslet.odata2.csdl.Type

Models the key and the collection of properties that define a set of Entity

Key = None¶: the Key

ValidateExpansion(expand, select)¶

A utility method for data providers.

Checks the expand and select options, as described in EntityCollection.Expand() for validity raising ValueError if they violate the OData specification.

Specifically the following are checked:

That “*” only ever appears as the last item in a select

path
That nothing appears after a simple property in a select

path
That all names are valid property names
That all expanded names are those of navigation properties

class pyslet.odata2.csdl.Key(parent)¶

Bases: pyslet.odata2.csdl.CSDLElement

Models the key fields of an EntityType

PropertyRef = None¶: a list of PropertyRef

class pyslet.odata2.csdl.PropertyRef(parent)¶

Bases: pyslet.odata2.csdl.CSDLElement

Models a reference to a single property within a Key.

name = None¶: the name of this (key) property

property = None¶: the Property instance of this (key) property

UpdateTypeRefs(scope, stopOnErrors=False)¶: Sets property

class pyslet.odata2.csdl.Property(parent)¶

Bases: pyslet.odata2.csdl.CSDLElement

Models a property of an EntityType or ComplexType.

Instances of this class are callable, taking an optional string literal. They return a new EDMValue instance with a value set from the optional literal or NULL if no literal was supplied. Complex values can’t be created from a literal.

name = None¶: the declared name of the property

type = None¶: the name of the property’s type

simpleTypeCode = None¶: one of the SimpleType constants if the property has a simple type

complexType = None¶: the associated ComplexType if the property has a complex type

nullable = None¶: if the property may have a null value

defaultValue = None¶: a string containing the default value for the property or None if no default is defined

maxLength = None¶: the maximum length permitted for property values

fixedLength = None¶: a boolean indicating that the property must be of length maxLength

precision = None¶: a positive integer indicating the maximum number of decimal digits (decimal values)

scale = None¶: a non-negative integer indicating the maximum number of decimal digits to the right of the point

unicode = None¶: a boolean indicating that a string property contains unicode data

Documentation = None¶: the optional Documentation

class pyslet.odata2.csdl.ComplexType(parent)¶

Bases: pyslet.odata2.csdl.Type

Models the collection of properties that define a Complex value.

This class is a trivial sub-class of Type

class pyslet.odata2.csdl.NavigationProperty(parent)¶

Bases: pyslet.odata2.csdl.CSDLElement

Models a navigation property of an EntityType.

name = None¶: the declared name of the navigation property

fromRole = None¶: the name of this link’s source role

toRole = None¶: the name of this link’s target role

from_end = None¶: the AssociationEnd instance representing this link’s source

toEnd = None¶: the AssociationEnd instance representing this link’s target

backLink = None¶: the NavigationProperty that provides the back link (or None, if this link is one-way)

class pyslet.odata2.csdl.Association(parent)¶

Bases: pyslet.odata2.csdl.NameTableMixin, pyslet.odata2.csdl.CSDLElement

Models an association.

This class inherits from NameTableMixin to enable it to behave like a scope in its own right. The contained AssociationEnd instances are declared in the association scope by role name.

name = None¶: the name declared for this association

Documentation = None¶: the optional Documentation

AssociationEnd = None¶: a list of AssociationEnd instances

class pyslet.odata2.csdl.AssociationEnd(parent)¶

Bases: pyslet.odata2.csdl.CSDLElement

Models one end of an Association.

name = None¶: the role-name given to this end of the link

type = None¶: name of the entity type this end links to

entityType = None¶: EntityType this end links to

multiplicity = None¶: a Multiplicity constant

otherEnd = None¶: the other AssociationEnd of this link

class pyslet.odata2.csdl.Documentation(parent)¶

Bases: pyslet.odata2.csdl.CSDLElement

Used to document elements in the metadata model

Misc Definitions¶

pyslet.odata2.csdl.ValidateSimpleIdentifier(identifier)¶: Validates a simple identifier, returning the identifier unchanged or raising ValueError.

class pyslet.odata2.csdl.SimpleType¶

Bases: pyslet.xsdatatypes20041028.Enumeration

SimpleType defines constants for the core data types defined by CSDL

SimpleType.Boolean
SimpleType.DEFAULT == None

For more methods see Enumeration

The canonical names for these constants uses the Edm prefix, for example, “Edm.String”. As a result, the class has attributes of the form “SimpleType.Edm.Binary” which are inaccessible to python unless getattr is used. To workaround this problem (and because the Edm. prefix seems to be optional) we also define aliases without the Edm. prefix. As a result you can use, e.g., SimpleType.Int32 as the symbolic representation in code but the following are all True:

SimpleType.DecodeValue(u"Edm.Int32")==SimpleType.Int32
SimpleType.DecodeValue(u"Int32")==SimpleType.Int32
SimpleType.EncodeValue(SimpleType.Int32)==u"Edm.Int32"  

PythonType = {<type 'str'>: 14, <type 'unicode'>: 14, <type 'long'>: 7, <type 'float'>: 8, <type 'bool'>: 2, <type 'int'>: 13}¶

A python dictionary that maps a type code (defined by the types module) to a constant from this class indicating a safe representation in the EDM. For example:

SimpleType.PythonType[types.IntType]==SimpleType.Int64

class pyslet.odata2.csdl.ConcurrencyMode¶

Bases: pyslet.xsdatatypes20041028.Enumeration

ConcurrencyMode defines constants for the concurrency modes defined by CSDL

ConcurrencyMode.Fixed
ConcurrencyMode.DEFAULT == ConcurrencyMode.none

Note that although ‘Fixed’ and ‘None’ are the correct values lower-case aliases are also defined to allow the value ‘none’ to be accessible through normal attribute access. In most cases you won’t need to worry as a test such as the following is sufficient:

if property.concurrencyMode==ConcurrencyMode.Fixed:

# do something with concurrency tokens

For more methods see Enumeration

pyslet.odata2.csdl.DecodeMaxLength(value)¶

Decodes a maxLength value from a unicode string.

“The maxLength facet accepts a value of the literal string “max” or a positive integer with value ranging from 1 to 2^31”

The value ‘max’ is returned as the value MAX

pyslet.odata2.csdl.EncodeMaxLength(value)¶: Encodes a maxLength value as a unicode string.

pyslet.odata2.csdl.MAX = -1¶: we define the constant MAX to represent the special ‘max’ value of maxLength

class pyslet.odata2.csdl.Multiplicity¶: Defines constants for representing association end multiplicities.

pyslet.odata2.csdl.DecodeMultiplicity(src)¶

Decodes a Multiplicity value from a unicode string.

The valid strings are “0..1”, “1” and “*”

pyslet.odata2.csdl.EncodeMultiplicity(value)¶: Encodes a Multiplicity value as a unicode string.

class pyslet.odata2.csdl.Parser(source)¶

Bases: pyslet.unicode5.BasicParser

A CSDL-specific parser, mainly for decoding literal values of simple types.

The individual parsing methods may raise ValueError in cases where parsed value has a value that is out of range.

ParseBinaryLiteral()¶: Parses a binary literal, returning a binary string

ParseBooleanLiteral()¶: Parses a boolean literal returning True, False or None if no boolean literal was found.

ParseByteLiteral()¶

Parses a byteLiteral, returning a python integer.

We are generous in what we accept, ignoring leading zeros. Values outside the range for byte return None.

ParseDateTimeLiteral()¶

Parses a DateTime literal, returning a pyslet.iso8601.TimePoint instance.

Returns None if no DateTime literal can be parsed. This is a generous way of parsing iso8601-like values, it accepts omitted zeros in the date, such as 4-7-2001.

ParseGuidLiteral()¶

Parses a Guid literal, returning a UUID instance from the uuid module.

Returns None if no Guid can be parsed.

ParseNumericLiteral()¶

Parses a numeric literal returning a named tuple of strings:

( sign, lDigits, rDigits, expSign, eDigits )

An empty string indicates a component that was not present except that rDigits will be None if no decimal point was present. Likewise, eDigits may be None indicating that no exponent was found.

Although both lDigits and rDigits can be empty they will never both be empty strings. If there are no digits present then the method returns None, rather than a tuple. Therefore, forms like “E+3” are not treated as being numeric literals whereas, perhaps oddly, 1E+ is parsed as a numeric literal (even though it will raise ValueError later when setting any of the numeric value types).

Representations of infinity and not-a-number result in lDigits being set to ‘inf’ and ‘nan’ respectively. They always result in rDigits and eDigits being None.

ParseTimeLiteral()¶

Parses a Time literal, returning a pyslet.iso8601.Time instance.

Returns None if no Time literal can be parsed. This is a generous way of parsing iso8601-like values, it accepts omitted zeros in the leading field, such as 7:45:00.

Utility Classes¶

These classes are not specific to the EDM but are used to support the implementation. They are documented to allow them to be reused in other modules.

class pyslet.odata2.csdl.NameTableMixin¶

Bases: pyslet.odata2.csdl.DictionaryLike

A mix-in class to help other objects become named scopes.

Using this mix-in the class behaves like a read-only named dictionary with string keys and object values. If the dictionary contains a value that is itself a NameTableMixin then keys can be compounded to look-up items in sub-scopes.

For example, if the name table contains a value with key “X” that is itself a name table containing a value with key “Y” then both “X” and “X.Y” are valid keys, the latter performing a ‘deep lookup’ in the nested scope.

name = None¶: the name of this name table (in the context of its parent)

nameTable = None¶: a dictionary mapping names to child objects

__getitem__(key)¶: Looks up key in nameTable and, if not found, in each child scope with a name that is a valid scope prefix of key. For example, if key is “My.Scope.Name” then a child scope with name “My.Scope” would be searched for “Name” or a child scope with name “My” would be searched for “Scope.Name”.

__iter__()¶: Yields all keys defined in this scope and all compounded keys from nested scopes. For example, a child scope with name “My.Scope” which itself has a child “Name” would generate two keys: “My.Scope” and “My.Scope.Name”.

__len__()¶: Returns the number of keys in this scope including all compounded keys from nested scopes.

Declare(value)¶

Declares a value in this named scope.

value must have a name attribute which is used to declare it in the scope; duplicate keys are not allowed and will raise DuplicateKey.

Values are always declared in the top-level scope, even if they contain the compounding character ‘.’, however, you cannot declare “X” if you have already declared “X.Y” and vice versa.

Undeclare(value)¶

Removes a value from the named scope.

Values can only be removed from the top-level scope.

class pyslet.odata2.csdl.DictionaryLike¶

Bases: object

An abstract class for behaving like a dictionary.

Derived classes must override __iter__() and __getitem__() and if the dictionary is writable __setitem__() and probably __delitem__() too. These methods all raise NotImplementedError by default.

Dervied classes should also override __len__() and clear() as the default implementations are inefficient.

A note on thread safety. Unlike native Python dictionaries, DictionaryLike objects can not be treated as thread safe for updates. The implementations of the read-only methods (including the iterators) are designed to be thread safe so, once populated, they can be safely shared. Derived classes should honour this contract when implementing __iter__(), __getitem__() and __len__() or clearly document that the object is not thread-safe at all.

Finally, one other difference worth noting is touched on in a comment from the following question on Stack Overflow: http://stackoverflow.com/questions/3358770/python-dictionary-is-thread-safe

This question is about whether a dictionary can be modified during iteration. Although not typically a thread-safety issue the commenter says:

I think they are related. What if one thread iterates and the other modifies the dict?

To recap, native Python dictionaries limit the modifications you can make during iteration, quoting from the docs:

The dictionary p should not be mutated during iteration. It is safe (since Python 2.1) to modify the values of the keys as you iterate over the dictionary, but only so long as the set of keys does not change

You should treat DictionaryLike objects with the same respect but the behaviour is not defined at this abstract class level and will vary depending on the implementation. Derived classes are only dictionary-like, they are not actually Python dictionaries!

__getitem__(key)¶

Implements self[key]

This method must be overridden to make a concrete implementation

__setitem__(key, value)¶

Implements assignment to self[key]

This method must be overridden if you want your dictionary-like object to be writable.

__delitem__(key)¶

Implements del self[key]

This method should be overridden if you want your dictionary-like object to be writable.

__iter__()¶

Returns an object that implements the iterable protocol on the keys

This method must be overridden to make a concrete implementation

__len__()¶

Implements len(self)

The default implementation simply counts the keys returned by __iter__ and should be overridden with a more efficient implementation if available.

__contains__(key)¶

Implements: key in self

The default implementation uses __getitem__ and returns False if it raises a KeyError.

iterkeys()¶: Returns an iterable of the keys, simple calls __iter__

itervalues()¶

Returns an iterable of the values.

The default implementation is a generator function that iterates over the keys and uses __getitem__ to yield each value.

keys()¶

Returns a list of keys.

This is a copy of the keys in no specific order. Modifications to this list do not affect the object. The default implementation uses iterkeys()

values()¶

Returns a list of values.

This is a copy of the values in no specific order. Modifications to this list do not affect the object. The default implementation uses itervalues().

iteritems()¶

Returns an iterable of the key,value pairs.

The default implementation is a generator function that uses __iter__() and __getitem__ to yield the pairs.

items()¶

Returns a list of key,value pair tuples.

This is a copy of the items in no specific order. Modifications to this list do not affect the object. The default implementation uses iteritems.

has_key(key)¶: Equivalent to: key in self

get(key, default=None)¶

Equivalent to: self[key] if key in self else default.

Implemented using __getitem__

setdefault(key, value=None)¶

Equivalent to: self[key] if key in self else value; ensuring self[key]=value

Implemented using __getitem__ and __setitem__.

pop(key, value=None)¶

Equivalent to: self[key] if key in self else value; ensuring key not in self.

Implemented using __getitem__ and __delitem__.

clear()¶

Removes all items from the object.

The default implementation uses keys() and deletes the items one-by-one with __delitem__. It does this to avoid deleting objects while iterating as the results are generally undefined. A more efficient implementation is recommended.

popitem()¶

Equivalent to: self[key] for some random key; removing key.

This is a rather odd implementation but to avoid iterating over the whole object we create an iterator with __iter__, use __getitem__ once and then discard it. If an object is found we use __delitem__ to delete it, otherwise KeyError is raised.

bigclear()¶

Removes all the items from the object (alternative for large dictionary-like objects).

This is an alternative implementation more suited to objects with very large numbers of keys. It uses popitem() repeatedly until KeyError is raised. The downside is that popitem creates (and discards) one iterator object for each item it removes. The upside is that we never load the list of keys into memory.

copy()¶

Makes a shallow copy of this object.

This method must be overridden if you want your dictionary-like object to support the copy operation.

update(items)¶: Iterates through items using __setitem__ to add them to the set.

__weakref__¶: list of weak references to the object (if defined)

Exceptions¶

class pyslet.odata2.csdl.NonExistentEntity¶

Bases: pyslet.odata2.csdl.EDMError

Raised when attempting to perform a restricted operation on an entity that doesn’t exist yet. For example, getting the value of a navigation property.

class pyslet.odata2.csdl.EntityExists¶

Bases: pyslet.odata2.csdl.EDMError

Raised when attempting to perform a restricted operation on an entity that already exists. For example, inserting it into the base collection.

class pyslet.odata2.csdl.ConstraintError¶

Bases: pyslet.odata2.csdl.EDMError

General error raised when a constraint has been violated.

class pyslet.odata2.csdl.NavigationError¶

Bases: pyslet.odata2.csdl.ConstraintError

Raised when attempting to perform an operation on an entity and a violation of a navigation property’s relationship is encountered. For example, adding multiple links when only one is allowed or failing to add a link when one is required.

class pyslet.odata2.csdl.ConcurrencyError¶

Bases: pyslet.odata2.csdl.ConstraintError

Raised when attempting to perform an update on an entity and a violation of a concurrency control constraint is encountered.

class pyslet.odata2.csdl.ModelIncomplete¶

Bases: pyslet.odata2.csdl.EDMError

Raised when a model element has a missing reference.

For example, an EntitySet that is bound to an undeclared :EntityType.

class pyslet.odata2.csdl.ModelConstraintError¶

Bases: pyslet.odata2.csdl.EDMError

Raised when an issue in the model other than completeness prevents an action being performed.

For example, an entity type that is dependent on two unbound principals (so can never be inserted).

class pyslet.odata2.csdl.DuplicateName¶

Bases: pyslet.odata2.csdl.EDMError

Raised by NameTableMixin when attempting to declare a name in a context where the name is already declared.

This might be raised if your metadata document incorrectly defines two objects with the same name in the same scope, for example

class pyslet.odata2.csdl.IncompatibleNames¶

Bases: pyslet.odata2.csdl.DuplicateName

A special type of DuplicateName exception raised by NameTableMixin when attempting to declare a name which might hide, or be hidden by, another name already declared.

CSDL’s definition of SimpleIdentifier allows ‘.’ to be used in names but also uses it for qualifying names. As a result, it is possible to define a scope with a name like “My.Scope” which precludes the later definition of a scope called simply “My” (and vice versa).

class pyslet.odata2.csdl.EDMError¶

Bases: exceptions.Exception

General exception for all CSDL model errors.

Constants¶

pyslet.odata2.csdl.EDM_NAMESPACE = 'http://schemas.microsoft.com/ado/2009/11/edm'¶: Namespace to use for CSDL elements

OData Core Classes¶

OData Metadata Classes¶

OData Client¶

Overview¶

Warning: this client doesn’t support certificate validation when accessing servers through https URLs. This feature is coming soon...

Using the Client¶

The client implementation uses Python’s logging module to provide logging, when learning about the client it may help to turn logging up to “INFO” as it makes it clearer what the client is doing. “DEBUG” would show exactly what is passing over the wire.:

>>> import logging
>>> logging.basicConfig(level=logging.INFO)

To create a new client simply instantiate a Client object. You can pass the URL of the service root you wish to connect to directly to the constructor which will then call the service to download the list of feeds and the metadata document from which it will set the Client.model.

>>> from pyslet.odata2.client import Client
>>> c=Client("http://services.odata.org/V2/Northwind/Northwind.svc/")
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/ HTTP/1.1
INFO:root:Finished Response, status 200
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/$metadata HTTP/1.1
INFO:root:Finished Response, status 200
>>>

The Client.feeds attribute is a dictionary mapping the exposed feeds (by name) onto EntitySet instances. This makes it easy to open the feeds as EDM collections. In your code you’d typically use the with statement when opening the collection but for clarity we’ll continue on the python command line:

>>> products=c.feeds['Products'].OpenCollection()
>>> for p in products: print p
...
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products HTTP/1.1
INFO:root:Finished Response, status 200
1
2
3
... [and so on]
...
20
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products?$skiptoken=20 HTTP/1.1
INFO:root:Finished Response, status 200
21
22
23
... [and so on]
...
76
77
>>>

Note that products behaves like a dictionary, iterating through it iterates through the keys in the dictionary. In this case these are the keys of the entities in the collection of products. Notice that the client logs several requests to the server interspersed with the printed output. Subsequent requests use $skiptoken because the server is limiting the maximum page size. These calls are made as you iterate through the collection allowing you to iterate through very large collections.

The keys alone are of limited interest, let’s try a similar loop but this time we’ll print the product names as well:

>>> for k,p in products.iteritems(): print k,p['ProductName'].value
...
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products HTTP/1.1
INFO:root:Finished Response, status 200
1 Chai
2 Chang
3 Aniseed Syrup
...
...
20 Sir Rodney's Marmalade
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products?$skiptoken=20 HTTP/1.1
INFO:root:Finished Response, status 200
21 Sir Rodney's Scones
22 Gustaf's Knäckebröd
23 Tunnbröd
...
...
76 Lakkalikööri
77 Original Frankfurter grüne Soße
>>>

Sir Rodney’s Scones sound interesting, we can grab an individual record in the usual way:

>>> scones=products[21]
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products(21) HTTP/1.1
INFO:root:Finished Response, status 200
>>> for k,v in scones.data_items(): print k,v.value
...
ProductID 21
ProductName Sir Rodney's Scones
SupplierID 8
CategoryID 3
QuantityPerUnit 24 pkgs. x 4 pieces
UnitPrice 10.0000
UnitsInStock 3
UnitsOnOrder 40
ReorderLevel 5
Discontinued False
>>>

Well, I’ve simply got to have some of these, let’s use one of the navigation properties to load information about the supplier:

>>> supplier=scones['Supplier'].GetEntity()
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products(21)/Supplier HTTP/1.1
INFO:root:Finished Response, status 200
>>> for k,v in supplier.data_items(): print k,v.value
...
SupplierID 8
CompanyName Specialty Biscuits, Ltd.
ContactName Peter Wilson
ContactTitle Sales Representative
Address 29 King's Way
City Manchester
Region None
PostalCode M14 GSD
Country UK
Phone (161) 555-4448
Fax None
HomePage None

Attempting to load a non existent entity results in a KeyError of course:

>>> p=products[211]
INFO:root:Sending request to services.odata.org
INFO:root:GET /V2/Northwind/Northwind.svc/Products(211) HTTP/1.1
INFO:root:Finished Response, status 404
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/pyslet/odata2/client.py", line 165, in __getitem__
        raise KeyError(key)
KeyError: 211
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/pyslet/odata2/client.py", line 165, in __getitem__
        raise KeyError(key)
KeyError: 211

Finally, when we’re done, it is a good idea to close the open collection:

>>> products.close()

Reference¶

class pyslet.odata2.client.Client(serviceRoot=None)¶

Bases: pyslet.rfc5023.Client

An OData client.

Can be constructed with an optional URL specifying the service root of an OData service. The URL is passed directly to LoadService().

service = None¶: a pyslet.rfc5023.Service instance describing this service

feeds = None¶: a dictionary of feed titles, mapped to csdl.EntitySet instances

model = None¶: a metadata.Edmx instance containing the model for the service

LoadService(serviceRoot)¶

Configures this client to use the service at serviceRoot

serviceRoot is a string or pyslet.rfc2396.URI instance.

Exceptions¶

class pyslet.odata2.client.ClientException¶

Bases: exceptions.Exception

Base class for all client-specific exceptions.

class pyslet.odata2.client.AuthorizationRequired¶

Bases: pyslet.odata2.client.ClientException

The server returned a response code of 401 to the request.

class pyslet.odata2.client.UnexpectedHTTPResponse¶

Bases: pyslet.odata2.client.ClientException

The server returned an unexpected response code, typically a 500 internal server error. The error message contains details of the error response returned.

An In-Memory Data Service¶

SQL Database-based Data Services¶

This module defines a general (but abstract) implementation of the EDM-based data-access-layer (DAL) using Python’s DB API: http://www.python.org/dev/peps/pep-0249/

It also contains a concrete implementation derived from the above that uses the standard SQLite module for storage. For more information about SQLite see: http://www.sqlite.org/

Data Access Layer API¶

Entity Containers¶

There are primarily two use cases here:

Create a derived class of SQLEntityContainer to provide platform specific modifications to the way SQL queries are constructed and database connections are created and managed.
Create a derived class of SQLEntityContainer to provide modified name mappings for a specific database and metadata model.

These two use cases can be supported through multiple (diamond) inheritance. This makes it easier for you to separate the code required. In practice, implementations for different database platforms are likely to be shared (perhaps as part of future releases of Pyslet itself) whereas modifications to the name mangler to map this API to an existing database will be project specific.

For example, to achieve platform specific modifications you’ll override SQLEntityContainer and provide new implementations for methods such as SQLEntityContainer.get_collection_class():

class MyDBContainer(SQLEntityContainer):

        def get_collection_class(self):
                return MyDBEntityCollection

To achieve modified property name mappings you’ll override SQLEntityContainer and provide new implementations for methods such as SQLEntityContainer.mangle_name():

class SouthwindDBContainer(SQLEntityContainer):

        def mangle_name(self,source_path):
                # do some custom name mangling here....

Normally, you’ll want to achieve both use cases, so to actually instantiate your database you’ll select the container class that represents the database platform and then combine it with the class that contains your data-specific modifications:

import MyDB, Southwind

# easy to configure constants at the top of your script
DBCONTAINER_CLASS=MyDB.MyDBContainer
DBCONTAINER_ARGS={
        'username':"southwind",
        'password':"password"
        }

MAX_CONNECTIONS=100

class SouthwindDB(Southwind.SouthwindDBContainer,DBCONTAINER_CLASS):
        pass

# .... load the metadata from disk and then do something like this
db=SouthwindDB(container=SouthwindMetadata,max_connections=MAX_CONNECTIONS,**DBCONTAINER_ARGS)

class pyslet.odata2.sqlds.SQLEntityContainer(container, dbapi, streamstore=None, max_connections=10, field_name_joiner=u'_', **kwargs)¶

Bases: object

Object used to represent an Entity Container (aka Database).

Keyword arguments on construction:

container

The EntityContainer that defines this database.

streamstore

An optional StreamStore that will be used to store media resources in the container. If absent, media resources actions will generate NotImplementedError.

dbapi

The DB API v2 compatible module to use to connect to the database.

This implementation is compatible with modules regardless of their thread-safety level (provided they declare it correctly!).

max_connections (optional)

The maximum number of connections to open to the database. If your program attempts to open more than this number (defaults to 10) then it will block until a connection becomes free. Connections are always shared within the same thread so this argument should be set to the expected maximum number of threads that will access the database.

If using a module with thread-safety level 0 max_connections is ignored and is effectively 1, so use of the API is then best confined to single-threaded programs. Multi-threaded programs can still use the API but it will block when there is contention for access to the module and context switches will force the database connection to be closed and reopened.

field_name_joiner (optional)

The character used by the name mangler to join compound names, for example, to obtain the column name of a complex property like “Address/City”. The default is “_”, resulting in names like “Address_City” but it can be changed here. Note: all names are quoted using quote_identifier() before appearing in SQL statements.

This class is designed to work with diamond inheritance and super. All derived classes must call __init__ through super and pass all unused keyword arguments. For example:

class MyDBContainer:
        def __init__(self,myDBConfig,**kwargs):
                super(MyDBContainer,self).__init__(**kwargs)
                # do something with myDBConfig....

streamstore = None¶: the EntityContainer

dbapi = None¶: the optional StreamStore

module_lock = None¶: the DB API compatible module

fk_table = None¶

A mapping from an entity set name to a FK mapping of the form:

{<association set end>: (<nullable flag>, <unique keys flag>),...}

The outer mapping has one entry for each entity set (even if the corresponding foreign key mapping is empty).

Each foreign key mapping has one entry for each foreign key reference that must appear in that entity set’s table. The key is an AssociationSetEnd that is bound to the entity set (the other end will be bound to the target entity set). This allows us to distinguish between the two ends of a recursive association.

aux_table = None¶

A mapping from the names of symmetric association sets to:

(<entity set A>, <name prefix A>, <entity set B>,

<name prefix B>, <unique keys>)

mangled_names = None¶

A mapping from source path tuples to mangled and quoted names to use in SQL queries. For example:

(u'Customer'):u'"Customer"'
(u'Customer', u'Address', u'City') : u"Address_City"
(u'Customer', u'Orders') : u"Customer_Orders"

Note that the first element of the tuple is the entity set name but the default implementation does not use this in the mangled name for primitive fields as they are qualified in contexts where a name clash is possible. However, mangled navigation property names do include the table name prefix as they used as pseudo-table names.

field_name_joiner = None¶: Default string used to join complex field names in SQL queries, e.g. Address_City

ro_names = None¶: The set of names that should be considered read only by the SQL insert and update generation code. The items in the set are source paths, as per mangled_names. The set is populated on construction using the ro_name() method.

mangle_name(source_path)¶

Mangles a source path into a quoted SQL name

This is a key extension point to use when you are wrapping an existing database with the API. It allows you to control the names used for entity sets (tables) and properties (columns) in SQL queries.

source_path

A tuple or list of strings describing the path to a property in the metadata model.

For entity sets, this is a tuple with a single entry in it, the entity set name.

For data properties this is a tuple containing the path, including the entity set name e.g., (“Customers”,”Address”,”City”) for the City property in a complex property ‘Address’ in entity set “Customers”.

For navigation properties the tuple is the navigation property name prefixed with the entity set name, e.g., (“Customers”,”Orders”). This name is only used as a SQL alias for the target table, to remove ambiguity from certain queries that include a join across the navigation property. The mangled name must be distinct from the entity set name itself. from other such aliases and from other column names in this table.

Foreign key properties contain paths starting with both the entity set and the association set names (see SQLForeignKeyCollection for details) unless the association is symmetric, in which case they also contain the navigation property name (see SQLAssociationCollection for details of these more complex cases).

The default implementation strips the entity set name away and uses the default joining character to create a compound name before calling quote_identifier() to obtain the SQL string. All names are mangled once, on construction, and from then on looked up in the dictionary of mangled names.

If you need to override this method to modify the names used in your database you should ensure all other names (including any unrecognized by your program) are passed to the default implementation for mangling.

ro_name(source_path)¶

Test if a source_path identifies a read-only property

This is a an additional extension point to use when you are wrapping an existing database with the API. It allows you to manage situations where an entity property has an implied value and should be treated read only.

There are two key use cases, auto-generated primary keys (such as auto-increment integer keys) and foreign keys which are exposed explicitly as foreign keys and should only be updated through an associated navigation property.

source_path: A tuple or list of strings describing the path to a property in the metadata model. See mangle_name() for more information.

The default implementation returns False.

If you override this method you must ensure all other names (including any unrecognized by your program) are passed to the default implementation using super.

source_path_generator(entity_set)¶: Utility generator for source path tuples for entity_set

get_collection_class()¶

Returns the collection class used to represent a generic entity set.

Override this method to provide a class derived from SQLEntityCollection when you are customising this implementation for a specific database engine.

get_symmetric_navigation_class()¶

Returns the collection class used to represent a symmetric relation.

Override this method to provide a class derived from SQLAssociationCollection when you are customising this implementation for a specific database engine.

get_fk_class()¶

Returns the class used when the FK is in the source table.

Override this method to provide a class derived from SQLForeignKeyCollection when you are customising this implementation for a specific database engine.

get_rk_class()¶

Returns the class used when the FK is in the target table.

Override this method to provide a class derived from SQLReverseKeyCollection when you are customising this implementation for a specific database engine.

create_all_tables()¶

Creates all tables in this container.

Tables are created in a sensible order to ensure that foreign key constraints do not fail but this method is not compatible with databases that contain circular references though, e.g., Table A -> Table B with a foreign key and Table B -> Table A with a foreign key. Such databases will have to be created by hand. You can use the create_table_query methods to act as a starting point for your script.

open()¶

Creates and returns a new connection object.

Must be overridden by database specific implementations because the underlying DB ABI does not provide a standard method of connecting.

break_connection(connection)¶

Called when closing or cleaning up locked connections.

This method is called when the connection is locked (by a different thread) and the caller wants to force that thread to relinquish control.

The assumption is that the database is stuck in some lengthy transaction and that break_connection can be used to terminate the transaction and force an exception in the thread that initiated it - resulting in a subsequent call to release_connection() and a state which enables this thread to acquire the connection’s lock so that it can close it.

The default implementation does nothing, which might cause the close method to stall until the other thread relinquishes control normally.

close(timeout=5)¶

Closes this database.

This method goes through each open connection and attempts to acquire it and then close it. The object is put into a mode that disables acquire_connection() (it returns None from now on).

timeout

Defaults to 5 seconds. If connections are locked by other running threads we wait for those threads to release them, calling break_connection() to speed up termination if possible.

If None (not recommended!) this method will block indefinitely until all threads properly call release_connection().

Any locks we fail to acquire in the timeout are ignored and the connections are left open for the python garbage collector to dispose of.

quote_identifier(identifier)¶

Given an identifier returns a safely quoted form of it.

By default we strip double quote and then use them to enclose it. E.g., if the string u’Employee_Name’ is passed then the string u‘“Employee_Name”’ is returned.

prepare_sql_type(simple_value, params, nullable=None)¶

Given a simple value, returns a SQL-formatted name of its type.

Used to construct CREATE TABLE queries.

simple_value: A pyslet.odata2.csdl.SimpleValue instance which should have been created from a suitable pyslet.odata2.csdl.Property definition.
params: A SQLParams object. If simple_value is non-NULL, a DEFAULT value is added as part of the type definition.
nullable: Optional Boolean that can be used to override the nullable status of the associated property definition.

For example, if the value was created from an Int32 non-nullable property and has value 0 then this might return the string u’INTEGER NOT NULL DEFAULT ?’ with 0 being added to params

You should override this implementation if your database platform requires special handling of certain datatypes. The default mappings are given below.

EDM Type	SQL Equivalent
Edm.Binary	BINARY(MaxLength) if FixedLength specified
Edm.Binary	VARBINARY(MaxLength) if no FixedLength
Edm.Boolean	BOOLEAN
Edm.Byte	SMALLINT
Edm.DateTime	TIMESTAMP
Edm.DateTimeOffset	CHARACTER(20), ISO 8601 string representation is used
Edm.Decimal	DECIMAL(Precision,Scale), defaults 10,0
Edm.Double	FLOAT
Edm.Guid	BINARY(16)
Edm.Int16	SMALLINT
Edm.Int32	INTEGER
Edm.Int64	BIGINT
Edm.SByte	SMALLINT
Edm.Single	REAL
Edm.String	CHAR(MaxLength) or VARCHAR(MaxLength)
Edm.String	NCHAR(MaxLength) or NVARCHAR(MaxLength) if Unicode=”true”
Edm.Time	TIME

Parameterized CREATE TABLE queries are unreliable in my experience so the current implementation of the native create_table methods ignore default values when calling this method.

prepare_sql_value(simple_value)¶

Returns a python object suitable for passing as a parameter

simple_value: A pyslet.odata2.csdl.SimpleValue instance.

You should override this method if your database requires special handling of parameter values. The default implementation performs the following conversions

EDM Type	Python value added as parameter
NULL	None
Edm.Binary	(byte) string
Edm.Boolean	True or False
Edm.Byte	int
Edm.DateTime	Timestamp instance from DB API module
Edm.DateTimeOffset	string (ISO 8601 basic format)
Edm.Decimal	Decimal instance
Edm.Double	float
Edm.Guid	(byte) string
Edm.Int16	int
Edm.Int32	int
Edm.Int64	long
Edm.SByte	int
Edm.Single	float
Edm.String	(unicode) string
Edm.Time	Time instance from DB API module

read_sql_value(simple_value, new_value)¶

Updates simple_value from new_value.

simple_value: A pyslet.odata2.csdl.SimpleValue instance.
new_value: A value returned by the underlying DB API, e.g., from a cursor fetch operation

This method performs the reverse transformation to prepare_sql_value() and may need to be overridden to convert new_value into a form suitable for passing to the underlying set_from_value() method.

new_from_sql_value(sql_value)¶

Returns a new simple value with value sql_value

The return value is a pyslet.odata2.csdl.SimpleValue instance.

sql_value: A value returned by the underlying DB API, e.g., from a cursor fetch operation

This method creates a new instance, selecting the most appropriate type to represent sql_value. By default pyslet.odata2.csdl.EDMValue.NewSimpleValueFromValue() is used.

You may need to override this method to identify the appropriate value type.

For an example of how to create a platform-specific implementation see SQLite below.

Entity Collections¶

These classes are documented primarily to facilitate the creation of alternative implementations designed to run over other DB API based data layers. The documentation goes a bit further than is necessary to help promote an understanding of the way the API is implemented.

class pyslet.odata2.sqlds.SQLEntityCollection(container, qualify_names=False, **kwargs)¶

Bases: pyslet.odata2.sqlds.SQLCollectionBase

Represents a collection of entities from an EntitySet.

This class is the heart of the SQL implementation of the API, constructing and executing queries to implement the core methods from pyslet.odata2.csdl.EntityCollection.

insert_entity(entity)¶

Inserts entity into the collection.

We override this method, rerouting it to a SQL-specific implementation that takes additional arguments.

insert_entity_sql(entity, from_end=None, fk_values=None, transaction=None)¶

Inserts entity into the collection.

This method is not designed to be overridden by other implementations but it does extend the default functionality for a more efficient implementation and to enable better transactional processing. The additional parameters are documented here.

from_end

An optional pyslet.odata2.csdl.AssociationSetEnd bound to this entity set. If present, indicates that this entity is being inserted as part of a single transaction involving an insert or update to the other end of the association.

This suppresses any check for a required link via this association (as it is assumed that the link is present, or will be, in the same transaction).

fk_values

If the association referred to by from_end is represented by a set of foreign keys stored in this entity set’s table (see SQLReverseKeyCollection) then fk_values is the list of (mangled column name, value) tuples that must be inserted in order to create the link.

transaction

An optional transaction. If present, the connection is left uncommitted.

The method functions in three phases.

Process all bindings for which we hold the foreign key. This includes inserting new entities where deep inserts are being used or calculating foreign key values where links to existing entities have been specified on creation.

In addition, all required links are checked and raise errors if no binding is present.
A simple SQL INSERT statement is executed to add the record to the database along with any foreign keys generated in (1) or passed in fk_values.
Process all remaining bindings. Although we could do this using the update_bindings() method of DeferredValue we handle this directly to retain transactional integrity (where supported).

Links to existing entities are created using the insert_link method available on the SQL-specific SQLNavigationCollection.

Deep inserts are handled by a recursive call to this method. After step 1, the only bindings that remain are (a) those that are stored at the other end of the link and so can be created by passing values for from_end and fk_values in a recursive call or (b) those that are stored in a separate table which are created by combining a recursive call and a call to insert_link.

Required links are always created in step 1 because the overarching mapping to SQL forces such links to be represented as foreign keys in the source table (i.e., this table) unless the relationship is 1-1, in which case the link is created in step 3 and our database is briefly in violation of the model. If the underlying database API does not support transactions then it is possible for this state to persist resulting in an orphan entity or entities, i.e., entities with missing required links. A failed rollback() call will log this condition along with the error that caused it.

update_entity(entity)¶

Updates entity

This method follows a very similar pattern to InsertMethod(), using a three-phase process.

Process all bindings for which we hold the foreign key.

This includes inserting new entities where deep inserts are being used or calculating foreign key values where links to existing entities have been specified on update.
A simple SQL UPDATE statement is executed to update the

record in the database along with any updated foreign keys generated in (1).
Process all remaining bindings while retaining transactional

integrity (where supported).

Links to existing entities are created using the insert_link or replace methods available on the SQL-specific SQLNavigationCollection. The replace method is used when a navigation property that links to a single entity has been bound. Deep inserts are handled by calling insert_entity_sql before the link is created.

The same transactional behaviour as insert_entity_sql() is exhibited.

update_link(entity, link_end, target_entity, no_replace=False, transaction=None)¶

Updates a link when this table contains the foreign key

entity: The entity being linked from (must already exist)
link_end: The AssociationSetEnd bound to this entity set that represents this entity set’s end of the assocation being modified.
target_entity: The entity to link to or None if the link is to be removed.
no_replace: If True, existing links will not be replaced. The affect is to force the underlying SQL query to include a constraint that the foreign key is currently NULL. By default this argument is False and any existing link will be replaced.
transaction: An optional transaction. If present, the connection is left uncommitted.

delete_entity(entity, from_end=None, transaction=None)¶

Deletes an entity

Called by the dictionary-like del operator, provided as a separate method to enable it to be called recursively when doing cascade deletes and to support transactions.

from_end

An optional AssociationSetEnd bound to this entity set that represents the link from which we are being deleted during a cascade delete.

The purpose of this parameter is prevent cascade deletes from doubling back on themselves and causing an infinite loop.

transaction

An optional transaction. If present, the connection is left uncommitted.

delete_link(entity, link_end, target_entity, transaction=None)¶

Deletes the link between entity and target_entity

The foreign key for this link must be held in this entity set’s table.

entity: The entity in this entity set that the link is from.
link_end: The AssociationSetEnd bound to this entity set that represents this entity set’s end of the assocation being modified.
target_entity: The target entity that defines the link to be removed.
transaction: An optional transaction. If present, the connection is left uncommitted.

clear_links(link_end, target_entity, transaction=None)¶

Deletes all links to target_entity

The foreign key for this link must be held in this entity set’s table.

link_end: The AssociationSetEnd bound to this entity set that represents this entity set’s end of the assocation being modified.
target_entity: The target entity that defines the link(s) to be removed.
transaction: An optional transaction. If present, the connection is left uncommitted.

create_table_query()¶: Returns a SQL statement and params for creating the table.

create_table()¶: Executes the SQL statement create_table_query()

class pyslet.odata2.sqlds.SQLCollectionBase(container, qualify_names=False, **kwargs)¶

Bases: pyslet.odata2.core.EntityCollection

A base class to provide core SQL functionality.

Additional keyword arguments:

container: A SQLEntityContainer instance.
qualify_names: An optional boolean (defaults to False) indicating whether or not the column names must be qualified in all queries.

On construction a data connection is acquired from container, this may prevent other threads from using the database until the lock is released by the close() method.

table_name = None¶: the parent container (database) for this collection

dbc = None¶: a connection to the database

close()¶: Closes the cursor and database connection if they are open.

set_page(top, skip=0, skiptoken=None)¶

Sets the values for paging.

Our implementation uses a special format for skiptoken. It is a comma-separated list of simple literal values corresponding to the values required by the ordering augmented with the key values to ensure uniqueness.

For example, if $orderby=A,B on an entity set with key K then the skiptoken will typically have three values comprising the last values returned for A,B and K in that order. In cases where the resulting skiptoken would be unreasonably large an additional integer (representing a further skip) may be appended and the whole token expressed relative to an earlier skip point.

join_clause()¶

A utility method to return the JOIN clause.

Defaults to an empty expression.

where_clause(entity, params, use_filter=True, use_skip=False, null_cols=())¶

A utility method that generates the WHERE clause for a query

entity: An optional entity within this collection that is the focus of this query. If not None the resulting WHERE clause will restrict the query to this entity only.
params: The SQLParams object to add parameters to.
use_filter: Defaults to True, indicates if this collection’s filter should be added to the WHERE clause.
use_skip: Defaults to False, indicates if the skiptoken should be used in the where clause. If True then the query is limited to entities appearing after the skiptoken’s value (see below).
null_cols: An iterable of mangled column names that must be NULL (defaults to an empty tuple). This argument is used during updates to prevent the replacement of non-NULL foreign keys.

The operation of the skiptoken deserves some explanation. When in play the skiptoken contains the last value of the order expression returned. The order expression always uses the keys to ensure unambiguous ordering. The clause added is best served with an example. If an entity has key K and an order expression such as “tolower(Name) desc” then the query will contain something like:

SELECT K, Nname, DOB, LOWER(Name) AS o_1, K ....
        WHERE (o_1 < ? OR (o_1 = ? AND K > ?))

The values from the skiptoken will be passed as parameters.

where_entity_clause(where, entity, params)¶

Adds the entity constraint expression to a list of SQL expressions.

where: The list to append the entity expression to.
entity: An expression is added to restrict the query to this entity

where_skiptoken_clause(where, params)¶

Adds the entity constraint expression to a list of SQL expressions.

where: The list to append the skiptoken expression to.

set_orderby(orderby)¶

Sets the orderby rules for this collection.

We override the default implementation to calculate a list of field name aliases to use in ordered queries. For example, if the orderby expression is “tolower(Name) desc” then each SELECT query will be generated with an additional expression, e.g.:

SELECT ID, Name, DOB, LOWER(Name) AS o_1 ...
    ORDER BY o_1 DESC, ID ASC

The name “o_1” is obtained from the name mangler using the tuple:

(entity_set.name,'o_1')

Subsequent order expressions have names ‘o_2’, ‘o_3’, etc.

Notice that regardless of the ordering expression supplied the keys are always added to ensure that, when an ordering is required, a defined order results even at the expense of some redundancy.

orderby_clause()¶

A utility method to return the orderby clause.

params: The SQLParams object to add parameters to.

orderby_cols(column_names, params, force_order=False)¶

A utility to add the column names and aliases for the ordering.

column_names: A list of SQL column name/alias expressions
params: The SQLParams object to add parameters to.
force_order: Forces the addition of an ordering by key if an orderby expression has not been set.

insert_fields(entity)¶

A generator for inserting mangled property names and values.

entity: Any instance of Entity

The yielded values are tuples of (mangled field name, SimpleValue instance).

Read only fields are never generated, even if they are keys. This allows automatically generated keys to be used and also covers the more esoteric use case where a foreign key constraint exists on the primary key (or part thereof) - in the latter case the relationship should be marked as required to prevent unexpected constraint violations.

Otherwise, only selected fields are yielded so if you attempt to insert a value without selecting the key fields you can expect a constraint violation unless the key is read only.

auto_fields(entity)¶

A generator for selecting auto mangled property names and values.

entity: Any instance of Entity

The yielded values are tuples of (mangled field name, SimpleValue instance).

Only fields that are read only are yielded with the caveat that they must also be either selected or keys. The purpose of this method is to assist with reading back automatically generated field values after an insert or update.

key_fields(entity)¶

A generator for selecting mangled key names and values.

entity: Any instance of Entity

The yielded values are tuples of (mangled field name, SimpleValue instance). Only the keys fields are yielded.

select_fields(entity)¶

A generator for selecting mangled property names and values.

entity: Any instance of Entity

The yielded values are tuples of (mangled field name, SimpleValue instance). Only selected fields are yielded with the caveat that the keys are always selected.

update_fields(entity)¶

A generator for updating mangled property names and values.

entity: Any instance of Entity

The yielded values are tuples of (mangled field name, SimpleValue instance).

Neither read only fields nor key are generated. All other fields are yielded but unselected fields are set to NULL before being yielded. This implements OData’s PUT semantics. See merge_fields() for an alternative.

merge_fields(entity)¶

A generator for merging mangled property names and values.

entity: Any instance of Entity

The yielded values are tuples of (mangled field name, SimpleValue instance).

Neither read only fields, keys nor unselected fields are generated. All other fields are yielded implementing OData’s MERGE semantics. See update_fields() for an alternative.

stream_field(entity)¶

Returns information for selecting the stream ID.

entity: Any instance of Entity

Returns a tuples of (mangled field name, SimpleValue instance).

sql_expression(expression, params, context='AND')¶

Converts an expression into a SQL expression string.

expression: A pyslet.odata2.core.CommonExpression instance.
params: A SQLParams object of the appropriate type for this database connection.
context: A string containing the SQL operator that provides the context in which the expression is being converted, defaults to ‘AND’. This is used to determine if the resulting expression must be bracketed or not. See sql_bracket() for a useful utility function to illustrate this.

This method is basically a grand dispatcher that sends calls to other node-specific methods with similar signatures. The effect is to traverse the entire tree rooted at expression.

The result is a string containing the parameterized expression with appropriate values added to the params object in the same sequence that they appear in the returned SQL expression.

When creating derived classes to implement database-specific behaviour you should override the individual evaluation methods rather than this method. All related methods have the same signature.

Where methods are documented as having no default implementation, NotImplementedError is raised.

sql_bracket(query, context, operator)¶

A utility method for bracketing a SQL query.

query: The query string
context: A string representing the SQL operator that defines the context in which the query is to placed. E.g., ‘AND’
operator: The dominant operator in the query.

This method is used by operator-specific conversion methods. The query is not parsed, it is merely passed in as a string to be bracketed (or not) depending on the values of context and operator.

The implementation is very simple, it checks the precedence of operator in context and returns query bracketed if necessary:

collection.sql_bracket("Age+3","*","+")=="(Age+3)"
collection.sql_bracket("Age*3","+","*")=="Age*3" 

sql_expression_member(expression, params, context)¶

Converts a member expression, e.g., Address/City

This implementation does not support the use of navigation properties but does support references to complex properties.

It outputs the mangled name of the property, qualified by the table name if qualify_names is True.

sql_expression_cast(expression, params, context)¶: Converts the cast expression: no default implementation

sql_expression_generic_binary(expression, params, context, operator)¶

A utility method for implementing binary operator conversion.

The signature of the basic sql_expression() is extended to include an operator argument, a string representing the (binary) SQL operator corresponding to the expression object.

sql_expression_mul(expression, params, context)¶: Converts the mul expression: maps to SQL “*”

sql_expression_div(expression, params, context)¶: Converts the div expression: maps to SQL “/”

sql_expression_mod(expression, params, context)¶: Converts the mod expression: no default implementation

sql_expression_add(expression, params, context)¶: Converts the add expression: maps to SQL “+”

sql_expression_sub(expression, params, context)¶: Converts the sub expression: maps to SQL “-“

sql_expression_lt(expression, params, context)¶: Converts the lt expression: maps to SQL “<”

sql_expression_gt(expression, params, context)¶: Converts the gt expression: maps to SQL “>”

sql_expression_le(expression, params, context)¶: Converts the le expression: maps to SQL “<=”

sql_expression_ge(expression, params, context)¶: Converts the ge expression: maps to SQL “>=”

sql_expression_isof(expression, params, context)¶: Converts the isof expression: no default implementation

sql_expression_eq(expression, params, context)¶: Converts the eq expression: maps to SQL “=”

sql_expression_ne(expression, params, context)¶: Converts the ne expression: maps to SQL “<>”

sql_expression_and(expression, params, context)¶: Converts the and expression: maps to SQL “AND”

sql_expression_or(expression, params, context)¶: Converts the or expression: maps to SQL “OR”

sql_expression_endswith(expression, params, context)¶

Converts the endswith function: maps to “op[0] LIKE ‘%’+op[1]”

This is implemented using the concatenation operator

sql_expression_indexof(expression, params, context)¶: Converts the indexof method: maps to POSITION( op[0] IN op[1] )

sql_expression_replace(expression, params, context)¶: Converts the replace method: no default implementation

sql_expression_startswith(expression, params, context)¶

Converts the startswith function: maps to “op[0] LIKE op[1]+’%’”

This is implemented using the concatenation operator

sql_expression_tolower(expression, params, context)¶: Converts the tolower method: maps to LOWER function

sql_expression_toupper(expression, params, context)¶: Converts the toupper method: maps to UCASE function

sql_expression_trim(expression, params, context)¶: Converts the trim method: maps to TRIM function

sql_expression_substring(expression, params, context)¶

Converts the substring method

maps to SUBSTRING( op[0] FROM op[1] [ FOR op[2] ]

sql_expression_substringof(expression, params, context)¶

Converts the substringof function

maps to “op[1] LIKE ‘%’+op[0]+’%’”

To do this we need to invoke the concatenation operator.

This method has been poorly defined in OData with the parameters being switched between versions 2 and 3. It is being withdrawn as a result and replaced with contains in OData version 4. We follow the version 3 convention here of “first parameter in the second parameter” which fits better with the examples and with the intuitive meaning:

substringof(A,B) == A in B

sql_expression_concat(expression, params, context)¶: Converts the concat method: maps to ||

sql_expression_length(expression, params, context)¶: Converts the length method: maps to CHAR_LENGTH( op[0] )

sql_expression_year(expression, params, context)¶: Converts the year method: maps to EXTRACT(YEAR FROM op[0])

sql_expression_month(expression, params, context)¶: Converts the month method: maps to EXTRACT(MONTH FROM op[0])

sql_expression_day(expression, params, context)¶: Converts the day method: maps to EXTRACT(DAY FROM op[0])

sql_expression_hour(expression, params, context)¶: Converts the hour method: maps to EXTRACT(HOUR FROM op[0])

sql_expression_minute(expression, params, context)¶: Converts the minute method: maps to EXTRACT(MINUTE FROM op[0])

sql_expression_second(expression, params, context)¶: Converts the second method: maps to EXTRACT(SECOND FROM op[0])

sql_expression_round(expression, params, context)¶: Converts the round method: no default implementation

sql_expression_floor(expression, params, context)¶: Converts the floor method: no default implementation

sql_expression_ceiling(expression, params, context)¶: Converts the ceiling method: no default implementation

class pyslet.odata2.sqlds.SQLNavigationCollection(aset_name, **kwargs)¶

Bases: pyslet.odata2.sqlds.SQLCollectionBase, pyslet.odata2.core.NavigationCollection

Abstract class representing all navigation collections.

Additional keyword arguments:

aset_name: The name of the association set that defines this relationship. This additional parameter is used by the name mangler to obtain the field name (or table name) used for the foreign keys.

insert_link(entity, transaction=None)¶

Inserts a link to entity into this collection.

transaction: An optional transaction. If present, the connection is left uncommitted.

replace_link(entity, transaction=None)¶

Replaces all links with a single link to entity.

transaction: An optional transaction. If present, the connection is left uncommitted.

delete_link(entity, transaction=None)¶

A utility method that deletes the link to entity in this collection.

This method is called during cascaded deletes to force-remove a link prior to the deletion of the entity itself.

transaction: An optional transaction. If present, the connection is left uncommitted.

class pyslet.odata2.sqlds.SQLForeignKeyCollection(**kwargs)¶

Bases: pyslet.odata2.sqlds.SQLNavigationCollection

The collection of entities obtained by navigation via a foreign key

This object is used when the foreign key is stored in the same table as from_entity. This occurs when the relationship is one of:

0..1 to 1
Many to 1
Many to 0..1

The name mangler looks for the foreign key in the field obtained by mangling:

(entity set name, association set name, key name)

For example, suppose that a link exists from entity set Orders[*] to entity set Customers[0..1] and that the key field of Customer is “CustomerID”. If the association set that binds Orders to Customers with this link is called OrdersToCustomers then the foreign key would be obtained by looking up:

('Orders','OrdersToCustomers','CustomerID')

By default this would result in the field name:

'OrdersToCustomers_CustomerID'

This field would be looked up in the ‘Orders’ table. The operation of the name mangler can be customised by overriding the SQLEntityContainer.mangle_name() method in the container.

join_clause()¶

Overridden to provide a join to from_entity‘s table.

The join clause introduces an additional name that is looked up by the name mangler. To avoid name clashes when the relationship is recursive the join clause introduces an alias for the table containing from_entity. To continue the example above, if the link from Orders to Customers is bound to a navigation property in the reverse direction called, say, ‘AllOrders’ in the target entity set then this alias is looked up using:

('Customers','AllOrders')

By default this would just be the string ‘AllOrders’ (the name of the navigation property). The resulting join looks something like this:

SELECT ... FROM Customers
INNER JOIN Orders AS AllOrders ON
    Customers.CustomerID=AllOrders.OrdersToCustomers_CustomerID
...
WHERE AllOrders.OrderID = ?;

The value of the OrderID key property in from_entity is passed as a parameter when executing the expression.

There is an awkward case when the reverse navigation property has not been bound, in this case the link’s role name is used instead, this provides a best guess as to what the navigation property name would have been had it been bound; it must be unique within the context of target entity_set’s type - a benign constraint on the model’s metadata description.

where_clause(entity, params, use_filter=True, use_skip=False)¶

Adds the constraint for entities linked from from_entity only.

We continue to use the alias set in the join_clause() where an example WHERE clause is illustrated.

class pyslet.odata2.sqlds.SQLReverseKeyCollection(**kwargs)¶

Bases: pyslet.odata2.sqlds.SQLNavigationCollection

The collection of entities obtained by navigation to a foreign key

This object is used when the foreign key is stored in the target table. This occurs in the reverse of the cases where SQLReverseKeyCollection is used, i.e:

1 to 0..1 1 to Many 0..1 to Many

The implementation is actually simpler in this direction as no JOIN clause is required.

where_clause(entity, params, use_filter=True, use_skip=False)¶: Adds the constraint to entities linked from from_entity only.

delete_link(entity, transaction=None)¶

Called during cascaded deletes.

This is actually a no-operation as the foreign key for this association is in the entity’s record itself and will be removed automatically when entity is deleted.

clear_links(transaction=None)¶

Deletes all links from this collection’s from_entity

transaction: An optional transaction. If present, the connection is left uncommitted.

class pyslet.odata2.sqlds.SQLAssociationCollection(**kwargs)¶

Bases: pyslet.odata2.sqlds.SQLNavigationCollection

The collection obtained by navigation using an auxiliary table

This object is used when the relationship is described by two sets of foreign keys stored in an auxiliary table. This occurs mainly when the link is Many to Many but it is also used for 1 to 1 relationships. This last use may seem odd but it is used to represent the symmetry of the relationship. In practice, a single set of foreign keys is likely to exist in one table or the other and so the relationship is best modelled by a 0..1 to 1 relationship even if the intention is that the records will always exist in pairs.

The name of the auxiliary table is obtained from the name mangler using the association set’s name. The keys use a more complex mangled form to cover cases where there is a recursive Many to Many relation (such as a social network of friends between User entities). The names of the keys are obtained by mangling:

( association set name, target entity set name,
    navigation property name, key name )

An example should help. Suppose we have entities representing sports Teams(TeamID) and sports Players(PlayerID) and that you can navigate from Player to Team using the “PlayedFor” navigation property and from Team to Player using the “Players” navigation property. Both navigation properties are collections so the relationship is Many to Many. If the association set that binds the two entity sets is called PlayersAndTeams then the the auxiliary table name will be mangled from:

('PlayersAndTeams')

and the fields will be mangled from:

('PlayersAndTeams','Teams','PlayedFor','TeamID')
('PlayersAndTeams','Players','Players','PlayerID')

By default this results in column names ‘Teams_PlayedFor_TeamID’ and ‘Players_Players_PlayerID’. If you are modelling an existing database then ‘TeamID’ and ‘PlayerID’ on their own are more likely choices. You would need to override the SQLEntityContainer.mangle_name() method in the container to catch these cases and return the shorter column names.

join_clause()¶

Overridden to provide the JOIN to the auxiliary table.

Unlike the foreign key JOIN clause there is no need to use an alias in this case as the auxiliary table is assumed to be distinct from the the table it is being joined to.

where_clause(entity, params, use_filter=True, use_skip=False)¶: Provides the from_entity constraint in the auxiliary table.

insert_entity(entity)¶: Rerouted to a SQL-specific implementation

insert_entity_sql(entity, transaction=None)¶

Inserts entity into the base collection and creates the link.

This is always done in two steps, bound together in a single transaction (where supported). If this object represents a 1 to 1 relationship then, briefly, we’ll be in violation of the model. This will only be an issue in non-transactional systems.

delete_link(entity, transaction=None)¶

Called during cascaded deletes to force-remove a link prior to the deletion of the entity itself.

This method is also re-used for simple deletion of the link in this case as the link is in the auxiliary table itself.

clear_links(transaction=None)¶

Deletes all links from this collection’s from_entity

transaction: An optional transaction. If present, the connection is left uncommitted.

classmethod clear_links_unbound(container, from_end, from_entity, transaction)¶

Special class method for deleting all the links from from_entity

This is a class method because it has to work even if there is no navigation property bound to this end of the association.

container: The SQLEntityContainer containing this association set.
from_end: The AssociationSetEnd that represents the end of the association that from_entity is bound to.
from_entity: The entity to delete links from
transaction: The current transaction (required)

This is a class method because it has to work even if there is no navigation property bound to this end of the association. If there was a navigation property then an instance could be created and the simpler clear_links() method used.

classmethod create_table_query(container, aset_name)¶

Returns a SQL statement and params to create the auxiliary table.

This is a class method to enable the table to be created before any entities are created.

classmethod create_table(container, aset_name)¶: Executes the SQL statement create_table_query()

SQLite¶

This module also contains a fully functional implementation of the API based on the sqlite3 module. The first job with any SQL implementation is to create a base collection class that implements any custom expression handling.

In the case of SQLite we override a handful of the standard SQL functions only. Notice that this class is derived from SQLCollectionBase, an abstract class. If your SQL platform adheres to the SQL standard very closely, or you are happy for SQL-level errors to be generated when unsupported SQL syntax is generated by some filter or orderby expressions then you can skip the process of creating customer collection classes completely.

class pyslet.odata2.sqlds.SQLiteEntityCollectionBase(container, qualify_names=False, **kwargs)¶

Bases: pyslet.odata2.sqlds.SQLCollectionBase

Base class for SQLite SQL custom mappings.

This class provides some SQLite specific mappings for certain functions to improve compatibility with the OData expression language.

sql_expression_length(expression, params, context)¶: Converts the length method: maps to length( op[0] )

sql_expression_year(expression, params, context)¶

Converts the year method

maps to CAST(strftime(‘%Y’,op[0]) AS INTEGER)

sql_expression_month(expression, params, context)¶

Converts the month method

maps to CAST(strftime(‘%m’,op[0]) AS INTEGER)

sql_expression_day(expression, params, context)¶

Converts the day method

maps to CAST(strftime(‘%d’,op[0]) AS INTEGER)

sql_expression_hour(expression, params, context)¶

Converts the hour method

maps to CAST(strftime(‘%H’,op[0]) AS INTEGER)

sql_expression_minute(expression, params, context)¶

Converts the minute method

maps to CAST(strftime(‘%M’,op[0]) AS INTEGER)

sql_expression_second(expression, params, context)¶

Converts the second method

maps to CAST(strftime(‘%S’,op[0]) AS INTEGER)

sql_expression_tolower(expression, params, context)¶

Converts the tolower method

maps to lower(op[0])

sql_expression_toupper(expression, params, context)¶

Converts the toupper method

maps to upper(op[0])

To ensure that our custom implementations are integrated in to all the collection classes we have to create specific classes for all collection types. These classes have no implementation!

class pyslet.odata2.sqlds.SQLiteEntityCollection(container, qualify_names=False, **kwargs)¶

Bases: pyslet.odata2.sqlds.SQLiteEntityCollectionBase, pyslet.odata2.sqlds.SQLEntityCollection

SQLite-specific collection for entity sets

class pyslet.odata2.sqlds.SQLiteForeignKeyCollection(**kwargs)¶

Bases: pyslet.odata2.sqlds.SQLiteEntityCollectionBase, pyslet.odata2.sqlds.SQLForeignKeyCollection

SQLite-specific collection for navigation from a foreign key

class pyslet.odata2.sqlds.SQLiteReverseKeyCollection(**kwargs)¶

Bases: pyslet.odata2.sqlds.SQLiteEntityCollectionBase, pyslet.odata2.sqlds.SQLReverseKeyCollection

SQLite-specific collection for navigation to a foreign key

class pyslet.odata2.sqlds.SQLiteAssociationCollection(**kwargs)¶

Bases: pyslet.odata2.sqlds.SQLiteEntityCollectionBase, pyslet.odata2.sqlds.SQLAssociationCollection

SQLite-specific collection for symmetric association sets

Finally, we can override the main container class to provide a complete implementation of our API using the sqlite3 module.

class pyslet.odata2.sqlds.SQLiteEntityContainer(file_path, sqlite_options={}, **kwargs)¶

Bases: pyslet.odata2.sqlds.SQLEntityContainer

Creates a container that represents a SQLite database.

Additional keyword arguments:

file_path

The path to the SQLite database file.

sqlite_options

A dictionary of additional options to pass as named arguments to the connect method. It defaults to an empty dictionary, you won’t normally need to pass additional options and you shouldn’t change the isolation_level as the collection classes have been designed to work in the default mode.

For more information see sqlite3

All other keyword arguments required to initialise the base class must be passed on construction except dbapi which is automatically set to the Python sqlite3 module.

get_collection_class()¶: Overridden to return SQLiteEntityCollection

get_symmetric_navigation_class()¶: Overridden to return SQLiteAssociationCollection

get_fk_class()¶: Overridden to return SQLiteForeignKeyCollection

get_rk_class()¶: Overridden to return SQLiteReverseKeyCollection

open()¶

Calls the underlying connect method.

Passes the file_path used to construct the container as the only parameter. You can pass the string ‘:memory:’ to create an in-memory database.

Other connection arguments are not currently supported, you can derive a more complex implementation by overriding this method and (optionally) the __init__ method to pass in values for .

break_connection(connection)¶: Calls the underlying interrupt method.

prepare_sql_type(simple_value, params, nullable=None)¶

Performs SQLite custom mappings

We inherit most of the type mappings but the following types use custom mappings:

EDM Type	SQLite Equivalent
Edm.Decimal	TEXT
Edm.Guid	BLOB
Edm.Time	REAL
Edm.Int64	INTEGER

prepare_sql_value(simple_value)¶

Returns a python value suitable for passing as a parameter.

We inherit most of the value mappings but the following types have custom mappings.

EDM Type	Python value added as parameter
Edm.Binary	buffer object
Edm.Decimal	string representation obtained with str()
Edm.Guid	buffer object containing bytes representation
Edm.Time	value of `pyslet.iso8601.Time.GetTotalSeconds()`

Our use of buffer type is not ideal as it generates warning when Python is run with the -3 flag (to check for Python 3 compatibility) but it seems unavoidable at the current time.

read_sql_value(simple_value, new_value)¶: Reverses the transformation performed by prepare_sql_value

new_from_sql_value(sql_value)¶

Returns a new simple value instance initialised from sql_value

Overridden to ensure that buffer objects returned by the underlying DB API are converted to strings. Otherwise sql_value is passed directly to the parent.

Utility Classes¶

Some miscellaneous classes documented mainly to make the implementation of the collection classes easier to understand.

class pyslet.odata2.sqlds.SQLTransaction(api, dbc)¶

Bases: object

Class used to model a transaction.

Python’s DB API uses transactions by default, hiding the details from the caller. Essentially, the first execute call on a connection issues a BEGIN statement and the transaction ends with either a commit or a rollback. It is generally considered a bad idea to issue a SQL command and then leave the connection with an open transaction.

The purpose of this class is to help us write methods that can operate either as a single transaction or as part of sequence of methods that form a single transaction. It also manages cursor creation and closing and logging.

Essentially, the class is used as follows:

t=SQLTransaction(db_module,db_connection)
try:
        t.begin()
        t.execute("UPDATE SOME_TABLE SET SOME_COL='2'")
        t.commit()
except Exception as e:
        t.rollback(e)
finally:
        t.close(e)

The transaction object can be passed to a sub-method between the begin and commit calls provided that method follows the same pattern as the above for the try, except and finally blocks. The object keeps track of these ‘nested’ transactions and delays the commit or rollback until the outermost method invokes them.

api = None¶: the database module

dbc = None¶: the database connection

cursor = None¶: the database cursor to use for executing commands

noCommit = None¶: used to manage nested transactions

queryCount = None¶: records the number of successful commands

begin()¶

Begins a transaction

If a transaction is already in progress a nested transaction is started which has no affect on the database connection itself.

execute(sqlcmd, params)¶

Executes sqlcmd as part of this transaction.

sqlcmd: A string containing the query
params: A SQLParams object containing any parameterized values.

commit()¶

Ends this transaction with a commit

Nested transactions do nothing.

rollback(err=None, swallow=False)¶

Calls the underlying database connection rollback method.

Nested transactions do not rollback the connection, they do nothing except re-raise err (if required).

If rollback is not supported the resulting error is absorbed.

err

The exception that triggered the rollback. If not None then this is logged at INFO level when the rollback succeeds.

If the transaction contains at least one successfully executed query and the rollback fails then err is logged at ERROR rather than INFO level indicating that the data may now be in violation of the model.

swallow

A flag (defaults to False) indicating that err should be swallowed, rather than re-raised.

close()¶

Closes this transaction after a rollback or commit.

Each call to begin() MUST be balanced with one call to close.

class pyslet.odata2.sqlds.SQLParams¶

Bases: object

An abstract class used to build parameterized queries.

Python’s DB API support three different conventions for specifying parameters and each module indicates the convention in use. The SQL construction methods in this module abstract away this variability for maximum portability using different implementations of the basic SQLParams class.

add_param(value)¶

Adds a value to this set of parameters

Returns the string to include in the query in place of this value.

value:: The native representation of the value in a format suitable for passing to the underlying DB API.

class pyslet.odata2.sqlds.QMarkParams¶

Bases: pyslet.odata2.sqlds.SQLParams

A class for building parameter lists using ‘?’ syntax.

class pyslet.odata2.sqlds.NumericParams¶

Bases: pyslet.odata2.sqlds.SQLParams

A class for building parameter lists using ‘:1’, ‘:2’,... syntax

class pyslet.odata2.sqlds.NamedParams¶

Bases: pyslet.odata2.sqlds.SQLParams

A class for building parameter lists using ‘:A’, ‘:B”,... syntax

Although there is more freedom with named parameters, in order to support the ordered lists of the other formats we just invent parameter names using ‘:p1’, ‘:p2’, etc.

Misc Definitions¶

pyslet.odata2.sqlds.SQL_TIMEOUT = 90¶

int(x=0) -> int or long int(x, base=10) -> int or long

Convert a number or string to an integer, or return 0 if no arguments are given. If x is floating point, the conversion truncates towards zero. If x is outside the integer range, the function returns a long instead.

If x is not a number or if base is given, then x must be a string or Unicode object representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-‘ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer literal. >>> int(‘0b100’, base=0) 4

class pyslet.odata2.sqlds.UnparameterizedLiteral(value)¶

Bases: pyslet.odata2.core.LiteralExpression

Class used as a flag that this literal is safe and does not need to be parameterized.

This is used in the query converter to prevent things like this happening when the converter itself constructs a LIKE expression:

"name" LIKE ?+?+? ; params=[u'%',u"Smith",u'%']

pyslet.odata2.sqlds.SQLOperatorPrecedence = {'>=': 4, '<>': 4, '<=': 4, 'AND': 2, 'LIKE': 4, '+': 5, '*': 6, '-': 5, ',': 0, '/': 6, 'OR': 1, 'NOT': 3, '=': 4, '<': 4, '>': 4}¶

Look-up table for SQL operator precedence calculations.

The keys are strings representing the operator, the values are integers that allow comparisons for operator precedence. For example:

SQLOperatorPrecedence['+']<SQLOperatorPrecedence['*']
SQLOperatorPrecedence['<']==SQLOperatorPrecedence['>']

class pyslet.odata2.sqlds.DummyLock¶

Bases: object

An object to use in place of a real Lock, can always be acquired

Exceptions¶

class pyslet.odata2.sqlds.DatabaseBusy¶

Bases: pyslet.odata2.sqlds.SQLError

Raised when a database connection times out.

class pyslet.odata2.sqlds.SQLError¶

Bases: exceptions.Exception

Base class for all module exceptions.

OData Server Reference¶

XML: Basic Constructs¶

This module defines classes for working with XML documents. The version of the standard implemented is the Extensible Markup Language (Fifth Edition), for more info see: http://www.w3.org/TR/xml/

XML is an integral part of many standards for LET but Pyslet takes a slightly different approach from the pre-existing XML support in the Python language. XML elements are represented by instances of a basic Element class which can be used as a base class to customize document processing for specific types of XML document. It also allows these XML elements to ‘come live’ with additional methods and behaviours.

Documents¶

class pyslet.xml20081126.structures.Node(parent)¶

Bases: object

Base class for Element and Document shared attributes.

XML documents are defined hierarchicaly, each element has a parent which is either another element or an XML document.

parent = None¶: The parent of this element, for XML documents this attribute is used as a sentinel to simplify traversal of the hierarchy and is set to None.

GetChildren()¶: Returns an iterator over this object’s children.

classmethod GetElementClass(name)¶

Returns a class object suitable for representing element name

name is a unicode string representing the element name.

The default implementation returns None - for elements this has the effect of deferring the call to the parent document (where this method is overridden to return Element).

This method is called immediately prior to ChildElement() and (when applicable) GetChildClass().

The real purpose of this method is to allow an element class to directly control the way the name of a child element maps to a class to represent it. You would normally override this method in the Document to map element names to classes but in some cases you may want to tweek the mapping at the individual element level. For example, if the same element name is used for two different purposes in the same XML document, although confusing, this is allowed in XML schema.

GetChildClass(stagClass)¶

Returns the element class implied by the STag for stagClass in this context.

This method is only called when the XMLParser.sgmlOmittag option is in effect. It is called prior to ChildElement() below and gives the context (the parent element or document) a chance to modify the child element that will be created (or reject it out-right, by returning None).

For well-formed XML documents the default implementation is sufficient as it simply returns stagClass.

The XML parser may pass None for stagClass indicating that PCDATA has been found in element content. This method should return the first child element that may contain (directly or indirectly) PCDATA or None if no children may contain PCDATA (or SGML-style omittag is not supported)

ChildElement(childClass, name=None)¶

Returns a new child of the given class attached to this object.

childClass is a class (or callable) used to create a new instance

of Element.
name is the name given to the element (by the caller). If no name

is given then the default name for the child is used. When the child returned is an existing instance, name is ignored.

ProcessingInstruction(target, instruction='')¶

Abstract method for handling processing instructions encountered by the parser while parsing this object’s content.

By default, processing instructions are ignored.

class pyslet.xml20081126.structures.Document(root=None, baseURI=None, reqManager=None, **args)¶

Bases: pyslet.xml20081126.structures.Node

Base class for all XML documents.

Initialises a new Document from optional keyword arguments.

With no arguments, a new Document is created with no baseURI or root element.

If root is a class object (descended from Element) it is used to create the root element of the document.

If root is an orphan instance of Element (i.e., it has no parent) is is used as the root element of the document and its Element.AttachToDocument() method is called.

baseURI can be set on construction (see SetBase) and a reqManager object can optionally be passed for managing and http(s) connections.

baseURI = None¶: The base uri of the document.

lang = None¶: The default language of the document.

declaration = None¶: The XML declaration (or None if no XMLDeclaration is used)

dtd = None¶: The dtd associated with the document.

root = None¶: The root element or None if no root element has been created yet.

GetChildren()¶: If the document has a root element it is returned in a single item list, otherwise an empty list is returned.

XMLParser(entity)¶

Returns an XMLParser instance suitable for parsing this type of document.

This method allows some document classes to override the parser used to parse them. This method is only used when parsing existing document instances (see Read() for more information).

Classes that override this method may still register themselves with RegisterDocumentClass() but if they do then the default XMLParser object will be used when the this document class is automatically created when parsing an unidentified XML stream.

classmethod GetElementClass(name)¶

Returns a class object suitable for representing name

name is a unicode string representing the element name.

The default implementation returns Element.

ChildElement(childClass, name=None)¶

Creates the root element of the given document.

If there is already a root element it is detached from the document first using Element.DetachFromDocument().

SetBase(baseURI)¶

Sets the baseURI of the document to the given URI.

baseURI should be an instance of pyslet.rfc2396.URI or an object that can be passed to its constructor.

Relative file paths are resolved relative to the current working directory immediately and the absolute URI is recorded as the document’s baseURI.

GetBase()¶: Returns a string representation of the document’s baseURI.

SetLang(lang)¶: Sets the default language for the document.

GetLang()¶: Returns the default language for the document.

ValidationError(msg, element, data=None, aname=None)¶

Called when a validation error is triggered by element.

This method is designed to be overriden to implement custom error handling or logging (which is likely to be added in future to this module).

msg contains a brief message suitable for describing the error in a log file. data and aname have the same meanings as Element.ValidationError.

Read(src=None, **args)¶

Reads this document, parsing it from a source stream.

With no arguments the document is read from the baseURI which must have been specified on construction or with a call to the SetBase() method.

You can override the document’s baseURI by passing a value for src which may be an instance of XMLEntity or an object that can be passed as a valid source to its constructor.

Create(dst=None, **args)¶

Creates the Document.

Create outputs the document as an XML stream. The stream is written to the baseURI by default but if the ‘dst’ argument is provided then it is written directly to there instead. dst can be any object that supports the writing of unicode strings.

Currently only documents with file type baseURIs are supported. The file’s parent directories are created if required. The file is always written using the UTF-8 as per the XML standard.

Update(**args)¶

Updates the Document.

Update outputs the document as an XML stream. The stream is written to the baseURI which must already exist! Currently only documents with file type baseURIs are supported.

DiffString(otherDoc, before=10, after=5)¶: Compares this document to otherDoc and returns first point of difference.

pyslet.xml20081126.structures.RegisterDocumentClass(docClass, rootName, publicID=None, systemID=None)¶

Registers a document class for use by XMLParser.ParseDocument().

This module maintains a single table of document classes which can be used to identify the correct class to use to represent a document based on the information obtained from the DTD.

docClass

is the class object being registered, it must be derived from Document
rootName

is the name of the root element or None if this class can be used with any root element.
publicID

is the public ID of the doctype, or None if any doctype can be used with this document class.
systemID

is the system ID of the doctype, this will usually be None indicating that the document class can match any system ID.

Characters¶

pyslet.xml20081126.structures.IsChar(c)¶

Tests if the character c matches the production for [2] Char.

If c is None IsChar returns False.

pyslet.xml20081126.structures.IsDiscouraged(c)¶

Tests if the character c is one of the characters discouraged in the specification.

Note that this test is currently limited to the range of unicode characters available in the narrow python build.

Common Syntactic Constructs¶

pyslet.xml20081126.structures.IsS(c)¶: Tests if a single character c matches production [3] S

pyslet.xml20081126.structures.IsWhiteSpace(data)¶: Tests if every character in data matches production [3] S

pyslet.xml20081126.structures.ContainsS(data)¶: Tests if data contains any characters matching production [3] S

pyslet.xml20081126.structures.StripLeadingS(data)¶: Returns data with leading S removed.

pyslet.xml20081126.structures.NormalizeSpace(data)¶

Returns data normalized according to the further processing rules for attribute-value normalization:

”...by discarding any leading and trailing space (#x20) characters, and by replacing sequences of space (#x20) characters by a single space (#x20) character”

pyslet.xml20081126.structures.CollapseSpace(data, sMode=True, sTest=<function IsS at 0x7fe07c532e60>)¶

Returns data with all spaces collapsed to a single space.

sMode determines the fate of any leading space, by default it is True and leading spaces are ignored provided the string has some non-space characters.

You can override the test of what consitutes a space by passing a function for sTest, by default we use IsS.

Note on degenerate case: this function is intended to be called with non-empty strings and will never return an empty string. If there is no data then a single space is returned (regardless of sMode).

pyslet.xml20081126.structures.IsNameStartChar(c)¶: Tests if the character c matches production [4] NameStartChar.

pyslet.xml20081126.structures.IsNameChar(c)¶: Tests if a single character c matches production [4a] NameChar

pyslet.xml20081126.structures.IsValidName(name)¶: Tests if name is a string matching production [5] Name

pyslet.xml20081126.structures.IsReservedName(name)¶: Tests if name is reserved for future standardization, e.g., if it begins with ‘xml’.

pyslet.xml20081126.structures.IsPubidChar(c)¶: Tests if the character c matches production for [13] PubidChar.

Character Data and Markup¶

pyslet.xml20081126.structures.EscapeCharData(src, quote=False)¶

Returns a unicode string with XML reserved characters escaped.

We also escape return characters to prevent them being ignored. If quote is True then the string is returned as a quoted attribute value.

pyslet.xml20081126.structures.EscapeCharData7(src, quote=False)¶: Returns a unicode string with reserved and non-ASCII characters escaped.

CDATA Sections¶

pyslet.xml20081126.structures.EscapeCDSect(src)¶

Returns a unicode string enclosed in <!CDATA[[ ]]> with ]]> by the clumsy sequence: ]]>]]><!CDATA[[

Degenerate case: an empty string is returned as an empty string

Prolog and Document Type Declaration¶

class pyslet.xml20081126.structures.XMLDTD¶

Bases: object

An object that models a document type declaration.

The document type declaration acts as a container for the entity, element and attribute declarations used in a document.

name = None¶: The declared Name of the root element

parameterEntities = None¶: A dictionary of XMLParameterEntity instances keyed on entity name.

generalEntities = None¶: A dictionary of XMLGeneralEntity instances keyed on entity name.

notations = None¶: A dictionary of XMLNotation instances keyed on notation name.

elementList = None¶: A dictionary of ElementType definitions keyed on the name of element.

attributeLists = None¶: A dictionary of dictionaries, keyed on element name. Each of the resulting dictionaries is a dictionary of XMLAttributeDefinition keyed on attribute name.

DeclareEntity(entity)¶

Declares an entity in this document.

The same method is used for both general and parameter entities. The value of entity can be either an XMLGeneralEntity or an XMLParameterEntity instance.

GetParameterEntity(name)¶

Returns the parameter entity definition matching name.

Returns an instance of XMLParameterEntity. If no parameter has been declared with name then None is returned.

GetEntity(name)¶

Returns the general entity definition matching name.

Returns an instance of XMLGeneralEntity. If no general has been declared with name then None is returned.

DeclareNotation(notation)¶

Declares a notation for this document.

The value of notation must be a XMLNotation instance.

GetNotation(name)¶

Returns the notation declaration matching name.

Returns an instance of XMLNotation. If no notation has been declared with name then None is returned.

DeclareElementType(eType)¶

Declares an element type.

eType is an ElementType instance containing the element definition.

GetElementType(elementName)¶

Looks up an element type definition.

elementName is the name of the element type to look up

The method returns an instance of ElementType or None if no element with that name has been declared.

DeclareAttribute(elementName, attributeDef)¶

Declares an attribute.

elementName

is the name of the element type which should have this attribute applied
attributeDef

is an XMLAttributeDefinition instance describing the attribute being declared.

GetAttributeList(name)¶

Returns a dictionary of attribute definitions for the element type name.

If there are no attributes declared for this element type, None is returned.

GetAttributeDefinition(elementName, attributeName)¶

Looks up an attribute definition.

elementName is the name of the element type in which to search

attributeName is the name of the attribute to search for.

The method returns an instance of XMLAttributeDefinition or None if no attribute matching this description has been declared.

class pyslet.xml20081126.structures.XMLDeclaration(version, encoding='UTF-8', standalone=False)¶

Bases: pyslet.xml20081126.structures.XMLTextDeclaration

Represents a full XML declaration.

Unlike the parent class, XMLTextDeclaration, the version is required. standalone defaults to False as this is the assumed value if there is no standalone declaration.

standalone = None¶: Whether an XML document is standalone.

Logical Structures¶

class pyslet.xml20081126.structures.Element(parent, name=None)¶

Bases: pyslet.xml20081126.structures.Node

Basic class that represents all XML elements.

Some aspects of the element’s XML serialisation behaviour are controlled by special class attributes that can be set on derived classes.

XMLNAME: the default name of the element the class represents.
XMLCONTENT: the default content model of the element; one of the ElementType constants.
ID: the name of the ID attribute if the element has a unique ID. With this class attribute set, ID handling is automatic (see SetID() and id below).

By default, attributes are simply stored as strings mapped in an internal dictionary. It is often more useful to map XML attributes on to python attributes, parsing and validating their values to python objects. This mapping can be provided using class attributes of the form XMLATTR_aname where aname is the name of the attribute as it would appear in the XML element start or empty element tag.

XMLATTR_aname=<string>

This form creates a simple mapping from the XML attribute ‘aname’ to a python attribute with a defined name. For example, you might want to create a mapping like this to avoid a python reserved word:
XMLATTR_class="styleClass"
This allows XML elements like this:
<element class="x"/>
To be parsed into python objects that behave like this:
element.styleClass=="x"     # True
If an instance is missing a python attribute corresponding to a defined XML attribute, or it’s value has been set to None, then the XML attribute is omitted from the element’s tag when generating XML output.

XMLATTR_aname=(<string>, decodeFunction, encodeFunction)

More complex attributes can be handled by setting XMLATTR_aname to a tuple. The first item is the python attribute name (as above); the decodeFunction is a simple callable that takes a string argument and returns the decoded value of the attribute and the encodeFunction performs the reverse transformation.

The encode/decode functions can be None to indicate a no-operation.

For example, you might want to create an integer attribute using something like:

<element apples="5"/>

# class attribute definition
XMLATTR_apples=('nApples',int,unicode)

# resulting object behaves like this...
element.nApples==5      # True

XMLATTR_aname=(<string>, decodeFunction, encodeFunction, type)

When XML attribute values are parsed from tags the optional type component of the tuple descriptor can be used to indicate a multi-valued attribute (for example, XML attributes defined using one of the plural forms, IDREFS, ENTITIES and NMTOKENS). If the type value is not None then the XML attribute value is first split by white-space, as per the XML specification, and then the decode function is applied to each resulting component. The instance attribute is then set depending on the value of type:

types.ListType
The instance attribute becomes a list, for example:

<element primes="2 3 5 7"/>

# class attribute definition
XMLATTR_primes=('primes',int,unicode)

# resulting object behaves like this...
element.primes==[2,3,5,7]       # True
types.DictType
The instance attribute becomes a dictionary mapping parsed values on to their frequency, for example:

<element fruit="apple pear orange pear"/>

# class attribute definition
XMLATTR_fruit=('fruit',None,None,types.DictType)

# resulting object behaves like this...
element.fruit=={'apple':1, 'orange':1, 'pear':2}        # True
In this case, the decode function (if given) must return a hashable object.
When creating XML output the reverse transformations are performed using the encode functions and the type (plain, list or dict) of the attribute’s current value. The declared multi-valued type is ignored. For dictionary values the order of the output values may not be the same as the order originally read from the XML input.

Warning: Empty lists and dictionaries result in XML attribute values which are present but with empty strings. If you wish to omit these attributes in the output XML you must set the attribute value to None in the instance.

XMLAMAP XMLARMAP

Internally, the XMLATTR_* descriptors are parsed into two mappings. The XMLAMAP maps XML attribute names onto a tuple of:

(<python attribute name>, decodeFunction, type)

The XMLARMAP maps python attribute names onto a tuple of:

(<xml attribute name>, encodeFunction)

The mappings are created automatically as needed.

For legacy reasons, the multi-valued rules can also be invoked by setting an instance member to either a list or dictionary prior to parsing the instance from XML (e.g., in a constructor).

XML attribute names may contain many characters that are not legal in Python method names and automated attribute processing is not supported for these attributes. In practice, the only significant limitation is the colon. The common xml-prefixed attributes such as xml:lang are handled using special purposes methods.

XMLCONTENT = 2¶: for consistency with the behaviour of the default methods we claim to be mixed content

reset(resetAttributes=False)¶: Clears all attributes and (optional) children.

GetDocument()¶

Returns the document that contains the element.

If the element is an orphan, or is the descendent of an orphan then None is returned.

SetID(id)¶

Sets the id of the element, registering the change with the enclosing document.

If the id is already taken then XMLIDClashError is raised.

MangleAttributeName(name)¶

Returns a mangled attribute name, used when setting attributes.

If name cannot be mangled, None is returned.

UnmangleAttributeName(mName)¶

Returns an unmangled attribute name, used when getting attributes.

If mName is not a mangled name, None is returned.

GetAttributes()¶

Returns a dictionary object that maps attribute names onto values.

Each attribute value is represented as a (possibly unicode) string. Derived classes should override this method if they define any custom attribute setters.

The dictionary returned represents a copy of the information in the element and so may be modified by the caller.

SetAttribute(name, value)¶

Sets the value of an attribute.

If value is None then the attribute is removed or, if an XMLATTR_ mapping is in place its value is set to an empty list, dictionary or None as appropriate.

GetAttribute(name)¶

Gets the value of a single attribute as a string.

If the element has no attribute with name then KeyError is raised.

IsEmpty()¶

Returns True/False indicating whether this element must be empty.

If the class defines the XMLCONTENT attribute then the model is taken from there and this method returns True only if XMLCONTENT is XMLEmpty.

Otherwise, the method defaults to False

IsMixed()¶

Indicates whether or not the element may contain mixed content.

If the class defines the XMLCONTENT attribute then the model is taken from there and this method returns True only if XMLCONTENT is XMLMixedContent.

Otherwise, the method default ot True

GetChildren()¶

Returns an iterable of the element’s children.

This method iterates through the internal list of children. Derived classes with custom factory elements MUST override this method.

Each child is either a string type, unicode string type or instance of Element (or a derived class thereof). We do not represent comments, processing instructions or other meta-markup.

GetCanonicalChildren()¶

A wrapper for GetChildren() that returns an iterable of the element’s children canonicalized for white space.

We check the current setting of xml:space, returning the same list of children as GetChildren() if ‘preserve’ is in force. Otherwise we remove any leading space and collapse all others to a single space character.

ChildElement(childClass, name=None)¶

Returns a new child of the given class attached to this element.

A new child is created and attached to the element’s model unless the model supports a single element of the given childClass and the element already exists, in which case the existing instance is returned.

childClass is a class (or callable) used to create a new instance.

name is the name given to the element (by the caller). If no name is given then the default name for the child is used. When the child returned is an existing instance, name is ignored.

The default implementation checks for a custom factory method and calls it if defined and does no further processing. A custom factory method is a method of the form ClassName or an attribute that is being used to hold instances of this child. The attribute must already exist and can be one of None (optional child, new child is created), a list (optional repeatable child, new child is created and appended) or an instance of childClass (required/existing child, no new child is created, existing instance returned).

When no custom factory method is found the class hierarchy of childClass is enumerated and the search continues for factory methods corresponding to these parent classes.

If no custom factory method is defined then the default processing simply creates an instance of child (if necessary) and attaches it to the internal list of children.

DeleteChild(child)¶

Deletes the given child from this element’s children.

We follow the same factory conventions as for child creation except that an attribute pointing to a single child (of this class) will be replaced with None. If a custom factory method is found then the corresponding Delete_ClassName method must also be defined.

FindChildren(childClass, childList, max=None)¶

Finds up to max children of class childClass from the element and its children.

Deprecated in favour of list(FindChildrenDepthFirst(childClass,False))

All matching children are added to childList. If specifing a max number of matches then the incoming list must originally be empty to prevent early termination.

Note that if max is None, the default, then all children of the given class are returned with the proviso that nested matches are not included. In other words, if the model of childClass allows further elements of type childClass as children (directly or indirectly) then only the top-level match is returned.

Effectively this method provides a depth-first list of children. For example, to get all <div> elements in an HTML <body> you would have to recurse over the resulting list calling FindChildren again until the list of matching children stops growing.

FindChildrenBreadthFirst(childClass, subMatch=True, maxDepth=1000)¶

A generator method that iterates over children of class childClass using a breadth first scan.

childClass may also be a tuple as per the definition of the builtin isinstance function in python.

If subMatch is True (the default) then matching elements are also scanned for nested matches. If False, only the outer-most matching element is returned.

maxDepth controls the depth of the scan with level 1 indicating direct children only. It must be a positive integer and defaults to 1000.

Warning: to reduce memory requirements when searching large documents this method performs a two-pass scan of the element’s children, i.e., GetChildren() will be called twice.

Given that XML documents tend to be broader than they are deep FindChildrenDepthFirst() is a better method to use for general purposes.

FindChildrenDepthFirst(childClass, subMatch=True, maxDepth=1000)¶

A generator method that iterates over children of class childClass using a depth first scan.

childClass may also be a tuple as per the definition of the builtin isinstance function in python.

If subMatch is True (the default) then matching elements are also scanned for nested matches. If False, only the outer-most matching element is returned.

maxDepth controls the depth of the scan with level 1 indicating direct children only. It must be a positive integer and defaults to 1000.

FindParent(parentClass)¶

Finds the first parent of class parentClass of this element.

If this element has no parent of the given class then None is returned.

AttachToParent(parent)¶

Called to attach an orphan element to a parent.

This method does not do any special handling of child elements, the caller takes responsibility for ensuring that this element will be returned by future calls to parent.GetChildren(). However, AttachToDocument() is called to ensure id registrations are made.

AttachToDocument(doc=None)¶

Called when the element is first attached to a document.

The default implementation ensures that any ID attributes belonging to this element or its descendents are registered.

DetachFromParent()¶

Called to detach an element from its parent, making it an orphan

This method does not do any special handling of child elements, the caller takes responsibility for ensuring that this element will no longer be returned by future calls to parent.GetChildren(). However, DetachFromDocument() is called to ensure id registrations are removed.

DetachFromDocument(doc=None)¶

Called when an element is being detached from a document.

The default implementation ensures that any ID attributes belonging to this element or its descendents are unregistered.

AddData(data)¶

Adds a string or unicode string to this element’s children.

This method raises a ValidationError if the element cannot take data children.

ContentChanged()¶

Notifies an element that its content has changed.

The default implementation tidies up the list of children to make future comparisons simpler and faster.

GenerateValue(ignoreElements=False)¶

A generator function that returns the strings that compromise this element’s value (useful when handling elements that contain a large amount of data). For more information see GetValue(). Note that:

string.join(e.GenerateValue(),u'')==e.GetValue()

GetValue(ignoreElements=False)¶

By default, returns a single unicode string representing the element’s data.

The default implementation is only supported for elements where mixed content is permitted (IsMixed()). It uses GetChildren() to iterate through the children.

If the element is empty an empty string is returned.

Derived classes may return more complex objects, such as values of basic python types or class instances, performing validation based on application-defined rules.

If the element contains child elements then XMLMixedContentError is raised. You can pass ignoreElements as True to override this behaviour in the unlikely event that you want:

<!-- elements like this... -->
<data>This is <em>the</em> value</data>

# to behave like this:
data.GetValue(True)==u"This is  value" 

SetValue(value)¶

Replaces the value of the element with the (unicode) value.

The default implementation is only supported for elements where mixed content is permitted (IsMixed()) and only affects the internally maintained list of children. Elements with more complex mixed models MUST override this method.

If value is None then the element becomes empty.

Derived classes may allow more complex values to be set, such as values of basic python types or class instances depending on the element type being represented in the application.

ValidationError(msg, data=None, aname=None)¶

Indicates that a validation error occurred in this element.

An error message indicates the nature of the error.

The data that caused the error may be given in data.

Furthermore, the attribute name may also be given indicating that the offending data was in an attribute of the element and not the element itself.

SortNames(nameList)¶

Given a list of element or attribute names, sorts them in a predictable order

The default implementation assumes that the names are strings or unicode strings so uses the default sort method.

Copy(parent=None)¶

Creates a new instance of this element which is a deep copy of this one.

parent is the parent node to attach the new element to. If it is None then a new orphan element is created.

This method mimics the process of serialisation and deserialisation (without the need to generate markup). As a result, element attributes are serialised and deserialised to strings during the copy process.

GetBase()¶: Returns the value of the xml:base attribute as a string.

SetBase(base)¶

Sets the value of the xml:base attribute from a string.

Changing the base of an element effects the interpretation of all relative URIs in this element and its children.

ResolveBase()¶

Returns a fully specified URI for the base of the current element.

The URI is calculated using any xml:base values of the element or its ancestors and ultimately relative to the baseURI.

If the element is not contained by a Document, or the document does not have a fully specified baseURI then the return result may be a relative path or even None, if no base information is available.

ResolveURI(uri)¶

Returns a fully specified URL, resolving uri in the current context.

The uri is resolved relative to the xml:base values of the element’s ancestors and ultimately relative to the document’s baseURI.

RelativeURI(href)¶

Returns href expressed relative to the element’s base.

If href is a relative URI then it is converted to a fully specified URL by interpreting it as being the URI of a file expressed relative to the current working directory.

If the element does not have a fully-specified base URL then href is returned as a fully-specified URL itself.

GetLang()¶: Returns the value of the xml:lang attribute as a string.

SetLang(lang)¶

Sets the value of the xml:lang attribute from a string.

See ResolveLang() for how to obtain the effective language of an element.

ResolveLang()¶

Returns the effective language for the current element.

The language is resolved using the xml:lang value of the element or its ancestors. If no xml:lang is in effect then None is returned.

PrettyPrint()¶

Indicates if this element’s content should be pretty-printed.

This method is used when formatting XML files to text streams. The behaviour can be affected by the xml:space attribute or by derived classes that can override the default behaviour.

If this element has xml:space set to ‘preserve’ then we return False. If self.parent.PrettyPrint() returns False then we return False.

Otherwise we return False if we know the element is (or should be) mixed content, True otherwise.

Note: an element of undetermined content model that contains only elements and white space is pretty printed.

WriteXMLAttributes(attributes, escapeFunction=<function EscapeCharData at 0x7fe07c5cf5f0>, root=False)¶

Adds strings representing the element’s attributes

attributes is a list of unicode strings. Attributes should be appended as strings of the form ‘name=”value”’ with values escaped appropriately for XML output.

GenerateXML(escapeFunction=<function EscapeCharData at 0x7fe07c5cf5f0>, indent='', tab='t', root=False)¶

A generator function that returns strings representing the serialised version of this element:

# the element's serialised output can be obtained as a single string
string.join(e.GenerateXML(),'')

class pyslet.xml20081126.structures.ElementType¶

Bases: object

An object for representing element type definitions.

Empty = 0¶: Content type constant for EMPTY

Any = 1¶: Content type constant for ANY

Mixed = 2¶: Content type constant for mixed content

SGMLCDATA = 4¶: Additional content type constant for SGML CDATA

name = None¶: The name of this element

contentType = None¶: The content type of this element, one of the constants defined above.

contentModel = None¶: A XMLContentParticle instance which contains the element’s content model or None in the case of EMPTY or ANY declarations.

particleMap = None¶: A mapping used to validate the content model during parsing. It maps the name of the first child element found to a list of XMLNameParticle instances that can represent it in the content model. For more information see XMLNameParticle.particleMap.

BuildModel()¶: Builds internal strutures to support model validation.

IsDeterministic()¶

Tests if the content model is deterministic.

For degenerates cases (elements declared with ANY or EMPTY) the method always returns True.

class pyslet.xml20081126.structures.XMLContentParticle¶

Bases: object

An object for representing content particles.

ZeroOrOne = 1¶: Occurrence constant for ‘?’

OneOrMore = 3¶: Occurrence constant for ‘+’

occurrence = None¶: One of the occurrence constants defined above.

BuildParticleMaps(exitParticles)¶

Abstract method that builds the particle maps for this node or its children.

For more information see XMLNameParticle.particleMap.

Although only name particles have particle maps this method is called for all particle types to allow the model to be built hierarchically from the root out to the terminal (name) nodes. exitParticles provides a mapping to all the following particles outside the part of the hierarchy rooted at the current node that are directly reachable from the particles inside.

SeekParticles(pMap)¶

Abstract method that adds all possible entry particles to pMap.

pMap is a mapping from element name to a list of XMLNameParticles XMLNameParticle.

Returns True if a required particle was added, False if all particles added are optional.

Like BuildParticleMaps(), this method is called for all particle types. The mappings requested represent all particles inside the part of the hierarchy rooted at the current node that are directly reachable from the preceeding particles outside.

AddParticles(srcMap, pMap)¶

A utility method that adds particles from srcMap to pMap.

Both maps are mappings from element name to a list of XMLNameParticles XMLNameParticle. All entries in srcMap not currently in pMap are added.

IsDeterministic(pMap)¶

A utility method for identifying deterministic particle maps.

A deterministic particle map is one in which name maps uniquely to a single content particle. A non-deterministic particle map contains an ambiguity, for example ((b,d)|(b,e)). The particle map created by SeekParticles() for the enclosing choice list is would have two entries for ‘b’, one to map the first particle of the first sequence and one to the first particle of the second sequence.

Although non-deterministic content models are not allowed in SGML they are tolerated in XML and are only flagged as compatibility errors.

class pyslet.xml20081126.structures.XMLNameParticle¶

Bases: pyslet.xml20081126.structures.XMLContentParticle

An object representing a content particle for a named element in the grammar

name = None¶: the name of the element type that matches this particle

particleMap = None¶

Each XMLNameParticle has a particle map that maps the name of the ‘next’ element found in the content model to the list of possible XMLNameParticles XMLNameParticle that represent it in the content model.

The content model can be traversed using ContentParticleCursor.

class pyslet.xml20081126.structures.XMLChoiceList¶

Bases: pyslet.xml20081126.structures.XMLContentParticle

An object representing a choice list of content particles in the grammar

class pyslet.xml20081126.structures.XMLSequenceList¶

Bases: pyslet.xml20081126.structures.XMLContentParticle

An object representing a sequence list of content particles in the grammar

class pyslet.xml20081126.structures.XMLAttributeDefinition¶

Bases: object

An object for representing attribute declarations.

CData = 0¶: Type constant representing CDATA

ID = 1¶: Type constant representing ID

IDRef = 2¶: Type constant representing IDREF

IDRefs = 3¶: Type constant representing IDREFS

Entity = 4¶: Type constant representing ENTITY

Entities = 5¶: Type constant representing ENTITIES

NmToken = 6¶: Type constant representing NMTOKEN

NmTokens = 7¶: Type constant representing NMTOKENS

Notation = 8¶: Type constant representing NOTATION

Implied = 0¶: Presence constant representing #IMPLIED

Required = 1¶: Presence constant representing #REQUIRED

Fixed = 2¶: Presence constant representing #FIXED

Default = 3¶: Presence constant representing a declared default value

entity = None¶: the entity in which this attribute was declared

name = None¶: the name of the attribute

type = None¶: One of the above type constants

values = None¶: An optional dictionary of values

defaultValue = None¶: An optional default value

Physical Structures¶

class pyslet.xml20081126.structures.XMLEntity(src=None, encoding=None, reqManager=None)¶

Bases: object

An object representing an entity.

This object serves two purposes, it acts as both the object used to store information about declared entities and also as a parser for feeding unicode characters to the main XMLParser.

Optional src, encoding and reqManager parameters can be provided, if src is not None then these parameters are used to open the entity reader immediately using one of the Open methods described below.

src may be a unicode string, a byte string, an instance of pyslet.rfc2396.URI or any object that supports file-like behaviour. If using a file, the file must support seek behaviour.

location = None¶: the location of this entity (used as the base URI to resolve relative links)

mimetype = None¶: the mime type of the entity, if known, or None

encoding = None¶: the encoding of the entity (text entities)

charSource = None¶: A unicode data reader used to read characters from the entity. If None, then the entity is closed.

bom = None¶: flag to indicate whether or not the byte order mark was detected. If detected the flag is set to True. An initial byte order mark is not reported in the_char or by the NextChar() method.

the_char = None¶: the character at the current position in the entity

lineNum = None¶: the current line number within the entity (first line is line 1)

linePos = None¶: the current character position within the entity (first char is 1)

buffText = None¶: used by XMLParser.PushEntity()

ChunkSize = 4096¶

Characters are read from the dataSource in chunks. The default chunk size is 4KB.

In fact, in some circumstances the entity reader starts more cautiously. If the entity reader expects to read an XML or Text declaration, which may have an encoding declaration then it reads one character at a time until the declaration is complete. This allows the reader to change to the encoding in the declaration without causing errors caused by reading too many characters using the wrong codec. See ChangeEncoding() and KeepEncoding() for more information.

GetName()¶: Abstract method to return a name to represent this entity in logs and error messages.

IsExternal()¶

Returns True if this is an external entity.

The default implementation returns True if location is not None, False otherwise.

Open()¶

Opens the entity for reading.

The default implementation uses OpenURI() to open the entity from location if available, otherwise it raises UnimplementedError.

IsOpen()¶: Returns True if the entity is open for reading.

OpenUnicode(src)¶: Opens the entity from a unicode string.

OpenString(src, encoding=None)¶

Opens the entity from a byte string.

The optional encoding is used to convert the string to unicode and defaults to None - meaning that the auto-detection method will be applied.

The advantage of using this method instead of converting the string to unicode and calling OpenUnicode() is that this method creates a unicode reader object to parse the string instead of making a copy of it in memory.

OpenFile(src, encoding='utf-8')¶

Opens the entity from an existing (open) binary file.

The optional encoding provides a hint as to the intended encoding of the data and defaults to UTF-8. Unlike other Open* methods we do not assume that the file is seekable however, you may set encoding to None for a seekable file thus invoking auto-detection of the encoding.

OpenURI(src, encoding=None, reqManager=None)¶

Opens the entity from a URI passed in src.

The file, http and https schemes are the only ones supported.

The optional encoding provides a hint as to the intended encoding of the data and defaults to UTF-8. For http(s) resources this parameter is only used if the charset cannot be read successfully from the HTTP headers.

The optional reqManager allows you to pass an existing instance of pyslet.rfc2616.Client for handling URI with http or https schemes.

OpenHTTPResponse(src, encoding='utf-8')¶

Opens the entity from an HTTP response passed in src.

The optional encoding provides a hint as to the intended encoding of the data and defaults to UTF-8. This parameter is only used if the charset cannot be read successfully from the HTTP response headers.

reset()¶: Resets an open entity, causing it to return to the first character in the entity.

GetPositionStr()¶

Returns a short string describing the current line number and character position.

For example, if the current character is pointing to character 6 of line 4 then it will return the string ‘Line 4.6’

NextChar()¶

Advances to the next character in an open entity.

This method takes care of the End-of-Line handling rules for XML which force us to remove any CR characters and replace them with LF if they appear on their own or to silenty drop them if they appear as part of a CR-LF combination.

AutoDetectEncoding(srcFile)¶

Auto-detects the character encoding in srcFile.

Should only be called for seek-able streams opened in binary mode.

ChangeEncoding(encoding)¶

Changes the encoding used to interpret the entity’s stream.

In many cases we can only guess at the encoding used in a file or other byte stream. However, XML has a mechanism for declaring the encoding as part of the XML or Text declaration. This declaration can typically be parsed even if the encoding has been guessed incorrectly initially. This method allows the XML parser to notify the entity that a new encoding has been declared and that future characters should be interpreted with this new encoding.

You can only change the encoding once. This method calls KeepEncoding() once the encoding has been changed.

KeepEncoding()¶

Tells the entity parser that the encoding will not be changed again.

This entity parser starts in a cautious mode, parsing the entity one character a time to avoid errors caused by buffering with the wrong encoding. This method should be called once the encoding is determined so that the entity parser can use its internal character buffer.

NextLine()¶

Called when the entity reader detects a new line.

This method increases the internal line count and resets the character position to the beginning of the line. You will not normally need to call this directly as line handling is done automatically by NextChar().

close()¶: Closes the entity.

class pyslet.xml20081126.structures.XMLGeneralEntity(name=None, definition=None, notation=None)¶

Bases: pyslet.xml20081126.structures.XMLDeclaredEntity

An object for representing general entities.

A general entity can be constructed with an optional name, definition and notation, used to initialise the following fields.

notation = None¶: the notation name for external unparsed entities

GetName()¶: Returns the name of the entity formatted as a general entity reference.

class pyslet.xml20081126.structures.XMLParameterEntity(name=None, definition=None)¶

Bases: pyslet.xml20081126.structures.XMLDeclaredEntity

An object for representing parameter entities.

A parameter entity can be constructed with an optional name and definition, used to initialise the following two fields.

NextChar()¶: Overrridden to provide trailing space during special parameter entity handling.

OpenAsPE()¶

Opens the parameter entity for reading in the context of a DTD.

This special method implements the rule that the replacement text of a parameter entity, when included as a PE, must be enlarged by the attachment of a leading and trailing space.

GetName()¶: Returns the name of the entity formatted as a parameter entity reference.

class pyslet.xml20081126.structures.XMLExternalID(public=None, system=None)¶

Bases: object

Used to represent external references to entities.

Returns an instance of XMLExternalID. One of public and system should be provided.

GetLocation(base=None)¶

Returns the absolute URI where the external entity can be found.

Returns a pyslet.rfc2396.URI resolved against base if applicable. If there is no system identifier then None is returned.

class pyslet.xml20081126.structures.XMLTextDeclaration(version='1.0', encoding='UTF-8')¶

Bases: object

Represents the text components of an XML declaration.

Both version and encoding are optional, though one or other are required depending on the context in which the declaration will be used.

class pyslet.xml20081126.structures.XMLNotation(name, externalID)¶

Bases: object

Represents an XML Notation

Returns an XMLNotation instance.

externalID is a XMLExternalID instance in which one of public or system must be provided.

name = None¶: the notation name

externalID = None¶: the external ID of the notation (an XMLExternalID instance)

Character Classes¶

pyslet.xml20081126.structures.IsLetter(c)¶: Tests if the character c matches production [84] Letter.

pyslet.xml20081126.structures.IsBaseChar(c)¶: Tests if the character c matches production [85] BaseChar.

pyslet.xml20081126.structures.IsIdeographic(c)¶: Tests if the character c matches production [86] Ideographic.

pyslet.xml20081126.structures.IsCombiningChar(c)¶: Tests if the character c matches production [87] CombiningChar.

pyslet.xml20081126.structures.IsDigit(c)¶: Tests if the character c matches production [88] Digit.

pyslet.xml20081126.structures.IsExtender(c)¶: Tests if the character c matches production [89] Extender.

XML: Parsing XML Documents¶

This module exposes a number of internal functions typically defined privately in XML parser implementations which make it easier to reuse concepts from XML in other modules. For example, the IsNameStartChar() tells you if a character matches the production for NameStartChar in the XML standard.

class pyslet.xml20081126.parser.XMLParser(entity)¶

Returns an XMLParser object constructed from the XMLEntity to parse.

XMLParser objects are used to parse entities for the constructs defined by the numbered productions in the XML specification.

XMLParser has a number of optional attributes, all of which default to False. Attributes with names started ‘check’ increase the strictness of the parser. All other parser flags, if set to True, will not result in a conforming XML processor.

DocumentClassTable = {}¶

A dictionary mapping doctype parameters onto class objects.

For more information about how this is used see GetDocumentClass() and RegisterDocumentClass().

RefModeNone = 0¶: Default constant used for setting refMode

RefModeInContent = 1¶: Treat references as per “in Content” rules

RefModeInEntityValue = 4¶: Treat references as per “in EntityValue” rules

RefModeInDTD = 5¶: Treat references as per “in DTD” rules

PredefinedEntities = {'amp': '&', 'lt': '<', 'gt': '>', 'apos': "'", 'quot': '"'}¶: A mapping from the names of the predefined entities (lt, gt, amp, apos, quot) to their replacement characters.

checkValidity = None¶

checks XML validity constraints

If checkValidity is True, and all other options are left at their default (False) setting then the parser will behave as a validating XML parser.

valid = None¶: Flag indicating if the document is valid, only set if checkValidity is True.

nonFatalErrors = None¶: A list of non-fatal errors discovered during parsing, only populated if checkValidity is True.

checkCompatibility = None¶: checks XML compatibility constraints; will cause checkValidity to be set to True when parsing.

checkAllErrors = None¶: checks all constraints; will cause checkValidity and checkCompatibility to be set to True when parsing.

raiseValidityErrors = None¶: treats validity errors as fatal errors

unicodeCompatibility = None¶: See http://www.w3.org/TR/unicode-xml/

sgmlNamecaseGeneral = None¶: option that simulates SGML’s NAMECASE GENERAL YES

sgmlNamecaseEntity = None¶: option that simulates SGML’s NAMECASE ENTITY YES

sgmlOmittag = None¶: option that simulates SGML’s OMITTAG YES

sgmlShorttag = None¶: option that simulates SGML’s SHORTTAG YES

sgmlContent = None¶

This option simulates some aspects of SGML content handling based on class attributes of the element being parsed.

Element classes with XMLCONTENT=:py:data:XMLEmpty are treated as elements declared EMPTY, these elements are treated as if they were introduced with an empty element tag even if they weren’t, as per SGML’s rules. Note that this SGML feature “has nothing to do with markup minimization” (i.e., sgmlOmittag.)

refMode = None¶

The current parser mode for interpreting references.

XML documents can contain five different types of reference: parameter entity, internal general entity, external parsed entity, (external) unparsed entity and character entity.

The rules for interpreting these references vary depending on the current mode of the parser, for example, in content a reference to an internal entity is replaced, but in the definition of an entity value it is not. This means that the behaviour of the ParseReference() method will differ depending on the mode.

The parser takes care of setting the mode automatically but if you wish to use some of the parsing methods in isolation to parse fragments of XML documents, then you will need to set the refMode directly using one of the RefMode* family of constants defined above.

entity = None¶: The current entity being parsed

declaration = None¶: The declaration parsed or None.

dtd = None¶

The documnet type declaration of the document being parsed.

This member is initialised to None as well-formed XML documents are not required to have an associated dtd.

doc = None¶: The document being parsed.

docEntity = None¶: The document entity.

element = None¶: The current element being parsed.

elementType = None¶: The element type of the current element.

NextChar()¶

Moves to the next character in the stream.

The current character can always be read from the_char. If there are no characters left in the current entity then entities are popped from an internal entity stack automatically.

PushEntity(entity)¶

Starts parsing entity

the_char is set to the current character in the entity’s stream. The current entity is pushed onto an internal stack and will be resumed when this entity has been parsed completely.

Note that in the degenerate case where the entity being pushed is empty (or is already positioned at the end of the file) then PushEntity does nothing.

CheckEncoding(entity, declaredEncoding)¶: Checks the entity against the declared encoding (if any) and the rules on entity encodings.

GetExternalEntity()¶

Returns the external entity currently being parsed.

If no external entity is being parsed then None is returned.

Standalone()¶

True if the document being parsed should be treated as standalone.

A document may be declared standalone or it may effectively be standalone due to the absence of a DTD, or the absence of an external DTD subset and parameter entity references.

DeclaredStandalone()¶: True if the current document was declared standalone.

WellFormednessError(msg='well-formedness error', errorClass=<class 'pyslet.xml20081126.structures.XMLWellFormedError'>)¶

Raises an XMLWellFormedError error.

Called by the parsing methods whenever a well-formedness constraint is violated. The method takes an optional message string, msg and an optional error class which must be a class object derived from py:class:XMLWellFormednessError.

The method raises an instance of errorClass and does not return. This method can be overridden by derived parsers to implement more sophisticated error logging.

ValidityError(msg='validity error', error=<class 'pyslet.xml20081126.structures.XMLValidityError'>)¶

Called when the parser encounters a validity error.

The method takes an optional message string, msg and an optional error class or instance which must be a (class) object derived from py:class:XMLValidityError.

The behaviour varies depending on the setting of the checkValidity and raiseValidityErrors options. The default (both False) causes validity errors to be ignored. When checking validity an error message is logged to nonFatalErrors and valid is set to False. Furthermore, if raiseValidityErrors is True error is raised (or a new instance of error is raised) and parsing terminates.

This method can be overridden by derived parsers to implement more sophisticated error logging.

CompatibilityError(msg='compatibility error')¶

Called when the parser encounters a compatibility error.

The method takes an optional message string, msg.

The behaviour varies depending on the setting of the checkCompatibility flag. The default (False) causes compatibility errors to be ignored. When checking compatibility an error message is logged to nonFatalErrors.

This method can be overridden by derived parsers to implement more sophisticated error logging.

ProcessingError(msg='Processing error')¶

Called when the parser encounters a general processing error.

The method takes an optional message string, msg and an optional error class or instance which must be a (class) object derived from py:class:XMLProcessingError.

The behaviour varies depending on the setting of the checkAllErrors flag. The default (False) causes processing errors to be ignored. When checking all errors an error message is logged to nonFatalErrors.

This method can be overridden by derived parsers to implement more sophisticated error logging.

ParseLiteral(match)¶

Parses a literal string, passed in match.

Returns True if match is successfully parsed and False otherwise. There is no partial matching, if match is not found then the parser is left in its original position.

ParseRequiredLiteral(match, production='Literal String')¶

Parses a required literal string raising a wellformed error if not matched.

production is an optional string describing the context in which the literal was expected.

ParseDecimalDigits()¶: Parses a, possibly empty, string of decimal digits matching [0-9]*.

ParseRequiredDecimalDigits(production='Digits')¶

Parses a required sring of decimal digits matching [0-9]+.

production is an optional string describing the context in which the digits were expected.

ParseHexDigits()¶: Parses a, possibly empty, string of hexadecimal digits matching [0-9a-fA-F].

ParseRequiredHexDigits(production='Hex Digits')¶

Parses a required sring of hexadecimal digits matching [0-9a-fA-F].

production is an optional string describing the context in which the hexadecimal digits were expected.

ParseQuote(q=None)¶

Parses the quote character, q, or one of “’” or ‘”’ if q is None.

Returns the character parsed or raises a well formed error.

ParseDocument(doc=None)¶

[1] document: parses an Document.

doc is the Document instance that will be parsed. The declaration, dtd and elements are added to this document. If doc is None then a new instance is created using GetDocumentClass() to identify the correct class to use to represent the document based on information in the prolog or, if the prolog lacks a declaration, the root element.

This method returns the document that was parsed, an instance of Document.

GetDocumentClass(dtd)¶

Returns a class object derived from Document suitable for representing a document with the given document type declaration.

In cases where no doctype declaration is made a dummy declaration is created based on the name of the root element. For example, if the root element is called “database” then the dtd is treated as if it was declared as follows:

<!DOCTYPE database>

This default implementation uses the following three pieces of information to locate class registered with RegisterDocumentClass(). The PublicID, SystemID and the name of the root element. If an exact match is not found then wildcard matches are attempted, ignoring the SystemID, PublicID and finally the root element in turn. If a document class still cannot be found then wildcard matches are tried matching only the PublicID, SystemID and root element in turn.

If no document class cab be found, Document is returned.

IsS()¶

By default just calls the module level IsS()

In Unicode compatibility mode the function maps the unicode white space characters at code points 2028 and 2029 to line feed and space respectively.

ParseS()¶

[3] S: Parses white space from the stream matching the production for S.

If there is no white space at the current position then an empty string is returned.

The productions in the specification do not make explicit mention of parameter entity references, they are covered by the general statement that “Parameter entity references are recognized anwhere in the DTD...” In practice, this means that while parsing the DTD, anywhere that an S is permitted a parameter entity reference may also be recognized. This method implements this behaviour, recognizing parameter entity references within S when refMode is RefModeInDTD.

ParseRequiredS(production='[3] S')¶

[3] S: Parses required white space from the stream.

If there is no white space then a well-formedness error is raised. production is an optional string describing the context in which the space was expected.

ParseName()¶

[5] Name: parses a Name

The name is returned as a unicode string. If no Name can be parsed then None is returned.

ParseRequiredName(production='Name')¶

[5] Name: Parses a required Name.

If no name can be parsed then a well-formed error is raised.

ParseNames()¶

[6] Names: parses a list of Names.

This method returns a tuple of unicode strings. If no names can be parsed then None is returned.

ParseNmtoken()¶

[7] Nmtoken: parses a single Nmtoken.

If no Nmtoken can be parsed then None is returned.

ParseNmtokens()¶

[8] Nmtokens: parses a list of Nmtokens.

This method returns a tuple of unicode strings. If no tokens can be parsed then None is returned.

ParseEntityValue()¶

[9] EntityValue: parses an EntityValue, returning it as a unicode string.

This method automatically expands other parameter entity references but does not expand general or character references.

ParseAttValue()¶

[10] AttValue: parses an attribute value.

The value is returned without the surrounding quotes and with any references expanded.

The behaviour of this method is affected significantly by the setting of the dontCheckWellFormedness flag. When set, attribute values can be parsed without surrounding quotes. For compatibility with SGML these values should match one of the formal value types (e.g., Name) but this is not enforced so values like width=100% can be parsed without error.

ParseSystemLiteral()¶

[11] SystemLiteral: Parses a literal value matching the production for SystemLiteral.

The value of the literal is returned as a string without the enclosing quotes.

ParsePubidLiteral()¶

[12] PubidLiteral: Parses a literal value matching the production for PubidLiteral.

The value of the literal is returned as a string without the enclosing quotes.

ParseCharData()¶

[14] CharData: parses a run of character data

The method adds the parsed data to the current element. In the default parsing mode it returns None.

When the parser option sgmlOmittag is selected the method returns any parsed character data that could not be added to the current element due to a model violation. Note that in this SGML-like mode any S is treated as being in the current element as the violation doesn’t occurr until the first non-S character (so any implied start tag is treated as being immediately prior to the first non-S).

parse_comment(gotLiteral=False)¶

[15] Comment: parses a comment.

If gotLiteral is True then the method assumes that the ‘<!–’ literal has already been parsed.

ParsePI(gotLiteral=False)¶

[16] PI: parses a processing instruction.

This method calls the Node.ProcessingInstruction() of the current element or of the document if no element has been parsed yet.

If gotLiteral is True the method assumes the ‘<?’ literal has already been parsed.

ParsePITarget()¶: [17] PITarget: parses a processing instruction target name

ParseCDSect(gotLiteral=False, cdEnd=u']]>')¶

[18] CDSect: parses a CDATA section.

This method adds any parsed data to the current element.

If gotLiteral is True then the method assumes the initial literal has already been parsed. (By default, CDStart.) The literal used to signify the end of the CDATA section can be overridden by passing an alternative literal in cdEnd.

ParseCDStart()¶: [19] CDStart: parses the literal that starts a CDATA section.

ParseCData(cdEnd=']]>')¶

[20] CData: parses a run of CData up to but not including cdEnd.

This method adds any parsed data to the current element.

ParseCDEnd()¶: [21] CDEnd: parses the end of a CDATA section.

ParseProlog()¶: [22] prolog: parses the document prolog, including the XML declaration and dtd.

ParseXMLDecl(gotLiteral=False)¶

[23] XMLDecl: parses an XML declaration.

This method returns an XMLDeclaration instance. Also, if an encoding is given in the declaration then the method changes the encoding of the current entity to match. For more information see ChangeEncoding().

If gotLiteral is True the initial literal ‘<?xml’ is assumed to have already been parsed.

ParseVersionInfo(gotLiteral=False)¶

[24] VersionInfo: parses XML version number.

The version number is returned as a string. If gotLiteral is True then it is assumed that the preceding white space and ‘version’ literal have already been parsed.

ParseEq(production='[25] Eq')¶: [25] Eq: parses an equal sign, optionally surrounded by white space

ParseVersionNum()¶: [26] VersionNum: parses the XML version number, returns it as a string.

ParseMisc()¶

[27] Misc: parses multiple Misc items.

This method parses everything that matches the production Misc*

ParseDoctypedecl(gotLiteral=False)¶

[28] doctypedecl: parses a doctype declaration.

This method creates a new instance of XMLDTD and assigns it to dtd, it also returns this instance as the result.

If gotLiteral is True the method assumes that the initial literal ‘<!DOCTYPE’ has already been parsed.

ParseDeclSep()¶: [28a] DeclSep: parses a declaration separator.

ParseIntSubset()¶: [28b] intSubset: parses an internal subset.

ParseMarkupDecl(gotLiteral=False)¶

[29] markupDecl: parses a markup declaration.

Returns True if a markupDecl was found, False otherwise.

ParseExtSubset()¶: [30] extSubset: parses an external subset

ParseExtSubsetDecl()¶: [31] extSubsetDecl: parses declarations in the external subset.

CheckPEBetweenDeclarations(checkEntity)¶

[31] extSubsetDecl: checks the well-formedness constraint on use of PEs between declarations.

checkEntity is the entity we should still be in!

ParseSDDecl(gotLiteral=False)¶

[32] SDDecl: parses a standalone declaration

Returns True if the document should be treated as standalone; False otherwise.

ParseElement()¶

[39] element: parses an element, including its content.

The class used to represent the element is determined by calling the GetElementClass() method of the current document. If there is no document yet then a new document is created automatically (see ParseDocument() for more information).

The element is added as a child of the current element using Node.ChildElement().

The method returns:

True: indicates that an element was parsed normally
False: indicates that the element is not allowed in this context

The second case only occurs when the sgmlOmittag option is in use and it indicates that the content of the enclosing element has ended. The Tag is buffered so that it can be reparsed when the stack of nested ParseContent() and ParseElement() calls is unwound to the point where it is allowed by the context.

CheckAttributes(name, attrs)¶

Checks attrs against the declarations for element name.

This method will add any omitted defaults to the attribute list. Also, checking the validity of the attributes may result in values being further normalized as per the rules for collapsing spaces in tokenized values.

MatchXMLName(element, name)¶

Tests if name is a possible name for this element.

This method is used by the parser to determine if an end tag is the end tag of this element. It is provided a separate method to allow it to be overridden by derived parsers

CheckExpectedParticle(name)¶

Tests if <name> fits with the cursor and raises a validity error if not.

An empty string for name indicates the enclosing end tag was found.

The method updates the current cursor as appropriate.

GetSTagClass(name, attrs=None)¶

[40] STag: returns information suitable for starting element name with attributes attrs in the current context

If there is no Document instance yet this method assumes that it is being called for the root element and selects an appropriate class based on the contents of the prolog and/or name.

When using the sgmlOmittag option name may be None indicating that the method should return information about the element implied by PCDATA in the current context (only called when an attempt to add data to the current context has already failed).

The result is a triple of:

elementClass: the element class that this STag must introduce or None if this STag does not belong (directly or indirectly) in the current context
elementName: the name of the element (to pass to ChildElement) or None to use the default
buffFlag: True indicates an omitted tag and that the triggering STag (i.e., the STag with name name) should be buffered.

ParseSTag()¶

[40] STag, [44] EmptyElemTag: parses a start tag or an empty element tag.

This method returns a triple of name, attrs, emptyFlag where:

name is the name of the element parsed.
attrs is a dictionary of attribute values keyed by attribute name
emptyFlag is a boolean; True indicates that the tag was an empty element tag.

ParseAttribute()¶

[41] Attribute: parses an attribute

Returns name, value where:

name is the name of the attribute or None if sgmlShorttag is True and a short form attribute value was supplied.
value is the attribute value.

If dontCheckWellFormedness the parser uses a very generous form of parsing attribute values to accomodate common syntax errors.

ParseETag(gotLiteral=False)¶

[42] ETag: parses an end tag

If gotLiteral is True then the method assumes the initial ‘</’ literal has been parsed alread.

The method returns the name of the end element parsed.

ParseContent()¶

[43] content: parses the content of an element.

The method returns:

True: indicates that the content was parsed normally
False: indicates that the content contained data or markup not allowed in this context

The second case only occurs when the sgmlOmittag option is in use and it indicates that the enclosing element has ended (i.e., the element’s ETag has been omitted). See py:meth:ParseElement for more information.

HandleData(data, cdata=False)¶

[43] content: handles character data in content.

When validating, the data is checked to see if it is optional white space. However, if cdata is True the data is treated as character data (even if it matches the production for S).

UnhandledData(data)¶

[43] content: manages unhandled data in content.

This method is only called when the sgmlOmittag option is in use. It processes data that occurs in a context where data is not allowed.

It returns a boolean result:

True: the data was consumed by a sub-element (with an omitted start tag)
False: the data has been buffered and indicates the end of the current content (an omitted end tag).

ParseEmptyElemTag()¶

[44] EmptyElemTag: there is no method for parsing empty element tags alone.

This method raises NotImplementedError. Instead, you should call ParseSTag() and examine the result. If it returns False then an empty element was parsed.

ParseElementDecl(gotLiteral=False)¶

[45] elementdecl: parses an element declaration

If gotLiteral is True the method assumes that the ‘<!ELEMENT’ literal has already been parsed.

ParseContentSpec(eType)¶: [46] contentspec: parses the content specification for an element type

ParseChildren(gotLiteral=False, groupEntity=None)¶

[47] children: parses an element content model comprising children.

If gotLiteral is True the method assumes that the initial ‘(‘ literal has already been parsed, including any following white space.

The method returns an instance of XMLContentParticle.

ParseCP()¶: [48] cp: parses a content particle

ParseChoice(firstChild=None, groupEntity=None)¶

[49] choice: parses a sequence of content particles.

firstChild is an optional XMLContentParticle instance. If present the method assumes that the first particle and any following white space has already been parsed. If firstChild is given then groupEntity must be the entity in which the opening ‘(‘ was parsed which started the choice group.

ParseSeq(firstChild=None, groupEntity=None)¶

[50] seq: parses a sequence of content particles.

firstChild is an optional XMLContentParticle instance. If present the method assumes that the first particle and any following white space has already been parsed.

ParseMixed(gotLiteral=False, groupEntity=None)¶

[51] Mixed: parses a mixed content type.

If gotLiteral is True the method assumes that the #PCDATA literal has already been parsed. In this case, groupEntity must be set to the entity which contained the opening ‘(‘ literal.

Returns an instance of XMLChoiceList with occurrence ZeroOrMore representing the list of elements that may appear in the mixed content model. If the mixed model contains #PCDATA only then the choice list will be empty.

ParseAttlistDecl(gotLiteral=False)¶

[52] AttlistDecl: parses an attribute list definition.

If gotLiteral is True the method assumes that the ‘<!ATTLIST’ literal has already been parsed.

ParseAttDef(gotS=False)¶

[53] AttDef: parses an attribute definition.

If gotS is True the method assumes that the leading S has already been parsed.

Returns an instance of XMLAttributeDefinition.

ParseAttType(a)¶

[54] AttType: parses an attribute type.

a must be an XMLAttributeDefinition instance. This method sets the type and values fields of a.

Note that, to avoid unnecessary look ahead, this method does not call ParseStringType() or ParseEnumeratedType().

ParseStringType(a)¶

[55] StringType: parses an attribute’s string type.

This method is provided for completeness. It is not called during normal parsing operations.

a must be an XMLAttributeDefinition instance. This method sets the type and values fields of a.

ParseTokenizedType(a)¶

[56] TokenizedType: parses an attribute’s tokenized type.

a must be an XMLAttributeDefinition instance. This method sets the type and values fields of a.

ParseEnumeratedType(a)¶

[57] EnumeratedType: parses an attribute’s enumerated type.

This method is provided for completeness. It is not called during normal parsing operations.

a must be an XMLAttributeDefinition instance. This method sets the type and values fields of a.

ParseNotationType(gotLiteral=False)¶

[58] NotationType: parses a notation type.

If gotLiteral is True the method assumes that the leading ‘NOTATION’ literal has already been parsed.

Returns a list of strings representing the names of the declared notations being referred to.

ParseEnumeration()¶

[59] Enumeration: parses an enumeration.

Returns a dictionary of strings representing the tokens in the enumeration.

ParseDefaultDecl(a)¶

[60] DefaultDecl: parses an attribute’s default declaration.

a must be an XMLAttributeDefinition instance. This method sets the defaultValue fields of a.

ParseConditionalSect(gotLiteralEntity=None)¶

[61] conditionalSect: parses a conditional section.

If gotLiteralEntity is set to an XMLEntity object the method assumes that the initial literal ‘<![‘ has already been parsed from that entity.

ParseIncludeSect(gotLiteralEntity=None)¶

[62] includeSect: parses an included section.

If gotLiteralEntity is set to an XMLEntity object the method assumes that the production, up to and including the keyword ‘INCLUDE’ has already been parsed and that the opening ‘<![‘ literal was parsed from that entity.

ParseIgnoreSect(gotLiteralEntity=None)¶

[63] ignoreSect: parses an ignored section.

If gotLiteralEntity is set to an XMLEntity object the method assumes that the production, up to and including the keyword ‘IGNORE’ has already been parsed and that the opening ‘<![‘ literal was parsed from that entity.

ParseIgnoreSectContents()¶

[64] ignoreSectContents: parses the contents of an ignored section.

The method returns no data.

ParseIgnore()¶

[65] Ignore: parses a run of characters in an ignored section.

This method returns no data.

ParseCharRef(gotLiteral=False)¶

[66] CharRef: parses a character reference.

If gotLiteral is True the method assumes that the leading ‘&’ literal has already been parsed.

The method returns a unicode string containing the character referred to.

ParseReference()¶

[67] Reference: parses a reference.

This method returns any data parsed as a result of the reference. For a character reference this will be the character referred to. For a general entity the data returned will depend on the parsing context. For more information see ParseEntityRef().

ParseEntityRef(gotLiteral=False)¶

[68] EntityRef: parses a general entity reference.

If gotLiteral is True the method assumes that the leading ‘&’ literal has already been parsed.

This method returns any data parsed as a result of the reference. For example, if this method is called in a context where entity references are bypassed then the string returned will be the literal characters parsed, e.g., “&ref;”.

If the entity reference is parsed successfully in a context where Entity references are recognized, the reference is looked up according to the rules for validating and non-validating parsers and, if required by the parsing mode, the entity is opened and pushed onto the parser so that parsing continues with the first character of the entity’s replacement text.

A special case is made for the predefined entities. When parsed in a context where entity references are recognized these entities are expanded immediately and the resulting character returned. For example, the entity & returns the ‘&’ character instead of pushing an entity with replacement text ‘&’.

Inclusion of an unescaped & is common so when we are not checking well- formedness we treat ‘&’ not followed by a name as if it were ‘&’. Similarly we are generous about the missing ‘;’.

LookupPredefinedEntity(name)¶

Utility function used to look up pre-defined entities, e.g., “lt”

This method can be overridden by variant parsers to implement other pre-defined entity tables.

ParsePEReference(gotLiteral=False)¶

[69] PEReference: parses a parameter entity reference.

If gotLiteral is True the method assumes that the initial ‘%’ literal has already been parsed.

This method returns any data parsed as a result of the reference. Normally this will be an empty string because the method is typically called in contexts where PEReferences are recognized. However, if this method is called in a context where PEReferences are not recognized the returned string will be the literal characters parsed, e.g., “%ref;”

If the parameter entity reference is parsed successfully in a context where PEReferences are recognized, the reference is looked up according to the rules for validating and non-validating parsers and, if required by the parsing mode, the entity is opened and pushed onto the parser so that parsing continues with the first character of the entity’s replacement text.

ParseEntityDecl(gotLiteral=False)¶

[70] EntityDecl: parses an entity declaration.

Returns an instance of either XMLGeneralEntity or XMLParameterEntity depending on the type of entity parsed. If gotLiteral is True the method assumes that the leading ‘<!ENTITY’ literal has already been parsed.

ParseGEDecl(gotLiteral=False)¶

[71] GEDecl: parses a general entity declaration.

Returns an instance of XMLGeneralEntity. If gotLiteral is True the method assumes that the leading ‘<!ENTITY’ literal and the required S have already been parsed.

ParsePEDecl(gotLiteral=False)¶

[72] PEDecl: parses a parameter entity declaration.

Returns an instance of XMLParameterEntity. If gotLiteral is True the method assumes that the leading ‘<!ENTITY’ literal and the required S have already been parsed.

ParseEntityDef(ge)¶

[73] EntityDef: parses the definition of a general entity.

The general entity being parsed must be passed in ge. This method sets the definition and notation fields from the parsed entity definition.

ParsePEDef(pe)¶

[74] PEDef: parses a parameter entity definition.

The parameter entity being parsed must be passed in pe. This method sets the definition field from the parsed parameter entity definition.

ParseExternalID(allowPublicOnly=False)¶

[75] ExternalID: parses an external ID returning an XMLExternalID instance.

An external ID must have a SYSTEM literal, and may have a PUBLIC identifier. If allowPublicOnly is True then the method will also allow an external identifier with a PUBLIC identifier but no SYSTEM literal. In this mode the parser behaves as it would when parsing the production:

(ExternalID | PublicID) S?

ResolveExternalID(externalID, entity=None)¶

[75] ExternalID: resolves an external ID, returning a URI reference.

Returns an instance of pyslet.rfc2396.URI or None if the external ID cannot be resolved.

entity can be used to force the resolution of relative URI to be relative to the base of the given entity. If it is None then the currently open external entity (where available) is used instead.

The default implementation simply calls GetLocation() with the entities base URL and ignores the public ID. Derived parsers may recognize public identifiers and resolve accordingly.

ParseNDataDecl(gotLiteral=False)¶

[76] NDataDecl: parses an unparsed entity notation reference.

Returns the name of the notation used by the unparsed entity as a string without the preceding ‘NDATA’ literal.

ParseTextDecl(gotLiteral=False)¶

[77] TextDecl: parses a text declataion.

Returns an XMLTextDeclaration instance.

ParseEncodingDecl(gotLiteral=False)¶

[80] EncodingDecl: parses an encoding declaration

Returns the declaration name without the enclosing quotes. If gotLiteral is True then the method assumes that the literal ‘encoding’ has already been parsed.

ParseEncName()¶

[81] EncName: parses an encoding declaration name

Returns the encoding name as a string or None if no valid encoding name start character was found.

ParseNotationDecl(gotLiteral=False)¶

[82] NotationDecl: Parses a notation declaration matching production NotationDecl

This method assumes that the literal ‘<!NOTATION’ has already been parsed. It declares the notation in the dtd.

ParsePublicID()¶

[83] PublicID: Parses a literal matching the production for PublicID.

The literal string is returned without the PUBLIC prefix or the enclosing quotes.

XML Schema Datatypes¶

This module implements some useful concepts drawn from http://www.w3.org/TR/xmlschema-2/

One of the main purposes of this module is to provide classes and functions for converting data between python-native representations of the value-spaces defined by this specification and the lexical representations defined in the specification.

The result is typically a pair of DecodeX and EncodeX functions that are used to define custom attribute handling in classes that are derived from xml20081126.structures.XMLElement. For example:

import xsdatatypes20041028 as xsi

class MyElement(XMLElement):
        XMLATTR_flag=('flag',xsi.DecodeBoolean,xsi.EncodeBoolean)

Primitive Datatypes¶

pyslet.xsdatatypes20041028.DecodeBoolean(src)¶

Decodes a boolean value from src.

Returns python constants True or False. As a convenience, if src is None then None is returned.

pyslet.xsdatatypes20041028.EncodeBoolean(src)¶

Encodes a boolean value using the canonical lexical representation.

src can be anything that can be resolved to a boolean except None, which raises ValueError.

pyslet.xsdatatypes20041028.DecodeDecimal(src)¶

Decodes a decimal value from a string returning a python float value.

If string is not a valid lexical representation of a decimal value then ValueError is raised.

pyslet.xsdatatypes20041028.EncodeDecimal(value, digits=None, stripZeros=True)¶

Encodes a decimal value into a string.

You can control the maximum number of digits after the decimal point using digits which must be greater than 0 - None indicates no maximum. This function always returns the canonical representation which means that it will strip trailing zeros in the fractional part. To override this behaviour and return exactly digits decimal places set stripZeros to False.

pyslet.xsdatatypes20041028.DecodeFloat(src)¶

Decodes a float value from a string returning a python float.

The precision of the python float varies depending on the implementation. It typically exceeds the precision of the XML schema float. We make no attempt to reduce the precision to that of schema’s float except that we return 0.0 or -0.0 for any value that is smaller than the smallest possible float defined in the specification. (Note that XML schema’s float canonicalizes the representation of zero to remove this subtle distinction but it can be useful to preserve it for numerical operations. Likewise, if we are given a representation that is larger than any valid float we return one of the special float values INF or -INF as appropriate.

pyslet.xsdatatypes20041028.EncodeFloat(value)¶

Encodes a python float value as a string.

To reduce the chances of our output being rejected by external applications that are strictly bound to a 32-bit float representation we ensure that we don’t output values that exceed the bounds of float defined by XML schema.

Therefore, we convert values that are too large to INF and values that are too small to 0.0E0.

pyslet.xsdatatypes20041028.DecodeDouble(src)¶

Decodes a double value from a string returning a python float.

The precision of the python float varies depending on the implementation. It may even exceed the precision of the XML schema double. The current implementation ignores this distinction.

pyslet.xsdatatypes20041028.EncodeDouble(value, digits=None, stripZeros=True)¶

Encodes a double value returning a unicode string.

digits controls the number of digits after the decimal point in the mantissa, None indicates no maximum and the precision of python’s float is used to determine the appropriate number. You may pass the value 0 - in which case no digits are given after the point and the point itself is omitted, but such values are not in their canonical form.

stripZeros determines whether or not trailing zeros are removed, if False then exactly digits digits will be displayed after the point. By default zeros are stripped (except there is always one zero left after the decimal point).

pyslet.xsdatatypes20041028.DecodeDateTime(src)¶: Returns an pyslet.iso8601.TimePoint instance.

pyslet.xsdatatypes20041028.EncodeDateTime(value)¶: Returns the canonical lexical representation of a pyslet.iso8601.TimePoint instance.

Derived Datatypes¶

pyslet.xsdatatypes20041028.DecodeName(src)¶: Decodes a name from a string. Returns the same string or raised ValueError.

pyslet.xsdatatypes20041028.EncodeName(src)¶: A convenience function, returns src unchanged.

pyslet.xsdatatypes20041028.DecodeInteger(src)¶

Decodes an integer value from a string returning an Integer or Long value.

If string is not a valid lexical representation of an integer then ValueError is raised.

pyslet.xsdatatypes20041028.EncodeInteger(value)¶: Encodes an integer value using the canonical lexical representation.

Constraining Facets¶

Enumeration¶

class pyslet.xsdatatypes20041028.Enumeration¶

An abstract class designed to make generating enumeration types easier. The class is not designed to be instantiated but to act as a method of defining constants to represent the values of an enumeration.

The basic usage of this class is to derive a class from it with a single class member called ‘decode’ which is a mapping from canonical strings to simple integers. You then call the function MakeEnumeration() to complete the declaration, after which, you can use the enumeration as if you had defined the constants as class members and call any of the following class methods to convert enumeration values to and from their string representations.

classmethod DecodeValue(src)¶

Decodes a string returning a value in this enumeration.

If no legal value can be decoded then ValueError is raised.

classmethod DecodeLowerValue(src)¶

Decodes a string, converting it to lower case first.

Returns a value in this enumeration. If no legal value can be decoded then ValueError is raised.

classmethod DecodeUpperValue(src)¶

Decodes a string, converting it to upper case first.

Returns a value in this enumeration. If no legal value can be decoded then ValueError is raised.

classmethod DecodeTitleValue(src)¶

Decodes a string, converting it to title case first.

Returns a value in this enumeration. If no legal value can be decoded then ValueError is raised.

classmethod DecodeValueList(decoder, src)¶

Decodes a space-separated string of values using decoder which must be one of the Decode*Value methods of the enumeration. The result is an ordered list of values (possibly containing duplicates).

Example usage:

fruit.DecodeValueList(fruit.DecodeLowerValue,"apples oranges, pears")
# returns [ fruit.apples, fruit.oranges, fruit.pears ]

classmethod DecodeValueDict(decoder, src)¶

Decodes a space-separated string of values using decoder which must be one of the Decode*Value methods of the enumeration. The result is a dictionary mapping the values found as keys onto the strings used to represent them. Duplicates are mapped to the first occurrence of the encoded value.

Example usage:

fruit.DecodeValueDict(fruit.DecodeLowerValue,"Apples oranges PEARS")
# returns...
{ fruit.apples:'Apples', fruit.oranges:'oranges', fruit.pears:'PEARS' }

classmethod EncodeValue(value)¶

Encodes one of the enumeration constants returning a string.

If value is None then the encoded default value is returned (if defined) or None.

classmethod EncodeValueList(valueList)¶

Encodes a list of enumeration constants returning a space-separated string.

If valueList is empty then an empty string is returned.

classmethod EncodeValueDict(valueDict, sortKeys=True)¶

Encodes a dictionary of enumeration constants returning a space-separated string.

If valueDict is empty then an empty string is returned. Note that the canonical representation of each value is used. Extending the example given in DecodeValueDict():

fruit.EncodeValueDict(fruit.DecodeValueDict(fruit.DecodeLowerValue,
        "Apples oranges PEARS"))
# returns...
"apples oranges pears"

The order of the encoded values in the string is determined by the sort order of the enumeration constants. This ensures that equivalent dictionaries are always encoded to equivalent strings. In the above example:

fruit.apples < fruit.oranges and fruit.oranges < fruit.pears

If you have large lists then you can skip the sorting step by passing False for sortKeys to improve performance at the expense of a predictable encoding.

pyslet.xsdatatypes20041028.MakeEnumeration(e, defaultValue=None)¶

Adds convenience attributes to the class ‘e’

This function assumes that e has an attribute ‘decode’ that is a dictionary which maps strings onto enumeration values. This function creates the reverse mapping called ‘encode’ and also defines constant attribute values that are equivalent to the keys of decode and can be used in code in the form e.key.

If defaultValue is not None then it must be on of the strings in the decode dictionary. It is then used to set the DEFAULT value.

pyslet.xsdatatypes20041028.MakeEnumerationAliases(e, aliases)¶

Adds aliases from a dictionary, declaring additional convenience attributes.

This function assumes that MakeEnumeration() has already been used to complete the declaration of the enumeration. The aliases are added to the decode dictionary but, for obvious reasons, not to the encode dictionary.

pyslet.xsdatatypes20041028.MakeLowerAliases(e)¶

Adds aliases by converting all keys to lower case.

Assumes that MakeEnumeration() has already been used to complete the declaration of the enumeration. You must call this function to complete the declaration before relying on calls to Enumeration.DecodeLowerValue().

pyslet.xsdatatypes20041028.MakeEnumeration(e, defaultValue=None)

Adds convenience attributes to the class ‘e’

This function assumes that e has an attribute ‘decode’ that is a dictionary which maps strings onto enumeration values. This function creates the reverse mapping called ‘encode’ and also defines constant attribute values that are equivalent to the keys of decode and can be used in code in the form e.key.

If defaultValue is not None then it must be on of the strings in the decode dictionary. It is then used to set the DEFAULT value.

pyslet.xsdatatypes20041028.MakeEnumeration(e, defaultValue=None)

Adds convenience attributes to the class ‘e’

This function assumes that e has an attribute ‘decode’ that is a dictionary which maps strings onto enumeration values. This function creates the reverse mapping called ‘encode’ and also defines constant attribute values that are equivalent to the keys of decode and can be used in code in the form e.key.

If defaultValue is not None then it must be on of the strings in the decode dictionary. It is then used to set the DEFAULT value.

WhiteSpace¶

pyslet.xsdatatypes20041028.WhiteSpaceReplace(value)¶: Replaces tab, line feed and carriage return with space.

pyslet.xsdatatypes20041028.WhiteSpaceCollapse(value)¶: Replaces all runs of white space with a single space. Also removes leading and trailing white space.

Regular Expressions¶

Appendix F of the XML Schema datatypes specification defines a regular expression language. This language differs from the native Python regular expression language but it is close enough to enable us to define a wrapper class which parses schema regular expressions and converts them to equivalent python regular expressions.

class pyslet.xsdatatypes20041028.RegularExpression(src)¶

Models a regular expression as defined by XML schema.

Regular expressions are constructed from unicode source strings. Internally they are parsed and converted to Python regular expressions to speed up matching. Warning: because the XML schema expression language contains concepts not supported by Python the python regular expression may not be very readable.

src = None¶: the original source string

match(target)¶: A convenience function, returns True if the expression matches target.

For completeness we also document the parser we use to do the conversion, it draws heavily on the pyslet.unicode5.CharClass concept.

class pyslet.xsdatatypes20041028.RegularExpressionParser(source)¶

Bases: pyslet.unicode5.BasicParser

A custom parser for XML schema regular expressions.

The parser is initialised from a source string, the string to be parsed.

ParseRegExp()¶: Returns a unicode string representing the regular expression.

ParseBranch()¶: Returns a unicode string representing this piece as a python regular expression.

ParseQuantifier()¶

Returns a tuple of n,m.

Symbolic values are expanded to the appropriate pair. The second value may be None indicating unbounded.

ParseQuantity()¶

Returns a tuple of n,m even if an exact quantity is given.

In other words, the exact quantity ‘n’ returns n,n. The second value may be None indicated unbounded.

ParseQuantExact()¶: Returns an integer.

ParseAtom()¶: Returns a unicode string representing this atom as a python regular expression.

IsChar(c=None)¶: The definition of this function is designed to be conservative with respect to the specification, which is clearly in error around production [10] as the prose and the BNF do not match. It appears that | was intended to be excluded in the prose but has been omitted, the reverse being true for the curly-brackets.

ParseCharClass()¶: Returns a CharClass instance representing this class.

ParseCharClassExpr()¶: Returns a CharClass instance representing this class expression.

ParseCharGroup()¶: Returns a CharClass representing this group. This method also handles the case of a class subtraction directly to reduce the need for look-ahead. If you specifically want to parse a subtraction you can do this with ParseCharClassSub().

ParsePosCharGroup()¶: Returns a CharClass representing a positive range

ParseNegCharGroup()¶: Returns a CharClass representing this range.

ParseCharClassSub()¶: Returns a CharClass representing this range - this method is not normally used by the parser as in present for completeness. See ParseCharGroup().

ParseCharRange()¶: Returns a CharClass representing this range.

ParseSERange()¶: Returns a CharClass representing this range.

ParseCharOrEsc()¶: Returns a single unicode character.

ParseCharClassEsc()¶: Returns a CharClass instance representing one of the escape sequences.

ParseSingleCharEsc()¶: Returns a single unicode character parsed from a single char escape.

ParseCatEsc()¶: Returns a CharClass, parsing a category escape.

ParseComplEsc()¶: Returns a CharClass, parsing the complement of a category escape.

ParseCharProp()¶: Returns a CharClass, parsing an IsCategory or IsBlock.

ParseIsCategory()¶: Returns a CharClass corresponding to one of the character categories or raises an error.

ParseIsBlock()¶: Returns a CharClass corresponding to one of the Unicode blocks.

ParseMultiCharEsc()¶: Returns a CharClass corresponding to one of the multichar escapes, if parsed.

ParseWildcardEsc()¶: Returns a CharClass corresponding to the wildcard ‘.’ character if parsed.

HTML¶

This module defines functions and classes for working with HTML documents. The version of the standard implemented is, loosely speaking, the HTML 4.0.1 Specification: http://www.w3.org/TR/html401/

This module contains code that can help parse HTML documents into classes based on the basic xml20081126 XML module, acting as a gateway to XHTML.

This module also exposes a number of internal functions typically defined privately in HTML parser implementations which make it easier to reuse concepts from HTML in other modules. For example, the LengthType used for storing HTML lengths (which can be pixel or relative) is used extensively by imsqtiv2p1.

pyslet.html40_19991224.HTML40_PUBLICID¶: The public ID to use in the declaration of an HTML document

pyslet.html40_19991224.XHTML_NAMESPACE¶: The namespace to use in the delcaration of an XHTML document

(X)HTML Documents¶

This module contains an experimental class for working with HTML documents. At the time of writing the implementation is designed to provide just enough HTML parsing to support the use of HTML within other standards (such as Atom and QTI).

class pyslet.html40_19991224.XHTMLDocument(**args)¶

Bases: pyslet.xmlnames20091208.XMLNSDocument

Represents an HTML document.

Although HTML documents are not always represented using XML they can be, and therefore we base our implementation on the pyslet.xmlnames20091208.XMLNSDocument class - a namespace-aware variant of the basic pyslet.xml20081126.XMLDocument class.

classMap = {('http://www.w3.org/1999/xhtml', 'dl'): <class 'pyslet.html40_19991224.DL'>, ('http://www.w3.org/1999/xhtml', 'ins'): <class 'pyslet.html40_19991224.Ins'>, ('http://www.w3.org/1999/xhtml', 'optgroup'): <class 'pyslet.html40_19991224.OptGroup'>, ('http://www.w3.org/1999/xhtml', 'thead'): <class 'pyslet.html40_19991224.THead'>, ('http://www.w3.org/1999/xhtml', 'var'): <class 'pyslet.html40_19991224.Var'>, ('http://www.w3.org/1999/xhtml', 'h2'): <class 'pyslet.html40_19991224.H2'>, ('http://www.w3.org/1999/xhtml', 'frameset'): <class 'pyslet.html40_19991224.Frameset'>, ('http://www.w3.org/1999/xhtml', 'acronym'): <class 'pyslet.html40_19991224.Acronym'>, ('http://www.w3.org/1999/xhtml', 'br'): <class 'pyslet.html40_19991224.Br'>, ('http://www.w3.org/1999/xhtml', 'param'): <class 'pyslet.html40_19991224.Param'>, ('http://www.w3.org/1999/xhtml', 'input'): <class 'pyslet.html40_19991224.Input'>, ('http://www.w3.org/1999/xhtml', 'fieldset'): <class 'pyslet.html40_19991224.FieldSet'>, ('http://www.w3.org/1999/xhtml', 'basefont'): <class 'pyslet.html40_19991224.BaseFont'>, ('http://www.w3.org/1999/xhtml', 'u'): <class 'pyslet.html40_19991224.U'>, ('http://www.w3.org/1999/xhtml', 'strong'): <class 'pyslet.html40_19991224.Strong'>, ('http://www.w3.org/1999/xhtml', 'noscript'): <class 'pyslet.html40_19991224.NoScript'>, ('http://www.w3.org/1999/xhtml', 'small'): <class 'pyslet.html40_19991224.Small'>, ('http://www.w3.org/1999/xhtml', 'caption'): <class 'pyslet.html40_19991224.Caption'>, ('http://www.w3.org/1999/xhtml', 'sup'): <class 'pyslet.html40_19991224.Sup'>, ('http://www.w3.org/1999/xhtml', 'big'): <class 'pyslet.html40_19991224.Big'>, ('http://www.w3.org/1999/xhtml', 'em'): <class 'pyslet.html40_19991224.Em'>, ('http://www.w3.org/1999/xhtml', 'form'): <class 'pyslet.html40_19991224.Form'>, ('http://www.w3.org/1999/xhtml', 'meta'): <class 'pyslet.html40_19991224.Meta'>, ('http://www.w3.org/1999/xhtml', 'blockquote'): <class 'pyslet.html40_19991224.Blockquote'>, ('http://www.w3.org/1999/xhtml', 'a'): <class 'pyslet.html40_19991224.A'>, ('http://www.w3.org/1999/xhtml', 'strike'): <class 'pyslet.html40_19991224.Strike'>, ('http://www.w3.org/1999/xhtml', 'legend'): <class 'pyslet.html40_19991224.Legend'>, ('http://www.w3.org/1999/xhtml', 'tt'): <class 'pyslet.html40_19991224.TT'>, ('http://www.w3.org/1999/xhtml', 'h3'): <class 'pyslet.html40_19991224.H3'>, ('http://www.w3.org/1999/xhtml', 'area'): <class 'pyslet.html40_19991224.Area'>, ('http://www.w3.org/1999/xhtml', 'tfoot'): <class 'pyslet.html40_19991224.TFoot'>, ('http://www.w3.org/1999/xhtml', 'script'): <class 'pyslet.html40_19991224.Script'>, ('http://www.w3.org/1999/xhtml', 'center'): <class 'pyslet.html40_19991224.Center'>, ('http://www.w3.org/1999/xhtml', 'q'): <class 'pyslet.html40_19991224.Q'>, ('http://www.w3.org/1999/xhtml', 'cite'): <class 'pyslet.html40_19991224.Cite'>, ('http://www.w3.org/1999/xhtml', 'frame'): <class 'pyslet.html40_19991224.Frame'>, ('http://www.w3.org/1999/xhtml', 'address'): <class 'pyslet.html40_19991224.Address'>, ('http://www.w3.org/1999/xhtml', 'hr'): <class 'pyslet.html40_19991224.HR'>, ('http://www.w3.org/1999/xhtml', 'li'): <class 'pyslet.html40_19991224.LI'>, ('http://www.w3.org/1999/xhtml', 'map'): <class 'pyslet.html40_19991224.Map'>, ('http://www.w3.org/1999/xhtml', 'h4'): <class 'pyslet.html40_19991224.H4'>, ('http://www.w3.org/1999/xhtml', 'td'): <class 'pyslet.html40_19991224.TD'>, ('http://www.w3.org/1999/xhtml', 'table'): <class 'pyslet.html40_19991224.Table'>, ('http://www.w3.org/1999/xhtml', 'span'): <class 'pyslet.html40_19991224.Span'>, ('http://www.w3.org/1999/xhtml', 'ul'): <class 'pyslet.html40_19991224.UL'>, ('http://www.w3.org/1999/xhtml', 'head'): <class 'pyslet.html40_19991224.Head'>, ('http://www.w3.org/1999/xhtml', 'samp'): <class 'pyslet.html40_19991224.Samp'>, ('http://www.w3.org/1999/xhtml', 'tr'): <class 'pyslet.html40_19991224.TR'>, ('http://www.w3.org/1999/xhtml', 'sub'): <class 'pyslet.html40_19991224.Sub'>, ('http://www.w3.org/1999/xhtml', 's'): <class 'pyslet.html40_19991224.S'>, ('http://www.w3.org/1999/xhtml', 'select'): <class 'pyslet.html40_19991224.Select'>, ('http://www.w3.org/1999/xhtml', 'col'): <class 'pyslet.html40_19991224.Col'>, ('http://www.w3.org/1999/xhtml', 'dd'): <class 'pyslet.html40_19991224.DD'>, ('http://www.w3.org/1999/xhtml', 'iframe'): <class 'pyslet.html40_19991224.IFrame'>, ('http://www.w3.org/1999/xhtml', 'abbr'): <class 'pyslet.html40_19991224.Abbr'>, ('http://www.w3.org/1999/xhtml', 'font'): <class 'pyslet.html40_19991224.Font'>, ('http://www.w3.org/1999/xhtml', 'tbody'): <class 'pyslet.html40_19991224.TBody'>, ('http://www.w3.org/1999/xhtml', 'img'): <class 'pyslet.html40_19991224.Img'>, ('http://www.w3.org/1999/xhtml', 'object'): <class 'pyslet.html40_19991224.Object'>, ('http://www.w3.org/1999/xhtml', 'bdo'): <class 'pyslet.html40_19991224.BDO'>, ('http://www.w3.org/1999/xhtml', 'body'): <class 'pyslet.html40_19991224.Body'>, ('http://www.w3.org/1999/xhtml', 'dt'): <class 'pyslet.html40_19991224.DT'>, ('http://www.w3.org/1999/xhtml', 'base'): <class 'pyslet.html40_19991224.Base'>, ('http://www.w3.org/1999/xhtml', 'th'): <class 'pyslet.html40_19991224.TH'>, ('http://www.w3.org/1999/xhtml', 'label'): <class 'pyslet.html40_19991224.Label'>, ('http://www.w3.org/1999/xhtml', 'textarea'): <class 'pyslet.html40_19991224.TextArea'>, ('http://www.w3.org/1999/xhtml', 'dfn'): <class 'pyslet.html40_19991224.Dfn'>, ('http://www.w3.org/1999/xhtml', 'button'): <class 'pyslet.html40_19991224.Button'>, ('http://www.w3.org/1999/xhtml', 'ol'): <class 'pyslet.html40_19991224.OL'>, ('http://www.w3.org/1999/xhtml', 'h5'): <class 'pyslet.html40_19991224.H5'>, ('http://www.w3.org/1999/xhtml', 'link'): <class 'pyslet.html40_19991224.Link'>, ('http://www.w3.org/1999/xhtml', 'pre'): <class 'pyslet.html40_19991224.Pre'>, ('http://www.w3.org/1999/xhtml', 'colgroup'): <class 'pyslet.html40_19991224.ColGroup'>, ('http://www.w3.org/1999/xhtml', 'style'): <class 'pyslet.html40_19991224.Style'>, ('http://www.w3.org/1999/xhtml', 'div'): <class 'pyslet.html40_19991224.Div'>, ('http://www.w3.org/1999/xhtml', 'h6'): <class 'pyslet.html40_19991224.H6'>, ('http://www.w3.org/1999/xhtml', 'i'): <class 'pyslet.html40_19991224.I'>, ('http://www.w3.org/1999/xhtml', 'title'): <class 'pyslet.html40_19991224.Title'>, ('http://www.w3.org/1999/xhtml', 'code'): <class 'pyslet.html40_19991224.Code'>, ('http://www.w3.org/1999/xhtml', 'del'): <class 'pyslet.html40_19991224.Del'>, ('http://www.w3.org/1999/xhtml', 'kbd'): <class 'pyslet.html40_19991224.Kbd'>, ('http://www.w3.org/1999/xhtml', 'html'): <class 'pyslet.html40_19991224.HTML'>, ('http://www.w3.org/1999/xhtml', 'option'): <class 'pyslet.html40_19991224.Option'>, ('http://www.w3.org/1999/xhtml', 'p'): <class 'pyslet.html40_19991224.P'>, ('http://www.w3.org/1999/xhtml', 'h1'): <class 'pyslet.html40_19991224.H1'>, ('http://www.w3.org/1999/xhtml', 'b'): <class 'pyslet.html40_19991224.B'>}¶: Data member used to store a mapping from element names to the classes used to represent them. This mapping is initialized when the module is loaded.

XMLParser(entity)¶

We override the basic XML parser to use a custom parser that is intelligent about the use of omitted tags, elements defined to have CDATA content and other SGML-based variations. Note that if the document starts with an XML declaration then the normal XML parser is used instead.

You won’t normally need to call this method as it is invoked automatically when you call pyslet.xml20081126.XMLDocument.Read().

The result is always a proper element hierarchy rooted in an HTML node, even if no tags are present at all the parser will construct an HTML document containing a single Div element to hold the parsed text.

GetChildClass(stagClass)¶: Always returns HTML.

Basic Types¶

Length Values¶

Length values are used in many places in HTML, the most common being the width and height values on images. There are two ways of specifying a Length, a simple integer number of pixels or a percentage of some base length defined by the context (such as the width of the browser window).

class pyslet.html40_19991224.LengthType(value, valueType=None)¶

Bases: object

Represents the HTML Length:

<!ENTITY % Length "CDATA" -- nn for pixels or nn% for percentage length -->

value can be either an integer value, another LengthType instance or a

string.
if value is an integer then valueType can be used to select Pixel or

Percentage
if value is a string then it is parsed for the length as per the format

defined for length attributes in HTML.

By default values are assumed to be Pixel lengths but valueType can be used to force such a value to be a Percentage if desired.

Pixel = 0¶: data constant used to indicate pixel co-ordinates

Percentage = 1¶: data constant used to indicate relative (percentage) co-ordinates

type = None¶: type is one of the the LengthType constants: Pixel or Percentage

value = None¶: value is the integer value of the length

__nonzero__()¶: Length values are non-zero if they have a non-zero value (pixel or percentage).

__str__()¶: Formats the length as a string of form nn for pixels or nn% for percentage.

__unicode__()¶: Formats the length as a unicode string of form nn for pixels or nn% for percentage.

GetValue(dim=None)¶: Returns the value of the Length, dim is the size of the dimension used for interpreting percentage values. I.e., 100% will return dim.

Add(value)¶

Adds value to the length.

If value is another LengthType instance then its value is added to the value of this instances’ value only if the types match. If value is an integer it is assumed to be a value of pixel type - a mismatch raises ValueError.

__weakref__¶: list of weak references to the object (if defined)

Coordinate Values¶

Coordinate values are simple lists of Lengths. In most cases Pyslet doesn’t define special types for lists of basic types but coordinates are represented in attribute values using comma separation, not space-separation. As a result they require special processing in order to be decoded/encoded correctly from/to XML streams.

class pyslet.html40_19991224.Coords(values=None)¶

Represents HTML Coords values

<!ENTITY % Coords "CDATA" -- comma-separated list of lengths -->

Instances can be initialized from an existing list of LengthType, or a list of any object that can be used to construct a LengthType. It can also be constructed from a string formatted as per the HTML attribute definition.

The resulting object behaves like a list of LengthType instances, for example:

x=Coords("10, 50, 60%,75%")
len(x)==4
x[0].value==10
x[2].type==LengthType.Percentage
str(x[3])=="75%"
# items are also assignable...
x[1]="40%"
x[1].type==LengthType.Percentage
x[1].value==40

values = None¶: a list of LengthType values

__unicode__()¶: Formats the Coords as comma-separated unicode string of Length values.

__str__()¶: Formats the Coords as a comma-separated string of Length values.

TestRect(x, y, width, height)¶

Tests an x,y point against a rect with these coordinates.

HTML defines the rect co-ordinates as: left-x, top-y, right-x, bottom-y

TestCircle(x, y, width, height)¶

Tests an x,y point against a circle with these coordinates.

HTML defines a circle as: center-x, center-y, radius.

The specification adds the following note:

When the radius value is a percentage value, user agents should calculate the final radius value based on the associated object’s width and height. The radius should be the smaller value of the two.

TestPoly(x, y, width, height)¶

Tests an x,y point against a poly with these coordinates.

HTML defines a poly as: x1, y1, x2, y2, ..., xN, yN.

The specification adds the following note:

The first x and y coordinate pair and the last should be the same to close the polygon. When these coordinate values are not the same, user agents should infer an additional coordinate pair to close the polygon.

The algorithm used is the “Ray Casting” algorithm described here: http://en.wikipedia.org/wiki/Point_in_polygon

URIs¶

URIs are represented by instances of the underlying pyselt.rfc2396.URI class, these functions provide a simple wrapper around the functions defined in that module.

pyslet.html40_19991224.DecodeURI(src)¶

Decodes a URI from src:

<!ENTITY % URI "CDATA"  -- a Uniform Resource Identifier -->

Note that we adopt the algorithm recommended in Appendix B of the specification, which involves replacing non-ASCII characters with percent-encoded UTF-sequences.

For more information see psylet.rfc2396.EncodeUnicodeURI()

pyslet.html40_19991224.EncodeURI(uri)¶

Encoding a URI means just converting it into a string.

By definition, a URI will only result in ASCII characters that can be freely converted to Unicode by the default encoding. However, it does mean that this function doesn’t adhere to the principal of using the ASCII encoding only at the latest possible time.

Uniform Resource Identifiers (RFC2396)¶

This module defines functions and classes for working with URIs as defined by RFC2396: http://www.ietf.org/rfc/rfc2396.txt

In addition to parsing and formating URI from strings of octets, this module also supports computing and resolving relative URI. To do this we define two notaional operators.

The resolve operator:

U = B [*] R

calculates a new URI ‘U’ from a base URI ‘B’ and a relative URI ‘R’.

The relative operator:

U [/] B = R

calcualtes the relative URI ‘R’ formed by expressing ‘U’ relative to ‘B’.

Clearly the Relative operator defines the reverse of the resolve operator, however note that in some cases several different values of R can resolve to the same URL with a common base URI.

Creating URI Instances¶

pyslet.rfc2396.URIFactory¶: An instance of URIFactoryClass that can be used for creating URI instances.

class pyslet.rfc2396.URIFactoryClass¶

A factory class that contains methods for creating URI instances.

URI(octets)¶: Creates an instance of URI from a string of octets.

URLFromPathname(path)¶

Converts a local file path into a URI instance.

If the path is not absolute it is made absolute by resolving it relative to the current working directory before converting it to a URI.

Under Windows, the URL is constructed according to the recommendations on this blog post: http://blogs.msdn.com/b/ie/archive/2006/12/06/file-uris-in-windows.aspx So UNC paths are mapped to both the network location and path components of the resulting URI.

URLFromVirtualFilePath(path)¶: Converts a virtual file path into a URI instance.

Resolve(b, r)¶

Evaluates the resolve operator B [*] R, resolving R relative to B

The input parameters are converted to URI objects if necessary.

Relative(u, b)¶

Evaluates the relative operator U [/] B, returning U relative to B

The input parameters are converted to URI objects if necessary.

pyslet.rfc2396.EncodeUnicodeURI(uSrc)¶

Takes a unicode string that is supposed to be a URI and returns an octent string.

The encoding algorithm used is the same as the one adopted by HTML: utf-8 and then %-escape. This is not part of the RFC standard which only defines the behaviour for streams of octets.

URI¶

class pyslet.rfc2396.URI(octets)¶

Bases: object

Class to represent URI Reference.

octets = None¶: The octet string representing this URI.

fragment = None¶: The fragment string that was appended to the URI or None if no fragment was given.

scheme = None¶: The URI scheme, if present.

authority = None¶: The authority (e.g., host name) of a hierarchical URI

absPath = None¶: The absolute path of a hierarchical URI (None if the path is relative)

query = None¶: The optional query associated with a hierarchical URI.

schemeSpecificPart = None¶: The scheme specific part of the URI.

relPath = None¶: The relative path of a hierarchical URI (None if the path is absolute)

opaquePart = None¶: None if the URI is hierarchical, otherwise the same as schemeSpecificPart.

GetFileName()¶: Returns the file name associated with this resource or None if the URL scheme does not have the concept. By default the file name is extracted from the last component of the path. Note the subtle difference between returning None and returning an empty string (indicating that the URI represents a directory-like object).

GetCanonicalRoot()¶

Returns a new URI comprised of the scheme and authority only.

Only valid for absolute URIs.

Resolve(base, currentDocRef=None)¶

Resolves a (relative) URI relative to base returning a new URI instance

If the base URI is also relative then the result is a relative URI, otherwise the result is an absolute URI. The RFC does not actually go into the procedure for combining relative URIs but if B is an absolute URI and R1 and R2 are relative URIs then using the resolve operator:

U1 = B [*] R1
U2 = U1 [*] R2
U2 = ( B [*] R1 ) [*] R2

The last expression prompts the issue of associativity, in other words, is the following expression also valid?

U2 = B [*] ( R1 [*] R2 )

For this to work it must be possible to use the resolve operator to combine two relative URIs to make a third, which is what we allow here.

The optional currentDocRef allows you to handle the special case of resolving the empty URI. Strictly speaking, fragments are not part of the URI itself so a relative URI consisting of the empty string, or a relative URI consisting of just a fragment both refer to the current document. By default, currentDocRef is assumed to be the same as base but there are cases where the base URI is not the same as the URI used to originally retrieve the document and the optional parameter allows you to cope with those cases.

Relative(base)¶

Evaluates the Relative operator, returning the URI expressed relative to base.

As we also allow the Resolve method for relative paths it makes sense for the Relative operator to also be defined:

R3 = R1 [*] R2
R3 [/] R1 = R2

Note that there are some restrictions....

U = B [*] R

If R is absolute, or simply more specified than B on the following scale:

absolute URI > authority > absolute path > relative path

then U = R regardless of the value of B and therefore:

U [/] B = U if B is less specified than U.

Also note that if U is a relative URI then B cannot be absolute. In fact B must always be less than, or equally specified to U because B is the base URI from which U has been derived.

U [/] B = undefined if B is more specified than U

Therefore the only interesting cases are when B is equally specified to U. To give a concrete example:

U = /HD/User/setting.txt
B = /HD/folder/file.txt

/HD/User/setting.txt [\] /HD/folder/file.txt = ../User/setting.txt
/HD/User/setting.txt = /HD/folder/file.txt [*] ../User/setting.txt

And for relative paths:

U = User/setting.txt
B = User/folder/file.txt

User/setting.txt [\] User/folder/file.txt = ../setting.txt
User/setting.txt = User/folder/file.txt [*] ../setting.txt              

canonicalize()¶: Returns a canonical form of this URI

match(otherURI)¶: Compares this URI against otherURI returning True if they match.

IsAbsolute()¶: Returns True if this URI is absolute, i.e., fully specified with a scheme name.

URI.__str__()¶

URI are always returned as a string (of bytes), not a unicode string.

The reason for this restriction is best illustrated with an example:

The URI %E8%8B%B1%E5%9B%BD.xml is a UTF-8 and URL-encoded path segment using the Chinese word for United Kingdom. When we remove the URL-encoding we get the string ‘\xe8\x8b\xb1\xe5\x9b\xbd.xml’ which must be interpreted with utf-8 to get the intended path segment value: u’\u82f1\u56fd.xml’. However, if the URL was marked as being a unicode string of characters then this second stage would not be carried out and the result would be the unicode string u’\xe8\x8b\xb1\xe5\x9b\xbd’, which is a meaningless string of 6 characters taken from the European Latin-1 character set.

Hypertext Transfer Protocol (RFC2616)¶

This sub-package defines functions and classes for working with HTTP as defined by RFC2616: http://www.ietf.org/rfc/rfc2616.txt and RFC2617: http://www.ietf.org/rfc/rfc2617.txt

The purpose of this module is to expose some of the basic constructs (including the synax of protocol components) to allow them to be used normatively in other contexts. The module also contains a functional HTTP client designed to support non-blocking and persistent HTTP client operations.

HTTP Client¶

Sending Requests¶

Here is a simple example of Pyslet’s HTTP support in action from the python interpreter:

>>> import pyslet.http.client as http
>>> c = http.Client()
>>> r = http.ClientRequest('http://odata.pyslet.org')
>>> c.process_request(r)
>>> r.response.status
200
>>> print r.response.get_content_type()
text/html; charset=UTF-8
>>> print r.response.entity_body.getvalue()
<html>
<head><title>Pyslet Home</title></head>
<body>
<p><a href="http://qtimigration.googlecode.com/"><img src="logoc-large.png" width="1024"/></a></p>
</body>
</html>
>>> c.close()

In its simplest form there are three steps required to make an HTTP request, firstly you need to create a Client object. The purpose of the Client object is sending requests and receiving responses. The second step is to create a ClientRequest object describing the request you want to make. For a simple GET request you only need to specify the URL. The third step is to instruct the Client to process the request. Once this method returns you can examine the request’s associated response. The response’s entity body is written to a StringIO object by default.

The request and response objects are both derived classes of a basic HTTP Message class. This class has methods for getting and setting headers. You can use the basic get_header() and set_header() to set headers from strings or, where provided, you can use special wrapper methods such as get_content_type() to get and set headers using special-purpose class objects that represent parsed forms of the expected value. In the case of Content-Type headers the result is a MediaType() object. Providing these special object types is one of the main reasons why Pyslet’s HTTP support is different from other clients. By exposing these structures you can reuse HTTP concepts in other contexts, particularly useful when other technical specifications make normative references to them.

Here is a glimpse of what you can do with a parsed media type, continuing the above example:

>>> type = r.response.get_content_type()
>>> type
MediaType('text','html',{'charset': ('charset', 'UTF-8')})
>>> type.type
'text'
>>> type.subtype
'html'
>>> type['charset']
'UTF-8'
>>> type['name']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyslet/http/params.py", line 382, in __getitem__
    repr(key))
KeyError: "MediaType instance has no parameter 'name'"
>>>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyslet/http/params.py", line 382, in __getitem__
    repr(key))
KeyError: "MediaType instance has no parameter 'name'"

There are lots of other special get_ and set_ methods on the Message, Request and Response objects.

Pipelining¶

One of the use cases that Pyslet’s HTTP client is designed to cover is reusing an HTTP connection to make multiple requests to the same host. The example above takes care to close the Client object when we’re done because otherwise it would leave the connection to the server open ready for another request.

Reference¶

The client module imports the grammar, params, messages and auth modules and these can therefore be accessed using a single import in your code. For example:

import pyslet.http.client as http
type = http.params.MediaType('application', 'xml')

For more details of the objects exposed by those modules see pyslet.http.grammar, pyslet.http.params, pyslet.http.messages and pyslet.http.auth.

class pyslet.http.client.Client(max_connections=100, ca_certs=None, timeout=None)¶

Bases: pyslet.pep8.PEP8Compatibility, object

An HTTP client

Note

In Pyslet 0.4 and earlier the name HTTPRequestManager was used, this name is still available as an alias for Client.

The object manages the sending and receiving of HTTP/1.1 requests and responses respectively. There are a number of keyword arguments that can be used to set operational parameters:

max_connections

The maximum number of HTTP connections that may be open at any one time. The method queue_request() will block (or raise RequestManagerBusy) if an attempt to queue a request would cause this limit to be exceeded.

timeout

The maximum wait time on the connection. This is not the same as a limit on the total time to receive a request but a limit on the time the client will wait with no activity on the connection before assuming that the server is no longer responding. Defaults to None, no timeout.

ca_certs

The file name of a certificate file to use when checking SSL connections. For more information see http://docs.python.org/2.7/library/ssl.html

In practice, there seem to be serious limitations on SSL connections and certificate validation in Python distributions linked to earlier versions of the OpenSSL library (e.g., Python 2.6 installed by default on OS X and Windows).

Warning

By default, ca_certs is optional and can be passed as None. In this mode certificates will not be checked and your connections are not secure from man in the middle attacks. In production use you should always specify a certificate file if you expect to use the object to make calls to https URLs.

Although max_connections allows you to make multiple connections to the same host+port the request manager imposes an additional restriction. Each thread can make at most 1 connection to each host+port. If multiple requests are made to the same host+port from the same thread then they are queued and will be sent to the server over the same connection using HTTP/1.1 pipelining. The manager (mostly) takes care of the following restriction imposed by RFC2616:

Clients SHOULD NOT pipeline requests using non-idempotent methods or non-idempotent sequences of methods

In other words, a POST (or CONNECT) request will cause the pipeline to stall until all the responses have been received. Users should beware of non-idempotent sequences as these are not automatically detected by the manager. For example, a GET,PUT sequence on the same resource is not idempotent. Users should wait for the GET request to finish fetching the resource before queuing a PUT request that overwrites it.

In summary, to take advantage of multiple simultaneous connections to the same host+port you must use multiple threads.

ConnectionClass¶: alias of Connection

httpUserAgent = None¶

The default User-Agent string to use, defaults to a string derived from the installed version of Pyslet, e.g.:

pyslet 0.5.20140727 (http.client.Client)

queue_request(request, timeout=None)¶

Starts processing an HTTP request

request

A messages.Request object.

timeout

Number of seconds to wait for a free connection before timing out. A timeout raises RequestManagerBusy

None means wait forever, 0 means don’t block.

The default implementation adds a User-Agent header from httpUserAgent if none has been specified already. You can override this method to add other headers appropriate for a specific context but you must pass this call on to this implementation for proper processing.

active_count()¶: Returns the total number of active connections.

thread_active_count()¶: Returns the total number of active connections associated with the current thread.

thread_task(timeout=None)¶

Processes all connections bound to the current thread then blocks for at most timeout (0 means don’t block) while waiting to send/receive data from any active sockets.

Each active connection receives one call to Connection.connection_task() There are some situations where this method may still block even with timeout=0. For example, DNS name resolution and SSL handshaking. These may be improved in future.

Returns True if at least one connection is active, otherwise returns False.

thread_loop(timeout=60)¶: Repeatedly calls thread_task() until it returns False.

process_request(request, timeout=60)¶

Process an messages.Message object.

The request is queued and then thread_loop() is called to exhaust all HTTP activity initiated by the current thread.

idle_cleanup(max_inactive=15)¶: Cleans up any idle connections that have been inactive for more than max_inactive seconds.

active_cleanup(max_inactive=90)¶

Clean up active connections that have been inactive for more than max_inactive seconds.

This method can be called from any thread and can be used to remove connections that have been abandoned by their owning thread. This can happen if the owning thread stops calling thread_task() leaving some connections active.

Inactive connections are killed using Connection.kill() and then removed from the active list. Should the owning thread wake up and attempt to finish processing the requests a socket error or messages.HTTPException will be reported.

close()¶

Closes all connections and sets the manager to a state where new connections cannot not be created.

Active connections are killed, idle connections are closed.

add_credentials(credentials)¶

Adds a pyslet.http.auth.Credentials instance to this manager.

Credentials are used in response to challenges received in HTTP 401 responses.

remove_credentials(credentials)¶

Removes credentials from this manager.

credentials: A pyslet.http.auth.Credentials instance previously added with add_credentials().

If the credentials can’t be found then they are silently ignored as it is possible that two threads may independently call the method with the same credentials.

dnslookup(host, port)¶

Given a host name (string) and a port number performs a DNS lookup using the native socket.getaddrinfo function. The resulting value is added to an internal dns cache so that subsequent calls for the same host name and port do not use the network unnecessarily.

If you want to flush the cache you must do so manually using flush_dns().

flush_dns()¶: Flushes the DNS cache.

find_credentials(challenge)¶: Searches for credentials that match challenge

find_credentials_by_url(url)¶: Searches for credentials that match url

class pyslet.http.client.ClientRequest(url, method='GET', res_body=None, protocol=<pyslet.http.params.HTTPVersion object at 0x7fe07c5fcd10>, auto_redirect=True, max_retries=3, **kwargs)¶

Bases: pyslet.http.messages.Request

Represents an HTTP request.

To make an HTTP request, create an instance of this class and then pass it to an Client instance using either Client.queue_request() or Client.process_request().

url: An absolute URI using either http or https schemes. A pyslet.rfc2396.URI instance or an object that can be passed to its constructor.

And the following keyword arguments:

method: A string. The HTTP method to use, defaults to “GET”
entity_body: A string or stream-like object containing the request body. Defaults to None meaning no message body. For stream-like objects the tell and seek methods must be supported to enable resending the request if required.
res_body: A stream-like object to write data to. Defaults to None, in which case the response body is returned as a string the res_body.
protocol: An params.HTTPVersion object, defaults to HTTPVersion(1,1)
autoredirect: Whether or not the request will follow redirects, defaults to True.
max_retries: The maximum number of times to attempt to resend the request following an error on the connection or an unexpected hang-up. Defaults to 3, you should not use a value lower than 1 because it is always possible that the server has gracefully closed the socket and we don’t notice until we’ve sent the request and get 0 bytes back on recv. Although ‘normal’ this scenario counts as a retry.

manager = None¶: the Client object that is managing us

connection = None¶: the Connection object that is currently sending us

status = None¶: the status code received, 0 indicates a failed or unsent request

error = None¶: If status == 0, the error raised during processing

scheme = None¶: the scheme of the request (http or https)

hostname = None¶: the hostname of the origin server

port = None¶: the port on the origin server

url = None¶: the full URL of the requested resource

res_body = None¶: the response body received (only used if not streaming)

auto_redirect = None¶: whether or not auto redirection is in force for 3xx responses

max_retries = None¶: the maximum number of retries we’ll attempt

nretries = None¶: the number of retries we’ve had

response = None¶: the associated ClientResponse

set_url(url)¶

Sets the URL for this request

This method sets the Host header and the following local attributes: scheme, hostname, port and request_uri.

can_retry()¶

Called after each connection-related failure

For idempotent methods we lose a life and check that it’s not game over. For non-idempotent methods (e.g., POST) we always return False.

set_client(client)¶

Called when we are queued for processing.

client: an Client instance

set_connection(connection)¶: Called when we are assigned to an HTTPConnection

disconnect()¶

Called when the connection has finished sending us

This may be before or after the response is received and handled!

finished()¶: Called when we have a final response and have disconnected from the connection There is no guarantee that the server got all of our data, it might even have returned a 2xx series code and then hung up before reading the data, maybe it already had what it needed, maybe it thinks a 2xx response is more likely to make us go away. Whatever. The point is that you can’t be sure that all the data was transmitted just because you got here and the server says everything is OK

class pyslet.http.client.ClientResponse(request, **kwargs)¶

Bases: pyslet.http.messages.Response

handle_headers()¶

Hook for response header processing.

This method is called when a set of response headers has been received from the server, before the associated data is received! After this call, recv will be called zero or more times until handle_message or handle_disconnect is called indicating the end of the response.

Override this method, for example, if you want to reject or invoke special processing for certain responses (e.g., based on size) before the data itself is received. To abort the response, close the connection using Connection.request_disconnect().

Override the Finished() method instead to clean up and process the complete response normally.

handle_message()¶: Hook for normal completion of response

handle_disconnect(err)¶

Hook for abnormal completion of the response

Called when the server disconnects before we’ve completed reading the response. Note that if we are reading forever this may be expected behaviour and err may be None.

We pass this information on to the request.

Exceptions¶

class pyslet.http.client.RequestManagerBusy¶

Bases: pyslet.http.messages.HTTPException

The HTTP client is busy

Raised when attempting to queue a request and no connections become available within the specified timeout.

HTTP Authentication¶

HTTP Messages¶

This modules defines objects that represent the values of HTTP messages and message headers and a special-purpose parser for parsing them from strings of octets.

Messages¶

class pyslet.http.messages.Request(**kwargs)¶

Bases: pyslet.http.messages.Message

method = None¶: the http method, always upper case, e.g., ‘POST’

request_uri = None¶: the request uri as it appears in the start line

response = None¶: the associated response

send_start()¶: Returns the start-line for this message

send_transferlength()¶

Adds request-specific processing for transfer-length

Request messages that must not have a message body are automatically detected and will raise an exception if they have a non-None body.

Request messages that may have a message body but have a transfer-length of 0 bytes will have a Content-Length header of 0 added if necessary

get_start()¶: Returns the start line

is_idempotent()¶: Returns True if this is an idempotent request

extract_authority()¶

Extracts the authority from the request

If the request_uri is an absolute URL then it is updated to contain the absolute path only and the Host header is updated with the authority information (host[:port]) extracted from it, otherwise the Host header is read for the authority information. If there is no authority information in the request None is returned.

If the url contains user information it raises NotImplementedError

get_accept()¶: Returns an AcceptList instance or None if no “Accept” header is present.

set_accept(accept_value)¶

Sets the “Accept” header, replacing any existing value.

accept_value: A AcceptList instance or a string that one can be parsed from.

get_accept_charset()¶: Returns an AcceptCharsetList instance or None if no “Accept-Charset” header is present.

set_accept_charset(accept_value)¶

Sets the “Accept-Charset” header, replacing any existing value.

accept_value: A AcceptCharsetList instance or a string that one can be parsed from.

get_accept_encoding()¶: Returns an AcceptEncodingList instance or None if no “Accept-Encoding” header is present.

set_accept_encoding(accept_value)¶

Sets the “Accept-Encoding” header, replacing any existing value.

accept_value: A AcceptEncodingList instance or a string that one can be parsed from.

class pyslet.http.messages.Response(request, **kwargs)¶

Bases: pyslet.http.messages.Message

REASON = {200: 'OK', 201: 'Created', 202: 'Accepted', 203: 'Non-Authoritative Information', 204: 'No Content', 205: 'Reset Content', 206: 'Partial Content', 400: 'Bad Request', 401: 'Unauthorized', 402: 'Payment Required', 403: 'Forbidden', 404: 'Not Found', 405: 'Method Not Allowed', 406: 'Not Acceptable', 407: 'Proxy Authentication Required', 408: 'Request Time-out', 409: 'Conflict', 410: 'Gone', 411: 'Length Required', 412: 'Precondition Failed', 413: 'Request Entity Too Large', 414: 'Request-URI Too Large', 415: 'Unsupported Media Type', 416: 'Requested range not satisfiable', 417: 'Expectation Failed', 100: 'Continue', 101: 'Switching Protocols', 300: 'Multiple Choices', 301: 'Moved Permanently', 302: 'Found', 303: 'See Other', 304: 'Not Modified', 305: 'Use Proxy', 307: 'Temporary Redirect', 500: 'Internal Server Error', 501: 'Not Implemented', 502: 'Bad Gateway', 503: 'Service Unavailable', 504: 'Gateway Time-out', 505: 'HTTP Version not supported'}¶: A dictionary mapping status code integers to their default message defined by RFC2616

send_start()¶: Returns the start-line for this message

get_accept_ranges()¶: Returns an AcceptRanges instance or None if no “Accept-Ranges” header is present.

set_accept_ranges(accept_value)¶

Sets the “Accept-Ranges” header, replacing any existing value.

accept_value: A AcceptRanges instance or a string that one can be parsed from.

get_age()¶: Returns an integer or None if no “Age” header is present.

set_age(age)¶

Sets the “Age” header, replacing any existing value.

age: an integer or long value or None to remove the header

get_etag()¶: Returns a EntityTag instance parsed from the ETag header or None if no “ETag” header is present.

set_etag(etag)¶

Sets the “ETag” header, replacing any existing value.

etag: a EntityTag instance or None to remove any ETag header.

get_location()¶

Returns a pyslet.rfc2396.URI instance created from the Location header.

If no Location header was present None is returned.

set_location(location)¶

Sets the Location header

location:: a pyslet.rfc2396.URI instance or a string from which one can be parsed. If None, the Location header is removed.

get_www_authenticate()¶

Returns a list of Challenge instances.

If there are no challenges an empty list is returned.

set_www_authenticate(challenges)¶

Sets the “WWW-Authenticate” header, replacing any exsiting value.

challenges: a list of Challenge instances

class pyslet.http.messages.Message(entity_body=None, protocol=<pyslet.http.params.HTTPVersion object at 0x7fe07c5fcd10>)¶

Bases: pyslet.pep8.PEP8Compatibility, object

An abstract class to represent an HTTP message.

The methods of this class are thread safe, using a lock to protect all access to internal structures.

The generic syntax of a message involves a start line, followed by a number of message headers and an optional message body.

entity_body

The optional entity_body parameter is a byte string containing the entity body, a file like object or object derived from io.RawIOBase. There are restrictions on the use of non-seekable streams, in particular the absence of a working seek may affect redirects and retries.

There is a subtle difference between passing None, meaning no entity body and an empty string ‘’. The difference is that an empty string will generate a Content-Length header indicating a zero length message body when the message is sent, whereas None will not. Some message types are not allowed to have an entity body (e.g., a GET request) and these messages must not have a message body (even a zero length one) or an error will be raised.

File-like objects do not generate a Content-Length header automatically as there is no way to determine their size when sending, however, if a Content-Length header is set explicitly then it will be used to constrain the amount of data read from the entity_body.

GENERAL_HEADERS = {'transfer-encoding': 'Transfer-Encoding', 'connection': 'Connection', 'upgrade': 'Upgrade', 'pragma': 'Pragma', 'cache-control': 'Cache-Control', 'date': 'Date', 'warning': 'Warning', 'via': 'Via', 'trailer': 'Trailer'}¶: a mapping from lower case header name to preferred case name

MAX_READAHEAD = 131072¶: A constant used to control the maximum read-ahead on an entity body’s stream. Entity bodies of undetermined length that exceed this size cannot be sent in requests to HTTP/1.0 server.

lock = None¶: the lock used to protect multi-threaded access

keep_alive = None¶: by default we’ll keep the connection alive

set_protocol(version)¶

Sets the protocol

version: An params.HTTPVersion instance or a string that can be parsed for one.

clear_keep_alive()¶

Clears the keep_alive flag on this message

The flag always starts set to True and cannot be set once cleared.

start_sending(protocol=<pyslet.http.params.HTTPVersion object at 0x7fe07c5fcd10>)¶

Starts sending this message

protocol: The protocol supported by the target of the message, defaults to HTTP/1.1 but can be overridden when the recipient only supports HTTP/1.0. This has the effect of suppressing some features.

The message is sent using the send_ family of methods.

send_start()¶: Returns the start-line for this message

send_header()¶: Returns a data string ready to send to the server

send_transferlength()¶

Calculates the transfer length of the message

It will read the Transfer-Encoding or Content-Length headers to determine the length.

If the length of the entity body is known, this method will verify that it matches the Content-Length or set that header’s value accordingly.

If the length of the entity body is not known, this method will set a Transfer-Encoding header.

send_body()¶

Returns (part of) the message body

Returns an empty string when there is no more data to send.

Returns None if the message is read blocked.

start_receiving()¶

Starts receiving this message

The message is received using the recv_mode() and recv() methods.

RECV_HEADERS = -3¶: recv_mode constant for a set of header lines terminated by CRLF, followed by a blank line.

RECV_LINE = -2¶: recv_mode constant for a single CRLF terminated line

RECV_ALL = -1¶: recv_mode constant for unlimited data read

recv_mode()¶

Indicates the type of data expected during recv

The result is interpreted as follows, using the recv_mode constants defined above:

RECV_HEADERS: this message is expecting a set of headers, terminated by a blank line. The next call to recv must be with a list of binary CRLF terminated strings the last of which must the string CRLF only.
RECV_LINE: this message is expecting a single terminated line. The next call to recv must be with a binary string representing a single terminated line.
integer > 0: the minimum number of bytes we are waiting for when data is expected. The next call to recv must be with a binary string of up to but not exceeding integer number of bytes
0: we are currently write-blocked but still need more data, the next call to recv must pass None to give the message time to write out existing buffered data.
RECV_ALL: we want to read until the connection closes, the next call to recv must be with a binary string. The string can be of any length but an empty string signals the end of the data.
None: the message is not currently in receiving mode, calling recv will raise an error.

recv_start(start_line)¶

Receives the start-line

Implemented differently for requests and responses.

handle_headers()¶

Hook for processing the message headers

This method is called after all headers have been received but before the message body (if any) is received. Derived classes should always call this implementation first (using super) to ensure basic validation is performed on the message before the body is received.

recv_transferlength()¶

Called to calculate the transfer length when receiving

The values of transferlength and transferchunked are set by this method. The default implementation checks for a Transfer-Encoding header and then a Content-Length header in that order.

If it finds neither then behaviour is determined by the derived classes Request and Response which wrap this implementation.

RFC2616:

If a Transfer-Encoding header field is present and has any value other than “identity”, then the transfer-length is defined by use of the “chunked” transfer-coding, unless the message is terminated by closing the connection

This is a bit weird, if I have a non-identity value which fails to mention ‘chunked’ then it seems like I can’t imply chunked encoding until the connection closes. In practice, when we handle this case we assume chunked is not being used and read until connection close.

handle_message()¶

Hook for processing the message

This method is called after the entire message has been received, including any chunk trailer.

get_headerlist()¶

Returns all header names

The list is alphabetically sorted and lower-cased.

has_header(field_name)¶: True if this message has a header with field_name

get_header(field_name)¶

Returns the header with field_name as a string.

If there is no header with field_name then None is returned.

set_header(field_name, field_value, append_mode=False)¶

Sets the header with field_name to the string field_value.

If field_value is None then the header is removed (if present).

If a header already exists with field_name then the behaviour is determined by append_mode:

append_mode==True: field_value is joined to the existing value using ”, ” as a separator.
append_mode==False (Default): field_value replaces the existing value.

get_allow()¶: Returns an Allow instance or None if no “Allow” header is present.

set_allow(allowed)¶

Sets the “Allow” header, replacing any existing value.

allowed: A Allow instance or a string that one can be parsed from.

If allowed is None any existing Allow header is removed.

get_authorization()¶

Returns a Credentials instance.

If there are no credentials None returned.

set_authorization(credentials)¶

Sets the “Authorization” header

credentials: a Credentials instance

get_cache_control()¶: Returns an CacheControl instance or None if no “Cache-Control” header is present.

set_cache_control(cc)¶

Sets the “Cache-Control” header, replacing any existing value.

cc: A CacheControl instance or a string that one can be parsed from.

If cc is None any existing Cache-Control header is removed.

get_connection()¶

Returns a set of connection tokens from the Connection header

If no Connection header was present an empty set is returned.

set_connection(connection_tokens)¶

Set the Connection tokens from a an iterable set of connection_tokens

If the list is empty any existing header is removed.

get_content_encoding()¶

Returns a list of lower-cased content-coding tokens from the Content-Encoding header

If no Content-Encoding header was present an empty list is returned.

Content-codings are always listed in the order they have been applied.

set_content_encoding(content_codings)¶: Sets the Content-Encoding header from a an iterable list of content-coding tokens. If the list is empty any existing header is removed.

get_content_language()¶

Returns a list of LanguageTag instances from the Content-Language header

If no Content-Language header was present an empty list is returned.

set_content_language(lang_list)¶: Sets the Content-Language header from a an iterable list of LanguageTag instances.

get_content_length()¶

Returns the integer size of the entity from the Content-Length header

If no Content-Length header was present None is returned.

set_content_length(length)¶: Sets the Content-Length header from an integer or removes it if length is None.

get_content_location()¶

Returns a pyslet.rfc2396.URI instance created from the Content-Location header.

If no Content-Location header was present None is returned.

set_content_location(location)¶: Sets the Content-Location header from location, a pyslet.rfc2396.URI instance or removes it if location is None.

get_content_md5()¶

Returns a 16-byte binary string read from the Content-MD5 header or None if no Content-MD5 header was present.

The result is suitable for comparing directly with the output of the Python’s MD5 digest method.

set_content_md5(digest)¶: Sets the Content-MD5 header from a 16-byte binary string returned by Python’s MD5 digest method or similar. If digest is None any existing Content-MD5 header is removed.

get_content_range()¶

Returns a ContentRange instance parsed from the Content-Range header.

If no Content-Range header was present None is returned.

set_content_range(range)¶: Sets the Content-Range header from range, a ContentRange instance or removes it if range is None.

get_content_type()¶

Returns a MediaType instance parsed from the Content-Type header.

If no Content-Type header was present None is returned.

set_content_type(mtype=None)¶: Sets the Content-Type header from mtype, a MediaType instance, or removes it if mtype is None.

get_date()¶

Returns the value of the Date header.

The return value is a params.FullDate instance. If no Date header was present None is returned.

set_date(date=None)¶

Sets the value of the Date header

date: a params.FullDate instance or None to remove the Date header.

To set the date header to the current date use:

set_date(params.FullDate.FromNowUTC())

get_last_modified()¶

Returns the value of the Last-Modified header

The result is a params.FullDate instance. If no Last-Modified header was present None is returned.

set_last_modified(date=None)¶

Sets the value of the Last-Modified header field

date: a FullDate instance or None to remove the header

To set the Last-Modified header to the current date use:

set_last_modified(params.FullDate.FromNowUTC())

get_transfer_encoding()¶

Returns a list of params.TransferEncoding

If no TransferEncoding header is present None is returned.

set_transfer_encoding(field_value)¶

Set the Transfer-Encoding header

field_value: A list of params.TransferEncoding instances or a string from which one can be parsed. If None then the header is removed.

General Header Types¶

class pyslet.http.messages.CacheControl(*args)¶

Bases: object

Represents the value of a Cache-Control general header.

The built-in str function can be used to format instances according to the grammar defined in the specification.

Instances are immutable, they are constructed from a list of arguments which must not be empty. Arguments are treated as follows:

string: a simple directive with no parmeter
2-tuple of string and non-tuple: a directive with a simple parameter
2-tuple of string and tuple: a directive with a quoted list-style parameter

Instances behave like read-only lists implementing len, indexing and iteration in the usual way. Instances also support basic key lookup of directive names by implementing __contains__ and __getitem__ (which returns None for defined directives with no parameter and raises KeyError for undefined directives. Instances are not truly dictionary like.

classmethod from_str(source)¶: Create a Cache-Control value from a source string.

Request Header Types¶

class pyslet.http.messages.AcceptList(*args)¶

Bases: object

Represents the value of an Accept header

The built-in str function can be used to format instances according to the grammar defined in the specification.

Instances are immutable, they are constructed from one or more AcceptItem instances. There are no comparison methods.

Instances behave like read-only lists implementing len, indexing and iteration in the usual way.

select_type(mtype_list)¶

Returns the best match from mtype_list, a list of media-types

In the event of a tie, the first item in mtype_list is returned.

classmethod from_str(source)¶: Create an AcceptList from a source string.

class pyslet.http.messages.MediaRange(type='*', subtype='*', parameters={})¶

Bases: pyslet.http.params.MediaType

Represents an HTTP media-range.

The built-in str function can be used to format instances according to the grammar defined in the specification.

Instances are immutable, they define comparison methods and a hash implementation to allow them to be used as keys in dictionaries. Quoting from the specification:

“Media ranges can be overridden by more specific media ranges or specific media types. If more than one media range applies to a given type, the most specific reference has precedence.”

In other words, the following media ranges would be sorted in the order shown:

image/png
image/*
text/plain;charset=utf-8
text/plain
text/*
*/*

If we have two rules with identical precedence then we sort them alphabetically by type; sub-type and ultimately alphabetically by parameters

classmethod from_str(source)¶

Creates a media-rannge from a source string.

Unlike the parent media-type we ignore all spaces.

match_media_type(mtype)¶

Tests whether a media-type matches this range.

mtype: A MediaType instance to be compared to this range.

The matching algorithm takes in to consideration wild-cards so that */* matches all types, image/* matches any image type and so on.

If a media-range contains parameters then each of these must be matched exactly in the media-type being tested. Parameter names are treated case-insensitively and any additional parameters in the media type are ignored. As a result:

text/plain does not match the range text/plain;charset=utf-8
application/myapp;charset=utf-8;option=on does match the range application/myapp;option=on

class pyslet.http.messages.AcceptItem(range=MediaType('*', '*', {}), qvalue=1.0, extensions={})¶

Bases: pyslet.http.messages.MediaRange

Represents a single item in an Accept header

The built-in str function can be used to format instances according to the grammar defined in the specification.

Instances are immutable, they define comparison methods and a hash implementation to allow them to be used as keys in dictionaries.

Accept items are sorted by their media ranges. Equal media ranges sort by descending qvalue, for example:

text/plain;q=0.75 < text/plain;q=0.5

Extension parameters are ignored in all comparisons.

range = None¶: the MediaRange instance that is acceptable

q = None¶: the q-value (defaults to 1.0)

classmethod from_str(source)¶: Creates a single AcceptItem instance from a source string.

class pyslet.http.messages.AcceptCharsetItem(token='*', qvalue=1.0)¶

Bases: pyslet.http.messages.AcceptToken

Represents a single item in an Accept-Charset header

class pyslet.http.messages.AcceptCharsetList(*args)¶

Bases: pyslet.http.messages.AcceptTokenList

Represents an Accept-Charset header

ItemClass¶: alias of AcceptCharsetItem

select_token(token_list)¶: Overridden to provide default handling of iso-8859-1

class pyslet.http.messages.AcceptEncodingItem(token='*', qvalue=1.0)¶

Bases: pyslet.http.messages.AcceptToken

Represents a single item in an Accept-Encoding header

class pyslet.http.messages.AcceptEncodingList(*args)¶

Bases: pyslet.http.messages.AcceptTokenList

Represents an Accept-Encoding header

ItemClass¶: alias of AcceptEncodingItem

select_token(token_list)¶: Overridden to provide default handling of identity

class pyslet.http.messages.AcceptLanguageItem(token='*', qvalue=1.0)¶

Bases: pyslet.http.messages.AcceptToken

Represents a single item in an Accept-Language header.

class pyslet.http.messages.AcceptLanguageList(*args)¶

Bases: pyslet.http.messages.AcceptTokenList

Represents an Accept-Language header

ItemClass¶

the class used to create items in this token list

alias of AcceptLanguageItem

select_token(token_list)¶: Remapped to select_language()

class pyslet.http.messages.AcceptToken(token='*', qvalue=1.0)¶

Bases: object

Represents a single item in a token-based Accept-* header

The built-in str function can be used to format instances according to the grammar defined in the specification.

Instances are immutable, they define comparison methods and a hash implementation to allow them to be used as keys in dictionaries.

AcceptToken items are sorted by their token, with wild cards sorting behind specified tokens. Equal values sort by descending qvalue, for example:

iso-8859-2;q=0.75 < iso-8859-2;q=0.5

token = None¶: the token that is acceptable or “*” for any token

q = None¶: the q-value (defaults to 1.0)

classmethod from_str(source)¶: Creates a single AcceptToken instance from a source string.

class pyslet.http.messages.AcceptTokenList(*args)¶

Bases: object

Represents the value of a token-based Accept-* header

The built-in str function can be used to format instances according to the grammar defined in the specification.

Instances are immutable, they are constructed from one or more AcceptToken instances. There are no comparison methods.

Instances behave like read-only lists implementing len, indexing and iteration in the usual way.

ItemClass¶

the class used to create new items in this list

alias of AcceptToken

select_token(token_list)¶

Returns the best match from token_list, a list of tokens.

In the event of a tie, the first item in token_list is returned.

classmethod from_str(source)¶: Create an AcceptTokenList from a source string.

Response Header Types¶

class pyslet.http.messages.AcceptRanges(*args)¶

Bases: object

Represents the value of an Accept-Ranges response header.

The built-in str function can be used to format instances according to the grammar defined in the specification.

Instances are immutable, they are constructed from a list of string arguments. If the argument list is empty then a value of “none” is assumed.

Instances behave like read-only lists implementing len, indexing and iteration in the usual way. Comparison methods are provided.

classmethod from_str(source)¶: Create an AcceptRanges value from a source string.

Entity Header Types¶

class pyslet.http.messages.Allow(*args)¶

Bases: object

Represents the value of an Allow entity header.

The built-in str function can be used to format instances according to the grammar defined in the specification.

Instances are immutable, they are constructed from a list of string arguments which may be empty.

Instances behave like read-only lists implementing len, indexing and iteration in the usual way. Comparison methods are provided.

classmethod from_str(source)¶: Create an Allow value from a source string.

is_allowed(method)¶: Tests if method is allowed by this value.

class pyslet.http.messages.ContentRange(first_byte=None, last_byte=None, total_len=None)¶

Bases: object

Represents a single content range

first_byte: Specifies the first byte of the range
last_byte: Specifies the last byte of the range
total_len: Specifies the total length of the entity

With no arguments an invalid range representing an unsatisfied range request from an entity of unknown length is created.

If first_byte is specified on construction last_byte must also be specified or TypeError is raised.

The built-in str function can be used to format instances according to the grammar defined in the specification.

Instances are immutable.

first_byte = None¶: first byte in the range

last_byte = None¶: last byte in the range

classmethod from_str(source)¶: Creates a single ContentRange instance from a source string.

is_valid()¶

Returns True if this range is valid, False otherwise.

A valid range is any non-empty byte range wholly within the entity described by the total length. Unsatisfied content ranges are treated as invalid.

Parsing Header Values¶

In most cases header values will be parsed automatically when reading them from messages. For completeness a header parser is exposed to enable you to parse these values from more complex strings.

class pyslet.http.messages.HeaderParser(source, ignore_sp=True)¶

Bases: pyslet.http.params.ParameterParser

A special parser for parsing HTTP headers from TEXT

require_media_range()¶

Parses a MediaRange instance.

Raises BadSyntax if no media-type was found.

require_accept_item()¶

Parses a AcceptItem instance

Raises BadSyntax if no item was found.

require_accept_list()¶

Parses a AcceptList instance

Raises BadSyntax if no valid items were found.

require_accept_token(cls=<class 'pyslet.http.messages.AcceptToken'>)¶

Parses a single AcceptToken instance

Raises BadSyntax if no item was found.

cls: An optional sub-class of AcceptToken to create instead.

require_accept_token_list(cls=<class 'pyslet.http.messages.AcceptTokenList'>)¶

Parses a list of token-based accept items

Returns a AcceptTokenList instance. If no tokens were found then an empty list is returned.

cls: An optional sub-class of AcceptTokenList to create instead.

require_contentrange()¶: Parses a ContentRange instance.

Exceptions¶

class pyslet.http.messages.HTTPException¶

Bases: exceptions.Exception

Class for all HTTP message-related errors.

HTTP Protocol Parameters¶

This section defines functions for handling basic parameters used by HTTP. Refer to Section 3 of RFC2616 for details.

The approach taken by this module is provide classes for each of the parameter types. Most classes have a class method ‘from_str’ which returns a new instance parsed from a string and performs the reverse transformation to the builtin str function. Instances are generally immutable objects which is consistent with them representing values of parameters in the protocol.

class pyslet.http.params.HTTPVersion(major=1, minor=None)¶

Bases: object

Represents the HTTP Version.

major: The (optional) major version
minor: The (optional) minor version

The default instance, HTTPVersion(), represents HTTP/1.1

HTTPVersion objects are immutable, they define comparison functions (such that 1.1 > 1.0 and 1.2 < 1.25) and a hash implementation is provided.

On conversion to a string the output is of the form:

HTTP/<major>.<minor>

For convenience, the constants HTTP_1p1 and HTTP_1p0 are provided for comparisons, e.g.:

if HTTPVersion.from_str(version_str) == HTTP_1p0:
    # do something to support a legacy system...

major = None¶: major protocol version (read only)

minor = None¶: minor protocol version (read only)

classmethod from_str(source)¶: Constructs an HTTPVersion object from a string.

class pyslet.http.params.HTTPURL(octets='http://localhost/')¶

Bases: pyslet.rfc2396.ServerBasedURL

Represents http URLs

DEFAULT_PORT = 80¶: the default HTTP port

canonicalize()¶: Returns a canonical form of this URI

class pyslet.http.params.HTTPSURL(octets='https://localhost/')¶

Bases: pyslet.http.params.HTTPURL

Represents https URLs

DEFAULT_PORT = 443¶: the default HTTPS port

class pyslet.http.params.FullDate(src=None, date=None, time=None)¶

Bases: pyslet.iso8601.TimePoint

A special sub-class for HTTP-formatted dates

classmethod from_http_str(source)¶: Returns an instance parsed from an HTTP formatted string

class pyslet.http.params.TransferEncoding(token='chunked', parameters={})¶

Bases: object

Represents an HTTP transfer-encoding.

token: The transfer encoding identifier, defaults to “chunked”
parameters: A parameter dictionary mapping parameter names to tuples of strings: (parameter name, parameter value)

The built-in str function can be used to format instances according to the grammar defined in the specification.

Instances are immutable, they define comparison methods and a hash implementation.

token = None¶: the lower-cased transfer-encoding token (defaults to “chunked”)

parameters = None¶: declared extension parameters

classmethod from_str(source)¶

Parses the transfer-encoding from a source string.

If the encoding is not parsed correctly BadSyntax is raised.

classmethod list_from_str(source)¶

Creates a list of transfer-encodings from a string

Transfer-encodings are comma-separated

class pyslet.http.params.Chunk(size=0, extensions=None)¶

Bases: object

Represents an HTTP chunk header

size: The size of this chunk (defaults to 0)
extensions: A parameter dictionary mapping parameter names to tuples of strings: (chunk-ext-name, chunk-ext-val)

The built-in str function can be used to format instances according to the grammar defined in the specification. The resulting string does not include the trailing CRLF.

Instances are immutable, they define comparison methods and a hash implementation.

size = None¶: the chunk-size

classmethod from_str(source)¶

Parses the chunk header from a source string of TEXT.

If the chunk header is not parsed correctly BadSyntax is raised. The header includes the chunk-size and any chunk-extension parameters but it does not include the trailing CRLF or the chunk-data

class pyslet.http.params.MediaType(type='application', subtype='octet-stream', parameters={})¶

Bases: object

Represents an HTTP media-type.

The built-in str function can be used to format instances according to the grammar defined in the specification.

type: The type code string, defaults to ‘application’
subtype: The sub-type code, defaults to ‘octet-stream’
parameters: A dictionary such as would be returned by grammar.WordParser.parse_parameters() containing the media type’s parameters.

Instances are immutable and support parameter value access by lower-case key, returning the corresponding value or raising KeyError. E.g., mtype[‘charset’]

Instances also define comparison methods and a hash implementation. Media-types are compared by type, subtype and ultimately parameters.

classmethod from_str(source)¶

Creates a media-type from a source string.

Enforces the following rule from the specification:

Linear white space (LWS) MUST NOT be used between the type and subtype, nor between an attribute and its value

class pyslet.http.params.ProductToken(token=None, version=None)¶

Bases: object

Represents an HTTP product token.

The built-in str function can be used to format instances according to the grammar defined in the specification.

Instances are immutable, they define comparison methods and a hash implementation.

The comparison operations use a more interesting sort than plain text on version in order to provide a more intuitive ordering. As it is common practice to use dotted decimal notation for versions (with some alphanumeric modifiers) the version string is exploded (see explode()) internally on construction and this exploded value is used in comparisons. The upshot is that version 1.0.3 sorts before 1.0.10 as you would expect and 1.0a < 1.0 < 1.0.3a3 < 1.0.3a20 < 1.0.3b1 < 1.0.3; there are limits to this algorithm. 1.0dev > 1.0b1 even though it looks like it should be the other way around. Similarly 1.0-live < 1.0-prod etc.

You shouldn’t use this comparison as a definitive way to determine that one release is more recent or up-to-date than another unless you know that the product in question uses a numbering scheme compatible with these rules.

token = None¶: the product’s token

version = None¶: the product’s version

classmethod explode(version)¶

Returns an exploded version string.

Version strings are split by dot and then by runs of non-digit characters resulting in a list of tuples. Examples will help:

explode("2.15")==((2),(15))
explode("2.17b3")==((2),(17,"b",3))
explode("2.b3")==((2),(-1,"b",3))

Note that a missing leading numeric component is treated as -1 to force “a3” to sort before “0a3”.

classmethod from_str(source)¶: Creates a product token from a source string.

classmethod list_from_str(source)¶

Creates a list of product tokens from a source string.

Individual tokens are separated by white space.

class pyslet.http.params.LanguageTag(primary, *subtags)¶

Bases: object

Represents an HTTP language-tag.

The built-in str function can be used to format instances according to the grammar defined in the specification.

Instances are immutable, they define comparison methods and a hash implementation.

partial_match(range)¶

True if this tag is a partial match against range

range: A tuple of lower-cased subtags. An empty tuple matches all instances.

For example:

lang=LanguageTag("en",("US","Texas"))
lang.partial_match(())==True
lang.partial_match(("en",)==True
lang.partial_match(("en","us")==True
lang.partial_match(("en","us","texas")==True
lang.partial_match(("en","gb")==False
lang.partial_match(("en","us","tex")==False

classmethod from_str(source)¶

Creates a language tag from a source string.

Enforces the following rules from the specification:

White space is not allowed within the tag

classmethod list_from_str(source)¶: Creates a list of language tags from a source string.

class pyslet.http.params.EntityTag(tag, weak=True)¶

Represents an HTTP entity-tag.

tag: The opaque tag
weak: A boolean indicating if the entity-tag is a weak or strong entity tag. Defaults to True.

The built-in str function can be used to format instances according to the grammar defined in the specification.

Instances are immutable, they define comparison methods and a hash implementation.

weak = None¶: True if this is a weak tag

tag = None¶: the opaque tag

classmethod from_str(source)¶: Creates an entity-tag from a source string.

Parsing Parameter Values¶

In most cases parameter values will be parsed directly by the class methods provided in the parameter types themselves. For completeness a parameter parser is exposed to enable you to parse these values from more complex strings.

class pyslet.http.params.ParameterParser(source, ignore_sp=True)¶

Bases: pyslet.http.grammar.WordParser

An extended parser for parameter values

This parser defines attributes for dealing with English date names that are useful beyond the basic parsing functions to allow the formatting of date information in English regardless of the locale.

parse_http_version()¶

Parses an HTTPVersion instance

Returns None if no version was found.

wkday = ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun']¶: A list of English day-of-week abbreviations: wkday[0] == “Mon”, etc.

weekday = ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday']¶: A list of English day-of-week full names: weekday[0] == “Monday”

month = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec']¶: A list of English month names: month[0] == “Jan”, etc.

require_fulldate()¶

Parses a FullDate instance.

Raises BadSyntax if none is found.

There are three supported formats as described in the specification:

"Sun, 06 Nov 1994 08:49:37 GMT"
"Sunday, 06-Nov-94 08:49:37 GMT"
"Sun Nov  6 08:49:37 1994"

The first of these is the preferred format.

parse_delta_seconds()¶: Parses a delta-seconds value, see WordParser.parse_integer()

parse_charset()¶: Parses a charset, see WordParser.parse_tokenlower()

parse_content_coding()¶: Parses a content-coding, see WordParser.parse_tokenlower()

require_transfer_encoding()¶

Parses a TransferEncoding instance

Returns None if no transfer-encoding was found.

require_chunk()¶

Parses a chunk header

Returns a Chunk instance or None if no chunk was found.

require_media_type()¶

Parses a MediaType instance.

Raises BadSyntax if no media-type was found.

require_product_token()¶

Parses a ProductToken instance.

Raises BadSyntax if no product token was found.

parse_qvalue()¶

Parses a qvalue returning a float

Returns None if no qvalue was found.

require_language_tag()¶: Parses a language tag returning a LanguageTag instance. Raises BadSyntax if no language tag was found.

require_entity_tag()¶: Parses an entity-tag returning a EntityTag instance. Raises BadSyntax if no language tag was found.

HTTP Grammar¶

This section defines functions for handling basic elements of the HTTP grammar, refer to Section 2.2 of RFC2616 for details.

The HTTP protocol only deals with octets but as a convenience, and due to the blurring of octet and character strings in Python 2.x we process characters as if they were octets.

pyslet.http.grammar.is_octet(c)¶: Returns True if a character matches the production for OCTET.

pyslet.http.grammar.is_char(c)¶: Returns True if a character matches the production for CHAR.

pyslet.http.grammar.is_upalpha(c)¶: Returns True if a character matches the production for UPALPHA.

pyslet.http.grammar.is_loalpha(c)¶: Returns True if a character matches the production for LOALPHA.

pyslet.http.grammar.is_alpha(c)¶: Returns True if a character matches the production for ALPHA.

pyslet.http.grammar.is_digit(c)¶: Returns True if a character matches the production for DIGIT.

pyslet.http.grammar.is_digits(src)¶

Returns True if all characters match the production for DIGIT.

Empty strings return False

pyslet.http.grammar.is_ctl(c)¶: Returns True if a character matches the production for CTL.

LWS and TEXT productions are handled by OctetParser

pyslet.http.grammar.is_hex(c)¶: Returns True if a characters matches the production for HEX.

pyslet.http.grammar.is_hexdigits(src)¶

Returns True if all characters match the production for HEX.

Empty strings return False

pyslet.http.grammar.check_token(t)¶: Raises ValueError if t is not a valid token

pyslet.http.grammar.is_separator(c)¶: Returns True if a character is a separator

pyslet.http.grammar.decode_quoted_string(qstring)¶

Decodes a quoted string, returning the unencoded string.

Surrounding double quotes are removed and quoted characters (characters preceded by ) are unescaped.

pyslet.http.grammar.quote_string(s, force=True)¶

Places a string in double quotes, returning the quoted string.

This is the reverse of decode_quoted_string(). Note that only the double quote, and CTL characters other than SP and HT are quoted in the output.

If force is False then valid tokens are not quoted.

pyslet.http.grammar.format_parameters(parameters)¶

Formats a dictionary of parameters

This function is suitable for formatting parameter dictionaries parsed by WordParser.parse_parameters().

Parameter values are quoted only if their values require it, that is, only if their values are not valid tokens.

Using the Grammar¶

The functions and data definitions above are exposed to enable normative use in other modules but use of the grammar is typically through use of a parser. There are two types of parser, an OctetParser that is used for parsing raw strings (or octets) such as those obtained from the HTTP connection itself and a WordParser that tokenizes the input string first and then provides a higher-level word-based parser.

class pyslet.http.grammar.OctetParser(source)¶

Bases: pyslet.unicode5.BasicParser

A special purpose parser for parsing HTTP productions.

parse_lws()¶

Parses a single instance of the production LWS

The return value is the LWS string parsed or None if there is no LWS.

parse_onetext(unfold=False)¶

Parses a single TEXT instance.

Parses a single character or run of LWS matching the production TEXT. The return value is the matching character, LWS string or None if no TEXT was found.

If unfold is True then any folding LWS is replaced with a single SP. It defaults to False

parse_text(unfold=False)¶

Parses TEXT

Parses a run of characters matching the production TEXT. The return value is the matching TEXT string (including any LWS) or None if no TEXT was found.

If unfold is True then any folding LWS is replaced with a single SP. It defaults to False

parse_token()¶

Parses a token.

Parses a single instance of the production token. The return value is the matching token string or None if no token was found.

parse_comment(unfold=False)¶

Parses a comment.

Parses a single instance of the production comment. The return value is the entire matching comment string (including the brackets, quoted pairs and any nested comments) or None if no comment was found.

If unfold is True then any folding LWS is replaced with a single SP. It defaults to False

parse_ctext(unfold=False)¶

Parses ctext.

Parses a run of characters matching the production ctext. The return value is the matching ctext string (including any LWS) or None if no ctext was found.

If unfold is True then any folding LWS is replaced with a single SP. It defaults to False

Although the production for ctext would include the backslash character we stop if we encounter one as the grammar is ambiguous at this point.

parse_quoted_string(unfold=False)¶

Parses a quoted-string.

Parses a single instance of the production quoted-string. The return value is the entire matching string (including the quotes and any quoted pairs) or None if no quoted-string was found.

If unfold is True then any folding LWS is replaced with a single SP. It defaults to False

parse_qdtext(unfold=False)¶

Parses qdtext.

Parses a run of characters matching the production qdtext. The return value is the matching qdtext string (including any LWS) or None if no qdtext was found.

If unfold is True then any folding LWS is replaced with a single SP. It defaults to False

Although the production for qdtext would include the backslash character we stop if we encounter one as the grammar is ambiguous at this point.

parse_quoted_pair()¶

Parses a single quoted-pair.

The return value is the matching string including the backslash so it will always be of length 2 or None if no quoted-pair was found.

class pyslet.http.grammar.WordParser(source, ignore_sp=True)¶

Bases: object

A word-level parser and tokeniser for the HTTP grammar.

source is the string to be parsed into words. It will normally be valid TEXT but it can contain control characters if they are escaped as part of a comment or quoted string.

LWS is unfolded automatically. By default the parser ignores spaces according to the rules for implied LWS in the specification and neither SP nor HT will be stored in the word list. If you set ignore_sp to False then LWS is not ignored and each run of LWS is returned as a single SP in the word list.

If the source contains a CRLF (or any other non-TEXT character) that is not part of a folding or escape sequence it raises ValueError

The resulting words may be a token, a single separator character, a comment or a quoted string. To determine the type of word, look at the first character.

‘(‘ means the word is a comment, surrounded by ‘(‘ and ‘)’
a double quote means the word is an encoded quoted string (use py:func:decode_quoted_string to decode it)
other separator chars are just themselves and only appear as single character strings. (HT is never returned.)
Any other character indicates a token.

Methods of the form require_* raise BadSyntax if the production is not found.

pos = None¶: a pointer to the current word in the list

the_word = None¶: the current word or None

setpos(pos)¶

Sets the current position of the parser.

Example usage for look-ahead:

# wp is a WordParser instance
savepos=wp.pos
try:
        # parse a token/sub-token combination
        token=wp.require_token()
        wp.require_separator('/')
        subtoken=wp.require_token()
        return token,subtoken
except BadSyntax:
        wp.setpos(savepos)
        return None,None

peek()¶

Returns the next word

If there are no more words, returns None.

syntax_error(expected)¶

Raises BadSyntax.

expected: a descriptive string indicating the expected production.

require_production(result, production=None)¶

Returns result if result is not None

If result is None, raises BadSyntax.

production: can be used to customize the error message with the name of the expected production.

parse_production(require_method, *args)¶

Executes the bound method require_method passing args.

If successful the result of the method is returned. If BadSyntax is raised, the exception is caught, the parser rewound and None is returned.

require_production_end(result, production=None)¶

Checks for a required production and the end of the word list

Returns result if result is not None and parsing is now complete, otherwise raises BadSyntax.

production: can be used to customize the error message with the name of the expected production.

require_end(production=None)¶

Checks for the end of the word list

If the parser is not at the end of the word list BadSyntax is raised.

production: can be used to customize the error message with the name of the production being parsed.

parse_word()¶

Parses any word from the list

Returns the word parsed or None if the parser was already at the end of the word list.

is_token()¶: Returns True if the current word is a token

parse_token()¶

Parses a token from the list of words

Returns the token or None if the next word was not a token.

parse_tokenlower()¶

Returns a lower-cased token parsed from the word list

Returns None if the next word was not a token.

parse_tokenlist()¶

Parses a list of tokens

Returns the list or [] if no tokens were found. Lists are defined by RFC2616 as being comma-separated. Note that empty items are ignored, so string such as “x,,y” return just [“x”, “y”].

require_token(expected='token')¶

Returns the current token or raises BadSyntax

expected: the name of the expected production, it defaults to “token”.

is_integer()¶: Returns True if the current word is an integer token

parse_integer()¶

Parses an integer token from the list of words

Return the integer’s value or None.

require_integer(expected='integer')¶

Parses an integer or raises BadSyntax

expected: can be set to the name of the expected object, defaults to “integer”.

is_hexinteger()¶: Returns True if the current word is a hex token

parse_hexinteger()¶

Parses a hex integer token from the list of words

Return the hex integer’s value or None.

require_hexinteger(expected='hex integer')¶

Parses a hex integer or raises BadSyntax

expected: can be set to the name of the expected object, defaults to “hex integer”.

is_separator(sep)¶: Returns True if the current word matches sep

parse_separator(sep)¶

Parses a sep from the list of words.

Returns True if the current word matches sep and False otherwise.

require_separator(sep, expected=None)¶

Parses sep or raises BadSyntax

expected: can be set to the name of the expected object

is_quoted_string()¶: Returns True if the current word is a quoted string.

parse_quoted_string()¶

Parses a quoted string from the list of words.

Returns the decoded value of the quoted string or None.

parse_sp()¶

Parses a SP from the list of words.

Returns True if the current word is a SP and False otherwise.

parse_parameters(parameters, ignore_allsp=True, case_sensitive=False, qmode=None)¶

Parses a set of parameters

parameters: the dictionary in which to store the parsed parameters
ignore_allsp: a boolean (defaults to True) which causes the function to ignore all LWS in the word list. If set to False then space around the ‘=’ separator is treated as an error and raises BadSyntax.
case_sensitive: controls whether parameter names are treated as case sensitive, defaults to False.
qmode: allows you to pass a special parameter name that will terminate parameter parsing (without being parsed itself). This is used to support headers such as the “Accept” header in which the parameter called “q” marks the boundary between media-type parameters and Accept extension parameters. Defaults to None

Updates the parameters dictionary with the new parameter definitions. The key in the dictionary is the parameter name (converted to lower case if parameters are being dealt with case insensitively) and the value is a 2-item tuple of (name, value) always preserving the original case of the parameter name.

parse_remainder(sep='')¶

Parses the rest of the words

The result is a single string representing the remaining words joined with sep, which defaults to an empty string.

Returns an empty string if the parser is at the end of the word list.

class pyslet.http.grammar.BadSyntax¶

Raised when a syntax error is encountered by the parsers

This is just a trivial sub-class of the built-in ValueError.

The Atom Syndication Format (RFC4287)¶

This module defines functions and classes for working with the Atom Syndication Format as defined by RFC4287: http://www.ietf.org/rfc/rfc4287.txt

Reference¶

Elements¶

class pyslet.rfc4287.Feed(parent)¶

Bases: pyslet.rfc4287.Source

Represents an Atom feed.

This is the document (i.e., top-level) element of an Atom Feed Document, acting as a container for metadata and data associated with the feed

AtomIdClass¶: alias of AtomId

TitleClass¶: alias of Title

UpdatedClass¶: alias of Updated

Entry = None¶: atomEntry

class pyslet.rfc4287.Source(parent)¶

Bases: pyslet.rfc4287.Entity

Metadata from the original source feed of an entry.

This class is also used a base class for Feed.

Generator = None¶: atomGenerator

Icon = None¶: atomIcon

Logo = None¶: atomLogo

Subtitle = None¶: atomSubtitle

class pyslet.rfc4287.Entry(parent)¶

Bases: pyslet.rfc4287.Entity

An individual entry, acting as a container for metadata and data associated with the entry.

AtomIdClass¶: alias of AtomId

TitleClass¶: alias of Title

UpdatedClass¶: alias of Updated

LinkClass¶: alias of Link

class pyslet.rfc4287.Entity(parent)¶

Bases: pyslet.rfc4287.AtomElement

Base class for feed, entry and source elements.

LinkClass¶: alias of Link

AtomId = None¶

the atomId of the object

Note that we qualify the class name used to represent the id to avoid confusion with the existing ‘id’ attribute in Element.

Author = None¶: atomAuthor

Link = None¶: atomLink

Title = None¶: atomTitle

class pyslet.rfc4287.Author(parent)¶

Bases: pyslet.rfc4287.Person

A Person construct that indicates the author of the entry or feed.

class pyslet.rfc4287.Category(parent)¶

Bases: pyslet.rfc4287.AtomElement

Information about a category associated with an entry or feed.

term = None¶: a string that identifies the category to which the entry or feed belongs

scheme = None¶

an IRI that identifies a categorization scheme.

This is not converted to a pyslet.rfc2396.URI instance as it is not normally resolved to a resource. Instead it defines a type of namespace.

label = None¶: a human-readable label for display in end-user applications

class pyslet.rfc4287.Content(parent)¶

Bases: pyslet.rfc4287.Text

Contains or links to the content of the entry.

Although derived from Text this class overloads the meaning of the Text.type attribute allowing it to be a media type.

src = None¶: link to remote content

GetValue()¶

Gets a single unicode string representing the value of the element.

Overloads the basic GetValue(), if type is a media type rather than one of the text types then a ValueError is raised.

class pyslet.rfc4287.Contributor(parent)¶

Bases: pyslet.rfc4287.Person

A Person construct that indicates a person or other entity who contributed to the entry or feed.

class pyslet.rfc4287.Generator(parent)¶

Bases: pyslet.rfc4287.AtomElement

Identifies the agent used to generate a feed, for debugging and other purposes.

uri = None¶: the uri of the tool used to generate the feed

version = None¶: the version of the tool used to generate the feed

SetPysletInfo()¶: Sets this generator to a default representation of this Pyslet module.

class pyslet.rfc4287.Icon(parent)¶

Bases: pyslet.rfc4287.AtomElement

Identifies an image that provides iconic visual identification for a feed.

uri = None¶: a URI instance representing the URI of the icon

GetValue()¶: Overrides GetValue(), returning a pyslet.rfc2396.URI instance.

SetValue(value)¶

Overrides SetValue(), enabling the value to be set from a pyslet.rfc2396.URI instance.

If value is a string it is used to set the element’s content, ContentChanged() is then called to update the value of uri. If value is a URI instance then uri is set directory and it is then converted to a string and used to set the element’s content.

ContentChanged()¶: Re-reads the value of the element and sets uri accordingly.

class pyslet.rfc4287.AtomId(parent, name=None)¶

Bases: pyslet.rfc4287.AtomElement

A permanent, universally unique identifier for an entry or feed.

class pyslet.rfc4287.Link(parent)¶

Bases: pyslet.rfc4287.AtomElement

A reference from an entry or feed to a Web resource.

href = None¶: a URI instance, the link’s IRI

rel = None¶: a string indicating the link relation type

type = None¶: an advisory media type

title = None¶: human-readable information about the link

class pyslet.rfc4287.Logo(parent)¶

Bases: pyslet.rfc4287.Icon

An image that provides visual identification for a feed.

class pyslet.rfc4287.Published(parent)¶

Bases: pyslet.rfc4287.Date

A Date construct indicating an instant in time associated with an event early in the life cycle of the entry.

class pyslet.rfc4287.Rights(parent)¶

Bases: pyslet.rfc4287.Text

A Text construct that conveys information about rights held in and over an entry or feed.

class pyslet.rfc4287.Subtitle(parent)¶

Bases: pyslet.rfc4287.Text

A Text construct that conveys a human-readable description or subtitle for a feed.

class pyslet.rfc4287.Summary(parent)¶

Bases: pyslet.rfc4287.Text

A Text construct that conveys a short summary, abstract, or excerpt of an entry.

class pyslet.rfc4287.Title(parent)¶

Bases: pyslet.rfc4287.Text

A Text construct that conveys a human-readable title for an entry or feed.

class pyslet.rfc4287.Updated(parent)¶

Bases: pyslet.rfc4287.Date

A Date construct indicating the most recent instant in time when an entry or feed was modified in a way the publisher considers significant.

Base Classes¶

class pyslet.rfc4287.Person(parent)¶

Bases: pyslet.rfc4287.AtomElement

An element that describes a person, corporation, or similar entity

NameClass¶: alias of Name

URIClass¶: alias of URI

class pyslet.rfc4287.Name(parent, name=None)¶

Bases: pyslet.rfc4287.AtomElement

A human-readable name for a person.

class pyslet.rfc4287.URI(parent, name=None)¶

Bases: pyslet.rfc4287.AtomElement

An IRI associated with a person

class pyslet.rfc4287.Email(parent, name=None)¶

Bases: pyslet.rfc4287.AtomElement

An e-mail address associated with a person

class pyslet.rfc4287.Text(parent)¶

Bases: pyslet.rfc4287.AtomElement

Base class for atomPlainTextConstruct and atomXHTMLTextConstruct.

SetValue(value, type=1)¶

Sets the value of the element. type must be a value from the TextType enumeration

Overloads the basic SetValue() implementation, adding an additional type attribute to enable the value to be set to either a plain TextType.text, TextType.html or TextType.xhtml value. In the case of an xhtml type, value is parsed for the required XHTML div element and this becomes the only child of the element. Given that the div itself is not considered to be part of the content the value can be given without the enclosing div, in which case it is generated automatically.

GetValue()¶

Gets a single unicode string representing the value of the element.

Overloads the basic GetValue() implementation to add support for text of type xhtml.

When getting the value of TextType.xhtml text the child div element is not returned as it is not considered to be part of the content.

class pyslet.rfc4287.TextType¶

Bases: pyslet.xsdatatypes20041028.Enumeration

text type enumeration:

"text" | "html" | "xhtml"

This enumeration is used for setting the Text.type attribute.

Usage: TextType.text, TextType.html, TextType.xhtml

class pyslet.rfc4287.Date(parent)¶

Bases: pyslet.rfc4287.AtomElement

An element conforming to the definition of date-time in RFC3339.

This class is modeled using the iso8601 module.

date = None¶: a TimePoint instance representing this date

GetValue()¶: Overrides GetValue(), returning a pyslet.iso8601.TimePoint instance.

SetValue(value)¶

Overrides SetValue(), enabling the value to be set from a pyslet.iso8601.TimePoint instance.

If value is a string the behaviour is unchanged, if value is a TimePoint instance then it is formatted using the extended format of ISO 8601 in accordance with the requirements of the Atom specification.

ContentChanged()¶: Re-reads the value of the element and sets date accordingly.

class pyslet.rfc4287.AtomElement(parent, name=None)¶

Bases: pyslet.xmlnames20091208.XMLNSElement

Base class for all APP elements.

All atom elements can have xml:base and xml:lang attributes, these are handled by the Element base class.

See GetLang() and SetLang(), GetBase() and SetBase()

Constants¶

pyslet.rfc4287.ATOM_NAMESPACE = 'http://www.w3.org/2005/Atom'¶: The namespace to use for Atom Document elements

pyslet.rfc4287.ATOM_MIMETYPE = 'application/atom+xml'¶: The mime type for Atom Document

The Atom Publishing Protocol (RFC5023)¶

This module defines functions and classes for working with the Atom Publishing Protocl as defined by RFC5023: http://www.ietf.org/rfc/rfc5023.txt

Reference¶

Elements¶

class pyslet.rfc5023.Service(parent)¶

Bases: pyslet.rfc5023.APPElement

The container for service information

Associated with one or more Workspaces.

Workspace = None¶: a list of Workspace instances

class pyslet.rfc5023.Workspace(parent)¶

Bases: pyslet.rfc5023.APPElement

Workspaces are server-defined groups of Collections.

Title = None¶: the title of this workspace

Collection = None¶: a list of Collection

class pyslet.rfc5023.Collection(parent)¶

Bases: pyslet.rfc5023.APPElement

Describes a collection (feed).

Title = None¶: the URI of the collection (feed)

Accept = None¶: the human readable title of the collection

Categories = None¶: list of Accept media ranges that can be posted to the collection

GetFeedURL()¶: Returns a fully resolved URL for the collection (feed).

class pyslet.rfc5023.Categories(parent)¶

Bases: pyslet.rfc5023.APPElement

The root of a Category Document.

A category document is a document that describes the categories allowed in a collection.

fixed = None¶: an optional URI to the category

scheme = None¶: indicates whether the list of categories is a fixed set. By default they’re open.

Category = None¶: identifies the default scheme for categories defined by this element

Base Classes¶

class pyslet.rfc5023.Accept(parent, name=None)¶

Bases: pyslet.rfc5023.APPElement

Represents the accept element.

class pyslet.rfc5023.Document(**args)¶

Bases: pyslet.rfc4287.AtomDocument

Class for working with APP documents.

This call can represent both APP and Atom documents.

ValidateMimeType(mimetype)¶: Checks mimetype against the APP or Atom specifications.

classmethod GetElementClass(name)¶

Returns the APP or Atom class used to represent name.

Overrides GetElementClass() when the namespace is APP_NAMESPACE.

class pyslet.rfc5023.APPElement(parent, name=None)¶

Bases: pyslet.xmlnames20091208.XMLNSElement

Base class for all APP elements.

All APP elements can have xml:base, xml:lang and/or xml:space attributes. These are handled by the base Element base class.

Constants¶

pyslet.rfc5023.APP_NAMESPACE = 'http://www.w3.org/2007/app'¶: The namespace to use for Atom Publishing Protocol elements

pyslet.rfc5023.ATOMSVC_MIMETYPE = 'application/atomsvc+xml'¶: The mime type for service documents

pyslet.rfc5023.ATOMCAT_MIMETYPE = 'application/atomcat+xml'¶: The mime type for category documents

ISO 8601 Dates and Times¶

This module defines special classes for handling ISO 8601 dates and times.

class pyslet.iso8601.Date(src=None, base=None, century=None, decade=None, year=None, month=None, day=None, week=None, weekday=None, ordinalDay=None, absoluteDay=None)¶

Bases: object

A class for representing ISO dates.

Values can be represent dates with reduced precision, for example:

Date(century=20,year=13,month=12)

represents December 2013, no specific day.

There are a number of different forms of the constructor based on named parameters, the simplest is:

Date(century=19,year=69,month=7,day=20)

You can also use weekday format (note that decade must be provided separately):

Date(century=19,decade=6,year=9,week=29,weekday=7)

Ordinal format (where day 1 is 1st Jan):

Date(century=19,year=69,ordinalDay=201)

Absolute format (where day 1 is the notional 1st Jan 0001):

Date(absoluteDay=718998)

An empty constructor is equivalent to:

Date()==Date(absoluteDay=1)

All constructors except the absolute form allow the passing of a base date which allows the most-significant values to be omitted, for example:

base=Date(century=19,year=69,month=7,day=20)
newDate=Date(day=21,base=base)  #: 21st July 1969

Note that base always represents a date before the newly constructed date, so:

base=Date(century=19,year=99,month=12,day=31)
newDate=Date(day=5,base=base)

constructs a Date representing the 5th January 2000

century = None¶: the century, 0..99

year = None¶: the year, 0..99

month = None¶: the month, 1..12 (for dates stored in calendar form)

week = None¶: the week (for dates stored in week form)

day = None¶: the day, 1..31 (or 1..7 when week is not None)

GetAbsoluteDay()¶: Return a notional day number - with 1 being the 0001-01-01 which is the base day of our calendar.

GetCalendarDay()¶: Returns a tuple of: (century,year,month,day)

GetOrdinalDay()¶: Returns a tuple of (century,year,ordinalDay)

GetWeekDay()¶: Returns a tuple of (century,decade,year,week,weekday), note that Monday is 1 and Sunday is 7

classmethod FromStructTime(t)¶: Constructs a Date from a struct_time, such as might be returned from time.gmtime() and related functions.

UpdateStructTime(t)¶: UpdateStructTime changes the year, month, date, wday and ydat fields of t, a struct_time, to match the values in this date.

classmethod FromNow()¶: Constructs a Date from the current local time.

Offset(centuries=0, years=0, months=0, weeks=0, days=0)¶: Constructs a Date from the current date + a given offset

classmethod from_str(src, base=None)¶: Parses a Date instance from a src string.

classmethod FromStringFormat(src, base=None)¶

Similar to from_str() except that a tuple is returned, the first item is the resulting Date instance, the second is a string describing the format parsed. For example:

d,f=Date.FromStringFormat("1969-07-20")
# f is set to "YYYY-MM-DD". 

GetCalendarString(basic=False, truncation=0)¶

Formats this date using calendar form, for example 1969-07-20

basic: True/False, selects basic form, e.g., 19690720. Default is False
truncation: One of the Truncation constants used to select truncated forms of the date. For example, if you specify Truncation.Year you’ll get –07-20 or –0720. Default is NoTruncation.

Note that Calendar format only supports Century, Year and Month truncated forms.

GetOrdinalString(basic=False, truncation=0)¶

Formats this date using ordinal form, for example 1969-201

basic: True/False, selects basic form, e.g., 1969201. Default is False
truncation: One of the Truncation constants used to select truncated forms of the date. For example, if you specify Truncation.Year you’ll get -201. Default is NoTruncation.

Note that ordinal format only supports century and year truncated forms.

GetWeekString(basic=False, truncation=0)¶

Formats this date using week form, for example 1969-W29-7

basic: True/False, selects basic form, e.g., 1969W297. Default is False
truncation: One of the Truncation constants used to select truncated forms of the date. For example, if you specify Truncation.Year you’ll get -W297. Default is NoTruncation.

Note that week format only supports century, decade, year and week truncated forms.

classmethod FromJulian(year, month, day)¶: Constructs a Date from a year, month and day expressed in the Julian calendar.

GetJulianDay()¶: Returns a tuple of: (year,month,day) representing the equivalent date in the Julian calendar.

LeapYear()¶

LeapYear returns True if this date is (in) a leap year and False otherwise.

Note that leap years fall on all years that divide by 4 except those that divide by 100 but including those that divide by 400.

Complete()¶: Returns True if this date has a complete representation, i.e., does not use one of the reduced precision forms.

GetPrecision()¶: Returns one of the Precision constants representing the precision of this date.

class pyslet.iso8601.Time(src=None, hour=None, minute=None, second=None, totalSeconds=None, zDirection=None, zHour=None, zMinute=None)¶

Bases: object

A class for representing ISO times

Values can be represent times with reduced precision, for example:

Time(hour=20)

represents 8pm without a specific minute/seconds value.

There are a number of different forms of the constructor based on named parameters, the simplest is:

Time(hour=20,minute=17,second=40)

To indicate UTC (Zulu time) by providing a zone direction of 0:

Time(hour=20,minute=17,second=40,zDirection=0)

To indicate a UTC offset provide additional values for hours (and optionally minutes):

Time(hour=15,minute=17,second=40,zDirection=-1,zHour=5,zMinute=0)

A UTC offset of 0 hours and minutes results in a value that compares as equal to the corresponding Zulu time but is formatted using an explicit offset by str() or unicode() rather than using the canonical “Z” form.

You may also specify a total number of seconds past midnight (no zone):

Time(totalSeconds=73060)

If totalSeconds overflows an error is raised. To create a time from an arbitrary number of seconds and catch overflow use Offset instead:

Time(totalSeconds=159460)
# raises DateTimeError

t,overflow=Time().Offset(seconds=159460)
# sets t to 20:40:17 and overflow=1

Time supports two representations of midnight: 00:00:00 and 24:00:00 in keeping with the ISO specification. These are considered equivalent by comparisons!

Truncated forms can be created directly from the base time, see Extend() for more information.

hour = None¶: the hour, 0..24

minute = None¶: the minute, 0..59

second = None¶: the seconds, 0..60 (to include leap seconds)

zOffset = None¶: the difference in minutes to UTC

GetTotalSeconds()¶: Note that leap seconds are handled as if they were invisible, e.g., 23:00:60 returns the same total seconds as 23:00:59.

GetTime()¶

Returns a tuple of (hour,minute,second).

Times with reduced precision will return None for second and or minute.

GetZone()¶

Returns a tuple of:

(zDirection,zOffset)

zDirection is defined as per Time’s constructor, zOffset is a non-negative integer minute offset or None, if the zone is unspecified for this Time.

GetZoneOffset()¶: Returns a single integer representing the zone offset (in minutes) or None if this time does not have a time zone offset.

GetZone3()¶

Returns a tuple of:

(zDirection,zHour,zMinute)

These values are defined as per Time’s constructor.

GetCanonicalZone()¶

Returns a tuple of:

(zDirection,zHour,zMinute)

These values are defined as per Time’s constructor but zero offsets always return zDirection=0. If present, the zone is always returned with complete (minute) precision.

GetTimeAndZone()¶: Returns a tuple of (hour,minute,second,zone direction,zone offset) as defined in GetTime and GetZone.

Extend(hour=None, minute=None, second=None)¶

Constructs a Time instance from an existing time, extended a (possibly) truncated hour/minute/second value.

The time zone is always copied if present. The result is a tuple of (<Time instance>,overflow) where overflow 0 or 1 indicating whether or not the time overflowed. For example:

# set base to 20:17:40Z
base=Time(hour=20,minute=17,second=40,zDirection=0)
t,overflow=base.Extend(minute=37)
# t is 20:37:40Z, overflow is 0
t,overflow=base.Extend(minute=7)
# t is 21:07:40Z, overflow is 0
t,overflow=base.Extend(hour=19,minute=7)
# t is 19:07:40Z, overflow is 1

Offset(hours=0, minutes=0, seconds=0)¶

Constructs a Time instance from an existing time and an offset number of hours, minutes and or seconds.

The time zone is always copied (if present). The result is a tuple of (<Time instance>,overflow) where overflow is 0 or 1 indicating whether or not the time overflowed. For example:

# set base to 20:17:40Z
base=Time(hour=20,minute=17,second=40,zDirection=0)
t,overflow=base.Offset(minutes=37)
# t is 20:54:40Z, overflow is 0
t,overflow=base.Offset(hours=4,minutes=37)
# t is 00:54:40Z, overflow is 1

WithZone(zDirection, zHour=None, zMinute=None)¶

Constructs a Time instance from an existing time but with the time zone specified.

The time zone of the existing time is ignored. Pass zDirection=None to strip the zone information completely.

ShiftZone(zDirection, zHour=None, zMinute=None)¶

Constructs a Time instance from an existing time but shifted so that it is in the time zone specified. The return value is a tuple of:

(<Time instance>, overflow)

overflow is one of -1, 0 or 1 indicating if the time over- or under-flowed as a result of the time zone shift.

classmethod FromStructTime(t)¶: Constructs a zone-less Time from a struct_time, such as might be returned from time.gmtime() and related functions.

UpdateStructTime(t)¶

UpdateStructTime changes the hour, minute, second and isdst fields of t, a struct_time, to match the values in this time.

isdst is always set to -1

classmethod FromNow()¶: Constructs a Time from the current local time.

classmethod from_str(src, base=None)¶: Constructs a Time instance from a string representation, truncated forms are returned as the earliest time on or after base and may have overflowed. See FromStringFormat() for more.

WithZoneString(zoneStr)¶: Constructs a Time instance from an existing time but with the time zone parsed from zoneStr. The time zone of the existing time is ignored.

WithZoneStringFormat(zoneStr)¶

Constructs a Time instance from an existing time but with the time zone parsed from zoneStr. The time zone of the existing time is ignored.

Returns a tuple of: (<Time instance>,format)

classmethod FromStringFormat(src, base=None)¶

Constructs a Time instance from a string representation, truncated forms are returned as the earliest time on or after base.

Returns a tuple of (<Time instance>,overflow,format) where overflow is 0 or 1 indicating whether or not a truncated form overflowed and format is a string representation of the format parsed, e.g., “hhmmss”.

GetString(basic=False, truncation=0, ndp=0, zonePrecision=7, dp=', ')¶

Formats this time, including zone, for example 20:17:40

basic: True/False, selects basic form, e.g., 201740. Default is False
truncation: One of the Truncation constants used to select truncated forms of the time. For example, if you specify Truncation.Hour you’ll get -17:40 or -1740. Default is NoTruncation.
ndp: Specifies the number of decimal places to display for the least significant component, the default is 0.
dp: The character to use as the decimal point, the default is the comma, as per the ISO standard.
zonePrecision: One of Precision.Hour or Precision.Complete to control the precision of the zone offset.

Note that time formats only support Minute and Second truncated forms.

GetZoneString(basic=False, zonePrecision=7)¶

Formats this time’s zone, for example -05:00.

basic: True/False, selects basic form, e.g., -0500. Default is False
zonePrecision: One of Precision.Hour or Precision.Complete to control the precision of the zone offset.

Times constructed with a zDirection value of 0 are always rendered using “Z” for Zulu time (the name is taken from the phonetic alphabet). To force use of the offset format you must construct the time with a non-zero value for zDirection.

Complete()¶

Returns True if this date has a complete representation, i.e., does not use one of the reduced precision forms.

(Whether or not a time is complete refers only to the precision of the time value, it says nothing about the presence or absence of a time zone offset.)

GetPrecision()¶: Returns one of the Precision constants representing the precision of this time.

WithPrecision(precision, truncate=False)¶

Constructs a Time instance from an existing time but with the precision specified by precision.

precision is one of the Precision constants, only hour, minute and complete precision are valid.

truncate is True/False indicating whether or not the time value should be truncated so that all values are integers. For example:

t=Time(hour=20,minute=17,second=40)
tm=t.WithPrecision(Precision.Minute,False)
print tm.GetString(ndp=3)
#   20:17,667
tm=t.WithPrecision(Precision.Minute,True)
print tm.GetString(ndp=3)
#   20:17,000       

class pyslet.iso8601.TimePoint(src=None, date=None, time=None)¶

Bases: object

A class for representing ISO timepoints

TimePoints are constructed from a date and a time (which may or may not have a time zone), for example:

TimePoint(date=Date(year=1969,month=7,day=20),
                time=Time(hour=20,minute=17,second=40,zDirection=0))

If the date is missing then the date origin is used, Date() or 0001-01-01. Similarly, if the time is missing then the time origin is used, Time() or 00:00:00

Times may be given with reduced precision but the date must be complete. In other words, there is no such thing as a timepoint with, month precision, use Date instead.

GetCalendarTimePoint()¶

Returns a tuple of:

(century,year,month,day,hour,minute,second)

GetOrdinalTimePoint()¶

Returns a tuple of:

(century,year,ordinalDay,hour,minute,second)

GetWeekDayTimePoint()¶

Returns a tuple of:

(century,decade,year,week,weekday,hour,minute,second)

GetZone()¶

Returns a tuple of

(zDirection,zOffset)

See Time.GetZone() for details.

WithZone(zDirection, zHour=None, zMinute=None)¶: Constructs a TimePoint instance from an existing TimePoint but with the time zone specified. The time zone of the existing TimePoint is ignored.

ShiftZone(zDirection, zHour=None, zMinute=None)¶: Constructs a TimePoint instance from an existing TimePoint but shifted so that it is in the time zone specified.

UpdateStructTime(t)¶: UpdateStructTime changes the year, month, date, hour, minute and second fields of t, a struct_time, to match the values in this date.

classmethod FromStructTime(t)¶: Constructs a TimePoint from a struct_time, such as might be returned from time.gmtime() and related functions.

classmethod from_str(src, base=None, tDesignators='T')¶: Constructs a TimePoint from a string representation. Truncated forms are parsed with reference to base.

classmethod FromStringFormat(src, base=None, tDesignators='T')¶

Similar to from_str() except that a tuple is returned, the first item is the resulting TimePoint instance, the second is a string describing the format parsed. For example:

tp,f=TimePoint.FromStringFormat("1969-07-20T20:40:17")
# f is set to "YYYY-MM-DDTmm:hh:ss".

GetCalendarString(basic=False, truncation=0, ndp=0, zonePrecision=7, dp=', ', tDesignator='T')¶

Formats this TimePoint using calendar form, for example 1969-07-20T20:17:40

basic: True/False, selects basic form, e.g., 19690720T201740. Default is False
truncation: One of the Truncation constants used to select truncated forms of the date. For example, if you specify Truncation.Year you’ll get –07-20T20:17:40 or –0720T201740. Default is NoTruncation. Note that Calendar format only :supports Century, Year and Month truncated forms, the time component cannot be truncated.
ndp, dp and zonePrecision: As specified in Time.GetString()

GetOrdinalString(basic=0, truncation=0, ndp=0, zonePrecision=7, dp=', ', tDesignator='T')¶

Formats this TimePoint using ordinal form, for example 1969-201T20-17-40

basic: True/False, selects basic form, e.g., 1969201T201740. Default is False
truncation: One of the Truncation constants used to select truncated forms of the date. For example, if you specify Truncation.Year you’ll get -201T20-17-40. Default is NoTruncation. Note that ordinal format only supports century and year truncated forms, the time component cannot be truncated.
ndp, dp and zonePrecision: As specified in Time.GetString()

GetWeekString(basic=0, truncation=0, ndp=0, zonePrecision=7, dp=', ', tDesignator='T')¶

Formats this TimePoint using week form, for example 1969-W29-7T20:17:40

basic: True/False, selects basic form, e.g., 1969W297T201740. Default is False
truncation: One of the Truncation constants used to select truncated forms of the date. For example, if you specify Truncation.Year you’ll get -W297T20-17-40. Default is NoTruncation. Note that week format only supports century, decade, year and week truncated forms, the time component cannot be truncated.
ndp, dp and zonePrecision: As specified in Time.GetString()

classmethod FromUnixTime(unixTime)¶

Constructs a TimePoint from unixTime, the number of seconds since the time origin. The resulting time has no zone.

This method uses python’s gmtime(0) to obtain the Unix origin time.

get_unixtime()¶: Returns a unix time value representing this time point.

classmethod FromNow()¶: Constructs a TimePoint from the current local date and time.

classmethod FromNowUTC()¶: Constructs a TimePoint from the current UTC date and time.

Complete()¶

Returns True if this TimePoint has a complete representation, i.e., does not use one of the reduced precision forms.

(Whether or not a TimePoint is complete refers only to the precision of the time value, it says nothing about the presence or absence of a time zone offset.)

GetPrecision()¶: Returns one of the Precision constants representing the precision of this TimePoint.

WithPrecision()¶: Constructs a TimePoint instance from an existing TimePoint but with the precision specified by precision. For more details see Time.WithPrecision()

class pyslet.iso8601.Duration(value=None)¶: A class for representing ISO durations

Supporting Constants¶

class pyslet.iso8601.Truncation¶

Defines constants to use when formatting to truncated forms.

No = 0¶: constant for no truncation

Year = 3¶: constant for truncating to year

Month = 4¶: constant for truncating to month

Week = 5¶: constant for truncating to week

Hour = 6¶: constant for truncating to hour

class pyslet.iso8601.Precision¶

Defines constants for representing reduced precision.

Year = 2¶: constant for year precision

Month = 3¶: constant for month precision

Week = 4¶: constant for week precision

Hour = 5¶: constant for hour precision

Utility Functions¶

pyslet.iso8601.LeapYear(year)¶

LeapYear returns True if year is a leap year and False otherwise.

Note that leap years famously fall on all years that divide by 4 except those that divide by 100 but including those that divide by 400.

pyslet.iso8601.DayOfWeek(year, month, day)¶: DayOfWeek returns the day of week 1-7, 1 being Monday for the given year, month and day

pyslet.iso8601.WeekCount(year)¶: Week count returns the number of calendar weeks in a year. Most years have 52 weeks of course, but if the year begins on a Thursday or a leap year begins on a Wednesday then it has 53.

Compatibility¶

Pyslet requires Python 2.6 or Python 2.7, with Python 2.7 being preferred.

Python 2.6¶

When run under Python 2.6 Pyslet will patch the following modules to make them more compatible with Python 2.7 code.

zipfile: Patches is_zipfile to add support for passing open files which is allowed under Python 2.7 but not under 2.6.
wsgiref.simple_server: Modifies the behaviour of the WSGI server when procssing HEAD requests so that Content-Length headers are not stripped. There is an issue in Python 2.6 that causes HEAD requests to return a Content-Length of 0 if the WSGI application does not return any data. The behaviour changed in Python 2.7 to be more as expected.
io: Benign addition of the SEEK_* constants as defined in Python 2.7.

This patching is done at run time by the pyslet.py26 module and will affect any script that uses Pyslet. It does not modify your Python installation!

Earlier versions of Python 2.6 have typically been built with a version of sqlite3 that does not support validation of foreign key constraints, the unittests have been designed to skip these tests when such a version is encountered.

Note

When run under Python 2.6, Pyslet may not support certificate validation of HTTP connections properly, this seems to depend on the version of OpenSSL that Python is linked to. If you have successfully used pip to install Pyslet then your Python is probably unaffected though.

Please be aware of the following bug in Python 2.6: http://bugs.python.org/issue2531 this problem caused a number of Pyslet’s tests to fail initially and remains a potential source of problems if you are using Decimal types in OData models.

Python 3¶

Pyslet is not currently compatible with Python 3, though some work has been towards a Python 3 version.

PEP-8¶

The code is not currently PEP-8 compliant but it is slowly being refactored for compliance as modules are touched during development. Where critical methods are renamed from CamelCase to PEP-8 compliant lower_case_form then the old names are defined as wrappers which raise deprecation warnings.

Distribution¶

Pyslet is developed on github: https://github.com/swl10/pyslet but it can be downloaded and installed from the popular PyPi package distribution site: https://pypi.python.org/pypi/pyslet using pip.

Users of older Python builds (e.g., the current Python 2.6 installed on OS X as of August 2014) should be aware that pip may well fell to install itself or other modules due to a failure to connect to the PyPi repository. Fixing this is hard and installing from source is recommended instead if you are afflicted by this issue.

Pyslet is distributed under the ‘New’ BSD license: http://opensource.org/licenses/BSD-3-Clause

Installing from Source¶

The Pyslet package contains a setup.py script so you can install it by downloading the compressed archive, uncompressing it and then running the following command inside the package:

python setup.py install

Format of the Documentation¶

The documentation has been written using ReStructuredText, a simple format created as part of the docutils package on SourceForge. The documentation files you are most likely reading have been generated using Sphinx. Parts of the documentation are auto-generated from the Python source files to make it easier to automatically discover the documentation using other tools capable of reading Python docstrings. However, this requires that the docstrings be written using ReStructuredText too, which means there is some additional markup for python-cross referencing in the code that may not be interpretable by other system (see below for details).

ReStructuredText Primer: http://docutils.sourceforge.net/docs/user/rst/quickstart.html
- Quick Reference: http://docutils.sourceforge.net/docs/user/rst/quickref.html
- Sphinx: http://sphinx.pocoo.org/
  - Autodoc externsion: http://sphinx.pocoo.org/ext/autodoc.html
  - Python-cross references: http://sphinx.pocoo.org/domains.html#python-roles