logo

Pydoc Filter

← Back to Filter List

Pydoc


Returns introspected python data in key-value storage format. Where input is a .txt file, this is assumed to be the name of an installed python module. Where input is a .py file, the file itself is loaded and parsed.

Aliases for this filter

  • pydoc

Converts from file formats:

  • .txt
  • .py

To file formats:

  • .sqlite3
  • .json

Available settings:

SettingDescriptionDefault
add-new-filesBoolean or list of extensions/patterns to match.False
added-in-versionDexy version when this filter was first available.
additional-dirsAdditional source directories to load, relative to package root. Useful for tests/[]
additional-doc-filtersFilters to apply to additional documents created as side effects.{}
additional-doc-settingsSettings to apply to additional documents created as side effects.{}
data-typeAlias of custom data class to use to store filter output.keyvalue
error-on-import-failShould an exception be raised if importing a specified module or file fails?False
examplesTemplates which should be used as examples for this filter.[]
exclude-add-new-filesList of patterns to skip even if they match add-new-files.[]
exclude-new-files-from-dirList of directories to skip when adding new files.[]
extFile extension to output.None
extension-mapDictionary mapping input extensions to default output extensions.None
helpHelpstring for plugin.Returns introspected python data in key-value storage format. Where input is a .txt file, this is assumed to be the name of an installed python module. Where input is a .py file, the file itself is loaded and parsed.
input-extensionsList of extensions which this filter can accept as input.[u'.txt', u'.py']
keep-originalsWhether, if additional-doc-filters are specified, the original unmodified docs should also be added.False
mkdirA directory which should be created in working dir.None
mkdirsA list of directories which should be created in working dir.[]
nodocWhether filter should be excluded from documentation.False
outputWhether to output results of this filter by default by reporters such as 'output' or 'website'.False
output-extensionsList of extensions which this filter can produce as output.[u'.sqlite3', u'.json']
override-workspace-exclude-filtersIf True, document will be populated to other workspaces ignoring workspace-exclude-filters.False
preserve-prior-data-classWhether output data class should be set to match the input data class.False
require-outputShould dexy raise an exception if no output is produced by this filter?True
tagsTags which describe the filter.[]
variablesA dictionary of variable names and values to make available to this filter.{}
varsA dictionary of variable names and values to make available to this filter.{}
workspace-exclude-filtersFilters whose output should be excluded from workspace.[u'pyg']
workspace-includesIf set to a list of filenames or extensions, only these will be populated to working dir.None
Filter Source Code
class Pydoc(PythonIntrospection):
    """
    Returns introspected python data in key-value storage format.

    Where input is a .txt file, this is assumed to be the name of an installed
    python module.

    Where input is a .py file, the file itself is loaded and parsed.
    """
    aliases = ["pydoc"]
    _settings = {
            'additional-dirs' : ("Additional source directories to load, relative to package root. Useful for tests/", [])
            }

    def append_item_content(self, key, item):
        self.log_debug("appending content for %s" % key)

        try:
            source = inspect.getsource(item)
            self.output_data.append("%s:source" % key, source)
        except (TypeError, IOError, sqlite3.ProgrammingError):
            pass

        try:
            doc = inspect.getdoc(item)
            self.output_data.append("%s:doc" % key, doc)
        except (TypeError, IOError, sqlite3.ProgrammingError):
            pass

        try:
            comment = inspect.getcomments(item)
            self.output_data.append("%s:comments" % key, comment)
        except (TypeError, IOError, sqlite3.ProgrammingError):
            pass

        try:
            value = json.dumps(item)
            self.output_data.append("%s:value" % key, value)
        except TypeError:
            pass

    def is_defined_in_module(self, mod, mod_name, item):
        if mod_name and hasattr(item, '__module__'):
            return item.__module__.startswith(mod_name)
        else:
            return True

    def process_members(self, mod):
        mod_name = mod.__name__

        if mod_name == 'dummy':
            mod_name = None

        for k, m in inspect.getmembers(mod):
            if mod_name:
                key = "%s.%s" % (mod_name, k)
            else:
                key = k

            is_class = inspect.isclass(m)
            is_def = self.is_defined_in_module(mod, mod_name, m)

            if not is_def:
                # this is something imported, not defined in the module
                # so we don't want to document it here
                self.log_debug("skipping %s for module %s" % (k, mod_name))
                continue

            if not is_class:
                self.append_item_content(key, m)

            else:
                self.append_item_content(key, m)
                for ck, cm in inspect.getmembers(m):
                    self.append_item_content("%s.%s" % (key, ck), cm)

    def process_module(self, package_name, name):
        self.log_debug("processing module %s" % name)
        mod = self.load_module(name)
        self.append_item_content(name, mod)
        if mod:
            self.process_members(mod)
        else:
            self.log_warn("no mod from %s" % name)

    def process_package(self, package):
        """
        Iterates over all modules included in the package and processes them.
        """
        self.log_debug("processing package %s" % package)
        package_name = package.__name__

        # Process top level package
        self.process_module(package_name, package_name)

        # Process sub-packages and modules
        if hasattr(package, '__path__'):
            path = package.__path__
            prefix = "%s." % package_name
            for loader, name, ispkg in pkgutil.walk_packages(path, prefix=prefix):
                self.process_module(package_name, name)

    def process_packages(self):
        package_names = unicode(self.input_data).split()
        packages = [__import__(name) for name in package_names]

        for package in packages:
            self.process_package(package)

    def process_file(self):
        mod = self.load_source_file()
        if mod:
            self.process_members(mod)

    def process(self):
        if self.prev_ext == '.txt':
            self.process_packages()
        elif self.prev_ext == '.py':
            self.process_file()
        else:
            raise InternalDexyProblem("Should not have ext %s" % self.prev_ext)

        self.output_data.save()

Content © 2013 Dr. Ana Nelson | Site Design © Copyright 2011 Andre Gagnon | All Rights Reserved.