logo

Idio Filter

← Back to Filter List

Idio


↓ examples

Splits files into sections based on comments like ### "foo" Replacement for idiopidae. Should be fully backwards-compatible. For more information about the settings starting with ply-, see the PLY YACC parser documentation http://www.dabeaz.com/ply/ply.html#ply_nn36

Aliases for this filter

  • idio
  • id
  • idiopidae
  • htmlsections

Converts from file formats:

  • .*

To file formats:

  • .html
  • .tex
  • .svg
  • .txt
  • .png
  • .bmp
  • .gif
  • .jpg

Available settings:

SettingDescriptionDefault
add-new-filesBoolean or list of extensions/patterns to match.False
added-in-versionDexy version when this filter was first available.
additional-doc-filtersFilters to apply to additional documents created as side effects.{}
additional-doc-settingsSettings to apply to additional documents created as side effects.{}
allow-unknown-extWhether to allow unknown file extensions to be parsed with the TextLexer by default instead of raising an exception.True
allow-unprintable-inputWhether to allow unprintable input to be replaced with dummy text instead of raising an exception.True
data-typeAlias of custom data class to use to store filter output.generic
examplesTemplates which should be used as examples for this filter.[u'htmlsections']
exclude-add-new-filesList of patterns to skip even if they match add-new-files.[]
exclude-new-files-from-dirList of directories to skip when adding new files.[]
extFile extension to output.None
extension-mapDictionary mapping input extensions to default output extensions.None
formatter-settingsList of all settings which will be passed to the formatter constructor.[u'style', u'full', u'linenos', u'noclasses']
fullPygments formatter option: output a 'full' document including header/footer tags.None
helpHelpstring for plugin.Splits files into sections based on comments like ### "foo" Replacement for idiopidae. Should be fully backwards-compatible. For more information about the settings starting with ply-, see the PLY YACC parser documentation http://www.dabeaz.com/ply/ply.html#ply_nn36
highlightWhether to apply syntax highlighting to sectional output.None
input-extensionsList of extensions which this filter can accept as input.[u'.*']
keep-originalsWhether, if additional-doc-filters are specified, the original unmodified docs should also be added.False
lexerThe name of the pygments lexer to use (will normally be determined automatically, only use this if you need to override the default setting or your filename isn't mapped to the lexer you want to use.None
lexer-argsDictionary of custom arguments to be passed directly to the lexer.{}
lexer-settingsList of all settings which will be passed to the lexer constructor.[]
line-numbersAlternative name for 'linenos'.None
linenosWhether to include line numbers. May be set to 'table' or 'inline'.None
mkdirA directory which should be created in working dir.None
mkdirsA list of directories which should be created in working dir.[]
noclassesIf set to true, token tags will not use CSS classes, but inline styles.None
nodocWhether filter should be excluded from documentation.False
outputWhether to output results of this filter by default by reporters such as 'output' or 'website'.False
output-extensionsList of extensions which this filter can produce as output.[u'.html', u'.tex', u'.svg', u'.txt', u'.png', u'.bmp', u'.gif', u'.jpg']
override-workspace-exclude-filtersIf True, document will be populated to other workspaces ignoring workspace-exclude-filters.False
ply-debugThe 'debug' setting to pass to PLY. A setting of 1 will produce very verbose output.0
ply-lextabName of lexer tabfile (.py extension will be added) to be stored in ply-outputdir if ply-optimize is set to 1.id_lextab
ply-optimizeWhether to use optimized mode for the lexer.1
ply-outputdirLocation relative to where you run dexy in which ply will store table files. Defaults to dexy's log directory.None
ply-parsetabName of parser tabfile (.py extension will be added) to be stored in ply-outputdir if ply-write-tables set to 1.id_parsetab
ply-write-tablesWhether to generate parser table files (which will be stored in ply-outputdir and named ply-tabmodule).1
preserve-prior-data-classWhether output data class should be set to match the input data class.False
remove-leadingIf a document starts with empty section named '1', remove it.False
require-outputShould dexy raise an exception if no output is produced by this filter?True
skip-extensionsBecause |idio gets applied to *.*, need to make it easy to skip non-textual files..odt
styleFormatter style to output.default
tagsTags which describe the filter.[]
unprintable-input-textDummy text to use instead of unprintable binary input.not printable
variablesA dictionary of variable names and values to make available to this filter.{}
varsA dictionary of variable names and values to make available to this filter.{}
workspace-exclude-filtersFilters whose output should be excluded from workspace.[u'pyg']
workspace-includesIf set to a list of filenames or extensions, only these will be populated to working dir.None

htmlsections Filter

Here is some HTML split into sections:

<html>
    <head>
    </head>
    <body>
        <!-- section "foo" -->
        <div style="border: thin solid blue; width: 100px;">
            <p>hello!</p>
        </div>
        <!-- section "bar" -->
        <div style="border: thick dotted red; width: 100px;">
            <p>how are you?</p>
        </div>
        <!-- section "end" -->
    </body>
</html>

Here is another HTML document which includes some sections:

<html>
    <body>
        Here is the 'foo' section:

        {{ d['sections.html|htmlsections']['foo'] }}

        Here is the 'bar' section:

        {{ d['sections.html|htmlsections']['bar'] }}
    </body>
</html>

After running dexy, the sections are included:

<html>
    <body>
        Here is the 'foo' section:

                <div style="border: thin solid blue; width: 100px;">
            <p>hello!</p>
        </div>


        Here is the 'bar' section:

                <div style="border: thick dotted red; width: 100px;">
            <p>how are you?</p>
        </div>

    </body>
</html>

Here is how we specify this:

filters:
    - docs.html|jinja:
        - sections.html|htmlsections
Filter Source Code
class Id(PygmentsFilter):
    """
    Splits files into sections based on comments like ### "foo"

    Replacement for idiopidae. Should be fully backwards-compatible.

    For more information about the settings starting with ply-, see the PLY
    YACC parser documentation http://www.dabeaz.com/ply/ply.html#ply_nn36
    """
    aliases = ['idio', 'id', 'idiopidae', 'htmlsections']
    _settings = {
            'examples' : ['htmlsections'],
            'highlight' : ("Whether to apply syntax highlighting to sectional output.", None),
            'skip-extensions' : ("Because |idio gets applied to *.*, need to make it easy to skip non-textual files.", (".odt")),
            'remove-leading' : ("If a document starts with empty section named '1', remove it.", False),
            'ply-debug' : ("The 'debug' setting to pass to PLY. A setting of 1 will produce very verbose output.", 0),
            'ply-optimize' : ("Whether to use optimized mode for the lexer.", 1),
            'ply-write-tables' : ("Whether to generate parser table files (which will be stored in ply-outputdir and named ply-tabmodule).", 1),
            'ply-outputdir' : ("Location relative to where you run dexy in which ply will store table files. Defaults to dexy's log directory.", None),
            'ply-parsetab' : ("Name of parser tabfile (.py extension will be added) to be stored in ply-outputdir if ply-write-tables set to 1.", 'id_parsetab'),
            'ply-lextab' : ("Name of lexer tabfile (.py extension will be added) to be stored in ply-outputdir if ply-optimize is set to 1.", 'id_lextab'),
            'output-extensions' : PygmentsFilter.MARKUP_OUTPUT_EXTENSIONS + PygmentsFilter.IMAGE_OUTPUT_EXTENSIONS
            }

    def process(self):
        try:
            input_text = unicode(self.input_data)
        except UnicodeDecodeError:
            self.output_data['1'] = "non textual"
            self.output_data.save()
            return

        lexer.outputdir = self.setting('ply-outputdir')
        lexer.errorlog = self.doc.wrapper.log
        lexer.remove_leading = self.setting('remove-leading')
        parser.outputdir = self.setting('ply-outputdir')
        parser.errorlog = self.doc.wrapper.log
        parser.write_tables = self.setting('ply-write-tables')

        _lexer = lexer.clone()
        _lexer.sections = []
        _lexer.level = 0
        start_new_section(_lexer, 0, 0, _lexer.level)

        parser.parse(input_text + "\n", lexer=_lexer)
        strip_trailing_newline(_lexer)
        parser_output = _lexer.sections

        pyg_lexer = self.create_lexer_instance()
        pyg_formatter = self.create_formatter_instance()

        # TODO fix file extension if highlight is set to false
        do_highlight = self.setting('highlight')
        if do_highlight is None:
            if self.alias in ('htmlsections',):
                do_highlight = False
                if self.output_data.setting('canonical-output') is None:
                    self.output_data.update_settings({'canonical-output' : True})
            else:
                do_highlight = True

        for section in parser_output:
            if do_highlight:
                section['contents'] = highlight(section['contents'], pyg_lexer, pyg_formatter)
            self.output_data._data.append(section)
        self.output_data.save()

Content © 2013 Dr. Ana Nelson | Site Design © Copyright 2011 Andre Gagnon | All Rights Reserved.