idio

↓ examples

Splits files into sections based on comments like ### "foo" Replacement for idiopidae. Should be fully backwards-compatible. For more information about the settings starting with ply-, see the PLY YACC parser documentation http://www.dabeaz.com/ply/ply.html#ply_nn36

Aliases for this filter

idio
id
idiopidae
htmlsections

Converts from file formats:

.*

To file formats:

.html
.tex
.svg
.txt
.png
.bmp
.gif
.jpg

Available settings:

Setting	Description	Default
add-new-files	Boolean or list of extensions/patterns to match.	False
added-in-version	Dexy version when this filter was first available.
additional-doc-filters	Filters to apply to additional documents created as side effects.	{}
additional-doc-settings	Settings to apply to additional documents created as side effects.	{}
allow-unknown-ext	Whether to allow unknown file extensions to be parsed with the TextLexer by default instead of raising an exception.	True
allow-unprintable-input	Whether to allow unprintable input to be replaced with dummy text instead of raising an exception.	True
data-type	Alias of custom data class to use to store filter output.	generic
examples	Templates which should be used as examples for this filter.	['idio', 'htmlsections']
exclude-add-new-files	List of patterns to skip even if they match add-new-files.	[]
exclude-new-files-from-dir	List of directories to skip when adding new files.	[]
ext	File extension to output.	None
extension-map	Dictionary mapping input extensions to default output extensions.	None
formatter-settings	List of all settings which will be passed to the formatter constructor.	['style', 'full', 'linenos', 'noclasses']
full	Pygments formatter option: output a 'full' document including header/footer tags.	None
help	Helpstring for plugin.	Splits files into sections based on comments like ### "foo" Replacement for idiopidae. Should be fully backwards-compatible. For more information about the settings starting with ply-, see the PLY YACC parser documentation http://www.dabeaz.com/ply/ply.html#ply_nn36
highlight	Whether to apply syntax highlighting to sectional output.	None
input-extensions	List of extensions which this filter can accept as input.	['.*']
keep-originals	Whether, if additional-doc-filters are specified, the original unmodified docs should also be added.	False
lexer	The name of the pygments lexer to use (will normally be determined automatically, only use this if you need to override the default setting or your filename isn't mapped to the lexer you want to use.	None
lexer-args	Dictionary of custom arguments to be passed directly to the lexer.	{}
lexer-settings	List of all settings which will be passed to the lexer constructor.	[]
line-numbers	Alternative name for 'linenos'.	None
linenos	Whether to include line numbers. May be set to 'table' or 'inline'.	None
mkdir	A directory which should be created in working dir.	None
mkdirs	A list of directories which should be created in working dir.	[]
noclasses	If set to true, token tags will not use CSS classes, but inline styles.	None
nodoc	Whether filter should be excluded from documentation.	False
output	Whether to output results of this filter by default by reporters such as 'output' or 'website'.	False
output-extensions	List of extensions which this filter can produce as output.	['.html', '.tex', '.svg', '.txt', '.png', '.bmp', '.gif', '.jpg']
override-workspace-exclude-filters	If True, document will be populated to other workspaces ignoring workspace-exclude-filters.	False
ply-lextab	Name of lexer tabfile (.py extension will be added) to be stored in ply-outputdir if ply-optimize is set to 1.	id_lextab
ply-optimize	Whether to use optimized mode for the lexer.	1
ply-outputdir	Location relative to where you run dexy in which ply will store table files. Defaults to dexy's log directory.	None
ply-parsetab	Name of parser tabfile (.py extension will be added) to be stored in ply-outputdir if ply-write-tables set to 1.	id_parsetab
ply-write-tables	Whether to generate parser table files (which will be stored in ply-outputdir and named ply-tabmodule).	1
preserve-prior-data-class	Whether output data class should be set to match the input data class.	False
remove-leading	If a document starts with empty section named '1', remove it.	False
require-output	Should dexy raise an exception if no output is produced by this filter?	True
skip-extensions	Because \|idio gets applied to ., need to make it easy to skip non-textual files.	.odt
style	Formatter style to output.	default
tags	Tags which describe the filter.	[]
unprintable-input-text	Dummy text to use instead of unprintable binary input.	not printable
variables	A dictionary of variable names and values to make available to this filter.	{}
vars	A dictionary of variable names and values to make available to this filter.	{}
workspace-exclude-filters	Filters whose output should be excluded from workspace.	['pyg']
workspace-includes	If set to a list of filenames or extensions, only these will be populated to working dir.	None

Idio Example

The idio filter splits source code according to comments found in the code. Many different comment formats are supported, please let us know if you need additional formats supported for your programming language.

Shortcut Quoted Form

You can write idio comments by using three comment characters and putting the section name in quotes, like this:

### "foo"
x = 6

### "bar"
y = 7

The resulting section names are:

1, foo, bar

For languages which support // as a comment char, just use three slashes instead of two:

/// "include"
#include <stdio.h>

/// "main"
int main(int argc, char *argv[]) {
    puts("Hello world!");
    return 0;
}

Here are the section names:

1, include, main

Here is the script output:

Hello world!

(A /* */ multiline-style comment is also supported, see the next section.)

The ; comment character is supported for Clojure:

;;; "foo"
(println "hello")

;;; "bar"
(println "world")

Here is the output from running the full clojure script:

hello
world

Here is the REPL output from just the foo section of the clojure script:

user=> (println "hello")
hello
nil

The % comment character is supported, for LaTeX:

%%% "foo"
Some LaTeX.

%%% "bar"
Some more LaTeX.

Here are the section names:

1, foo, bar

Long Form

The original idio syntax uses the @export keyword followed by a name, which need not be in quotes. This format is still supported:

### @export foo
x = 6

### @export "bar"
y = 7

The resulting section names are:

1, foo, bar

To use a multi-line style comment, use three initial asterisks and the @export keyword:

/*** @export include */
#include <stdio.h>

/*** @export main */
int main(int argc, char *argv[]) {
    puts("Hello world!");
    return 0;
}

Here are the section names:

1, include, main

Here is the script output:

Hello world!

HTML style comments are also supported:

<!-- @export foo -->
foo

<!-- @export "bar" -->
bar

Here are the section names:

1, foo, bar

htmlsections Filter

Here is some HTML split into sections:

<html>
    <head>
    </head>
    <body>
        <!-- section "foo" -->
        <div style="border: thin solid blue; width: 100px;">
            <p>hello!</p>
        </div>
        <!-- section "bar" -->
        <div style="border: thick dotted red; width: 100px;">
            <p>how are you?</p>
        </div>
        <!-- section "end" -->
    </body>
</html>

Here is another HTML document which includes some sections:

<html>
    <body>
        Here is the 'foo' section:

        {{ d['sections.html|htmlsections']['foo'] }}

        Here is the 'bar' section:

        {{ d['sections.html|htmlsections']['bar'] }}
    </body>
</html>

After running dexy, the sections are included:

<html>
    <body>
        Here is the 'foo' section:

                <div style="border: thin solid blue; width: 100px;">
            <p>hello!</p>
        </div>


        Here is the 'bar' section:

                <div style="border: thick dotted red; width: 100px;">
            <p>how are you?</p>
        </div>

    </body>
</html>

Here is how we specify this:

filters:
    - docs.html|jinja:
        - sections.html|htmlsections

Filter Source Code

class Id(PygmentsFilter):
    """
    Splits files into sections based on comments like ### "foo"

    Replacement for idiopidae. Should be fully backwards-compatible.

    For more information about the settings starting with ply-, see the PLY
    YACC parser documentation http://www.dabeaz.com/ply/ply.html#ply_nn36
    """
    aliases = ['idio', 'id', 'idiopidae', 'htmlsections']
    _settings = {
            'examples' : ['idio', 'htmlsections'],
            'highlight' : ("Whether to apply syntax highlighting to sectional output.", None),
            'skip-extensions' : ("Because |idio gets applied to *.*, need to make it easy to skip non-textual files.", (".odt")),
            'remove-leading' : ("If a document starts with empty section named '1', remove it.", False),
            'ply-optimize' : ("Whether to use optimized mode for the lexer.", 1),
            'ply-write-tables' : ("Whether to generate parser table files (which will be stored in ply-outputdir and named ply-tabmodule).", 1),
            'ply-outputdir' : ("Location relative to where you run dexy in which ply will store table files. Defaults to dexy's log directory.", None),
            'ply-parsetab' : ("Name of parser tabfile (.py extension will be added) to be stored in ply-outputdir if ply-write-tables set to 1.", 'id_parsetab'),
            'ply-lextab' : ("Name of lexer tabfile (.py extension will be added) to be stored in ply-outputdir if ply-optimize is set to 1.", 'id_lextab'),
            'output-extensions' : PygmentsFilter.MARKUP_OUTPUT_EXTENSIONS + PygmentsFilter.IMAGE_OUTPUT_EXTENSIONS
            }

    def process(self):
        try:
            input_text = str(self.input_data)
        except UnicodeDecodeError:
            self.output_data['1'] = "non textual"
            self.output_data.save()
            return

        lexer.outputdir = self.setting('ply-outputdir')
        lexer.errorlog = self.doc.wrapper.log
        lexer.remove_leading = self.setting('remove-leading')
        parser.outputdir = self.setting('ply-outputdir')
        parser.errorlog = self.doc.wrapper.log
        parser.write_tables = self.setting('ply-write-tables')

        _lexer = lexer.clone()
        _lexer.sections = []
        _lexer.level = 0
        start_new_section(_lexer, 0, 0, _lexer.level)

        parser.parse(input_text + "\n", lexer=_lexer)
        strip_trailing_newline(_lexer)
        parser_output = _lexer.sections

        pyg_lexer = self.create_lexer_instance()
        pyg_formatter = self.create_formatter_instance()

        # TODO fix file extension if highlight is set to false
        do_highlight = self.setting('highlight')
        if do_highlight is None:
            if self.alias in ('htmlsections',):
                do_highlight = False
                if self.output_data.setting('canonical-output') is None:
                    self.output_data.update_settings({'canonical-output' : True})
            else:
                do_highlight = True

        for section in parser_output:
            if do_highlight:
                section['contents'] = highlight(section['contents'], pyg_lexer, pyg_formatter)
            self.output_data._data.append(section)
        self.output_data.save()

idio filter

idio

Aliases for this filter

Converts from file formats:

To file formats:

Available settings:

Idio Example

Shortcut Quoted Form

Long Form

htmlsections Filter

Filter Source Code

How to

Contact Us

Quick Note

About Dexy

Dexy is a free-form literate documentation tool for writing any kind of technical document incorporating code. Dexy helps you write correct documents, and to easily maintain them over time as your code changes.

Latest News

From Twitter

Thanks To

Dexy Sponsors