nvbn/py-backwards

•Updated over 9 years ago

Docker image for py-backwards

Image

1.5K

nvbn/py-backwards repository overview

⁠Py-backwards

Python to python compiler that allows you to use some Python 3.6 features in older versions, you can try it in the online demo⁠.

Requires Python 3.3+ to run, can compile down to 2.7.

⁠Supported features

Target 3.5:

formatted string literals⁠ like f'hi {x}'
variables annotations⁠ like x: int = 10 and x: int
underscores in numeric literals⁠ like 1_000_000 (works automatically)

Target 3.4:

starred unpacking⁠ like [*range(1, 5), *range(10, 15)] and print(*[1, 2], 3, *[4, 5])
dict unpacking⁠ like {1: 2, **{3: 4}}

Target 3.3:

import pathlib2⁠ instead of pathlib

Target 3.2:

Target 2.7:

functions annotations⁠ like def fn(a: int) -> str
imports from __future__⁠
super without arguments⁠
classes without base like class A: pass
imports from six moves⁠
metaclass
string/unicode literals (works automatically)
str to unicode
define encoding (not transformer)
dbm => anydbm and dbm.ndbm => dbm

For example, if you have some python 3.6 code, like:

def returning_range(x: int):
    yield from range(x)
    return x


def x_printer(x):
    val: int
    val = yield from returning_range(x)
    print(f'val {val}')


def formatter(x: int) -> dict:
    items: list = [*x_printer(x), x]
    print(*items, *items)
    return {'items': items}


result = {'x': 10, **formatter(10)}
print(result)


class NumberManager:
    def ten(self):
        return 10

    @classmethod
    def eleven(cls):
        return 11


class ImportantNumberManager(NumberManager):
    def ten(self):
        return super().ten()

    @classmethod
    def eleven(cls):
        return super().eleven()


print(ImportantNumberManager().ten())
print(ImportantNumberManager.eleven())

You can compile it for python 2.7 with:

➜ py-backwards -i input.py -o output.py -t 2.7

Got some ugly code⁠ and ensure that it works:

➜ python3.6 input.py
val 10
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
{'x': 10, 'items': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}
10
11
➜ python2 output.py                           
val 10
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
{'x': 10, 'items': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}
10
11

⁠Usage

Installation:

pip install py-backwards

Compile code:

py-backwards -i src -o compiled -t 2.7

⁠Testing compiled code

For testing compiled code with each supported python version you can use tox⁠ and tox-py-backwards⁠. You need to install them:

pip install tox tox-py-backwards

Fill tox.ini (py_backwards = true in testenv section enables py-backwards), like:

[tox]
envlist = py27,py33,py34,py35,py36

[testenv]
deps = pytest
commands = py.test
py_backwards = true

And run tests with:

tox

⁠Distributing compiled code

For distributing packages compiled with py-backwards you can use py-backwards-packager⁠. Install it with:

pip install py-backwards-packager

And change setup import in setup.py to:

try:
    from py_backwards_packager import setup
except ImportError:
    from setuptools import setup

By default all targets enabled, but you can limit them with:

setup(...,
      py_backwards_targets=['2.7', '3.3'])

After that your code will be automatically compiled on bdist and bdist_wheel.

⁠Running on systems without Python 3.3+

You can use docker for running py-backwards on systems without Python 3.3+, for example for testing on travis-ci with Python 2.7:

docker run -v $(pwd):/data/ nvbn/py-backwards -i example -o out -t 2.7

⁠Development

Setup:

pip install .
python setup.py develop
pip install -r requirements.txt

Run tests:

 py.test -vvvv --capture=sys --enable-functional

Run tests on systems without docker:

 py.test -vvvv

⁠Writing code transformers

First of all, you need to inherit from BaseTransformer, BaseNodeTransformer (if you want to use NodeTransfromer⁠ interface), or BaseImportRewrite (if you want just to change import).

If you use BaseTransformer, override class method def transform(cls, tree: ast.AST) -> TransformationResult, like:

from ..types import TransformationResult
from .base import BaseTransformer


class MyTransformer(BaseTransformer):
    @classmethod
    def transform(cls, tree: ast.AST) -> TransformationResult:
        return TransformationResult(tree=tree,
                                    tree_changed=True,
                                    dependencies=[])

If you use BaseNodeTransformer, override visit_* methods, for simplification this class have a whole tree in self._tree, you should also set self._tree_changed = True if the tree was changed:

from .base import BaseNodeTransformer


class MyTransformer(BaseNodeTransformer):
    dependencies = []  # additional dependencies

    def visit_FunctionDef(self, node: ast.FunctionDef) -> ast.FunctionDef:
        self._tree_changed = True  # Mark that transformer changed tree
        return self.generic_visit(node)

If you use BaseImportRewrite, just override rewrites, like:

from .base import BaseImportRewrite


class MyTransformer(BaseImportRewrite):
    dependencies = ['pathlib2']

    rewrites = [('pathlib', 'pathlib2')]

After that you need to add your transformer to transformers.__init__.transformers.

It's hard to write code in AST, because of that we have snippets⁠:

from ..utils.snippet import snippet, let, extend


@snippet
def my_snippet(class_name, class_body):
    class class_name:  # will be replaced with `class_name`
        extend(class_body)  # body of the class will be extended with `class_body`
        
        def fn(self):
            let(x)  # x will be replaced everywhere with unique name, like `_py_backwards_x_1`
            x = 10
            return x

And you can easily get content of snippet with:

my_snippet.get_body(class_name='MyClass',
                    class_body=[ast.Expr(...), ...])

Also please look at tree utils⁠, it contains such useful functions like find, get_parent and etc.

⁠License MIT

Tag summary

Recent tags

Content type

Image

Digest

Size

262.3 MB

Last updated

over 9 years ago

docker pull nvbn/py-backwards