Python EBNF Syntax to Match a Decorator

I have a very large project where I need to move some modules from bar to foo.bar, and subsequently update all functions calls to reflect new location.

This works fine against function calls, but I am having problems building a pattern to match decorators.

The code I like to match is::

  from unittest.mock import patch

  @patch('bar')
  def func(self, patcher):
      ...

I would like to turn the patch above to @patch('foo.bar').

My current pattern matches patch when it is a function call, but not when used as a decorator::

class FixRenameMockPatch(fixer_base.BaseFix):

    PATTERN = """
    power<
        'patch' args=trailer< '(' [any] ')' >
    >
    """

    def transform(self, node, results):
        ...

I am really struggling to understand the grammar on https://docs.python.org/3.8/reference/grammar.html so my fixers has largely been based on the source code of lib2to3.

How do I write the pattern above to match all uses of patch, including a decorator and a context manager? And a full qualified use of patch, such as mock.patch or unittest.mock.patch?

Answer

You can try this

PATTERN = "decorator< '@' 'patch' '(' [any] ')' any* >"

or alternatively

PATTERN = """
    decorator< '@' 'patch' '(' [any] ')' any* >
    | decorator< '@' dotted_name< 'mock' '.' 'patch' > '(' [any] ')' any* >
    """

if you want to match @mock.patch too.

There is a script (written in Python2, Py3 version here) to parse your file and shows the parsed pattern, which I find useful. For example, you create test.py as

from unittest.mock import patch

@patch('bar')
def func(self, patcher):
    print('foo')

Then run find_pattern.py, hit enter until the line you want to parse is shown, then enter some character (y in the case below), and hit enter to show the parsing result:

$ python find_pattern.py -f test.py
' unittest.mock'

'from unittest.mock import patch'

'from unittest.mock import patchn'

"n@patch('bar')n"
y
decorator< '@' 'patch' '(' "'bar'" ')' 'n' >