� !A?hsk��@�dZdZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z m Z mZddlmZddlmZmZdd lmZdd �ZGd �d e��Zd�Zd�Zd�Zd�Zd�ZGd�de��ZGd�de��Z Gd�de��Z!Gd�de ��Z"dS)z�Refactoring framework. Used as a main program, this can refactor any number of files and/or recursively descend down directories. Imported as a module, this provides infrastructure to write your own refactoring tool. z#Guido van Rossum �N)�chain�)�driver�tokenize�token)� find_root)�pytree�pygram)� btm_matcherTc���t|ggdg��}g}tj|j��D]<\}}}|�d��r!|r |dd�}|�|���=|S)zEReturn a sorted list of all available fix names in the given package.�*�fix_�N)� __import__�pkgutil� iter_modules�__path__� startswith�append)� fixer_pkg� remove_prefix�pkg� fix_names�finder�name�ispkgs �=/opt/alt/python-internal/lib64/python3.11/lib2to3/refactor.py�get_all_fix_namesrs��� �Y��B��� .� .�C��I�&�3�C�L�A�A�#�#����e� �?�?�6� "� "� #�� ��A�B�B�x�� � � �T� "� "� "�� ��c��eZdZdS)� _EveryNodeN��__name__� __module__� __qualname__�rrr!r!+��������Drr!c���t|tjtjf��r|j�t �|jhSt|tj��r"|jrt|j��St �t|tj ��rAt��}|jD])}|D]$}|� t|�����%�*|Std|z���)zf Accepts a pytree Pattern Node and returns a set of the pattern types which will match first. Nz$Oh no! I don't understand pattern %s) � isinstancer � NodePattern� LeafPattern�typer!�NegatedPattern�content�_get_head_types�WildcardPattern�set�update� Exception)�pat�r�p�xs rr/r//s����#��*�F�,>�?�@�@�� �8� �� ���z���#�v�,�-�-�� �;� 0�"�3�;�/�/� /����#�v�-�.�.�� �E�E���� -� -�A�� -� -�������+�+�,�,�,�,� -��� �:�S�A� B� B�Brc�Z�tjt��}g}|D]�}|jr[ t |j��}|D]}||�|����?#t $r|�|��Y�`wxYw|j�!||j�|����|�|����ttj j � ��tj j ��D]}||�|���t|��S)z^ Accepts a list of fixers and returns a dictionary of head node type --> fixer list. )� collections� defaultdict�list�patternr/rr!� _accept_typerr �python_grammar� symbol2number�values�tokens�extend�dict)� fixer_list� head_nodes�every�fixer�heads� node_types r�_get_headnode_dictrJKsL���(��.�.�J� �E�� $� $�� �=� $� 8�'�� �6�6��"'�8�8�I��y�)�0�0��7�7�7�7�8��� $� $� $�� � �U�#�#�#�#�#� $���� �!�-��5�-�.�5�5�e�<�<�<�<�� � �U�#�#�#�#��6�0�>�E�E�G�G�!�0�7�9�9�,�,� ��9��$�$�U�+�+�+�+� � � � �s�A�A?�>A?c�<���fd�t�d��D��S)zN Return the fully qualified names for fixers in the package pkg_name. c� ��g|] }�dz|z�� S��.r&)�.0�fix_name�pkg_names �r� z+get_fixers_from_package..hs8��� @� @� @�� �s�N�X� %� @� @� @rF)r)rQs`r�get_fixers_from_packagerSds@��� @� @� @� @�-�h��>�>� @� @� @�@rc��|S�Nr&)�objs r� _identityrWks�� �Jrc�h��d}tjtj|��j����fd�}t t jtjt j h��}t��} |��\}}||vr�|t j kr|rn�d}n�|t j kr�|dkr�|��\}}|t j ks|dkrn�|��\}}|t j ks|dkrn�|��\}}|t j kr|dkr |��\}}|t j krV|�|��|��\}}|t j ks|dkrn|��\}}|t j k�Vnn�� n#t$rYnwxYwt |��S) NFc�B��t���}|d|dfS)Nrr)�next)�tok�gens �r�advancez(_detect_future_features..advancers ����3�i�i���1�v�s�1�v�~�rT�from� __future__�import�(�,)r�generate_tokens�io�StringIO�readline� frozensetr�NEWLINE�NL�COMMENTr1�STRING�NAME�OP�add� StopIteration)�source�have_docstringr]�ignore�features�tp�valuer\s @r�_detect_future_featuresrvos�����N� � "�2�;�v�#6�#6�#?� @� @�C�������� �x�{�E�M�B� C� C�F��u�u�H� � ��� � �I�B���V�|�|���u�|�#�#�!���!%����u�z�!�!�e�v�o�o�#�G�I�I� ��E����#�#�u� �'<�'<��#�G�I�I� ��E����#�#�u��'8�'8��#�G�I�I� ��E����>�>�e�s�l�l� '�� � �I�B���E�J�&�&��L�L��'�'�'� '�� � �I�B���U�X�~�~��#���� '�� � �I�B�� �E�J�&�&���3 ���4 � � � � �� ���� �X� � �s�3D!F� F"�!F"c��eZdZdZdS)� FixerErrorzA fixer could not be loaded.N)r#r$r%�__doc__r&rrrxrx�s������&�&�&�&rrxc���eZdZdddd�ZdZdZdd�Zd�Zd�Zd �Z d �Z d �Z dd �Z dd �Z d�Zdd�Zd�Zd d�Zd�Zd�Z d!d�Zd"d�ZdZdZd�Zd�Zd�Zd�Zd�Zd�ZdS)#�RefactoringToolF)�print_function� exec_function�write_unchanged_files�FixrNc�P�||_|pg|_|j���|_|�|j�|��t j���|_|jdr|jj d=n|jdr |jj d=|j� d��|_ g|_ tjd��|_g|_d|_t%j|jt(j|j� ��|_|���\|_|_g|_t5j��|_g|_g|_t?|j|j��D]k}|j r|j�!|���$||jvr|j�"|���H||jvr|j�"|���ltG|j��|_$tG|j��|_%dS) z�Initializer. Args: fixer_names: a list of fixers to import options: a dict with configuration. explicit: a list of fixers to run even if they are explicit. Nr|�printr}�execr~r{F)�convert�logger)&�fixers�explicit�_default_options�copy�optionsr2r r>�grammar�keywords�getr~�errors�logging� getLoggerr�� fixer_log�wroter�Driverr r�� get_fixers� pre_order� post_order�files�bm� BottomMatcher�BM� bmi_pre_order�bmi_post_orderr� BM_compatible� add_fixerrrJ�bmi_pre_order_heads�bmi_post_order_heads)�self� fixer_namesr�r�rGs r�__init__zRefactoringTool.__init__�s��"�� � ��B�� ��,�1�1�3�3�� � � � �L� � �� (� (� (��,�1�1�3�3�� � �<�(� )� .�� �%�g�.�.� �\�/� *� .�� �%�f�-� &*�\�%5�%5�6M�%N�%N��"��� ��'�(9�:�:�� ������ ��m�D�L�,2�N�+/�;�8�8�8�� �+/�/�/�*;�*;�'������� ��"�$�$������ ����4�?�D�N�;�;� 2� 2�E��"� 2���!�!�%�(�(�(�(��$�.�(�(��"�)�)�%�0�0�0�0��$�/�)�)��#�*�*�5�1�1�1��#5�d�6H�#I�#I�� �$6�t�7J�$K�$K��!�!�!rc���g}g}|jD�]�}t|iidg��}|�dd��d}|�|j��r|t |j��d�}|�d��}|jd�d�|D����z} t||��}n$#t$rtd |�d|����d�wxYw||j |j ��} | jr*|jd ur!||jvr|�d |����!|�d |��| jd kr|�| ����Y| jdkr|�| ����{td| jz���t'jd��} |�| ���|�| ���||fS)aInspects the options to load the requested patterns and handlers. Returns: (pre_order, post_order), where pre_order is the list of fixers that want a pre-order AST traversal, and post_order is the list that want post-order traversal. r rNr�����N�_�c�6�g|]}|�����Sr&)�title)rOr6s rrRz.RefactoringTool.get_fixers..�s ��5O�5O�5O�A�a�g�g�i�i�5O�5O�5Orz Can't find TzSkipping optional fixer: %szAdding transformation: %s�pre�postzIllegal fixer order: %r� run_order��key)r�r�rsplitr� FILE_PREFIX�len�split� CLASS_PREFIX�join�getattr�AttributeErrorrxr�r�r�� log_message� log_debug�orderr�operator� attrgetter�sort) r��pre_order_fixers�post_order_fixers� fix_mod_path�modrP�parts� class_name� fix_classrG�key_funcs rr�zRefactoringTool.get_fixers�s"������ �K� J� J�L��\�2�r�C�5�9�9�C�#�*�*�3��2�2�2�6�H��"�"�4�#3�4�4� <�#�C��(8�$9�$9�$:�$:�;���N�N�3�'�'�E��*�R�W�W�5O�5O��5O�5O�5O�-P�-P�P�J� X�#�C��4�4� � ��!� X� X� X� �j�x�x�x���!L�M�M�SW�W� X�����I�d�l�D�N�;�;�E��~� �$�-�t�";�";� �� �5�5�� � �!>��I�I�I�� �N�N�6�� A� A� A��{�e�#�#� �'�'��.�.�.�.����&�&�!�(�(��/�/�/�/� �!:�U�[�!H�I�I�I��&�{�3�3�����(��+�+�+����8��,�,�,� �"3�4�4s �1C�!C#c���)zCalled when an error occurs.r&)r��msg�args�kwdss r� log_errorzRefactoringTool.log_errors�� rc�H�|r||z}|j�|��dS)zHook to log a message.N)r��info�r�r�r�s rr�zRefactoringTool.log_messages/�� � ���*�C� � ��������rc�H�|r||z}|j�|��dSrU)r��debugr�s rr�zRefactoringTool.log_debug s/�� � ���*�C� � ���#�����rc��dS)zTCalled with the old version, new version, and filename of a refactored file.Nr&)r��old_text�new_text�filename�equals r� print_outputzRefactoringTool.print_outputs �� �rc��|D]P}tj�|��r|�|||���9|�|||���QdS)z)Refactor a list of files and directories.N)�os�path�isdir� refactor_dir� refactor_file)r��items�write� doctests_only� dir_or_files r�refactorzRefactoringTool.refactorsn��!� F� F�K��w�}�}�[�)�)� F��!�!�+�u�m�D�D�D�D��"�"�;��}�E�E�E�E�  F� Frc���tjdz}tj|��D]�\}}}|�d|��|���|���|D]w}|�d��s`tj�|��d|kr7tj�||��} |� | ||���xd�|D��|dd�<��dS)z�Descends down a directory and refactor every Python file found. Python files are assumed to have a .py extension. Files and subdirectories starting with '.' are skipped. �pyzDescending into %srNrc�<�g|]}|�d���|��SrM)r)rO�dns rrRz0RefactoringTool.refactor_dir..2s)��K�K�K�"�� � �c�8J�8J�K�2�K�K�KrN) r��extsep�walkr�r�rr��splitextr�r�) r��dir_namer�r��py_ext�dirpath�dirnames� filenamesr�fullnames rr�zRefactoringTool.refactor_dir s����T�!��,.�G�H�,=�,=� L� L� (�G�X�y� �N�N�/�� 9� 9� 9� �M�M�O�O�O� �N�N� � � �!� G� G������,�,�G��G�$�$�T�*�*�1�-��7�7�!�w�|�|�G�T�:�:�H��&�&�x�� �F�F�F��K�K��K�K�K�H�Q�Q�Q�K�K� L� Lrc�� t|d��}n/#t$r"}|�d||��Yd}~dSd}~wwxYw tj|j��d}|���n#|���wxYwtj|d|d���5}|���|fcddd��S#1swxYwYdS) zG Do our best to decode a Python source file correctly. �rbzCan't open %s: %sN�NNrr5r���encoding�newline) �open�OSErrorr�r�detect_encodingrf�closerd�read)r�r��f�errr�s r�_read_python_sourcez#RefactoringTool._read_python_source4s"�� ��X�t�$�$�A�A��� � � � �N�N�.��#� >� >� >��:�:�:�:�:����� ���� ��/�� �;�;�A�>�H� �G�G�I�I�I�I��A�G�G�I�I�I�I���� �W�X�s�X�r� B� B� B� &�a��6�6�8�8�X�%� &� &� &� &� &� &� &� &� &� &� &� &���� &� &� &� &� &� &s.�� ?�:�?�A7�7B �(C � C�Cc���|�|��\}}|�dS|dz }|rl|�d|��|�||��}|js||kr|�|||||��dS|�d|��dS|�||��}|js |r7|jr0|�t|��dd�|||���dS|�d|��dS)zRefactors a file.N� zRefactoring doctests in %szNo doctest changes in %sr�)r�r�zNo changes in %s)r�r��refactor_docstringr~�processed_file�refactor_string� was_changed�str)r�r�r�r��inputr��output�trees rr�zRefactoringTool.refactor_fileDs@���2�2�8�<�<���x� �=� �F� �� �� � =� �N�N�7�� B� B� B��,�,�U�H�=�=�F��)� E�V�u�_�_��#�#�F�H�e�U�H�M�M�M�M�M����9�8�D�D�D�D�D��'�'��x�8�8�D��)� =�d� =�t�7G� =��#�#�C��I�I�c�r�c�N�H�*/�(�$�D�D�D�D�D����1�8�<�<�<�<�}|�d||jj |��Yd}~|j|j_dSd}~wwxYw |j|j_n#|j|j_wxYw||_ |� d|��|� ||��|S)aFRefactor a given input string. Args: data: a string holding the code to be refactored. name: a human-readable name for use in error/log messages. Returns: An AST corresponding to the refactored input stream; None if there were errors during the parse. r|zCan't parse %s: %s: %sNzRefactoring %s) rvr �!python_grammar_no_print_statementrr�� parse_stringr3r�� __class__r#�future_featuresr�� refactor_tree)r��datarrsrr�s rr�zRefactoringTool.refactor_string[s ��+�4�0�0�� �x� '� '�"(�"J�D�K� � /��;�+�+�D�1�1�D�D��� � � � �N�N�3����!7�� >� >� >� �F�F�F�"&�,�D�K� � � �����  ���� � #'�,�D�K� � ��$�,�D�K� � .� .� .� .�'��� ���'��.�.�.� ���4��&�&�&�� s/�A�B$� B�"B �2B$� B�B$�$B7c���tj���}|rh|�d��|�|d��}|js||kr|�|d|��dS|�d��dS|�|d��}|js |r-|jr&|�t|��d|��dS|�d��dS)NzRefactoring doctests in stdinzzNo doctest changes in stdinzNo changes in stdin) �sys�stdinr�r�r�r~r�r�r�r)r�r�rrrs r�refactor_stdinzRefactoringTool.refactor_stdinvs���� ��� � �� � 6� �N�N�:� ;� ;� ;��,�,�U�I�>�>�F��)� >�V�u�_�_��#�#�F�I�u�=�=�=�=�=����<�=�=�=�=�=��'�'��y�9�9�D��)� 6�d� 6�t�7G� 6��#�#�C��I�I�y�%�@�@�@�@�@����4�5�5�5�5�5rc��t|j|j��D]}|�||���|�|j|�����|�|j|�����|j�|� ����}t|� �����r�|jj D�]�}||v�r�||�r�||� tjjd���|jr+||� tjj���t'||��D�]8}|||vr||�|�� t+|��n#t,$rY�EwxYw|jr ||jvr�Z|�|��}|r�|�||��}|��|�|��|���D]*}|jsg|_|j�|���+|j�|� ����}|D],} | |vrg|| <|| �|| ���-��:���t|� �������t|j|j��D]}|�||���|jS)a�Refactors a parse tree (modifying the tree in place). For compatible patterns the bottom matcher module is used. Otherwise the tree is traversed node-to-node for matches. Args: tree: a pytree.Node instance representing the root of the tree to be refactored. name: a human-readable name for this tree. Returns: True if the tree was modified, False otherwise. T)r��reverser�)rr�r�� start_tree� traverse_byr�r�r��run�leaves�anyr@r�r�r �Base�depth�keep_line_order� get_linenor;�remover� ValueError�fixers_applied�match� transform�replacerrB� finish_treer�) r�rrrG� match_set�node�results�new� new_matches�fxrs rr zRefactoringTool.refactor_tree�s%�� �4�>�4�?�;�;� )� )�E� � � �T�4� (� (� (� (� ����1�4�>�>�3C�3C�D�D�D� ����2�D�O�O�4E�4E�F�F�F��G�K�K�� � � � �.�.� ��)�"�"�$�$�%�%�/ L����. L�. L���I�%�%�)�E�*:�%��e�$�)�)�f�k�.?��)�N�N�N��,�J�"�%�(�-�-�&�+�2H�-�I�I�I� $�Y�u�%5� 6� 6�$L�$L���9�U�#3�3�3�%�e�,�3�3�D�9�9�9�%�%�d�O�O�O�O��)�%�%�%�%�H�%����  �.�%�5�D��%A�>@��(;�$(�$7�$>�$>�u�$E�$E�$E�$E�/3�g�k�k�#�*�*�,�,�.G�.G� �+6�!L�!L�C�+.�)�+;�+;�79� �#��$-�c�N�$9�$9�+�c�:J�$K�$K�$K�$K���_�)�"�"�$�$�%�%�/ L�b�4�>�4�?�;�;� *� *�E� � � �d�D� )� )� )� )���s�F%�% F2�1F2c���|sdS|D]X}||jD]H}|�|��}|r/|�||��}|�|�|��|}�I�YdS)aTraverse an AST, applying a set of fixers to each node. This is a helper method for refactor_tree(). Args: fixers: a list of fixer instances. traversal: a generator that yields AST nodes. Returns: None N)r,rrr)r�r�� traversalr"rGr#r$s rrzRefactoringTool.traverse_by�s���� � �F�� #� #�D��� �*� #� #���+�+�d�+�+���#��/�/�$��8�8�C���� � �S�)�)�)�"���  #� #� #rc�^�|j�|��|�|�|��d}|�dS||k}|�||||��|r|�d|��|jsdS|r|�||||��dS|�d|��dS)zR Called when a file has been refactored and there may be changes. NrzNo changes to %szNot writing changes to %s)r�rr�r�r�r~� write_file)r�r�r�r�r�r�r�s rr�zRefactoringTool.processed_file�s��� � ���(�#�#�#� � ��/�/��9�9�!�<�H������H�$�� ���(�H�h��>�>�>� � � �N�N�-�x� 8� 8� 8��-� ��� � B� �O�O�H�h��(� C� C� C� C� C� �N�N�6�� A� A� A� A� Arc�� tj|d|d���}n/#t$r"}|�d||��Yd}~dSd}~wwxYw|5 |�|��n.#t$r!}|�d||��Yd}~nd}~wwxYwddd��n #1swxYwY|�d|��d|_dS) z�Writes a string to a file. It first shows a unified diff between the old text and the new text, and then rewrites the file; the latter is only done if the write option is set. �wr�r�zCan't create %s: %sNzCan't write %s: %szWrote changes to %sT)rdr�r�r�r�r�r�)r�r�r�r�r��fpr�s rr*zRefactoringTool.write_filesX�� ����3��2�F�F�F�B�B��� � � � �N�N�0�(�C� @� @� @� �F�F�F�F�F����� ����� D� D� D�����"�"�"�"��� D� D� D����3�X�s�C�C�C�C�C�C�C�C����� D���� D� D� D� D� D� D� D� D� D� D� D���� D� D� D� D� ���,�h�7�7�7��� � � sP�� A�A�A� B�A$�#B�$ B�.B �B� B�B�B"�%B"z>>> z... c ��g}d}d}d}d}|�d���D�])}|dz }|����|j��rW|�+|�|�||||����|}|g}|�|j��} |d| �}��|�V|�||jz��s#|||j���zdzkr|� |����|�+|�|�||||����d}d}|� |����+|�+|�|�||||����d� |��S)a�Refactors a docstring, looking for doctests. This returns a modified version of the input string. It looks for doctests, which start with a ">>>" prompt, and may be continued with "..." prompts, as long as the "..." is indented the same as the ">>>". (Unfortunately we can't use the doctest module's parser, since, like most parsers, it is not geared towards preserving the original source.) NrT��keependsrr�r�) � splitlines�lstripr�PS1rB�refactor_doctest�find�PS2�rstriprr�) r�rr��result�block� block_lineno�indent�lineno�line�is rr�z"RefactoringTool.refactor_docstrings�������� ������$�$�d�$�3�3� $� $�D� �a�K�F��{�{�}�}�'�'���1�1� $��$��M�M�$�"7�"7��|�8>��#J�#J�K�K�K�%� �����I�I�d�h�'�'���b�q�b�����$��?�?�6�D�H�#4�5�5�%��6�D�H�O�O�$5�$5�5��<�<�<�� � �T�"�"�"�"��$��M�M�$�"7�"7��|�8>��#J�#J�K�K�K������ � �d�#�#�#�#� � � �M�M�$�/�/��|�06��B�B� C� C� C��w�w�v���rc����� ��||���}n�#t$r�}�j�tj��r.|D]+}��d|�d�����,��d|||j j |��|cYd}~Sd}~wwxYw�� ||��r�t|��� d���}|d|dz �||dz d�}} | dg|dz zks J| ���|d�d��s|dxxdz cc<��jz|�d ��zg}|r|��fd �|D��z }|S) z�Refactors one doctest. A doctest is given as a block of lines, the first of which starts with ">>>" (possibly indented), while the remaining lines start with "..." (identically indented). z Source: %sr�z+Can't parse docstring in %s line %s: %s: %sNTr/rr�rc�*��g|]}��jz|z��Sr&)r6)rOr=r;r�s ��rrRz4RefactoringTool.refactor_doctest..^s%���C�C�C�t�&�4�8�+�d�2�C�C�Cr)� parse_blockr3r�� isEnabledForr��DEBUGr�r7r�rr#r rr1�endswithr3�pop) r�r9r<r;r�rr�r=r$�clippeds ` ` rr4z RefactoringTool.refactor_doctestDs����� ��#�#�E�6�6�:�:�D�D��� � � ��{�'�'�� �6�6� D�!�D�D�D��N�N�<����T�1B�1B�C�C�C�C� �N�N�H�#�V�S�]�-C�S� J� J� J��L�L�L�L�L�L�����  ���� � � �d�H� -� -� D��d�)�)�&�&��&�5�5�C��y��q��y�>�3�v�a�x�y�y�>�S�G��t�f��q��1�1�1�1�7�1�1�1��r�7�#�#�D�)�)� ��B����4������d�h�&�������3�4�E�� D��C�C�C�C�C�s�C�C�C�C��� s�� B'�A6B"�B'�"B'c�6�|jrd}nd}|js|�d|��n5|�d|��|jD]}|�|���|jr4|�d��|jD]}|�|���|jrut |j��dkr|�d��n(|�dt |j����|jD]\}}}|j|g|�Ri|���dSdS) N�werez need to bezNo files %s modified.zFiles that %s modified:z$Warnings/messages while refactoring:rzThere was 1 error:zThere were %d errors:)r�r�r�r�r�r�)r�rH�file�messager�r�r�s r� summarizezRefactoringTool.summarizeask�� �:� ��D�D��D��z� '� � � �4�d� ;� ;� ;� ;� � � �6�� =� =� =�� � '� '��� � ��&�&�&�&� �>� *� � � �C� D� D� D��>� *� *��� � ��)�)�)�)� �;� 5��4�;���1�$�$�� � �!5�6�6�6�6�� � �!8�#�d�k�:J�:J�K�K�K�#'�;� 5� 5���T�4� �� ��4�t�4�4�4�t�4�4�4�4�  5� 5�  5� 5rc��|j�|�|||����}t��|_|S)z�Parses a block into a tree. This is necessary to get correct line number / offset information in the parser diagnostics and embedded into the parse tree. )r� parse_tokens� wrap_toksrgr)r�r9r<r;rs rrAzRefactoringTool.parse_blockxs:�� �{�'�'����u�f�f�(M�(M�N�N��(�{�{���� rc#��K�tj|�||��j��}|D]+\}}\}}\} } } ||dz z }| |dz z } ||||f| | f| fV��,dS)z;Wraps a tokenize stream to systematically modify start/end.rN)rrc� gen_lines�__next__) r�r9r<r;rAr,ru�line0�col0�line1�col1� line_texts rrNzRefactoringTool.wrap_toks�s������)�$�.�.���*G�*G�*P�Q�Q��DJ� G� G� @�D�%��%�� ���y� �V�a�Z� �E� �V�a�Z� �E� ���t�}�u�d�m�Y�F� F� F� F� F� G� Grc#�K�||jz}||jz}|}|D]h}|�|��r|t|��d�V�n5||���dzkrdV�nt d|�d|�����|}�i dV��)z�Generates lines as expected by tokenize from a list of lines. This strips the first len(indent + self.PS1) characters off each line. Nr�zline=z , prefix=Tr�)r3r6rr�r7�AssertionError)r�r9r;�prefix1�prefix2�prefixr=s rrPzRefactoringTool.gen_lines�s����� �4�8�#���4�8�#����� � �D����v�&�&� L��3�v�;�;�<�<�(�(�(�(�(�������4�/�/�/�� � � � �$�n�T�T�T�6�6�%J�K�K�K��F�F� ��H�H�H� rr�)FF)F)NFNrU)r#r$r%r�r�r�r�r�r�r�r�r�r�r�r�r�r�rr rr�r*r3r6r�r4rKrArNrPr&rrr{r{�s�������+0�).�27�9�9���L��K�3L�3L�3L�3L�n&5�&5�&5�P������ ���  � � � F�F�F�F�L�L�L�L�(&�&�&� =�=�=�=�.���66�6�6�6� M �M �M �^#�#�#�.GL� $�B�B�B�B�*����* �C� �C�)�)�)�V���:5�5�5�.��� G� G� G�����rr{c��eZdZdS)�MultiprocessingUnsupportedNr"r&rrr]r]�r'rr]c�B��eZdZ�fd�Z d�fd� Z�fd�Z�fd�Z�xZS)�MultiprocessRefactoringToolc�d��tt|��j|i|��d|_d|_dSrU)�superr_r��queue� output_lock�r�r��kwargsrs �rr�z$MultiprocessRefactoringTool.__init__�s;���9��)�4�0�0�9�4�J�6�J�J�J��� �����rFrc������|dkr*tt����|||��S ddl�n#t$rt �wxYw�j�td��������_�� ���_ ��fd�t|��D��} |D]}|� ���tt����|||���j� ��t|��D]}�j�d���|D]*}|���r|� ���+d�_dS#�j� ��t|��D]}�j�d���|D]*}|���r|� ���+d�_wxYw)Nrrz already doing multiple processesc�F��g|]}���j�����S))�target)�Process�_child)rOr>�multiprocessingr�s ��rrRz8MultiprocessRefactoringTool.refactor..�s<���4�4�4��%�,�,�D�K�,�@�@�4�4�4r)rar_r�rk� ImportErrorr]rb� RuntimeError� JoinableQueue�Lockrc�range�startr��put�is_alive) r�r�r�r�� num_processes� processesr6r>rkrs ` @�rr�z$MultiprocessRefactoringTool.refactor�s/����� �A� � ��4�d�;�;�D�D��u�m�-�-� -� -� "� "� "� "� "��� -� -� -�,� ,� -���� �:� !��A�B�B� B�$�2�2�4�4�� �*�/�/�1�1���4�4�4�4�4�#�M�2�2�4�4�4� � �� � ����� � � � � �-�t� 4� 4� =� =�e�U�>K� M� M� M� �J�O�O� � � ��=�)�)� %� %��� ���t�$�$�$�$�� � ���:�:�<�<���F�F�H�H�H���D�J�J�J�� �J�O�O� � � ��=�)�)� %� %��� ���t�$�$�$�$�� � ���:�:�<�<���F�F�H�H�H���D�J� � � � s�:�A �4AE2�2A;G-c�4��|j���}|�{|\}} tt|��j|i|��|j���n#|j���wxYw|j���}|�ydSdSrU)rbr�rar_r�� task_done)r��taskr�rers �rrjz"MultiprocessRefactoringTool._child�s�����z�~�~�������L�D�&� '�F��1�4�8�8�F��%�#�%�%�%�� �$�$�&�&�&�&��� �$�$�&�&�&�&�����:�>�>�#�#�D������s � A�A8c���|j�|j�||f��dStt|��j|i|��SrU)rbrrrar_r�rds �rr�z)MultiprocessRefactoringTool.refactor_file�sV��� �:� !� �J�N�N�D�&�>� *� *� *� *� *�I�5�4�d�;�;�I��!��!�!� !r)FFr)r#r$r%r�r�rjr�� __classcell__)rs@rr_r_�s�������� � � � � � :?� �������: $� $� $� $� $�!�!�!�!�!�!�!�!�!rr_)T)#ry� __author__rdr�rr r�r�r9� itertoolsr�pgen2rrr� fixer_utilrr�r r r r�rr3r!r/rJrSrWrvrx�objectr{r]r_r&rr�r�s����3� � � � � � � � � ����� � � � �������������������+�*�*�*�*�*�*�*�*�*�!�!�!�!�!�!��������������� � � � � � � � � �� � � �C�C�C�8���2@�@�@����%�%�%�P'�'�'�'�'��'�'�'�F�F�F�F�F�f�F�F�F�R � � � � �� � � �4!�4!�4!�4!�4!�/�4!�4!�4!�4!�4!r