3
� Z!d?d@� Z"dAdB� Z#dCdD� Z$dEdF� Z%dGdH� Z&dIdJ� Z'dKdL� Z(dMdN� Z)dOdP� Z*dQdR� Z+dSdT� Z,dUdV� Z-dWdX� Z.dYdZ� Z/d[d\� Z0d]d^� Z1d_d`� Z2dadb� Z3dcdd� Z4dedf� Z5dgdh� Z6didj� Z7dkdl� Z8dmdn� Z9dodp� Z:dqdr� Z;dsdt� Zdydz� Z?d{d|� Z@d}d~dd�d�d�d�d�d�d�d�d�d�d�d�d�d�d��ZAd�d�d�d��ZBd�d�� ZCd�d�� ZDd�d�� ZEd�d�� ZFd�d�� ZGd�d�� ZHd�d�� ZId�d�� ZJd�d�� ZKd�S )�r c C s< d| _ d| _g | _| j� d| _d| _g | _d| _i | _dS )zInitialize the scanner.F� r TN�����) �done�
flow_level�tokens�fetch_stream_start� tokens_taken�indent�indents�allow_simple_key�possible_simple_keys)r r r r
r 0 s zScanner.__init__c G sJ x| j � r| j� qW | jrF|s$dS x |D ]}t| jd |�r*dS q*W dS )NTr F)�need_more_tokens�fetch_more_tokensr �
isinstance)r �choicesZchoicer r r
� check_tokenq s
zScanner.check_tokenc C s* x| j � r| j� qW | jr&| jd S d S )Nr )r r r )r r r r
�
peek_token} s
zScanner.peek_tokenc C s: x| j � r| j� qW | jr6| jd7 _| jjd�S d S )Nr r )r r r r �pop)r r r r
� get_token� s
zScanner.get_tokenc C s2 | j r
dS | jsdS | j� | j� | jkr.dS d S )NFT)r r �stale_possible_simple_keys�next_possible_simple_keyr )r r r r
r � s zScanner.need_more_tokensc C s� | j � | j� | j| j� | j� }|dkr4| j� S |dkrL| j� rL| j� S |dkrd| j� rd| j � S |dkr|| j
� r|| j � S |dkr�| j � S |dkr�| j
� S |dkr�| j� S |dkr�| j� S |d kr�| j� S |dkr�| j� r�| j� S |d
kr�| j� r�| j� S |d k�r| j� �r| j� S |d k�r*| j� S |d
k�r<| j� S |dk�rN| j� S |dk�rj| j �rj| j� S |dk�r�| j �r�| j � S |dk�r�| j � S |dk�r�| j � S | j � �r�| j � S t!dd d| | j"� ��d S )N� �%�-�.�[�{�]�}�,�?�:r �&�!�|�>�'�"z!while scanning for the next tokenz.found character %r that cannot start any token)#�scan_to_next_tokenr' �
unwind_indentr �peek�fetch_stream_end�check_directive�fetch_directive�check_document_start�fetch_document_start�check_document_end�fetch_document_end�fetch_flow_sequence_start�fetch_flow_mapping_start�fetch_flow_sequence_end�fetch_flow_mapping_end�fetch_flow_entry�check_block_entry�fetch_block_entry� check_key� fetch_key� check_value� fetch_value� fetch_alias� fetch_anchor� fetch_tagr �
fetch_literal� fetch_folded� fetch_single� fetch_double� check_plain� fetch_plainr �get_mark)r �chr r r
r � s^
zScanner.fetch_more_tokensc C s: d }x0| j D ]&}| j | }|d ks,|j|k r |j}q W |S )N)r r )r Zmin_token_number�level�keyr r r
r( s
z Scanner.next_possible_simple_keyc C s` xZt | j�D ]L}| j| }|j| jks6| j|j dkr |jrPtd|jd| j� ��| j|= q W d S )Ni zwhile scanning a simple keyzcould not find expected ':')�listr r r r
r r rX )r rZ r[ r r r
r' s
z"Scanner.stale_possible_simple_keysc C s^ | j o| j| jk}| jrZ| j� | jt| j� }t||| j | j
| j| j � �}|| j | j < d S )N)
r r r r �remove_possible_simple_keyr �lenr r r r rX r )r r
r r[ r r r
�save_possible_simple_key$ s z Scanner.save_possible_simple_keyc C s@ | j | jkr<| j| j }|jr2td|jd| j� ��| j| j = d S )Nzwhile scanning a simple keyzcould not find expected ':')r r r
r r rX )r r[ r r r
r] 5 s z"Scanner.remove_possible_simple_keyc C sD | j r
d S x4| j|kr>| j� }| jj� | _| jjt||�� q W d S )N)r r rX r r% r �appendZ
BlockEndToken)r r r r r r
r; B s zScanner.unwind_indentc C s& | j |k r"| jj| j � || _ dS dS )NTF)r r r` )r r r r r
�
add_indentZ s
zScanner.add_indentc C s$ | j � }| jjt||| jd�� d S )N)�encoding)rX r r` ZStreamStartTokenrb )r r r r r
r d s zScanner.fetch_stream_startc C sB | j d� | j� d| _i | _| j� }| jjt||�� d| _d S )Nr FTr ) r; r] r r rX r r` ZStreamEndTokenr )r r r r r
r= p s
zScanner.fetch_stream_endc C s, | j d� | j� d| _| jj| j� � d S )Nr Fr )r; r] r r r` �scan_directive)r r r r
r? � s
zScanner.fetch_directivec C s | j t� d S )N)�fetch_document_indicatorZDocumentStartToken)r r r r
rA � s z Scanner.fetch_document_startc C s | j t� d S )N)rd ZDocumentEndToken)r r r r
rC � s zScanner.fetch_document_endc C sH | j d� | j� d| _| j� }| jd� | j� }| jj|||�� d S )Nr F� r )r; r] r rX �forwardr r` )r �
TokenClass�
start_mark�end_markr r r
rd � s
z Scanner.fetch_document_indicatorc C s | j t� d S )N)�fetch_flow_collection_startZFlowSequenceStartToken)r r r r
rD � s z!Scanner.fetch_flow_sequence_startc C s | j t� d S )N)rj ZFlowMappingStartToken)r r r r
rE � s z Scanner.fetch_flow_mapping_startc C sJ | j � | jd7 _d| _| j� }| j� | j� }| jj|||�� d S )Nr T)r_ r r rX rf r r` )r rg rh ri r r r
rj � s z#Scanner.fetch_flow_collection_startc C s | j t� d S )N)�fetch_flow_collection_endZFlowSequenceEndToken)r r r r
rF � s z Scanner.fetch_flow_sequence_endc C s | j t� d S )N)rk ZFlowMappingEndToken)r r r r
rG � s z Scanner.fetch_flow_mapping_endc C sJ | j � | jd8 _d| _| j� }| j� | j� }| jj|||�� d S )Nr F)r] r r rX rf r r` )r rg rh ri r r r
rk � s z!Scanner.fetch_flow_collection_endc C s< d| _ | j� | j� }| j� | j� }| jjt||�� d S )NT)r r] rX rf r r` ZFlowEntryToken)r rh ri r r r
rH � s zScanner.fetch_flow_entryc C s� | j sF| js td d d| j� ��| j| j�rF| j� }| jjt||�� n d| _| j � | j� }| j
� | j� }| jjt ||�� d S )Nz%sequence entries are not allowed hereT) r r r rX ra r r r` ZBlockSequenceStartTokenr] rf ZBlockEntryToken)r r rh ri r r r
rJ � s
zScanner.fetch_block_entryc C s� | j sD| js td d d| j� ��| j| j�rD| j� }| jjt||�� | j | _| j � | j� }| j
� | j� }| jjt ||�� d S )Nz!mapping keys are not allowed here) r r r rX ra r r r` �BlockMappingStartTokenr] rf �KeyToken)r r rh ri r r r
rL s
zScanner.fetch_keyc C s | j | jkr|| j| j }| j| j = | jj|j| j t|j|j�� | j st| j|j �rt| jj|j| j t
|j|j�� d| _ n\| j s�| j s�t d d d| j
� ��| j s�| j| j �r�| j
� }| jjt
||�� | j | _ | j� | j
� }| j� | j
� }| jjt||�� d S )NFz#mapping values are not allowed here)r r r �insertr r rm r ra r rl r r rX r` r] rf Z
ValueToken)r r[ r rh ri r r r
rN s2
zScanner.fetch_valuec C s$ | j � d| _| jj| jt�� d S )NF)r_ r r r` � scan_anchorZ
AliasToken)r r r r
rO V s zScanner.fetch_aliasc C s$ | j � d| _| jj| jt�� d S )NF)r_ r r r` ro Z AnchorToken)r r r r
rP a s zScanner.fetch_anchorc C s" | j � d| _| jj| j� � d S )NF)r_ r r r` �scan_tag)r r r r
rQ l s zScanner.fetch_tagc C s | j dd� d S )Nr6 )�style)�fetch_block_scalar)r r r r
rR w s zScanner.fetch_literalc C s | j dd� d S )Nr7 )rq )rr )r r r r
rS z s zScanner.fetch_foldedc C s$ d| _ | j� | jj| j|�� d S )NT)r r] r r` �scan_block_scalar)r rq r r r
rr } s zScanner.fetch_block_scalarc C s | j dd� d S )Nr8 )rq )�fetch_flow_scalar)r r r r
rT � s zScanner.fetch_singlec C s | j dd� d S )Nr9 )rq )rt )r r r r
rU � s zScanner.fetch_doublec C s$ | j � d| _| jj| j|�� d S )NF)r_ r r r` �scan_flow_scalar)r rq r r r
rt � s zScanner.fetch_flow_scalarc C s" | j � d| _| jj| j� � d S )NF)r_ r r r` �
scan_plain)r r r r
rW � s zScanner.fetch_plainc C s | j dkrdS d S )Nr T)r )r r r r
r> � s
zScanner.check_directivec C s. | j dkr*| jd�dkr*| jd�dkr*dS d S )Nr re z---u
…

T)r �prefixr< )r r r r
r@ � s
z Scanner.check_document_startc C s. | j dkr*| jd�dkr*| jd�dkr*dS d S )Nr re z...u
…

T)r rw r< )r r r r
rB � s
zScanner.check_document_endc C s | j d�dkS )Nr u
…

)r< )r r r r
rI � s zScanner.check_block_entryc C s | j r
dS | jd�dkS d S )NTr u
…

)r r< )r r r r
rK � s zScanner.check_keyc C s | j r
dS | jd�dkS d S )NTr u
…

)r r< )r r r r
rM � s zScanner.check_valuec C s6 | j � }|dkp4| j d�dko4|dkp4| j o4|dkS )Nu
…

-?:,[]{}#&*!|>'"%@`r u
…

r+ z?:)r< r )r rY r r r
rV � s zScanner.check_plainc C s� | j dkr | j� dkr | j� d}xb|s�x| j� dkr@| j� q*W | j� dkrhx| j� dkrf| j� qPW | j� r~| js�d| _q$d}q$W d S )Nr u F� �#u
…

T)r r< rf �scan_line_breakr r )r �foundr r r
r: � s zScanner.scan_to_next_tokenc C s� | j � }| j� | j|�}d }|dkr:| j|�}| j � }n>|dkrV| j|�}| j � }n"| j � }x| j� dkrv| j� q`W | j|� t||||�S )NZYAMLZTAGu
…

)rX rf �scan_directive_name�scan_yaml_directive_value�scan_tag_directive_valuer< �scan_directive_ignored_lineZDirectiveToken)r rh �name�valueri r r r
rc s
zScanner.scan_directivec C s� d}| j |�}xfd| ko"dkn s`d| ko:dkn s`d| koRdkn s`|dkrt|d 7 }| j |�}qW |s�td
|d | | j� ��| j|�}| j|� | j � }|d kr�td
|d | | j� ��|S )
Nr �0�9�A�Z�a�zz-_r zwhile scanning a directivez6expected alphabetic or numeric character, but found %ru
…

)r< r rX rw rf )r rh �lengthrY r� r r r
r| # s$
J
zScanner.scan_directive_namec C s� x| j � dkr| j� qW | j|�}| j � dkrJtd|d| j � | j� ��| j� | j|�}| j � dkr�td|d| j � | j� ��||fS )Nrx r, zwhile scanning a directivez%expected a digit or '.', but found %ru
…

z%expected a digit or ' ', but found %r)r< rf �scan_yaml_directive_numberr rX )r rh �major�minorr r r
r} 8 s
z!Scanner.scan_yaml_directive_valuec C s� | j � }d| kodkn s6td|d| | j� ��d}x*d| j |� koTdkn rd|d7 }q| j� q(W | j � }|dkrftd|d| | j� ��| j� d S )Nrx ry u
…

zwhile scanning a directivez0expected a comment or a line break, but found %r)r< rf r rX rz )r rh rY r r r
r r s z#Scanner.scan_directive_ignored_linec C s | j � }| j� }|dkr d}nd}| j� d}| j|�}xfd| koLdkn s�d| koddkn s�d | ko|d
kn s�|d kr�|d 7 }| j|�}q:W |s�td
| |d| | j � ��| j|�}| j|� | j� }|dkr�td
| |d| | j � ��| j � }||||�S )Nr �aliasZanchorr r� r� r� r� r� r� z-_r zwhile scanning an %sz6expected alphabetic or numeric character, but found %ru
…

?:,]}%@`)rX r< rf r rw ) r rg rh Z indicatorr� r� rY r� ri r r r
ro � s2
J
zScanner.scan_anchorc C s& | j � }| jd�}|dkrdd }| jd� | jd|�}| j� dkrZtd|d| j� | j � ��| j� n~|dkr~d }d }| j� ndd}d
}x,|d kr�|d kr�d }P |d7 }| j|�}q�W d }|r�| jd|�}n d }| j� | jd|�}| j� }|d k�r
td
|d| | j � ��||f}| j � }t|||�S )Nr �<� �tagr7 zwhile parsing a tagzexpected '>', but found %ru
…

r5 Fu
…

Tzwhile scanning a tagzexpected ' ', but found %r)rX r< rf r� r r� ZTagToken) r rh rY r� �suffixr� Z
use_handler� ri r r r
rp � sH
zScanner.scan_tagc C s� |dkrd}nd}g }| j � }| j� | j|�\}}| j|� | jd }|dk rTd}|d krv| j� \}} }
t|| �} n|| d } | j| �\}}
d} x�| j| ko�| j � dk�rt|j
|� | j � dk}
d}x| j |�d kr�|d7 }q�W |j | j |�� | j|� | j
� } | j| �\}}
| j| k�rp| j � dk�rp|�rd| d
k�rd|
�rd| j � dk�rd|�sn|j d � n
|j | � q�P q�W |dk �r�|j | � |dk�r�|j
|� tdj|�d||
|�S ) Nr7 TFr � r) z r u
…

�
rx )rX rf � scan_block_scalar_indicators� scan_block_scalar_ignored_liner � scan_block_scalar_indentation�max�scan_block_scalar_breaksr r< �extendr` rw rz � ScalarToken�join)r rq Zfolded�chunksrh �chomping� incrementZ
min_indent�breaks�
max_indentri r �
line_breakZleading_non_spacer� r r r
rs � sP
zScanner.scan_block_scalarc C s� d }d }| j � }|dkrn|dkr&d}nd}| j� | j � }|dkr�t|�}|dkrdtd|d| j� ��| j� n\|dkr�t|�}|dkr�td|d| j� ��| j� | j � }|dkr�|dkr�d}nd}| j� | j � }|d kr�td|d
| | j� ��||fS ) Nz+-�+TF�
0123456789r z while scanning a block scalarz| j� q(W | j � }|dkrftd|d| | j� ��| j� d S )Nrx ry u
…

z while scanning a block scalarz0expected a comment or a line break, but found %r)r< rf r rX rz )r rh rY r r r
r� A s
z&Scanner.scan_block_scalar_ignored_linec C sh g }d}| j � }xL| j� dkr\| j� dkrB|j| j� � | j � }q| j� | j|kr| j}qW |||fS )Nr u
…

rx )rX r< r` rz rf r )r r� r� ri r r r
r� O s
z%Scanner.scan_block_scalar_indentationc C s� g }| j � }x"| j|k r.| j� dkr.| j� qW xJ| j� dkrz|j| j� � | j � }x"| j|k rv| j� dkrv| j� qVW q2W ||fS )Nrx u
…

)rX r r< rf r` rz )r r r� ri r r r
r� ^ s z Scanner.scan_block_scalar_breaksc C s� |dkrd}nd}g }| j � }| j� }| j� |j| j||�� x4| j� |krt|j| j||�� |j| j||�� qBW | j� | j � }tdj|�d|||�S )Nr9 TFr� )rX r< rf r� �scan_flow_scalar_non_spaces�scan_flow_scalar_spacesr� r� )r rq �doubler� rh Zquoteri r r r
ru k s zScanner.scan_flow_scalarr) ��� r� � � �
�rx r9 �\� …�  u 
u 
)r� r� �b�tr� �n�v�f�r�erx r9 r� �N�_�L�Pr� � � )�x�u�Uc C s� g }�x�d}x| j |�dkr&|d7 }qW |rF|j| j|�� | j|� | j � }| r�|dkr�| j d�dkr�|jd� | jd� q|r�|dks�| r�|dkr�|j|� | j� q|o�|dk�r�| j� | j � }|| jkr�|j| j| � | j� n�|| jk�r|| j| }| j� xBt|�D ]6}| j |�dk�rtd |d
|| j |�f | j� ���qW t | j|�d �}|jt
|�� | j|� n<|d k�r�| j � |j | j
||�� ntd |d
| | j� ��q|S qW d S )Nr u '"\
…

r r8 r� z"\r� �0123456789ABCDEFabcdefz%while scanning a double-quoted scalarz?expected escape sequence of %d hexdecimal numbers, but found %r� u
…

z!found unknown escape character %r)r< r` rw rf �ESCAPE_REPLACEMENTS� ESCAPE_CODES�ranger rX r� �chrrz r� �scan_flow_scalar_breaks)r r� rh r� r� rY �k�coder r r
r� � sL
z#Scanner.scan_flow_scalar_non_spacesc C s� g }d}x| j |�dkr"|d7 }q
W | j|�}| j|� | j � }|dkr\td|d| j� ��nT|dkr�| j� }| j||�}|dkr�|j|� n|s�|jd � |j|� n
|j|� |S )
Nr z r r) z while scanning a quoted scalarz found unexpected end of streamu
…

r� rx ) r< rw rf r rX rz r� r` r� ) r r� rh r� r� � whitespacesrY r� r� r r r
r� � s(
z Scanner.scan_flow_scalar_spacesc C s� g }xx| j d�}|dks |dkr@| jd�dkr@td|d| j� ��x| j� dkrX| j� qBW | j� dkrv|j| j� � q|S qW d S ) Nre z---z...u
…

z while scanning a quoted scalarz#found unexpected document separatorz u
…

)rw r< r rX rf r` rz )r r� rh r� rw r r r
r� � s
z Scanner.scan_flow_scalar_breaksc C sN g }| j � }|}| jd }g }�xd}| j� dkr4P xP| j|�}|dksx| j rj|dkrj| j|d �dksx| jrz|dkrzP |d7 }q6W | jr�|dkr�| j|d �dkr�| j|� td|d | j � d
��|dkr�P d | _|j|� |j| j |�� | j|� | j � }| j
||�}| �s4| j� dk�s4| j r"| j |k r"P q"W t d j
|�d
||�S )Nr r ry u
…

r3 z,:?[]{}u
…

,[]{}z while scanning a plain scalarzfound unexpected ':'zGPlease check http://pyyaml.org/wiki/YAMLColonInFlowContext for details.Fr� T)rX r r< r rf r r r� r` rw �scan_plain_spacesr r� r� )r r� rh ri r Zspacesr� rY r r r
rv � sF
zScanner.scan_plainc
C s. g }d}x| j |�dkr"|d7 }q
W | j|�}| j|� | j � }|dk�r| j� }d| _| jd�}|dksr|dkr�| j d�d kr�d S g } x`| j � d
kr�| j � dkr�| j� q�| j| j� � | jd�}|dks�|dkr�| j d�d kr�d S q�W |d kr�|j|� n| �s|jd� |j| � n|�r*|j|� |S ) Nr rx r u
…

Tre z---z...u
…

u
…

r� )r< rw rf rz r r` r� )
r r rh r� r� r� rY r� rw r� r r r
r� ! s>
zScanner.scan_plain_spacesc C s� | j � }|dkr*td| |d| | j� ��d}| j |�}|dkr�xfd| koTdkn s�d| kold kn s�d
| ko�d kn s�|d kr�|d7 }| j |�}qBW |dkr�| j|� td| |d| | j� ��|d7 }| j|�}| j|� |S )
Nr5 zwhile scanning a %szexpected '!', but found %rr rx r� r� r� r� r� r� z-_)r< r rX rf rw )r r� rh rY r� r� r r r
r� F s&
J
zScanner.scan_tag_handlec C s� g }d}| j |�}x�d| ko&dkn sdd| ko>dkn sdd| koVdkn sd|dkr�|d kr�|j| j|�� | j|� d}|j| j||�� n|d
7 }| j |�}qW |r�|j| j|�� | j|� d}|s�td | |d | | j� ��d
j|�S )Nr r� r� r� r� r� r� z-;/?:@&=+$,_.!~*'()[]%r* r zwhile parsing a %szexpected URI, but found %rr� )r< r` rw rf �scan_uri_escapesr rX r� )r r� rh r� r� rY r r r
r� ^ s(
J
zScanner.scan_tag_uric C s� g }| j � }xx| j� dkr�| j� x>td�D ]2}| j|�dkr,td| |d| j|� | j � ��q,W |jt| jd�d�� | jd� qW yt|�j d�}W n8 t
k
r� } z td| |t |�|��W Y d d }~X nX |S )Nr* r� r� zwhile scanning a %szBexpected URI escape sequence of 2 hexdecimal numbers, but found %rr� zutf-8) rX r< rf r� r r` r� rw �bytes�decode�UnicodeDecodeError�str)r r� rh Zcodesr r� r� �excr r r
r� w s
(zScanner.scan_uri_escapesc C sN | j � }|dkr6| jd�dkr*| jd� n| j� dS |dkrJ| j� |S dS )Nu
…r� z
r� u 

r� )r< rw rf )r rY r r r
rz � s zScanner.scan_line_breakN)Lr r r r r# r$ r& r r r( r' r_ r] r; ra r r= r? rA rC rd rD rE rj rF rG rk rH rJ rL rN rO rP rQ rR rS rr rT rU rt rW r> r@ rB rI rK rM rV r: rc r| r} r� r~ r� r� r ro rp rs r� r� r� r� ru r� r� r� r� r� rv r� r� r� r� rz r r r r
r . s� A
l
! 8
#
$)L(
+/%N)�__all__�errorr r r r r r r r r
� s