�
S�Ljc @ sy d Z d Z d d l Z d d l Z d d l m Z d Z e Z d � Z d � Z d � Z
d � Z d
� Z d � Z
e e � Z Z Z e e � Z Z e e e j � Z Z e
d � Z Z Z e
d
� Z Z Z e
d � Z d � Z d � Z d � Z d � Z d � Z! d � Z" d � Z# d � Z$ e# � Z% e$ � Z& d � Z' d � Z( d � Z) d � Z* d � Z+ d � Z, e) e j- e& e � Z. Z/ Z0 e) e j- e% e � Z1 Z2 e* e j- e% e e j � Z3 Z4 e+ e j5 d � Z6 e+ e j7 d � Z8 e+ e j5 d � Z9 e+ e j7 d � Z: e, e j5 d! � Z; e, e j7 d" � Z< d# � Z= d$ � Z> d% � Z? d& � Z@ d' � ZA d( � ZB d) � ZC d S(* s� Code for encoding protocol message primitives.
Contains the logic for encoding every logical protocol field type
into one of the 5 physical wire types.
This code is designed to push the Python interpreter's performance to the
limits.
The basic idea is that at startup time, for every field (i.e. every
FieldDescriptor) we construct two functions: a "sizer" and an "encoder". The
sizer takes a value of this field's type and computes its byte size. The
encoder takes a writer function and a value. It encodes the value into byte
strings and invokes the writer function to write those strings. Typically the
writer function is the write() method of a BytesIO.
We try to do as much work as possible when constructing the writer and the
sizer rather than when calling them. In particular:
* We copy any needed global functions to local variables, so that we do not need
to do costly global table lookups at runtime.
* Similarly, we try to do any attribute lookups at startup time if possible.
* Every field's tag is encoded to bytes at startup, since it can't change at
runtime.
* Whatever component of the field size we can compute at startup, we do.
* We *avoid* sharing code if doing so would make the code slower and not sharing
does not burden us too much. For example, encoders for repeated fields do
not just call the encoders for singular fields in a loop because this would
add an extra function call overhead for every loop iteration; instead, we
manually inline the single-value encoder into the loop.
* If a Python function lacks a return statement, Python actually generates
instructions to pop the result of the last statement off the stack, push
None onto the stack, and then return that. If we really don't care what
value is returned, then we can save two instructions by returning the
result of the last statement. It looks funny but it helps.
* We assume that type and bounds checking has happened at a higher level.
s [email protected] (Kenton Varda)i����N( t wire_formatg �c C s� | d k r d S| d k r d S| d k r0 d S| d k r@ d S| d k rP d
S| d k r` d S| d
k rp d S| d k r� d S| d k r� d Sd S( s# Compute the size of a varint value.i i i�? i i�� i i���i l �� i l ���i l ��� i l ����i l ���� i i
( ( t value( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt _VarintSizeR s& c C s� | d k r d S| d k r d S| d k r0 d S| d k r@ d S| d k rP d
S| d k r` d S| d
k rp d S| d k r� d S| d k r� d S| d k r� d Sd S( s* Compute the size of a signed varint value.i i
i i i�? i i�� i i���i l �� i l ���i l ��� i l ����i l ���� i ( ( R ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt _SignedVarintSize` s* c C s t t j | d � � S( sQ Returns the number of bytes required to serialize a tag with this field
number.i ( R R t PackTag( t field_number( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt _TagSizeo s c s � f d � } | S( s� A sizer which uses the function compute_value_size to compute the size of
each value. Typically compute_value_size is _VarintSize.c sg t | � � | r1 t � � � � f d � } | S| rM � � f d � } | S� � f d � } | Sd S( Nc s9 d } x | D] } | � | � 7} q
W| � | � � S( Ni ( ( R t resultt element( t compute_value_sizet local_VarintSizet tag_size( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt PackedFieldSize� s
c s5 � t | � } x | D] } | � | � 7} q W| S( N( t len( R R R ( R R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt RepeatedFieldSize� s
c s � � | � S( N( ( R ( R R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt FieldSize� s ( R R ( R t is_repeatedt is_packedR R R ( R ( R
R sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt
SpecificSizer� s ( ( R R ( ( R sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt _SimpleSizer~ s c s � � f d � } | S( s� Like SimpleSizer, but modify_value is invoked on each value before it is
passed to compute_value_size. modify_value is typically ZigZagEncode.c sp t | � � | r4 t � � � � � f d � } | S| rS � � � f d � } | S� � � f d � } | Sd S( Nc s? d } x$ | D] } | � � | � � 7} q
W| � | � � S( Ni ( ( R R R ( R R
t modify_valueR ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR � s
c s; � t | � } x$ | D] } | � � | � � 7} q W| S( N( R
( R R R ( R R R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR � s
c s � � � | � � S( N( ( R ( R R R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR � s ( R R ( R R R R R R ( R R ( R
R sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR � s ( ( R R R ( ( R R sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt _ModifiedSizer� s c s � f d � } | S( sW Like _SimpleSizer except for a fixed-size field. The input is the size
of one value.c su t | � � | r1 t � � � � f d � } | S| rT � � � � f d � } | S� � � � f d � } | Sd S( Nc s" t | � � } | � | � � S( N( R
( R R ( R
R t
value_size( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR � s c s t | � � S( N( R
( R ( t element_size( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR � s c s � S( N( ( R ( t
field_size( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR � s ( R R ( R R R R R R ( R ( R R R
R sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR � s
( ( R R ( ( R sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt _FixedSizer� s i i i c sa t | � � t � t � | s% t � | rD � � � f d � } | S� � � f d � } | Sd S( s# Returns a sizer for a string field.c sN � t | � } x7 | D]/ } � | j d � � } | � | � | 7} q W| S( Ns utf-8( R
t encode( R R R t l( R
t local_lenR ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR � s
c s' � | j d � � } � � | � | S( Ns utf-8( R ( R R ( R
R R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR � s N( R R R
t AssertionError( R R R R R ( ( R
R R sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt StringSizer� s
c sa t | � � t � t � | s% t � | rD � � � f d � } | S� � � f d � } | Sd S( s" Returns a sizer for a bytes field.c sE � t | � } x. | D]& } � | � } | � | � | 7} q W| S( N( R
( R R R R ( R
R R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR s
c s � | � } � � | � | S( N( ( R R ( R
R R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR s N( R R R
R ( R R R R R ( ( R
R R sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt
BytesSizer� s
c sM t | � d � | s t � | r6 � f d � } | S� f d � } | Sd S( s" Returns a sizer for a group field.i c s5 � t | � } x | D] } | | j � 7} q W| S( N( R
t ByteSize( R R R ( R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR s
c s � | j � S( N( R ( R ( R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR s N( R R ( R R R R R ( ( R sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt
GroupSizer s
c sU t | � � t � | s t � | r; � � f d � } | S� � f d � } | Sd S( s$ Returns a sizer for a message field.c sE � t | � } x. | D]& } | j � } | � | � | 7} q W| S( N( R
R ( R R R R ( R
R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR + s
c s | j � } � � | � | S( N( R ( R R ( R
R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR 3 s N( R R R ( R R R R R ( ( R
R sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt MessageSizer$ s
c sJ t d � d t d � t | � t d � � t � � � f d � } | S( s� Returns a sizer for extensions of MessageSet.
The message set message looks like this:
message MessageSet {
repeated group Item = 1 {
required int32 type_id = 2;
required string message = 3;
}
}
i i i c s | j � } � � | � | S( N( R ( R R ( R
t static_size( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR L s ( R R ( R R ( ( R
R# sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt MessageSetItemSizer= s
!
c s7 | j � t | j t t � � � � � f d � } | S( s Returns a sizer for a map field.c s` d } xS | D]K } | | } � j d | d | � } | � | � 7} � r
| j � q
q
W| S( Ni t keyR ( t _concrete_classR ( t map_valuet totalR% R t entry_msg( t is_message_mapt
message_sizert message_type( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyR _ s
( R, R" t numbert False( t field_descriptorR* R ( ( R* R+ R, sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt MapSizerW s c C s
d � } | S( sB Return an encoder for a basic varint value (does not include tag).c S s_ | d @} | d L} x5 | rK | t j d | B� � | d @} | d L} q W| t j | � � S( Ni i i� ( t sixt int2byte( t writeR t unused_deterministict bits( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt EncodeVarintw s
( ( R6 ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt _VarintEncodert s c C s
d � } | S( sK Return an encoder for a basic signed varint value (does not include
tag).c S sx | d k r | d 7} n | d @} | d L} x5 | rd | t j d | B� � | d @} | d L} q0 W| t j | � � S( Ni i i@ i i i� l ( R1 R2 ( R3 R R4 R5 ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt EncodeSignedVarint� s
( ( R8 ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt _SignedVarintEncoder� s c C s& g } t | j | t � d j | � S( s� Encode the given integer as a varint and return the bytes. This is only
called at startup time so it doesn't need to be fast.t ( t
_EncodeVarintt appendt Truet join( R t pieces( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt _VarintBytes� s c C s t j t t j | | � � � S( sC Encode the given tag and return the bytes. Only called at startup.( R1 t binary_typeR@ R R ( R t wire_type( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt TagBytes� s c s � � � f d � } | S( s_ Return a constructor for an encoder for fields of a particular type.
Args:
wire_type: The field's wire type, for encoding tags.
encode_value: A function which encodes an individual value, e.g.
_EncodeVarint().
compute_value_size: A function which computes the size of an individual
value, e.g. _VarintSize().
c s� | r: t | t j � � t � � � � � f d � } | S| re t | � � � � � f d � } | St | � � � � � f d � } | Sd S( Nc sf | � � d } x | D] } | � | � 7} q W� | | | � x | D] } � | | | � qH Wd S( Ni ( ( R3 R t
deterministict sizeR ( R t encode_valuet local_EncodeVarintt tag_bytes( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt EncodePackedField� s
c s/ x( | D] } | � � � | | | � q Wd S( N( ( R3 R RD R ( RF RH ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt EncodeRepeatedField� s
c s | � � � | | | � S( N( ( R3 R RD ( RF RH ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt EncodeField� s
( RC R t WIRETYPE_LENGTH_DELIMITEDR; ( R R R RI RJ RK ( R RF RB ( RG RH sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt SpecificEncoder� s ( ( RB RF R RM ( ( R RF RB sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt _SimpleEncoder� s c s � � � � f d � } | S( s� Like SimpleEncoder but additionally invokes modify_value on every value
before passing it to encode_value. Usually modify_value is ZigZagEncode.c s� | r= t | t j � � t � � � � � � f d � } | S| rk t | � � � � � � f d � } | St | � � � � � � f d � } | Sd S( Nc sr | � � d } x$ | D] } | � � | � � 7} q W� | | | � x$ | D] } � | � | � | � qN Wd S( Ni ( ( R3 R RD RE R ( R RF RG R RH ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRI � s
c s5 x. | D]& } | � � � | � | � | � q Wd S( N( ( R3 R RD R ( RF R RH ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRJ � s
c s | � � � | � | � | � S( N( ( R3 R RD ( RF R RH ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRK � s
( RC R RL R; ( R R R RI RJ RK ( R RF R RB ( RG RH sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRM � s ( ( RB RF R R RM ( ( R RF R RB sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt _ModifiedEncoder� s c s( t j � � � � � � f d � } | S( s� Return a constructor for an encoder for a fixed-width field.
Args:
wire_type: The field's wire type, for encoding tags.
format: The format string to pass to struct.pack().
c s� t j � | rF t | t j � � t � � � � � � f d � } | S| rt t | � � � � � � f d � } | St | � � � � � � f d � } | Sd S( Nc sL | � � � | t | � � | � x! | D] } | � � | � � q+ Wd S( N( R
( R3 R RD R ( t formatRG t local_struct_packRH R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRI s
c s2 x+ | D]# } | � � | � � | � � q Wd S( N( ( R3 R R4 R ( RP RQ RH ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRJ s
c s | � � | � � | � � S( N( ( R3 R R4 ( RP RQ RH ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRK s
( t structt packRC R RL R; ( R R R RI RJ RK ( RP R RB ( RG RQ RH sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRM s ( RR t calcsize( RB RP RM ( ( RP R RB sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt _StructPackEncoder� s c sk t j � � � � d k r' d � � n( � d k r? d � � n t d � � � � � � � f d � } | S( sq Return a constructor for an encoder for float fields.
This is like StructPackEncoder, but catches errors that may be due to
passing non-finite floating-point values to struct.pack, and makes a
second attempt to encode those values.
Args:
wire_type: The field's wire type, for encoding tags.
format: The format string to pass to struct.pack().
i c S sR | t k r | d � n5 | t k r2 | d � n | | k rK | d � n � d S( Ns �s ��s �( t _POS_INFt _NEG_INF( R3 R ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt EncodeNonFiniteOrRaise+ s
i c S sR | t k r | d � n5 | t k r2 | d � n | | k rK | d � n � d S( Ns �s ��s �( RV RW ( R3 R ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRX 6 s
sG Can't encode floating-point values that are %d bytes long (only 4 or 8)c s� t j � | rI t | t j � � t � � � � � � � f d � } | S| rz t | � � � � � � � f d � } | St | � � � � � � � f d � } | Sd S( Nc sq | � � � | t | � � | � xF | D]> } y | � � | � � Wq+ t k
rh � | | � q+ Xq+ Wd S( N( R
t SystemError( R3 R RD R ( RX RP RG RQ RH R ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRI H s
c sW xP | D]H } | � � y | � � | � � Wq t k
rN � | | � q Xq Wd S( N( RY ( R3 R R4 R ( RX RP RQ RH ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRJ U s
c sF | � � y | � � | � � Wn t k
rA � | | � n Xd S( N( RY ( R3 R R4 ( RX RP RQ RH ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRK _ s
( RR RS RC R RL R; ( R R R RI RJ RK ( RX RP R RB ( RG RQ RH sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRM C s
( RR RT t
ValueError( RB RP RM ( ( RX RP R RB sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt _FloatingPointEncoder s
$s | D]6 } | � � � | | j � | � | j | | � q Wd S( N( R Rd ( R3 R RD R ( RG Ra ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRJ � s
c s0 | � � � | | j � | � | j | | � S( N( R Rd ( R3 R RD ( RG Ra ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRK � s
N( RC R RL R; R ( R R R RJ RK ( ( RG Ra sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt MessageEncoder� s
c sv d j t d t j � t d t j � t | � t d t j � g � � t d t j � � t � � � � f d � } | S( s� Encoder for extensions of MessageSet.
The message set message looks like this:
message MessageSet {
repeated group Item = 1 {
required int32 type_id = 2;
required string message = 3;
}
}
R: i i i c s: | � � � | | j � | � | j | | � | � � S( N( R Rd ( R3 R RD ( t end_bytesRG t start_bytes( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRK s
( R> RC R Rg R^ R@ RL Rh R; ( R RK ( ( Rk RG Rl sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt MessageSetItemEncoder s c s4 | j � t | j t t � � � � f d � } | S( s� Encoder for extensions of MessageSet.
Maps always have a wire format like this:
message MapEntry {
key_type key = 1;
value_type value = 2;
}
repeated MapEntry map = N;
c s_ | r t | j � � n | } x: | D]2 } � j d | d | | � } � | | | � q% Wd S( NR% R ( t sortedt keysR& ( R3 R RD t
value_keysR% R) ( t encode_messageR, ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyRK 5 s
( R, Rj R- R. ( R/ RK ( ( Rq R, sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt
MapEncoder& s (D t __doc__t
__author__RR R1 t google.protobuf.internalR RV RW R R R R R R t
Int32Sizert
Int64Sizert EnumSizert UInt32Sizert UInt64Sizert ZigZagEncodet SInt32Sizert SInt64Sizert Fixed32Sizert
SFixed32Sizert
FloatSizert Fixed64Sizert
SFixed64Sizert DoubleSizert BoolSizerR R R! R" R$ R0 R7 R9 R; t _EncodeSignedVarintR@ RC RN RO RU R[ R^ t Int32Encodert Int64Encodert EnumEncodert
UInt32Encodert
UInt64Encodert
SInt32Encodert
SInt64Encodert WIRETYPE_FIXED32t Fixed32Encodert WIRETYPE_FIXED64t Fixed64Encodert SFixed32Encodert SFixed64Encodert FloatEncodert
DoubleEncoderR_ Rb Rc Ri Rj Rm Rr ( ( ( sD /usr/lib/python2.7/site-packages/google/protobuf/internal/encoder.pyt A sn !
) " & R %